Carlos
De la Guardia

All notes on one page

Now

Researching artificial general intelligence

I’m currently exploring a variety of questions, and sharing what I learn in these notes and in my newsletter (where I hope to improve my writing, guided by Steven Pinker’s A Sense of Style). I’m also working on a short book.

Making music

I’ve gotten much better at drumming over the last few months (thanks to Drumeo). Now I’m working on recording drum parts for my music, which is inspired by the prog rock band Rush and by various classical composers (e.g. Debussy). To improve my music, I’m also learning about music composition, music notation, and the art of storytelling in fiction. Inspired by Ichika Nito, I’m recording 100 microsongs to put on Youtube.

Roam book and Roam site generator

As I continue to use Roam for my all my personal and research notes, I’ve learned to use it more effectively and aim to eventually share those lessons in a book. I’ve also created a tool to easily publish a selection of my Roam notes online (this site was published that way, and this page was written in Roam). Once I make it easier to use, I aim to start selling it as a product others can use to publish a notes website like this one.

P.S. This page is inspired by Derek Sivers.

Writers can learn from functional programming

Functional programmers use functions to write code in small, self-contained chunks. Writers can Write small, self-contained notes.

There are different reasons to create a new function:

  • Break up a large and unmanageable one.
  • Avoid duplication by create a dedicated function for an common operation that would otherwise be repeated in multiple places. This helps one focus on creating a single, well-written implementation of the code.
  • Create a stub implementation. You may write the name for a function that you haven’t yet implemented, since you know you’ll need it, but don’t want to write it yet. The case of writing is a bit different from code, since the programmer usually knows what needs to happen, and then needs to figure out how to do it - they need to write the implementation. In the case of writing, one sometimes knows what to say, but not yet how to say it. That is like the programmer. But sometimes, while writing, you may just have a question, and have no idea where it leads. You could be making an argument, and maybe wonder if it is correct. In this case, you could continue writing the present note, create a stub note for the question you want to turn to next. This again helps you work on just one small idea at a time.

Writing prose is like writing code

There are many ways in which writing prose is like writing code. Both often involve ambitious projects, and encoding large amounts of precisely-stated knowledge in a complex and interconnected way. The mechanisms of abstraction, modularity, and so on are different, but the same fundamental problems appear in both domains. Many of these problems are the result of scale - of creating things which grow larger and large - more difficult to keep in mind and understand all at once.

Also, writers can make use of fundamental data structures like sequences, sets, outlines, and networks. I’m not sure writers need to keep these structures explicitly in mind, after all, it’s much more important to think in terms of things like paragraphs, sentences, and so on.

Avoid getting stuck

I think this is the primary danger for any individual. One can get stuck with respect to just one project or pursuit, or perhaps stuck with regard to everything. Because Stasis is hell, I think the main job of any tool or way of life, workflow, or whatever, is to avoid getting stuck.

Break the writing process into small steps

To Avoid getting stuck, one can to Break problems down into easier problems. When it comes to writing, one can Write small, self-contained notes, but that is not enough. A book contains many ideas, and has to be comprehensible to readers, not only the author. This introduces many new problems, the first one being how to arrange all those hundreds of ideas into a coherent structure. This can’t be done all at once. Ideas are related in all sorts of ways, and it’s a creative challenge just to figure out the order in which to present two ideas, much less hundreds.

Moreover, a book is not just the concatenation of existing ideas in the proper order. Some ideas will appear in multiple forms in multiple places. Some will be dropped. Some will be elaborated upon with background knowledge and helpful examples. New ideas will be introduced, and old ideas expressed anew, in the process of writing a manuscript others can read. The process of integrating all these ideas will turn up unforeseen conflicts and problems between ideas which hadn’t been considered together in such detail before. Trying to express an idea clearly and succinctly requires understanding it well, and making it sharper, and this can lead to all sorts of new ideas, and changes to old ones.

These are all problems of substance, but there are also problems of communication - of language. That is another layer of problems - creating sharp sentences, paragraphs, pages, and chapters. Putting things in perspective - emphasizing what is most important, and sequencing things so they are easy to understand, and one is exposed to one idea only after prerequisite ideas have been introduced.

In other words, a books’ worth of ideas is not a book. Making it into a book (or essay, or whatever) will introduce many new problems (in the good sense of the word, not necessarily distressing ones). Problems of substance and of communication. If you cannot break these problems down to a manageable size, you cannot write the book. You’ll get stuck.

So, how do you break these problems into a manageable size? What are the problems?

The curse of knowledge - providing the relevant background knowledge and context for others to understand not only what you are saying, but why it is important and interesting.

Basic clarity and concision - creating clear sentences, using the right words.

Coherence and continuity - making things flow from one sentence to the next.

Making things tractable for the reader - chapter titles and section breaks allow the reader to skip around, and to find and refer to sections they care about, to recall what they’re read.

The problems of structure and modularity - understanding the ideas at various levels of abstraction. The high-level patterns and relationships. Some ideas are more important than others, more high level, more fundamental, whatever. Some ideas belong in a certain sequence, or in a certain group. Maybe they are part of the same topic or argument. Just like pages allow you to make small, self-contained chunks, you need to create chapters and outlines and other things to create small (relative to the size of the whole book), self-contained chunks. Just like with notes, that is nontrivial and creative.

What about Chunk ideas into manageable groups using outlines?

Use your writing

While the act of writing can be useful on it own, the text one creates can be a great help in future thinking. If you Write small, self-contained notes, and give them good titles, they become easy to build upon in other notes. Treat your notes as an API, as building blocks and stepping stones to become familiar with and build upon. This chunking of ideas helps you think about things at higher levels of abstraction, and takes things which might have been difficult to think about before, and makes them easy. One note can also constrain other notes.

References

The Use of Knowledge

Prose and Programs

Break problems down into easier problems

Many problems are too difficult to solve in one go. To make progress on them - to Avoid getting stuck - one has to divide the problem into many smaller, simpler, easier ones. This is true of problem-solving in every domain, from AGI research to music and writing books.

If you don’t break the problem down, it may be needlessly difficult or painful to work on. It may even be so overwhelming you become stuck and make no progress at all.

Incidentally, breaking a problem down is creative (and fun!) and a special case of a more fundamental mental phenomenon: Minds divide the world into independent, interacting objects.

Examples

Break the writing process into small steps

Break the editing process into small steps

Write small, self-contained notes

Break the editing process into small steps

I sometimes find it difficult to improve a draft. To reorder, restate, and cut things. I think it’s because I mix up several different questions all at once.

  • Is this true?
  • Have I given a good argument for it? (Or against alternatives?)
  • What are the alternatives?
  • Is this clear?
  • What is a better way to write this?
  • Should I break these idea into separate notes?
  • Is this interesting, important, or useful?
  • If it is, have I explained why?
  • How should I start the piece?
  • What is the scope of the piece? What should be included or excluded?

Switching between many such questions in a frenzy is a poor way to work. Or, perhaps it’s not a problem for the process to be chaotic, but it should progress in small steps. By switching between all the above questions, things can seem overwhelming, and one can’t take even a single step forward.

How can you break up these things into small, manageable steps? How can you take a draft and start reshaping it slowly-but-surely into something better?

Well, one thing you can do is just go through a draft several times, each time trying to fix just one kind of problem. Maybe the first time, you just cut unnecessary things. Then you can try to rewrite ideas to be shorter. Then you can put things in the proper sequence. Then you can try to improve the points - or perhaps that should come first.

Maybe the really central and first point of revising should be to revise the ideas themselves. To find flaws and gaps. That is, after all, how I have approached writing blog posts. Try to state the problem or starting point clearly, and move from there in an orderly and logical way. I first thought I might avoid this as I write my notes. I thought that sort of writing might constitute an unnecessary and wasteful level of polish. But, perhaps it’s just the opposite, and expressing an idea from beginning to end in a clear and continuous chain of argument is the advantage of writing real prose rather than just bullet-point notes. So, I guess I’d like to revise my idea about sharing public notes.

Instead of sharing my half-baked ideas, the point is to improve them dramatically. To try to get to the crux of the matter, and to put them forward in an extremely clear, crisp, and compelling way. Each time I turn to a note can be an attempt to improve its quality in a fundamental way.

Having said that, this is a costly thing to do, and it means that I may not publish as often as I first thought. Then again, if you begin publishing one compelling note per day this way, that will constitute a dramatic increase in your output.

Real prose at a rate of 300 words per day yields 100k+ words per year. That’s no small thing.

The other side of this is that it means not everything should be a public note. Not everything merits the effort. Some things should remain private. This level of investment in each idea means that you’ll work on only the important ones. That is probably as it should be.

Don't confuse the listener

When composing music, there’s a constant tension between familiarity and novelty - repetition and change. Pure repetition is boring. Pure change is confusing.

I quite like Rush, and they’ve managed to produce music which is ambitious and full of variety and change, but also easy to follow. Transitions makes sense. One thing lead into to another. Things never seem haphazard or chaotic. There’s almost always a steady groove one can tap one’s foot to, even if it’s in an odd time signature.

On the other hand, much prog rock and classical music is very confusing (especially to the untrained ear). Things lack a steady pulse, speed up and slow down, change rhythms continually, take long pauses and restart unpredictably.

Whether one is creating melodies or drum parts, one can make things easy to follow by limiting major changes to those between sections (and by doing work on either side of section transitions to make them ‘make sense’). Within a section, one can explore a variety of ideas in the form of variations of a core idea. In this way, one introduces novelty and surprise in the details, but maintains a predictable overall structure.

Incidentally, it’s not only easier to listen to. It’s easier to compose variations than entirely new ideas.

The knowledge-based view

All living things depend on knowledge. This is especially true of people, since we so obviously rely on science, culture, and technology. But, even plants and animals depend on the knowledge embodied in genes, without which they couldn’t manage the kaleidoscopic chemical complexity of even the simplest cell.

But, there was once a time when no knowledge existed - not in any book, brain, or bacterium. It all had to be created. How? Variation and selection, or conjecture and criticism (in the words of Karl Popper).

Because all knowledge-creation depends on variation and selection, it’s possible to compare knowledge-creating systems. and their differences make such comparisons fruitful. Lessons learned in one domain can be applied to others. Many questions about knowledge apply to every system that depends on or creates it. I often try to apply ideas about AGI to biological evolution, machine learning, and business, and see whether they work. They often don’t, and I learn that they are special cases of more general things which do apply in every case. The same is true of many obstacles to knowledge-creation.

Prose and programs are fundamentally different

Programs aim at usefulness, while prose aims at truth, and truth and usefulness are fundamentally different.

This means that a collection of programs has a fundamentally different structure from a collection of prose. Also, the process of creating a large program is also unlike the process of creating a large work of prose.

Programming is fundamentally about processing information - doing something. Every bit of code takes something in and spits something out.

This is very different from prose. Prose aims at truth. To make an important point which is true, and to argue that it is true, or superior to alternative ideas.

Programming is usually not openended in the way writing is. This means that things have a fundamentally different structure. (I suppose one can come up with unexpected new features, etc. But in that case, the application is openended, but its implementation is not. The implementation has only one purpose: to instantiate the application.)

Write small, self-contained notes

It eliminates the distraction of other ideas which can and ought to be considered independently. It excludes things. It introduces boundaries.

Second, it allows you to use the idea. It has a clear title that turns it into a single chunk in your mind. It provides an interface which allows it to be used elsewhere. When your writing takes the form of chaotic mountains of text, the good ideas are hidden among the bad and irrelevant ones, and tend to die a quiet death because it is so difficult to come across them again.

Third, it forces you to put related, but distinct, ideas into separate notes that you link to. This makes the connections between these ideas clear and explicit - and also very limited and easy to understand.

Being forced to create a title means that you sharpen the idea and try to make a single point (and not a different point).

Keeping one’s notes small and self-contained is a challenge, but I think it’s one that’s creative and interesting, and entirely different from trying to work on a large and complex text. The first thing is to simply notice when a note is too long, complicated, and hard to work with. As a rule of thumb, if the entire idea can’t fit on screen, it’s probably too big and should be split up. The real challenge is figuring out how to split it up. The goal is to end up with two or more separate notes which each contain a single complete idea which can be considered on its own.

In some cases, this may be as simple as cutting and pasting a chunk of text into a new note, but often the process of dividing things is more involved. The two separate ideas might be all mixed together and have to be separated, and maybe restated so they are independent. Also, the ideas can change in the process of breaking them up. Each note may become more specific and focused, and the titles can change. This is very much like refactoring code. Once you’ve got everything in place, you can make it much clearer and simpler, although the initial process of creation is quite messy.

Since these ideas started off in a single note, they’ll almost certainly be closely related, and one (or both) will link to the other. Also, it’s often useful to add an idea to a small outline in a separate note (which is tagged as an outline). Often, one ends up working with a collection of ideas in a handful of related notes, and switching between them.

References

- [The Life of a File](https://www.youtube.com/watch?v=XpDsk374LDE), Evan Czaplicki
- [Scaling Elm Apps](https://www.youtube.com/watch?v=DoA4Txr4GUs&feature=emb_title) Richard Feldman

Create boundaries and interfaces between ideas

Boundaries make things tractable. They limit how much you have to keep in mind while you work on an idea. They help you Break problems down into easier problems. Interfaces make things usable. They provide an quick description of the content of an idea and a way to refer to it, and therefore build on it. This is why you should Write small, self-contained notes and link them together.

The clearest and perhaps most important boundary is the page boundary, but every spacebar, comma, period, paragraph/block break, or outline indent also creates an important boundary.

Similarly, the clearest and perhaps most important interface/interaction is the page link/reference. But, ideas interface with each other in many different ways and at many different scales. Words can be found in a single phrase, linked by the rules of grammar. Sentences can follow each other, linked by transitional phrases like ‘for example’. Paragraphs can be made adjacent, showing they are closely related.

Each of these boundaries and interfaces has a role to play. It would be preposterous to put everything in one sentence, paragraph, or page, or to put every idea in a separate page, say. In each case, the primary concern is about clarity. It’s hard to read things when they’re too long, whether they’re a phrase, sentence, paragraph, or page. When something is too long, break it down. Use a boundary of whatever sort is needed.

However, almost every time you create a boundary, you’re splitting up ideas which are related in an important way. After all, they were created in the same sentence, paragraph, or page for a reason. A boundary can make it hard to see how the ideas are related, or cause them to become separated from each other like a ripped halves of sheet of paper scattering in the wind. Both of these problems become more pronounced as you move from small boundaries like commas to large ones like separate notes. This is why it is important to tie sentences and paragraphs together with transitional phrases, and to tie notes together with links.

In practice, most writers will have already developed a sense for when and how to introduce boundaries and interfaces between ideas - except at the level of pages and links. Roam’s introduction of links - a kind of interface - changes the way you use boundaries, and it lets you break ideas into smaller chunks than you otherwise could. As I mentioned, breaking ideas apart always runs the risk of making their connections hard to see, or altogether losing the fact they are connected and should sometimes be considered together. Coping with that problem is uniquely easy with page links.

Ironically, linking to a note using its descriptive title makes the connections between ideas clearer. In a single note, you only know that one chunk of text follows another. It can be hard to tell where one idea ends and another begins. And, this problem becomes worse as more ideas are present in a document. So, page references offer a key tool that allows you to express connections in a very powerful way.

(All the same arguments apply to block references, too, though that is even more foreign than page links. At least with page links, we can rely on our knowledge of the web. Block references are a bit more like referencing tweets, which is less common.)

Working on boundaries and interfaces helps you better understand the ideas, incidentally, since you are working to find not only how to express them in a short, self-contained way, but also explicitly define the relationships between multiple such chunks. This has the effect of taking a very complex bundle of ideas and pulling it apart into distinct, coherent ideas with distinct, specific connections. So, this is largely about constraints. This is big with Elm, where the whole idea is that, as you scale up, you need to find ways to constrain things so that when you try to solve a bug, for instance, you can quickly determine a small set of possibilities, because only a few thing can possibly have affected the relevant code. If the system is less constrained, then many more things could have affected the code and caused the bug. Likewise here, one wants to specify the connections fairly precisely. Not quite for finding bugs.

By the way, tags allow you to put a note in a certain set of notes, and this is also a useful way to create boundaries. But, instead of being about making a single idea manageable, it’s about how to Chunk ideas into manageable groups using outlines.

References

Scaling Elm Apps Richard Feldman

Notes

Howdy! These are my notes on artificial general intelligence and related interests. You can explore them in several ways:

Btw, I use Roam to write my notes, and then I export some of them here using a script I wrote. Here’s a twitter thread with more info.