Discovering Maps of Meaning
This graph shows the arcs of desperation vs relief over the course of the Lord of the Rings trilogy:
(Despair is at the top and relief is at the bottom, and the time axis begins on the left — the yellow clouds just represent density. The highlighted point can be read to the right - this paragraph is interpreted as the most relieving moment in the trilogy!)
And this graph shows the arc between stagnation and the sense of adventure:
(Here points towards the top are more “stagnant” - funnily enough, the highlighted point, the peak, is the meeting with the Ents.)
And this graph shows the degree of superficiality of my writing on twitter over the last ~ decade:
(Here points towards the top are more “superficial”)
And this graph shows how loving they were:
(Points on the left are older, whereas points towards the top of the graph express more “struggle.”)
This graph shows how I swing between technical and emotional topics, and how my writing has shifted over time to be less technical:
(Here points towards the top are more “technical”)
If we analyze spiritual texts like the Bible or the Tao-Te-Ching, we can interpret it along axes like how “Spiritual” vs how “Mundane” the passages are. The Bible is interesting as you can see how different the semantics are between sections:
(Here the vertical axis represents mundanity vs spirituality, with the top of the graph representing the more spiritual. Points are colorized by chaos vs order, where the book of job stands out as particularly “chaotic.”)
Here’s that most chaotic paragraph:
(same as above)
And here we see the Tao Te Ching compared to the Bible - by this measure the Tao Te Ching is considered exceedingly mundane:
(Here the vertical axis represents the spiritual again, and the color represents whether the text is from the Bible vs the TTC)
How does this work?
New Systems for Understanding
AI tools, especially Large Language Models (LLMs), offer the promise of helping us dramatically reduce complexity. LLMs can process vast amounts of information and provide us with simple, concise summaries. An implication of these systems is that LLMs can turn content into numerical representations called "embeddings" that represent the underlying meanings and relationships of text. Think of these embeddings like coordinates on a map: similar concepts are placed near each other, while different ones are further apart. Unlike a flat map that we might hold in our hands with two axes, these embeddings operate on hundreds or even thousands of axes, allowing for incredibly nuanced representations.
Embeddings are more than just a technical detail—they represent a fundamental shift in how we engage with information. Embeddings allow us to work abstractly with structures of meaning, rather than exact word-matches or calculations. They increase how quickly we can find and organize information, and how quickly we can learn and solve problems.
Turning Symphonies into Solo-Pieces
So we know that we can turn content into coordinate systems - but turning text into many thousands of random-seeming numbers is not really that useful to us humans — so far we’ve just made things way less readable. The challenge now is bringing these many thousands of numbers into something we can actually understand. One place to start is by compressing the information contained in these numbers. There’s a fast, efficient technique called Principal Component Analysis (PCA), which works a bit like rewriting a symphony piece for just a handful of instruments—you can still capture key patterns and themes of the broader piece, but with far less complexity.
When we apply this distillation technique to embeddings some interesting things happen. We can turn our hundreds of axes into just a ~dozen or so. When we do this we can see ideas and themes cluster together, relationships become visible, and the threads of meaning begin to appear as coherent structures. Clear topics and dichotomies arcs arise. Our new PCA-compressed coordinates maximize “descriptive power” much like how image or audio compression tries to preserve it’s content. When we compress language, we see that the axes on our map represent important conceptual or philosophical differences across the body of text.
But we still have one issue - we might have turned our map of meaning into just a few axes, but we still don’t know what they actually represent — we don’t actually have any sense of what our “X’s” or “Y’s” even are. This has a fun solution - take the most extreme examples (say the top and bottom ten) for each axis, give these opposite sample sets to an AI, and have it describe the difference. When we do this, we see that our coordinate system represents axes like “Tension vs Release” or “Grief vs Joy.” These axes depend on what we’re analyzing — for example spiritual texts are explained by entirely different dimensions like “Belief vs Doubt” or “Destruction vs Salvation” whereas tweets are explained by axes like “frivolity” or “technicality.“
Here’s a gallery of some more examples!
Closing
After experimenting with these systems for a couple years I have some thoughts — I think a few things are likely to happen:
Identifying and organizing complex, abstract concepts will continue to get easier
The accuracy of these systems accelerate along with any advancements to LLM systems
We’ll be able to individually develop clearer and clearer understandings of complex concepts & their relationships
Complex concepts and relationships can be mapped out for us, automatically
So to paint a picture of what this might look like - the act of opening hundreds of tabs of things that interest you could automatically create a landscape of your interests — their relationships could be identified, visualized, or acted upon in meaningful ways. These maps could suggest you things that might interest you next. The books you read or movies you watch can be dynamically compared & contrasted by their themes or narrative arcs. You could visually see how major events in your life changed your personality and the things that interested you.
Systems like this mean that the interplay of cultural spheres and their organizing concepts can be mapped in relation to one-another, with key ideological differentiators between spheres clearly identified. Deeper understandings of social systems and organizational cultures might mean we can more accurately predict the behaviors of companies or large groups of people. The evolution and interaction of more central, organizing beliefs in groups can be more easily mapped over time, and the impact of major historical events on a society could be visualized and interpreted.
In many ways all of these are already happening - algorithmic systems model our habits and interests to suggest us social media posts, music, search results, and videos. We model engagement history and the frequency of words and topics in different spheres. The important thing to realize is that these systems have already been colossal organizing forces on our lives and societies, and any improvement in their depth and explanatory power means sweeping, foundational changes to our lives and cultures. Corporations use these systems to understand and influence our behavior, but I think we can also use these systems for self-discovery, expression, and for organizing our thoughts and lives.
As AI systems begin to understand on a deeper level not just what we value and think about, but how these concepts all intertwine and influence one-another, there’s tremendous opportunity and tremendous risk. AI systems are steadily integrating into every system we use on a regular basis. This comes with many individual opportunities and also implies societal vulnerabilities. I think one way to increase our resilience against the latter is to build tools for the former, where we increase our ability to interpret and synthesize the complexity of our lives and influences, making us less susceptible to algorithmic architectures of influence or control.
Thanks for reading! If you’d like to support me in my research or creative work consider ordering something from my shop or commissioning one of my services!
