Unlocking AI's Knowledge Work Potential
Conveying meaning and transforming information into knowledge
The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. Frequently the messages have meaning.
– Claude Shannon, the father of information theory
With generative AI now prevalent, James Gleick’s book, The Information: A History, a Theory, a Flood, is even more fascinating today than when it was published twelve years ago; it delves into information theory, which is concerned with how information is processed and used. For thousands of years, humanity’s main obsession was on sending messages quickly across vast distances and without information loss. In the last century, humans have perfected the science of transmission from dots and dashes over telegraph wires to digital bits flying around the globe losslessly at breakneck speeds. While some may believe that we have exhausted the possibilities of information theory, I believe that we are on the cusp of a new era, one that is yet to be fully explored.
The birth of information theory came with its ruthless sacrifice of meaning–the very quality that gives information its value and its purpose. Introducing The Mathematical Theory of Communication1, Shannon had to be blunt. He simply declared meaning to be “irrelevant to the engineering problem.” Forget human psychology; abandon subjectivity.
– James Gleick, a science writer and historian
If the last century was about sacrificing meaning in the pursuit of of transmitting information as-is, the next century will be focused on meaning itself. The subjective nature of information has been explored through philosophy for ages, but the process of conveying meaning and transforming information into knowledge was never considered an engineering problem that could solved with machines, until now. Concepts such as “prompt engineering” have gone mainstream and soon the world will realize that engineering prompts is only the beginning.
Conveying Meaning, Beyond Messages
Large language models can convey the core meaning of a message rather than the message verbatim. This is accomplished by morphing messages into various forms along the way.
This visual jest from Kevin Cannon took off on Twitter but in earnest, it illustrates the promise of AI as a communication tool. We all speak our own dialects in our own minds. Major languages such as English are a powerful yet imperfect way for transmitting meaning. As each of us transform thoughts into words or interpret writing into thoughts, meaning is lost in translation because we interpret specific language differently based on our own lens and lived experience.
Communication without AI
English is actually many different languages–as many, perhaps, as there are English speakers–each with different statistics. It also spawns artificial dialects: the language of symbolic logic, […] and the language one questioner called “Airplanese,” employed by control towers and pilots.
– James Gleick
In a world without AI assistance, we’re left to encode and decode meaning ourselves which can often lead to miscommunication.
Communication with AI
To correct for the multiverse of meanings that words contain, we can imbue AI with our own lens and have it act as a universal translator2. There are many implications here for industry. One example for software is that in the near future, AI could replace developers as the universal API and glue for software, freeing up teams to focus on higher leverage architectural decisions.
AI as a Reasoning Machine
At the end of AI Builder’s Handbook // Part 1 published back in February (which feels like two lifetimes ago on the generative AI timeline), I emphasized the importance of using AI for its reasoning abilities, rather than solely for text generation. In the past month, support for exploring AI’s reasoning abilities has poured in, with a notable endorsement coming from Sam Altman. In a recent interview, he mentions how not using AI as a reasoning engine is a missed opportunity.
We’re treating these models […] as a database instead of using the model as a reasoning engine.
Much like how physical machines help us with manual labor, reasoning machines will help us with knowledge work. To better understand knowledge work, let’s dig deeper into the semantics of information theory.
Information theorists define ‘information’ in a compelling yet counterintuitive way, where ‘information’ represents the amount of uncertainty, surprise, and entropy in a message. With this in mind, we can see how transforming the uncertainty of 'information' into knowledge is, at its core, knowledge work.
After reexamining the tweet, it’s apparent that AI is indeed performing knowledge work – the conversion of information between various states. It takes the sender's initial signal, expands it into a detail-rich email, and then condenses the email into a clear message for the receiver. AI, through reasoning, is capable of both expanding and condensing the uncertainty of a message to suit the needs of any communication channel.
It takes a human–or, let’s say, a “cognitive agent”–to take a signal and turn it into information. “Beauty is in the eye of the beholder, and information is in the head of the receiver,” says Fred Dretske.
– James Gleick
It was once thought that this kind of knowledge work was solely in the domain of humans. Today, we arrive at this moment, abreast with generative AI and large language models. We are no longer the only ones on this planet who can reason.
Next Time, A Framework
Now that we know the ‘why’ behind the transformative nature of AI, we'll return next week to examine the ‘how’ through a visual framework that helps you map out knowledge workflows. With that framework, we can identify the bottlenecks in your work that can be relieved with AI.
A Mathematical Theory of Communication is Claude Shannon’s seminal paper from 1948. Fun fact, Anthropic’s AI Claude is named after Claude Shannon.
If we add brain computer interfaces into the communication mix, we might be able to create telepathy. What is a thought other than an embedding in our brain? Of course, some may think it unwise to designate AI as the courier for all of our communication. That may leave us vulnerable to manipulation should we lose control over our AI messengers but one could argue that's already the status quo today. We have behemoth tech companies dictating “the algorithm” while individuals are left to fend for ourselves with our feeble minds. I digress; I’ll stop myself here before I fall into into a rabbit hole – a post for another time.