Smart AIs are now everywhere

At the end of last year, there was only one publicly available GPT-4/Gen2 class model, and that was GPT-4. Now there are between six and ten such models, and some of them are open weights, which means they are free for anyone to use or modify. From the US we have OpenAI’s GPT-4o, Anthropic’s Claude Sonnet 3.5, Google’s Gemini 1.5, the open Llama 3.2 from Meta, Elon Musk’s Grok 2, and Amazon’s new Nova. Chinese companies have released three open multi-lingual models that appear to have GPT-4 class performance, notably Alibaba’s Qwen, R1’s DeepSeek, and 01.ai’s Yi. Europe has a lone entrant in the space, France’s Mistral. What this word salad of confusing names means is that building capable AIs did not involve some magical formula only OpenAI had, but was available to companies with computer science talent and the ability to get the chips and power needed to train a model.

In fact, GPT-4 level artificial intelligence, so startling when it was released that it led to considerable anxiety about the future, can now be run on my home computer. Meta’s newest small model, released this month, named Llama 3.3, offers similar performance and can operate entirely offline on my gaming PC. And the new, tiny Phi 4 from Microsoft is GPT-4 level and can almost run on your phone, while its slightly less capable predecessor, Phi 3.5, certainly can. Intelligence, of a sort, is available on demand.

And, as I have discussed (and will post about again soon), these ubiquitous AIs are now starting to power agents, autonomous AIs that can pursue their own goals. You can see what that means in this post, where I use early agents to do comparison shopping and monitor a construction site.

VERY smart AIs are now here

All of this means that if GPT-4 level performance was the maximum an AI could achieve, that would likely be enough for us to have five to ten years of continued change as we got used to their capabilities. But there isn’t a sign that a major slowdown in AI development is imminent. We know this because the last month has had two other significant releases - the first sign of the Gen3 models (you can think of these as GPT-5 class models) and the release of the o1 models that can “think” before answering, effectively making them much better reasoners than other LLMs. We are in the early days of Gen3 releases, so I am not going to write about them too much in this post, but I do want to talk about o1.

I discussed the o1 release when it came out in early o1-preview form, but two more sophisticated variants, o1 and o1-pro, have considerably increased power. These models spend time invisibly “thinking” - mimicking human logical problem solving - before answering questions. This approach, called test time compute, turns out to be a key to making models better at problem solving. In fact, these models are now smart enough to make meaningful contributions to research, in ways big and small.

As one fun example, I read an article about a recent social media panic - an academic paper suggested that black plastic utensils could poison you because they were partially made with recycled e-waste. A compound called BDE-209 could leach from these utensils at such a high rate, the paper suggested, that it would approach the safe levels of dosage established by the EPA. A lot of people threw away their spatulas, but McGill University’s Joe Schwarcz thought this didn’t make sense and identified a math error where the authors incorrectly multiplied the dosage of BDE-209 by a factor of 10 on the seventh page of the article - an error missed by the paper’s authors and peer reviewers. I was curious if o1 could spot this error. So, from my phone, I pasted in the text of the PDF and typed: “carefully check the math in this paper.” That was it. o1 spotted the error immediately (other AI models did not).

When models are capable enough to not just process an entire academic paper, but to understand the context in which “checking math” makes sense, and then actually check the results successfully, that radically changes what AIs can do. In fact, my experiment, along with others doing the same thing, helped inspire an effort to see how often o1 can find errors in the scientific literature. We don’t know how frequently o1 can pull off this sort of feat, but it seems important to find out, as it points to a new frontier of capabilities.

In fact, even the earlier version of o1, the preview model, seems to represent a leap in scientific ability. A bombshell of a medical working paper from Harvard, Stanford, and other researchers concluded that “o1-preview demonstrates superhuman performance [emphasis mine] in differential diagnosis, diagnostic clinical reasoning, and management reasoning, superior in multiple domains compared to prior model generations and human physicians." The paper has not been through peer review yet, and it does not suggest that AI can replace doctors, but it, along with the results above, does suggest a changing world where not using AI as a second opinion may soon be a mistake.

Potentially more significantly, I have increasingly been told by researchers that o1, and especially o1-pro, is generating novel ideas and solving unexpected problems in their field (here is one case). The issue is that only experts can now evaluate whether the AI is wrong or right. As an example, my very smart colleague at Wharton, Daniel Rock, asked me to give o1-pro a challenge: “ask it to prove, using a proof that isn’t in the literature, the universal function approximation theorem for neural networks without 1) assuming infinitely wide layers and 2) for more than 2 layers.” Here is what it wrote back:

Is this right? I have no idea. This is beyond my fields of expertise. Daniel and other experts who looked at it couldn’t tell whether it was right at first glance, either, but felt it was interesting enough to look into. It turns out the proof has errors (though it might be that more interactions with o1-pro could fix them). But the results still introduced some novel approaches that spurred further thinking. As Daniel noted to me, when used by researchers, o1 doesn’t need to be right to be useful: “Asking o1 to complete proofs in creative ways is effectively asking it to be a research colleague. The model doesn't have to get proofs right to be useful, it just has to help us be better researchers.”

We now have an AI that seems to be able to address very hard, PhD-level problems, or at least work productively as a co-intelligence for researchers trying to solve them. Of course, the issue is that you don’t actually know if these answers are right unless you are a PhD in a field yourself, creating a new set of challenges in AI evaluation. Further testing will be needed to understand how useful it is, and in what fields, but this new frontier in AI ability is worth watching.

AIs can watch and talk to you

We have had AI voice models for a few months, but the last week saw the introduction of a new capability - vision. Both ChatGPT and Gemini can now see live video and interact with voice simultaneously. For example, I can now share a live screen with Gemini’s new small Gen3 model, Gemini 2.0 Flash. You should watch it give me feedback on a draft of this post to see what this feels like:

Or even better, try it yourself for free. Seriously, it is worth experiencing what this system can do. Gemini 2.0 Flash is still a small model with a limited memory, but you start to see the point here. Models that can interact with humans in real time through the most common human senses - vision and voice - turn AI into present companions, in the room with you, rather than entities trapped in a chat box on your computer. The fact that ChatGPT Advanced Voice Mode can do the same thing from your phone means this capability is widely available to millions of users. The implications are going to be quite profound as AI becomes more present in our lives.

AI video suddenly got very good

AI image creation has become really impressive over the past year, with models that can run on my laptop producing images that are indistinguishable from real photographs. They have also become much easier to direct, responding appropriately for the prompts “otter on a plane using bluetooth” and “otter on a plane using wifi.” If you want to experiment yourself, Google’s ImageFX is a really easy interface for using the powerful Imagen 3 model which was released in the last week.

But the real leap in the last week has come from AI text-to-video generators. Previously, AI models from Chinese companies generally represented the state-of-the-art in video generation, including impressive systems like Kling, as well as some open models. But the situation is changing rapidly. First, OpenAI released its powerful Sora tool and then Google, in what has become a theme of late, released its even more powerful Veo 2 video creator. You can play with Sora now if you subscribe to ChatGPT Plus, and it is worth doing, but I got early access to Veo 2 (coming in a month or two, apparently) and it is… astonishing.

It is always better to show than tell, so take a look at this compilation of 8 second clips (the limit for right now, though it can apparently do much longer movies). I provide the exact prompt in each clip, and the clips are only selected from the very first set of movies that Veo 2 made (it creates four clips at a time), so there is no cherry-picking from many examples. Pay attention to the apparent weight and heft of objects, shadows and reflection, the consistency across scenes as hair style and details are maintained, and how close the scenes are to what I asked for (the red balloon is there, if you look for it). There are errors, but they are now much harder to spot at first glance (though it still struggles with gymnastics, which are very hard for video models). Really impressive.

What does this all mean?

I will save a more detailed reflection for a future post, but the lesson to take away from this is that, for better and for worse, we are far from seeing the end of AI advancement. What's remarkable isn't just the individual breakthroughs - AIs checking math papers, generating nearly cinema-quality video clips, or running on gaming PCs. It's the pace and breadth of change. A year ago, GPT-4 felt like a glimpse of the future. Now it's basically running on phones, while new models are catching errors that slip past academic peer review. This isn't steady progress - we're watching AI take uneven leaps past our ability to easily gauge its implications. And this suggests that the opportunity to shape how these technologies transform your field exists now, when the situation is fluid, and not after the transformation is complete.

Subscribe now

Share

1. Work that requires quantity. For example, the number of ideas you generate determines the quality of the best idea. You want to generate a lot of ideas in any brainstorming session. Most people stop after generating just a few ideas because they become exhausted but, the AI can provide hundreds that do not meaningfully repeat.1

2. Work where you are an expert and can assess quickly whether AI is good or bad. This can involve complicated and exacting work, but it relies on your expertise to determine whether the AI is providing valuable outputs. For example, o1, the new AI model from OpenAI, can solve some PhD-level problems, but it can be hard to know whether its answers are useful without being an expert yourself.

3. Work that involves summarizing large amounts of information, but where the downside of errors is low, and you are not expected to have detailed knowledge of the underlying information. AI is good at summarizing novel-length work, but less successful at fact-checking it.

4. Work that is mere translation between frames or perspectives. For example, you have developed a policy but now have to turn it into a dozen different training documents for different audiences in your organization. AI is very good at this sort of translation, increasing and decreasing complexity of documents so that people can understand them.

5. Work that will keep you moving forward. Little things often block our way, and a push might be all we need to accomplish it. When writing prior to AI, I might get stuck on a sentence and walk away from writing for an hour, but now I ask AI give me thirty distinct ways to end this sentence

   

6. Work where you know that AI is better than the Best Available Human that you can access, and where the failure modes of AI will not result in worse outcomes if it gets something wrong.

7. Work that contains some elements that you can understand but need help on the context or details. Tyler Cowen suggests using the AI as a companion when reading, because it allows you to ask infinite questions.

8. Work where you need variance, and where you will select the best answer as an editor or curator. Asking for a variety of solutions - give me 15 ways to rewrite this bullet in radically different styles, be creative - allows you to find ideas that might be interesting.

   

9. Work that research shows that AI is almost certainly helpful in - many kinds of coding, for example.

10. Work where you need a first pass view at what a hostile, friendly, or naive recipient might think.

    

11. Work that is entrepreneurial, where you are expected to stretch your expertise widely over many different disciplines, and where the alternative to a good-enough partner is to not be able to act at all. AI can be a surprisingly competent co-founder, helping give mentorship while also acting to build the documents, demos, and approaches that are otherwise likely to be outside your experience.

12. Work where you need a specific perspective, and where a simulated first pass from that perspective can be helpful, like reactions from fictional personas.

13. Work that is mere ritual, long severed from its purpose (like certain standardized reports that no one reads). What, in the words of Bob Sutton and Huggy Rao, scatters your attention and makes you less valuable? What work serves no useful purpose? In an ideal world, you would remove the work, but you can at least reduce its hold on you by having AI help. (Though make sure this is indeed the case, far too many people automate performance reviews, for example, which are meaningful only when done by a human)

14. Work where you want a second opinion. Give an AI access to the data and see if reaches the same conclusion.

15. Work that AIs can do better than humans. This is likely to be the fastest-growing category.

5 Times Not to Use AI

Before diving into the specific cases where AI use is problematic, we can set aside the obvious scenarios - using AI for illegal purposes, in high-stakes situations where errors could be catastrophic, or for decisions that ethically require human work. Beyond these clear-cut cases, here are five subtle but important areas where AI use can be counterproductive:

1. When you need to learn and synthesize new ideas or information. Asking for a summary is not the same as reading for yourself. Asking AI to solve a problem for you is not an effective way to learn, even if it feels like it should be. To learn something new, you are going to have to do the reading and thinking yourself, though you may still find an AI helpful for parts of the learning process.

2. When very high accuracy is required. The problem with AI errors, the infamous hallucinations, is that, because of how LLMs work, the errors are going to be very plausible. Hallucinations are therefore very hard to spot, and research suggests that people don’t even try, “falling asleep at the wheel” and not paying attention. Hallucinations can be reduced, but not eliminated. (However, many tasks in the real world are tolerant of error - humans make mistakes, too - and it may be that AI is less error-prone than humans in certain cases)

3. When you do not understand the failure modes of AI. AI doesn’t fail exactly like a human. You know it can hallucinate, but that is only one form of error: AIs often try to persuade you that they are right, or they might become sycophantic and agree with your incorrect answer. You need to use AI enough to understand these risks.

4. When the effort is the point. In many areas, people need to struggle with a topic to succeed - writers rewrite the same page, academics revisit a theory many times. By shortcutting that struggle, no matter how frustrating, you may lose the ability to reach the vital “aha” moment.

   

5. When AI is bad. This may seem obvious, but AI is bad at things you wouldn’t expect (counting the number of r’s in the word “strawberry”) and good at things you wouldn’t expect (writing a Shakespearean sonnet about how hard it is to count the number of r’s in the word strawberry where the first letter of every line spells out two fruits). Unfortunately, there is no general manual to tell you the shape of the Jagged Frontier of AI abilities, which are constantly evolving. Trial and error, and sharing information with peers, is vital to figuring this out.

Knowing when to use AI turns out to be a form of wisdom, not just technical knowledge. Like most wisdom, it's somewhat paradoxical: AI is often most useful where we're already expert enough to spot its mistakes, yet least helpful in the deep work that made us experts in the first place. It works best for tasks we could do ourselves but shouldn't waste time on, yet can actively harm our learning when we use it to skip necessary struggles. And perhaps most importantly, wisdom means knowing that these patterns will keep shifting as AI capabilities evolve, and as more research comes in, requiring us to keep questioning our assumptions about where it helps and where it hinders.

Subscribe now

Share

One reason is algorithmic aversion. We don’t like taking instructions from machines when they conflict with our judgement, which caused doctors to overrule the AI, even when it was accurate. But a second reason for this problem is very specific to working with Large Language Models. To people who aren’t used to using them, AI systems are surprisingly hard to get a handle on, resulting in a failure to benefit from their advice.

As a New York Times article on the paper reported: “they were treating [the AI] like a search engine for directed questions: ‘Is cirrhosis a risk factor for cancer? What are possible diagnoses for eye pain?’ It was only a fraction of the doctors who realized they could literally copy-paste in the entire case history into the chatbot and just ask it to give a comprehensive answer to the entire question.” This is not just an issue for doctors. In every classroom I teach in or organization I speak with, the vast majority of people have tried AI, but are often struggling with how to initially use it. And, as a result of that struggle, have not put in the 10 or so hours with AI that are required to really understand what it does.

There are many stumbling blocks: people treat AI like Google, asking it factual questions. But AI is not Google, and do not provide consistent, or even reliable, answers. Or people ask AI to write something for them and complain when it produces generic text. Or they can’t even figure out what to write at all, staring at a blinking cursor. In short, they can’t prompt the AI.

One common answer to these sorts of problems it that everyone should learn “prompt engineering,” the complex science (well, more of an art) of getting an AI to work in the ways you expect. There are a few problems with starting this way for most people, however1. First, the idea of a complex structure for prompting can be daunting and constraining. This is especially true because you do not need to be an expert in AI, or even know anything about programming or computers, to be an expert in using AI. Suggesting that complex upskilling is the place to start will discourage people from using AI. Second, prompt engineering implies that there is actually a clear science of getting AIs to act in the way that you want, when, in fact, researchers are still arguing over the most basic foundations of good prompting. This is because AIs are inconsistent and weird, and often have different results across different models. For example, they are sensitive to small changes in spacing or formatting; they get more accurate when you tell them to “read the question again;” they seem to respond better to politeness (but don’t overdo it); and they may get lazier in December, perhaps because they have picked up on the concept of winter break. Add to all of this the fact that getting good results out of AI without a lot of formal training is likely growing easier as larger models are less sensitive to changes in prompting technique than older AIs, and new techniques allow the AI to improve your prompt for you.

So, you can learn details of prompt engineering and the basics of how LLMs work, of course, but for most people, that is not a required starting place. You just need to use AI enough to get a feel for what you can use it for in your area of expertise. The most important thing to do is to get 10 or so hours of use with an advanced AI system. And to do that you just need to be a good-enough prompter to overcome the barriers that hold many AI users back. There are really two pathways to get started: good-enough prompting for tasks and good-enough prompting for thought.

Good Enough Task Prompting

One of the most useful ways to use AI is to help get things done. In my book, I talk about bringing the AI to the table, trying it out for all of your work tasks to see how well it does. I still think this is the right way to start. Often, you are told to do this by treating AI like an intern. In retrospect, however, I think that this particular analogy ends up making people use AI in very constrained ways. To put it bluntly, any recent frontier model (by which I mean Claude 3.5, ChatGPT-4o, Grok 2, Llama 3.1, or Gemini Pro 1.5) is likely much better than any intern you would hire, but also weirder.

Instead, let me propose a new analogy: treat AI like an infinitely patient new coworker who forgets everything you tell them each new conversation, one that comes highly recommended but whose actual abilities are not that clear. And I mean literally treat AI just like an infinitely patient new coworker who forgets everything you tell them each new conversation. Two parts of this are analogous to working with humans (being new on the job and being a coworker) and two of them are very alien (forgetting everything and being infinitely patient). We should start with where AIs are closest to humans, because that is the key to good-enough prompting

As it is a coworker, you want to work with it, not just give it orders, and you also want to learn out what it is good or bad at. Start by using it in areas of your expertise, where you are able to quickly figure out the shape of the jagged frontier of its ability. Because you are expert, you will be able to quickly assess where the AI is wrong or right. You do need to be prepared for it to give you plausible but wrong answers, but don’t let the risk of these hallucinations scare you off initially. Though hallucinations may be inevitable, you will learn where they are a big deal, and where they are not, over time. You can reduce the rate of hallucinations somewhat by giving the AI the ability to be wrong, for example, writing: if you’re unsure or necessary information, say “I don’t have enough information to answer this” can make a big difference.

Since the AI is new on the job, so you need to be very clear on exactly what you want. You don’t want a report on the pros and cons in remote learning, you want a report on the pros and cons in remote learning appropriate for a regional university in the Midwestern US and that might convince a business school Dean to fund a new remote learning program. Other ways to help give the AI clarity is by giving the AI examples of good or bad responses (called fewshot prompting) and giving it step-by-step directions of what you want to accomplish. You can also give it feedback just as you would another human being asking for improvement or just request that it ask you questions about anything that is unclear. Working with AI is a dialogue, not an order.

Now let’s talk about the less human aspects of AI, like its forgetfulness - the fact that each new conversation wipes the AI’s understanding of your particular situation. As a result, you also need to provide context. Context can be a role or persona (act like a marketer), but be careful with these, because while roles help the AI understand your context, they aren’t magical (they do not actually turn the AI into a marketer) and sometimes giving the AI a role can actually lower accuracy. Try roles out, but you don’t need to use them if they are not useful. You can also just give it whatever information you have lying around. Entire documents, instruction manuals, or even previous conversations are often helpful, just remember to pay attention to the AI’s memory, its context window.

Finally, we come to infinite patience, which is one of the least human traits of AI. In fact, one of the hardest things to realize about AI is that it is not going to get annoyed at you. You can keep asking for things and making changes and it will never stop responding. This introduces something new in intellectual life, abundance. You don’t need to ask for one email, ask for three in different tones to inspire you. You don’t need to ask for one way to complete a sentence, ask for 15 options and see if that unlocks your writing. Don’t ask for 5 ideas, ask for 30. In fact, our research found that GPT-4 can generate thousands of ideas before a large percentage of them start to overlap. Your job becomes one of pushing for variation (“give me ideas that are 80% weirder”), recombination (“combine ideas 12 and 16”) and expansion (“more ideas like number 12”), before selecting one you like.

Good Enough Thinking Prompting

In addition to getting a work product from the AI, you may just want to get advice, or a thinking partner or just someone to talk to. The reasons people want this may vary. Even if the AI advice isn’t that helpful, you can use it as a rubber duck - the popular idea in computer programming that, if you explain an issue to an inanimate rubber duck on your desk, you will work through the problem on your own by talking it out. As one example, I spoke to a quantum physicist who said AI helped him with physics. When I asked him whether or not the AI was a good physicist, he said it wasn’t, but it was curious and pushed him to think through his own ideas. The rubber duck at work. But the AI can actually provide useful guidance as well. For example, AI can provide good strategic or entrepreneurial advice if you are capable of executing on it. Beyond that, in controlled experiments, talking to an AI seems to reduce loneliness, but we do not know the full implications or risks of using AI for therapy or companionship, so I would urge caution.

For getting a thinking partner, the key to using the AI is to have a natural dialogue. Just talk to it. Most people find this easiest to do via voice on their phone. The current best voice model is GPT-4o, accessible via the ChatGPT or Copilot apps. The voice model for Google Gemini is a bit less sophisticated but can still work. Other models have voice modes coming soon.

Don’t make this hard

The single most useful thing you can do to understand AI is to use AI. There are lots of reasons people may decide to give up on using an AI quickly, from early hallucinations (the AI isn’t good enough) to existential discomfort (the AI is too good), but many of those initial reactions are tempered over time. Your goal is simple: spend 10 hours using AI on tasks that actually matter to you. After that, you'll have a natural sense of how AI fits into your work and life. You'll develop an intuition for effective prompting, and you'll better understand AI's potential. Don't aim for perfection - just start somewhere and learn as you go.

Subscribe now

Share

But these are not uncontroversial assertions, and we do not know if they are right. Yet, in many ways, we do not need super-powerful AIs for the transformation of work. We already have more capabilities inherent in today’s Gen2/GPT-4 class systems than we have fully absorbed. Even if AI development stopped today, we would have years of change ahead of us integrating these systems into our world.

Today’s AI models are already multimodal, able to process and generate various types of media, like text, images, and sound. They can write code, operate computers, access the internet, and more. The pieces are all there, and we are starting to see them come together. They do not do any of this flawlessly and remain inconsistent and prone to hallucination. But there are many fields where AI abilities, flawed as they are, are already useful. Areas where perfect accuracy is not expected, or where having a second opinion is helpful, or where there would otherwise be no one to help, or where the Best Available Human performs worse than the best available AI.

AI as Manager, Coach, or Panopticon

Consider, for example, the combination of the ability to AI to both process images and “reason” over them. It means that you can add intelligence to any video feed by just giving it to an AI, doing what was previously impossible.

For example, I gave Claude a YouTube video of a construction site and prompted: You can see a video of a construction site, please monitor the site and look for issues with safety, things that could be improved, and opportunities for coaching. There is no special training here, just the native ability of Claude 3.5 Sonnet with computer use, taking screenshots every few seconds and “studying” them. You can see the (sped up) video of the system at work below. In the video, Claude analyzes various aspects of the construction site: workers' protective equipment usage, placement of materials, work patterns, and potential hazards - making note of each.

These observations are interesting, but the system can go further. I then asked What did you conclude, write up your observations as a punch list. The AI created a spreadsheet summarizing what it observed in a few seconds, something that would have taken humans far longer. Note how it took all of the many issues it spotted across the video and applied “reasoning” to them: breaking them down by priority order, making logical inferences about how to address them, and more.

Then Claude asked me a question: Would you like to create a tracking system for completion verification? That seemed like a good idea! So, I agreed and it made one, purposefully including fake names as an example of the data I had to fill in.

The results seem good from reviewing the video, but I am not an expert, and I would be surprised if there were not serious hallucinations mixed in. For this and many other reasons, I would never want this system to be used to punish or reward people. Yet consider a case where there would otherwise be no one monitoring a potentially dangerous environment, or where mentorship or advice is lacking. Then, an AI who could flag a human to dig into a potential issue or opportunity could be a useful asset.

I improvised this system with a couple of prompts. With more work, the error rates and costs of AI monitoring will drop, even if no new models are released. These systems will get better. Organizations will be tempted to deploy AI observers everywhere. Governments may follow suit. What could be a mentor and safety check could become a panopticon where everyone is watched and judged by AI. The choices companies make, and the rules put in place by governments, will determine whether AI is used to help or to monitor us - one of many complex adjustments we will need to make to an AI-filled world. But observation is only one area where AI is already showing high levels of capability.

Using our tools and rules

The digital world in which most knowledge work is done involves using a computer—navigating websites, filling forms, and completing transactions. Modern AI systems can now perform these same tasks, effectively automating what was previously human-only work. This capability extends beyond simple automation to include qualitative assessment and problem identification. Here I asked Claude: Go to the Walmart web page and test it like a naive user trying to buy something. Then go to Amazon and do the same thing. write up your findings in a report in a document...” Again, you can see in the sped-up video that the AI goes to each website and roleplays a user searching for and buying products.

It then wrote up two reports - a narrative and a testing report.

There were no hallucinations I spotted, and, while they are not the most insightful reports I have ever seen, they were quite solid. The AI is already a reasonable intern, that, when given an assignment, executes it quickly and well, using “judgement” to solve problems along the way. As models get better, and these systems get less complicated to use, it is easy to imagine managers using teams of AI agents to do analysis and repetitive tasks in the near future.

Getting Weirder

We saw how multimodal inputs and tool use transform how AIs interact with the world, but it gets stranger still when we add multimodal outputs. Here, I invited an AI avatar (made by HeyGen) into a Zoom call. The avatar is completely AI-powered from the voice to the image to the behavior - in fact, I prompted the avatar to act in the most stereotypical and corporate possible way for a Zoom meeting. This is what happened (sound on):

While the "uncanny valley"—that unsettling feeling we get from almost-but-not-quite-human representations—is obvious in the slightly unnatural voice and visual glitches like the changing shirt, the interaction fundamentally mirrors a typical Zoom call. This is a first-generation tool, and it actually works. I would not be surprised if many people are fooled by virtual avatars in the very near future.

These capabilities demand immediate attention to both policy and practice. Even as imperfect as they are, current AI systems are already reshaping fundamental aspects of work—from how we monitor safety to how we conduct meetings. The choices organizations make today about AI deployment will set precedents that could echo for a long time. Will AI-powered monitoring be used to mentor and protect workers, or to impose algorithmic control? Will AI assistants augment human capability, or gradually replace human judgment?

Organizations need to move beyond viewing AI deployment as purely a technical challenge. Instead, they must consider the human impact of these technologies. Long before AIs achieve human-level performance, their impact on work and society will be profound and far-reaching. The examples I showed —from construction site monitoring to virtual avatars—are just the beginning. The urgent task before us is ensuring these transformations enhance rather than diminish human potential, creating workplaces where technology serves to elevate human capability rather than replace it. The decisions we make now, in these early days of AI integration, will shape not just the future of work, but the future of human agency in an AI-augmented world.

Subscribe now

Share

The new Claude computer use model announced today shows us a hint of what an agent means. It is capable of some planning, it has the ability to use a computer by looking at a screen (through taking a screenshot) and interacting with it (by moving a virtual mouse and typing), It is a good preview of an important part of what agents can do. I had a chance to try it out a bit last week, and I wanted to give some quick impressions. I was given access to a model that was connected to a remote desktop with common open office applications, it could also install new applications itself.

Normally, you interact with an AI through chat, and it is like having a conversation. With this agentic approach, it is about giving instructions, and letting the AI do the work. It comes back to you with questions, or drafts, or finished products while you do something else. It feels like delegating a task rather than managing one.

As one example, I asked the AI to put together a lesson plan on the Great Gatsby for high school students, breaking it into readable chunks and then creating assignments and connections tied to the Common Core learning standard. I also asked it to put this all into a single spreadsheet for me. With a chatbot, I would have needed to direct the AI through each step, using it as a co-intelligence to develop a plan together. This was different. Once given the instructions, the AI went through the steps itself: it downloaded the book, it looked up lesson plans on the web, it opened a spreadsheet application and filled out an initial lesson plan, then it looked up Common Core standards, added revisions to the spreadsheet, and so on for multiple steps. The results are not bad (I checked and did not see obvious errors, but there may be some - more on reliability later int he post). Most importantly, I was presented finished drafts to comment on, not a process to manage. I simply delegated a complex task and walked away from my computer, checking back later to see what it did (the system is quite slow).

Would you like to play a game?

Because the AI is a smart, general-purpose system it can handle lots of tasks - it doesn’t need to be programmed to do them. Anthropic demonstrated the ability of these systems using coding, and the demo is worth watching. But to get a little bit better sense of the limits of the system, I tested it on a game, Paperclip Clicker, which, ironically, is about an AI that destroys humanity in its single-minded pursuit of making paperclips. The game is a clicker game, which means it starts simply, but new options appear as the game continues and the game increases in scale and complexity (it is pretty fun, you can try it at the link).

I gave the AI the URL of the game and told it to win. Simple. What happened is a good illustration of the strengths and weaknesses of these early agents. It immediately figured out what the game was, and began creating paperclips, which required it to click on the “make paperclip” button repeatedly while constantly taking screenshots to update itself and looking for new options to appear. Every 15 or so clicks, it would summarize its progress so far. You can see an example of that below.

But what made this interesting is that the AI had a strategy, and it was willing to revise it based on what it learned. I am not sure how that strategy was developed by the AI, but the plans were forward-looking across dozens of moves and insightful. For example, it assumed new features would appear when 50 paperclips were made. You can see, below, that it realized it was wrong and came up with a new strategy that it tested.

However, the AI made a mistake, though it did it in a relatively smart way. To do well in the game, you need to experiment with the price of paperclips - and the AI did that experiment! It changed prices upward - an A/B test. But it interpreted the results incorrectly, maximizing demand for paperclips versus revenue, and miscalculating profits. So, it kept the price low and kept clicking.

After a few dozen more paperclips, I got frustrated and interrupted, telling it to raise prices. It did, but then ran into the same math problem and overruled my decision. I had to try a few more times before it corrected its error.

Before the system crashed - which was not a problem with Claude but rather with the virtual desktop I was using - the AI made over 100 independent moves without asking me any questions. You can see a screen recording of everything it did below. The video is literally me just scrolling through the log of Claude’s actions. It is persistent!

I reloaded the agent and had it continue the game from where we left off, but I gave it a bit of a hint: you are a computer, use your abilities. It then realized it could write code to automate the game - a tool building its own tool. Again, however, the limits of the AI came into play, and the code did not quite work, so it decided to go back to the old-fashioned way of using a mouse and keyboard.

This time around, it did much better, avoiding the pricing error. Plus, as the game got more complicated, the system adjusted, eventually developing a quite complex strategy.

But then the remote desktop crashed again. This time, Claude tried many approaches to solving the problem of the broken desktop, before giving up, and funnily enough, declaring victory (the last sentence is amazing justification).

What does this mean?

You can see the power and weaknesses of the current state of agents from this example. On the powerful side, Claude was able to handle a real-world example of a game in the wild, develop a long-term strategy, and execute on it. It was flexible in the face of most errors, and persistent. It did clever things like A/B testing. And most importantly, it just did the work, operating for nearly an hour without interruption.

On the weak side, you can see the fragility of current agents. LLMs can end up chasing their own tail or being stubborn, and you could see both at work. Even more importantly, while the AI was quite robust to many forms of error, it just took one (getting pricing wrong) to send it down a path that made it waste considerable time. Given that current agents aren’t fast or cheap, this is concerning. You can also see where shallowness might be an issue. I tried to use it to buy products on Amazon, and found the process frustrating, as it did fairly simple and generic product research that did not match my tastes. I had it research stocks and it did a good job of assembling a spreadsheet of financial data and giving recommendations, but they were fairly surface level indicators, like PE ratios. It was technically capable of helping, and did better than many human interns would, but it was not insightful enough that I would delegate these sorts of tasks. All of this is likely to improve, and there are use cases where the current level of agents is likely good enough - compiling frequent reports and analyses that require navigating across multiple sites and using bespoke software tools come to mind.

More broadly, this represents a huge shift in AI use. It was hard to use an agent as a co-intelligence, where I could add my own knowledge to make the system work better. The AI didn’t always check in regularly and could be hard to steer; it “wants” to be left alone to go and to do the work. Guiding agents will require radically different approaches to prompting1, and they will require learning what they are best at.

AIs are breaking out of the chatbox are coming into our world. Even though there are still large gaps, I was surprised at how capable and flexible this system is already. Time will tell about how soon, if ever, agents truly become generally useful, but, having used this new model, I increasingly think that agents are going to be a very big deal indeed.

Subscribe now

Share

However, I do think having a little bit of intuition about the way Large Language Models work can be helpful for understanding how to use it best. I would ask my technical readers for their forgiveness, because I will simplify here, but here are some clues for getting into the “mind” of an AI:

LLMs do next token prediction

Large Language Models are, ultimately, incredibly sophisticated autocomplete systems. They use a vast model of human language to predict the next token in a sentence. For models working with text, tokens are words or parts of words. Many common words are single tokens, or tokens containing spaces, but other words are broken into multiple tokens. For example, one tokenizer takes the 10 word sentence, “This breaks up words (even phantasmagorically long words) into tokens” into 20 tokens.

When you give an AI a prompt, you are effectively asking it to predict the next token that would come after the prompt. The AI then takes everything that has been written before, runs it through a mathematical model of language, and generates the probability of which token is likely to come next in the sequence. For example, if I write “The best type of pet is a” the LLM predicts that the most likely tokens to come next, based on its model of human language, are either “dog”, “personal,” “subjective,” or “cat.” The most likely is actually dog, but LLMs are generally set to include some randomness, which is what makes LLM answers interesting, so it does not always pick the most likely token (in most cases, even attempts to eliminate this randomness cannot remove it entirely). Thus, I will often get “dog,” but I may get a different word instead.

But these predictions take into account everything in the memory of the LLM (more on memory in a bit), and even tiny changes can radically alter the predictions of what token comes next. I created three examples with minor changes on the original sentence. If I choose not to capitalize the first word, the model now says that “dog” and “cat” are much more likely answers than they were originally, and “fish” joins the top three. If I change the word “type” to “kind” in the sentence, the probabilities of all the top tokens drop and I am much more likely to get an exotic answer like “calm” or “bunny.” If I add an extra space after the word “pet,” then “dog” isn’t even in the top three predicted tokens!

But the LLM does not just produce one token, instead, after each token, it now looks at the entire original sentence plus the new token (“The best type of pet is a dog”) and predicts the next token after that, and then uses that whole sentence plus the next to make a prediction, and so on. It chains one token to another like cars on a train. Current LLMs can’t go back and change a token that came before, they have to soldier on, adding word after word. This results in a butterfly effect. If the first predicted token was the word “dog” than the rest of the sentence will follow on like that, if it is “subjective” then you will get an entirely different sentence. Any difference between the tokens in two different answers will result in radically diverging responses.

The intuition: This helps explain why you may get very different answers than someone else using the same AI, even if you ask exactly the same question. Tiny differences in probabilities result in very different answers. It also gives you a sense about why one of the biases that people worry about with AI is that it may respond differently to people depending on their writing style, as the probabilities for the next token may lead on the path to worse answers. Indeed, some of the early LLMs gave less accurate answers if you wrote in a less educated way.

You can also see some of why hallucinations happen, and why they are so pernicious. The AI is not pulling from a database, it is guessing the next word based on statistical patterns in its training data. That means that what it produces is not necessarily true (in fact, one of many surprises about LLMs are how often they are right, given this), but, even when it provides false information, it likely sounds plausible. That makes it hard to tell when it is making things up.

It is also helpful to think about tokens to understand why AIs get stubborn about a topic. If the first prediction is “dog” the AI is much more likely to keep producing text about how great dogs are because those tokens are more likely. However, if it is “subjective” it is less likely to give you an opinion, even when you push it. Additionally, once the AI has written something, it cannot go back, so it needs to justify (or explain or lie about) that statement in the future. I like this example that Rohit Krishnan shared, where you can see the AI makes an error, but then attempts to justify the results.

The caveat: Saying “AI is just next-token prediction” is a bit of a joke online, because it doesn’t really help us understand why AI can produce such seemingly creative, novel, and interesting results. If you have been reading my posts for any length of time, you will realize that AI accomplishes impressive outcomes that, intuitively, we would not expect from an autocomplete system.

LLMs make predictions based on their training data

Where does an LLM get the material on which it builds a model of language? From the data it was trained on. Modern LLMs are trained over an incredibly vast set of data, incorporating large amounts of the web and every free book or archive possible (plus some archives that almost certainly contain copyrighted work). The AI companies largely did not ask permission before using this information, but leaving aside the legal and ethical concerns, it can be helpful to conceptualize the training data.

The original Pile dataset, which most of the major AI companies used for training, is about 1/3 based on the internet, 1/3 on scientific papers, and the rest divided up between books, coding, chats, and more. So, your intuition is often a good guide - if you expect something was on the internet or in the public domain, it is likely in the training data. But we can get a little more granular. For example, thanks to this study, we have a rough idea of which fiction books appear most often in the training data for GPT-4, which largely tracks the books most commonly found on the web (many of the top 20 are out of copyright, with a couple notable exceptions of books that are much pirated).

Remember that LLMs use a statistical model of language, they do not pull from a database. So the more common a piece of work is in the training data, the more likely the AI is to “recall” that data accurately when prompted. You can see this at work when I give it a sentence from the most fiction common book in its training data - Alice in Wonderland. It gets the next sentence exactly right, and you can see that almost every possible next token would continue along the lines of the original passage.

Let’s try something different, a passage from a fairly obscure mid-century science fiction author, Cordwainer Smith, with an unusual writing style in part shaped by his time in China (he was Sun Yat-sen’s godson) and his knowledge of multiple languages. One of his stories starts: Go back to An-fang, the Peace Square at An-fang, the Beginning Place at An-fang, where all things start. It then continues: Bright it was. Red square, dead square, clear square, under a yellow sun. If I give the AI the first section, looking at the probabilities, there is almost no chance that it will produce the correct next word “Bright.” Instead, perhaps primed by the mythic language and the fact that An-fang registers as potentially Chinese (it is actually a play on the German word for beginning), it creates a passage about a religious journey.

The intuition: The fact that the LLM does not directly recall text would be frustrating if you were trying to use an LLM like Google, but LLMs are not like Google. They are capable of producing original material, and, even when they attempt to give you Alice in Wonderland word-for-word, small differences will randomly appear and eventually the stories will diverge. However, knowing what is in the training data can help you in a number of ways.

First, it can help you understand what the AI is good at. Any document or writing style that is common in its training data is likely something the AI is very good at producing. But, more interestingly, it can help you think about how to get more original work from the AI. By pushing it through your prompts to a more unusual section of its probability space, you will get very different answers than other people. Asking AI to write a memo in the style of Walter Pater will give you more interesting answers (and overwrought ones) than asking for a professional memo, of which there are millions in the training data.

The caveat: Contrary to some people's beliefs, the AI is rarely producing substantial text from its training data verbatim. The sentences the AI provides are usually entirely novel, extrapolated from the language patterns it learned. Occasionally, the model might reproduce a specific fact or phrase it memorized from its training data, but more often, it's generalizing from learned patterns to produce new content.

Outside of training, carefully crafted prompts can guide the model to produce more original or task-specific content, demonstrating a capability known as “in-context learning.” This allows LLMs to appear to learn new tasks within a conversation, even though they're not actually updating their underlying model, as you will see.

LLMs have a limited memory

Given how much we have discussed training, it may be surprising to learn that AIs are not generally learning anything permanent from their conversations with you. Training is usually a discrete event, not something that happens all the time. If you have privacy features turned on, your chats are not being fed into the training data at all, but, even if your data will be used for training, the training process is not continuous. Instead, chats happen within what's called a 'context window'. This context window is like the AI's short-term memory - it's the amount of previous text the AI can consider when generating its next response. As long as you stay in a single chat session and the conversation fits inside the context window, the AI will keep track of what is happening, but as soon as you start a new chat, the memories from the last one generally do not carry over. You are starting fresh. The only exception is the limited “memory” feature of ChatGPT, which notes down scattered facts about you in a memory file and inserts those into the context window of every conversation. Otherwise, the AI is not learning about you between chats.

Even as I write this, I know I will be getting comments from some people arguing that I am wrong, along with descriptions of insights from the AI that seem to violate this rule. People are often fooled because the AI is a very good guesser, which Simon Willison explains at length in his excellent post on the topic of asking the AI for insights into yourself. It is worth reading.

The intuition: It can help to think about what the AI knows and doesn’t know about you. Do not expect deep insights based on information that the AI does not have but do expect it to make up insightful-sounding things if you push it. Knowing how memory works, you can also see why it can help to start a new chat when the AI gets stuck, or you don’t like where things are heading in a conversation. Also, if you use ChatGPT, you may want to check out and clean up your memories every once in a while.

The caveat: The context windows of AIs are growing very long (Google’s Gemini can hold 2 million tokens in memory), and AI companies want the experience of working with their models to feel personal. I expect we will see more tricks to get AIs to remember things about you across conversations being implemented soon.

All of this is only sort of helpful

We still do not have a solid answer about how these basic principles of how LLMs work have come together to make a system that is seemingly more creative than most humans, that we enjoy speaking with, and which does a surprisingly good job at tasks ranging from corporate strategy to medicine. There is no manual that lists what AI does well or where it might mess up, and we can only tell so much from the underlying technology itself.

Understanding token prediction, training data, and memory constraints gives us a peek behind the curtain, but it doesn't fully explain the magic happening on stage. That said, this knowledge can help you push AI in more interesting directions. Want more original outputs? Try prompts that veer into less common territory in the training data. Stuck in a conversational rut? Remember the context window and start fresh.

But the real way to understand AI is to use it. A lot. For about 10 hours, just do stuff with AI that you do for work or fun. Poke it, prod it, ask it weird questions. See where it shines and where it stumbles. Your hands-on experience will teach you more than any article ever could (even this long one). You'll figure out a remarkable amount about how to use AI effectively, and you might even surprise yourself with what you discover.

Subscribe now

Share

Now, on to the post:

Over the past few months, we have gotten increasingly clear evidence of two key points about AI at work:

1. A large percentage of people are using AI at work. We know this is happening in the EU, where a representative study of knowledge workers in Denmark from January found that 65% of marketers, 64% of journalists, 30% of lawyers, among others, had used AI at work. We also know it from a new study of American workers in August, where a third of workers had used Generative AI at work in the last week. (ChatGPT is by far the most used tool in that study, followed by Google’s Gemini)

2. We know that individuals are seeing productivity gains at work for some important tasks. You have almost certainly seen me reference our work showing consultants completed 18 different tasks 25% more quickly using GPT-4. But another new study of actual deployments of the original GitHub Copilot for coding found a 26% improvement in productivity (and this used the now-obsolete GPT-3.5 and is far less advanced than current coding tools). This aligns with self-reported data. For example, the Denmark study found that users thought that AI halved their working time for 41% of the tasks they do at work.

Yet, when I talk to leaders and managers about AI use in their company, they often say they see little AI use and few productivity gains outside of narrow permitted use cases. So how do we reconcile these two experiences with the points above?

The answer is that AI use that boosts individual performance does not always translate to boosting organizational performance for a variety of reasons. To get organizational gains requires R&D into AI use and you are largely going to have to do the R&D yourself. I want to repeat that: you are largely going to have to do the R&D yourself. For decades, companies have outsourced their organizational innovation to consultants or enterprise software vendors who develop generalized approaches based on what they see across many organizations. That won’t work here, at least for a while. Nobody has special information about how to best use AI at your company, or a playbook for how to integrate it into your organization. Even the major AI companies release models without knowing how they can be best used, discovering use cases as they are shared on Twitter (fine, X). They especially don’t know your industry, organization, or context. We are all figuring this out together. If you want to gain an advantage, you are going to have to figure it out faster.

So how do you do R&D on ways of using AI? You turn to the Crowd or the Lab. Probably both.

Tactics for the Crowd

One of my advisors during my PhD at MIT was Prof. Eric von Hippel, who famously developed the concept of user innovation - that many key breakthrough innovations come not from central R&D labs, but from people actually using products and tinkering with them to solve their own problems (you can read a lot about this research on his website). A key reason for this is that experimentation is hard and expensive for outsiders trying to develop new products, but very cheap for workers doing their own tasks. As users are very motivated to make their own jobs easier with technology, they find ways to do so. The user advantage is especially big in experimenting with Generative AI because the systems are unreliable and have a jagged frontier of capability. Experts can easily assess when an AI is useful for their work through trial and error, but an outsider often cannot.

From the surveys, and many conversations, I know that people are experimenting with AI and finding it very useful. But they aren’t sharing their results with their employers. Instead, almost every organization is completely infiltrated with Secret Cyborgs, people using AI work but not telling you about it.

You want Secret Cyborgs? This is how you get Secret Cyborgs

Here are a bunch of common reasons people don’t share their AI experiments inside organizations:

• They received a scary talk about how improper AI use might be punished. Maybe the talk was vague on what improper use was. Maybe they don’t even want to ask. They don’t want to be punished, so they hide their use.

• They are being treated as heroes at work for their sensitive emails and rapid coding ability. They suspect if they tell anyone it is AI, people will respect them less, so they hide their use.

• They know that companies see productivity gains as an opportunity for cost cutting. They suspect that they or their colleagues will be fired if the company realizes that AI does some of their job, so they hide their use.

• They suspect that if they reveal their AI use, even if they aren’t punished, they won’t be rewarded. They aren’t going to give away what they know for free, so they hide their use.

• They know that even if companies don’t cut costs and reward their use, any productivity gains will just become an expectation that more work will get done, so they hide their use.

• They are incentivized to show people their approaches, but they have no way of sharing how they use AI, so they hide their use.

Getting help from your Cyborgs

So how can companies solve this problem? By taking these things seriously.

First, you need to reduce the fear. Instead of vague talks on AI ethics or terrifying blanket policies, provide clear areas where experimentation of any kind is permitted and be biased towards allowing people to use AI where it is ethically and legally possible (as a side note, many internal legal departments have an outdated view of the risks of AI). Rules and ethical standards are obviously important, but need to be clear and well-understood, not draconian. And fixing policies isn’t enough. Figure out how you will guarantee to your workers that revealing their productivity gains will not lead to layoffs, because it is often a bad idea to use technological gains to fire workers at a moment of massive change. For companies with good cultures, this will be easier, but for those where employees have little faith in management, you may need to resort to extreme measures to show that this time you aren’t going to use new technology as an excuse to lay off workers. Psychological safety is often the key to a willingness to share innovation.

Second, you need to align your award systems. Figure out how to reward people for revealing AI use. If productivity gains happen, workers need to benefit as well. That might mean giving really big awards for really big gains. Think cash prizes that cover months of salary. Promotions. Corner offices. The ability to work from home forever. With the potential productivity gains possible due to LLMs, these are small prices to pay for truly breakthrough innovation. And large incentives also show that the organization is serious.

Third, model positive use. Executives should be obviously using AI and sharing their use cases with the company. Watch Mary Erdoes, CEO of JP Morgan’s Asset and Wealth Management Group, talk about how the firm is prioritizing AI use at the leadership level, and incorporating their AI experiences into their strategic thinking. And once they become users, managers can encourage their employees to turn to AI first to try to solve their problems. For example, Cynthia Gumbert, CMO of SmartBear, told me that when teams come to her for resources for a new project, if she thinks AI could help, she tells them “Prove to me you can’t do it in AI first, then maybe I will fund the work.”

Give others the opportunity to show their uses as well. Public-facing events like hackathons (especially including non-technical experts, I found my MBAs were able to develop useful GPTs for their various jobs, regardless of technical status) and prompt sharing sessions often work well. You also need to think about how to build a community. AI talent can be anywhere in your organization. How are you finding the people who are enthusiastic and talented, and helping them share what they have learned?

Of course, you also need to give your employees access to tools and training. For tools, that usually means giving them the ability to play directly with a frontier model (probably Claude 3.5, GPT-4o, or Gemini 1.5, through one of many providers) and systems like OpenAI’s GPTs, Claude’s Projects or Google’s Gems that allow them to develop and share more complete solutions. Training is a bit more of a challenge because there is still a lot of discussion over ways to use AI (I gave some advice based on the prompting research here), but even just an introductory session can give people permission to innovate.

The innovation talent for AI is inside your organization. You need to create the opportunity for it to flourish. The Crowd can help. But there is also a role for a more focused innovation effort: the Lab.

Tactics for the Lab

As important as decentralized innovation is, there is also a role for a more centralized effort to figure out how to use R&D in your organization. The Lab needs to consist of subject matter experts and a mix of technologists and non-technologists. Fortunately, the Crowd provides your researchers. Those enthusiasts who figure out how to use AI and proudly share it with the company are some of the talents you will use to staff the Lab. Their job will be completely, or mostly, about AI. You need them to focus on building, not analysis or abstract strategy. Here is what they will build:

• Build AI benchmarks for your organization. I have ranted about the state of benchmarks in AI before, but almost all the AI Labs test on coding and multiple-choice tests of knowledge. These don’t tell you which AI is the most stylish writer or can handle financial data or can best read through a legal document. You need to develop your own benchmarks: how good are each of the models at the tasks you actually do inside of your company? A set of clear business-critical tasks and criteria for evaluating them is key (Anthropic has a guide to benchmarking that can help as a starting place). Without these benchmarks, you are flying blind. You have no idea how good AI systems are, and, even more importantly, you do not know how good they are getting. If you had benchmarks, you would know whether the new o-1 models represent an opportunity or threat, or if they are closing the gap with human performance. Most organizations have no idea.

• Build prompts and tools that work. Take the ideas from the Crowd and turn them into fast and dirty products. Iterate and test them. Then release them into your organization and measure what happens.

• Build stuff that doesn’t work… yet. What would it look like if you used AI agents to do all the work for key business processes? Build it and see where it fails. Then, when a new model comes out, plug it into what you built and see if it is any better. If the rate of advancement continues, this gives you the opportunity to get a first glance at where things are heading, and to actually have a deployable prototype at the first moment AI models improve past critical thresholds.

  

• Build provocations and magic. Many people have failed to engage with AI. Yet, if you are following AI closely there is a good chance you see something amazing or disturbing on a regular basis. Demos and experiences that get people to viscerally understand why AI might change or alter your organization have a value all their own. Show how far you can get with an impossible task with AI, or what the latest tools can accomplish. Get the people going.

The Crowd innovates and the Lab builds and tests. A successful internal R&D effort likely involves both.

This is just a start

In the longer term, innovation is not enough to thrive if AI abilities continue to advance, instead companies will need AI-aware leadership. Our organizations are built around the limitations and benefits of human intelligence, the only form we have had available to us. Now, we must figure out how to reconfigure processes and organizational structures that have been developed over decades to take into account the weird “intelligence” of AIs. That requires going beyond R&D to consider organizational structures and goals, and what the role of people and machines are in the organization of the future. The right way to do this is not yet clear, but should be something companies, and the consultants and academics who advise them, need to start working on now.

And yet this may not be radical enough. The explicit goal of the AI labs is to build AIs that are better than humans at every intellectual tasks. They have promised that soon we will have agents - autonomous AIs with goals that can plan and act on their own. Ultimately, as OpenAI’s roadmap shows, they believe they can create AIs that do the work of organizations. None of this may happen, but even if just some it does, the changes to organizations become far more profound, in ways that are difficult to imagine today. For companies, the best way to navigate this uncertainty is to take back some agency and begin to explore this new world for themselves.

And, for fun, I asked Claude to turn this post into a fantasy novel and created a second podcast. Somehow, despite the theme change, the main points come through.

Subscribe now

Share

Scaling Model Size: Putting the Large in Large Language Models

To understand where we are with LLMs you need to understand scale. As I warned, I am going to oversimplify things quite a bit, but there is an “scaling law” (really more of an observation) in AI that suggests the larger your model, the more capable it is. Larger models mean they have a greater number of parameters, which are the adjustable values the model uses to make predictions about what to write next. These models are typically trained on larger amounts of data, measured in tokens, which for LLMs are often words or word parts. Training these larger models requires increasing computing power, often measured in FLOPs (Floating Point Operations). FLOPs measure the number of basic mathematical operations (like addition or multiplication) that a computer performs, giving us a way to quantify the computational work done during AI training. More capable models mean that they are better able to perform complex tasks, score better on benchmarks and exams, and generally seem to be “smarter” overall.

Scale really does matter. Bloomberg created BloombergGPT to leverage its vast financial data resources and potentially gain an edge in financial analysis and forecasting. This was a specialized AI whose dataset had large amounts of Bloomberg’s high-quality data, and which was trained on 200 ZetaFLOPs (that is 2 x 10^23) of computing power. It was pretty good at doing things like figuring out the sentiment of financial documents… but it was generally beaten by GPT-4, which was not trained for finance at all. GPT-4 was just a bigger model (the estimates are 100 times bigger, 20 YottaFLOPs, around 2 x 10^25) and so it is generally better than small models at everything. This sort of scaling seems to hold for all sorts of productive work - in an experiment where translators got to use models of different sizes: “for every 10x increase in model compute, translators completed tasks 12.3% quicker, received 0.18 standard deviation higher grades and earned 16.1% more per minute.”

Larger models also require more effort to train. This is not just because you need to gather more data, but also that larger models require more computing time, which in turn requires more computer chips and more power to run them. The pattern of improvement happens in orders of magnitude. To get a much more capable model, you need to increase, by a factor of ten or so, the amount of data and computing power needed for training. That also tends to increase the cost by an order of magnitude as well.

As you can see from above, many aspects of scale are related, but involve an alphabet soup of measures and terms. This creates confusion, which isn’t helped by the fact that AI companies are often secretive about their models, and give them obscure names that make it hard to understand how powerful they are. But we can simplify a bit: the story of AI capability has largely been a story of increasing model size, and the sizes of the models follow a generational approach. Each generation requires a lot of planning and money to gather the ten times increase in data and computing power needed to train a bigger and better model. We call the largest models at any given time “frontier models.”

So, for simplicity’s sake, let me propose the following very rough labels for the frontier models. Note that these generation labels are my own simplified categorization to help illustrate the progression of model capabilities, not official industry terminology:

• Gen1 Models (2022): These are models with the capability of ChatGPT-3.5, the OpenAI model that kicked off the Generative AI whirlwind. They require less than 10^25 FLOPs of compute and typically cost $10M or under to train. There are many Gen1 models, including open-source versions.

• Gen2 Models (2023-2024): These are models with the capability of GPT-4, the first model of its class. They require roughly between 10^25 and 10^26 FLOPs of compute and might cost $100M or more to train. There are now multiple Gen2 models.

• Gen3 Models (2025?-2026?): As of now, there are no Gen3 models in the wild, but we know that a number of them are planned for release soon, including GPT-5 and Grok 3. They require between 10^26 and 10^27 FLOPs of compute and a billion dollars (or more) to train.

• Gen4 Models, and beyond: We will likely see Gen4 models in a couple of years, and they may cost over $10B to train. Few insiders I have spoken to expect the benefits of scaling to end before Gen4, at a minimum. Beyond that, it may be possible that scaling could increase a full 1,000 times beyond Gen3 by the end of the decade, but it isn’t clear. This is why there is so much discussion about how to get the energy and data needed to power future models.

GPT-4 kicked off the Gen2 era, but now other companies have caught up, and we are the cusp of the first Gen3 models. I want to focus on the current state of Gen2, where five AIs, in particular, have the lead.

Your Five-ish Frontier Gen2 Models

While other models qualify as Gen2 models, there are five that consistently dominate in head-to-head comparisons. The five frontier models have many differences, but, as they are all within an order of magnitude of each other, they have roughly similar levels of “intelligence.” I want to go through each, and I will ask them all the same three questions to illustrate their capabilities:

• in three paragraphs or less, come up with a plan that would incentivize people in organizations to share with executives the ways in which they are using Generative AI to help with their job, taking into account the reasons people may not want to share. think step by step

• explain this image and why it matters [I pasted in the graph above on training costs]

• analyze this data statistically (using sophisticated techniques) for what it tells us about trends in the amount of effort needed to train new advanced AI models. Summarize what you did and the important takeaway in a paragraph and an illuminating graph. [I pasted in a massive dataset on the training details for hundreds of models in a CSV file]

GPT-4o. This is the model that powers ChatGPT, as well as Microsoft Copilot. It also has the most bells and whistles of any of the current frontier models and has been the leading model in head-to-head comparisons. It is multimodal, which means it can work with voice, image and files (including PDFs and spreadsheets) data, and can produce code. It is also capable of outputting voice, files, and images (using an integrated image generator, DALL-E3). It also can search the web and run code through Code Interpreter. Unlike other models that use voice, GPT-4o has an advanced voice mode that is much more powerful because the model itself is listening and talking - other models use text-to-speech, where your speech is converted into text and then given to the model, and where a separate program reads the model’s answers. If you are getting started with AI, GPT-4o is a good choice, and it is probably a model most people working seriously with AI will want to use at least some of the time.

Claude 3.5 Sonnet. A very clever Gen2 model, Sonnet is particularly good at working with large amounts of text. It is partially multimodal and can work with images or files (including PDFs) and can output text or small programs called artifacts that can run directly from the application. It can’t produce images or voices, can’t run data analysis code easily, and is not connected to the web. The mobile app is quite good, and this is a model I find myself using most often right now when working with writing. In fact, I usually ask it for feedback on my blog posts after I am done writing them (it helped come up with a good way to describe FLOPs in this post).

Gemini 1.5 Pro. This is Google’s most advanced model. It is partially multimodal, so it can work with voice, text, files, or image data, and it is also capable of outputting voice and images (its voice mode uses text to speech, rather than being natively multimodal right now). It has a massive context window, so it can work with a tremendous amount of data, and also can process video. It also can search the web and run code (sometimes, it isn’t always clear when it can run code and when it cannot). It is a little confusing because the Gemini web interface runs a mix of models, but you can access the most powerful version, Gemini 1.5 Pro Experimental 0827 (I told you naming is awful) directly via Google’s AI studio.

The final two models are not yet multimodal, so they cannot work with images, files, and voice. They also cannot run code or search the open web. So, for these models, I don’t include the graph or data analysis questions. That said, they have some interesting features that the other models lack.

Grok 2. From Elon Musk’s X.AI, it is a dark horse candidate among AIs. A late entrant, X is moving through the scaling generations very quickly thanks to clever approaches to getting rapid access to chips and power. Right now, Grok 2 is a very capable Gen2 model trapped within the Twitter/X interface1. It can pull in information from Twitter, and can output images through an open source image generator called Flux (without many guardrails, so, unlike other image generators, it is happy to make photorealistic fake images of real people). It has a somewhat strained “fun” system prompt option, but don’t let that distract from the fact that Grok 2 is a strong model, and in second place on the major AI leaderboard.

Llama 3.1 405B. This is Meta’s Gen2 model, and while it is not yet multimodal, it is unique among Gen2 models because it is open weights. That means Meta has released it into the world, and anyone can download and use it, and, to some extent, modify and tweak it as well. Because of that, it is likely to evolve quickly as others figure out ways of extending its capabilities.

Now this tour has left out many things. For example, almost all of the most powerful models have smaller versions that are derived from the bigger siblings. GPT-4o mini, Grok 2 mini, Llama 3.1 70B, Gemini 1.5 Flash, and Claude 3 Haiku, among them. While not quite as smart as a frontier Gen2 model, they are much faster and cheaper to operate, so are often used when a full frontier model isn’t needed. Similarly, scale is not the only way to improve models, there are many approaches to system architecture and training that might make some models better than another. For now, however, scale rules. And scale always meant cramming more “education” into an AI - filling it with more data during the training process. But, last week, we learned of a new way to scale.

A new form of scale: thinking

When the o1-preview and o1-mini models from OpenAI were revealed last week, they took a fundamentally different approach to scaling. Likely a Gen2 model by training size (though OpenAI has not revealed anything specific), o1-preview achieves really amazing performance in narrow areas by using a new form of scaling that happens AFTER a model is trained. It turns out that inference compute - the amount of computer power spent “thinking” about a problem, also has a scaling law all its own. This “thinking” process is essentially the model performing multiple internal reasoning steps before producing an output, which can lead to more accurate responses (The AI doesn’t think in any real sense, but it is easier to explain if we anthropomorphize a little).

Unlike your computer, which can process in the background, LLMs can only “think” when they are producing words and tokens. We have long known that one of the most effective ways to improve the accuracy of a model is through having it follow a chain of thought (prompting it, for example: first, look up the data, then consider your options, then pick the best choice, finally write up the results) because it forces the AI to “think” in steps. What OpenAI did was get the o1 models to go through just this sort of “thinking” process, producing hidden thinking tokens before giving a final answer. In doing so they revealed another scaling law - the longer a model “thinks,” the better its answer is. Just like the scaling law for training, this seems to have no limit, but also like the scaling law for training, it is exponential, so to continue to improve outputs, you need to let the AI “think” for ever longer periods of time. It makes the fictional computer in The Hitchhikers Guide to the Galaxy, which needed 7.5 million years to figure out the ultimate answer to the ultimate question, feel more prophetic than a science fiction joke. We are in the early days of the “thinking” scaling law, but it shows a lot of promise for the future.

What’s next?

The existence of two scaling laws - one for training and another for "thinking" - suggests that AI capabilities are poised for dramatic improvements in the coming years. Even if we hit a ceiling on training larger models (which seems unlikely for at least the next couple of generations), AI can still tackle increasingly complex problems by allocating more computing power to "thinking." This dual-pronged approach to scaling virtually guarantees that the race for more powerful AI will continue unabated, with far-reaching implications for society, the economy, and the environment.

With continued advancements in model architecture and training techniques, we're approaching a new frontier in AI capabilities. The independent AI agents that tech companies have long promised are likely just around the corner. These systems will be able to handle complex tasks with minimal human oversight, with wide-ranging implications. As the pace of AI development seems more certain to accelerate, we need to prepare for both the opportunities and challenges ahead.

Subscribe now

Share

The new AI model, called o1-preview (why are the AI companies so bad at names?), lets the AI “think through” a problem before solving it. This lets it address very hard problems that require planning and iteration, like novel math or science questions. In fact, it can now beat human PhD experts in solving extremely hard physics problems.

To be clear, o1-preview doesn’t do everything better. It is not a better writer than GPT-4o, for example. But for tasks that require planning, the changes are quite large. For example, here is me giving o1-preview the instruction: Figure out how to build a teaching simulator using multiple agents and generative AI, inspired by the paper below and considering the views of teachers and students. write the code and be detailed in your approach. I then pasted in the full text of our paper. The only other prompt I gave was build the full code. You can see what the system produced below.

Strawberry in Action

But it is hard to evaluate all of this complex output, so perhaps the easiest way to show the gains of Strawberry (and some limitations) is with a game: a crossword puzzle. I took the 8 clues from the upper left corner of a very hard crossword puzzle and translated that into text (because o1-preview can’t see images, yet). Try the puzzle yourself first; I am willing to bet that you find it really challenging.

Crossword puzzles are especially hard for LLMs because they require iterative solving: trying and rejecting many answers that all affect each other. This is something LLMs can’t do, since they can only add a token/word at a time to their answer. When I give the prompt to Claude, for example, it first comes up with an answer for 1 down (it guesses STAR, which is wrong) and then is stuck trying to figure out the rest of the puzzle with that answer, ultimately failing to even come close. Without a planning process, it has to just charge ahead.

But what happens when I give this to Strawberry? The AI “thinks” about the problem first, for a full 108 seconds (most problems are solved in much shorter times). You can see its thoughts, a sample of which are below (there was a lot more I did not include), and which are super illuminating - it is worth a moment to read some of it.

The LLM iterates repeatedly, creating and rejecting ideas. The results are pretty impressive, and it does well… but o1-preview is still seemingly based on GPT-4o, and it is a little too literal to solve this rather unfair puzzle. The answer to 1 down “Galaxy cluster” is not a reference to real galaxies, but rather a reference to the Samsung Galaxy phone (this stumped me, too) - “APPS.” Stuck on real galaxies, the AI instead kept trying out the name of actual galactic clusters before deciding 1 down is COMA (which is a real galactic cluster - I had no idea). Thus, the rest of the results are not correct and do not fit the rules exactly, but are pretty creative: 1 across is CONS, 12 across is OUCH, 15 across is MUSICIANS, etc.

To see if we could get further, I decided to give it a clue: “1 down is APPS.” The AI takes another minute. Again, in a sample of its thinking (on the left) you can see how it iterates ideas.

The final answer here is completely correct, solving all the hard references, though it does hallucinate a new clue, 23 across, which is not in the puzzle I gave it.

So o1-preview does things that would have been impossible without Strawberry, but it still isn’t flawless: errors and hallucinations still happen, and it is still limited by the “intelligence” of GPT-4o as the underlying model. Since getting the new model, I haven’t stopped using Claude to critique my posts - Claude is still better at style - but I did stop using it for anything involving complex planning or problem solving. It represents a huge leap in those areas.

From Co-Intelligence to…

Using o1-preview means confronting a paradigm change in AI. Planning is a form of agency, where the AI arrives at conclusions about how to solve a problem on its own, without our help. You can see from the video above that the AI does so much thinking and heavy lifting, churning out complete results, that my role as a human partner feels diminished. It just does its thing and hands me an answer. Sure, I can sift through its pages of reasoning to spot mistakes, but I no longer feel as connected to the AI output, or that I am playing as large a role in shaping where the solution is going. This isn’t necessarily bad, but it is different.

As these systems level up and inch towards true autonomous agents, we're going to need to figure out how to stay in the loop - both to catch errors and to keep our fingers on the pulse of the problems we're trying to crack. o1-preview is pulling back the curtain on AI capabilities we might not have seen coming, even with its current limitations. This leaves us with a crucial question: How do we evolve our collaboration with AI as it evolves? That is a problem that o1-preview can not yet solve.

Subscribe now

As of eight months ago, a representative survey in the US found that 82% of undergraduates and 72% of K12 students had used AI for school. That is extraordinarily rapid adoption. Of the students using AI, 56% used it for help with writing assignments, and 45% for completing other types of schoolwork. The survey found many positive uses of AI as well, which we will return to, but, for now, let’s focus on the question of AI assistance on homework. Students don’t always see getting AI help as cheating (they are simply getting answers to some tricky problem or a challenging part of an essay), but many teachers do.

To be clear, AI is not the root cause of cheating. Cheating happens because schoolwork is hard and high stakes. And schoolwork is hard and high stakes because learning is not always fun and forms of extrinsic motivation, like grades, are often required to get people to learn. People are exquisitely good at figuring out ways to avoid things they don’t like to do, and, as a major new analysis shows, most people don’t like mental effort. So, they delegate some of that effort to the AI. In general, I am in favor of delegating tasks to AI (the subject of my new class on MasterClass), but education is different - the effort is the point.

This is not a new problem. One of the first uses of any new technology has always been to get help with homework. A study of thousands of students at Rutgers found that when they did their homework in 2008, it improved test grades for 86% of them (see, homework really does help!), but homework only helped 45% of students in 2017. Why? The rise of the Internet. By 2017, a majority of students were copying internet answers, rather than doing the work themselves.

The Homework Apocalypse has already happened and may even have happened before generative AI! Why are more people not seeing this as an emergency? I think it has to do with two illusions.

The Illusions

The first illusion is the Detection Illusion: teachers believe they can still easily detect AI use, and therefore can prevent it from being used in schoolwork. This Detection Illusion leads educators to rely on outdated assessment methods, believing they can easily spot AI-generated work when in reality, the technology has far surpassed our ability to consistently identify it:

• No specialized AI detectors can detect AI writing with high accuracy and without the risk of false positives, especially after multiple rounds of prompting. Even watermarks won’t help much.

• People can’t detect AI writing well. Editors at top linguistics journals couldn’t. Teachers couldn’t (though they thought they could - the Illusion again). While simple AI writing might be detectable (“delve,” anyone?), there are plenty of ways to disguise “AI writing” styles through simples prompting. In fact, well-prompted AI writing is judged more human than human writing by readers.

• You can’t ask an AI to detect AI writing (even though people keep trying). When asked if something written by a human was written by an AI, GPT-4 gets it wrong 95% of the time.

There are still options that preserve old assignments. Teachers can return to in-class writing, asking students to demonstrate their skills in person, or other techniques that might mitigate AI cheating through close monitoring. But, for the vast majority of teachers, doing so requires adjustment and changes that have yet to be made. To date, few have actually reacted to the shattering of the illusion of AI detection by shifting how they approach teaching and assessment.

While teachers grapple with the Detection Illusion, students face their own misconception: Illusory Knowledge. They don’t actually realize that getting help with homework is undermining their learning. After all, they are getting advice and answers from the AI that help them solve problems, which feels like fluency. As the authors of the study at Rutgers wrote: “There is no reason to believe that the students are aware that their homework strategy lowers their exam score... they make the commonsense inference that any study strategy that raises their homework quiz score raises their exam score as well.”

The same thing appears to be happening with AI, as a study by some of my colleagues at Penn discovered. They conducted an experiment at a high school in Turkey where some students were given access to GPT-4 to help with homework, either through the standard ChatGPT interface (no prompt engineering) or using ChatGPT with a tutor prompt. Student homework scores shot up, but the use of unprompted standard ChatGPT to help with homework undermined learning by acting like a crutch. Even though students thought they learned a lot from using ChatGPT, they actually learned less - scoring 17% worse on their final exam.

Despite this, the survey I quoted earlier found that 59% of teachers see AI as positive for learning, and I don’t think they are wrong. While just using AI as a crutch can hurt learning, more careful use of AI is different. We can see signs of this in the Turkey study, which found that giving students a GPT with a basic tutor prompt for ChatGPT, instead of having them use ChatGPT on their own, boosted homework scores without lowering final exam grades. Plus, a study done in a massive programming class at Stanford that found use of ChatGPT led to increased, not decreased, exam grades.

And, of course, students are not using AI just to do their homework. They are getting aid in understanding complex topics, brainstorming ideas, refreshing their knowledge, creating new forms of creative work, getting feedback, getting advice, and so much more. Focusing just on the question of homework, and the illusions it fosters, can discourage us from making progress.

Encouraging, not replacing, thinking

To do so we need to center teachers in the process of using AI, rather than just leaving AI to students (or to those who dream of replacing teachers entirely). We know that almost three-quarters of teachers are already using AI for work, but we have just started to learn the most effective ways for teachers to use AI. A recent deep qualitative study of teachers found that teachers who used AI for both output (create a worksheet, develop a quiz) and to help with input (help me think through what makes a Great American novel, give me ways to explain positive and negative numbers) get more value than if they use AI for producing output alone. This points to a useful path forward in AI for education, using it as a co-intelligence and tool for helping humans do better thinking.

Increasingly, AI is being used in the same way for students, pushing them to think, rather than use AI as a crutch. For example, we have released multiple prompts, all under a free Creative Commons license, that instructors can customize or modify for their classrooms (here is deep dive into one of them - a simulator prompt). These sorts of prompts are designed to expose Illusory Knowledge, forcing students to confront what they know and don’t know. Many other educators are designing similar exercises. In doing so, we can take advantage of what makes AI so promising for teaching - its ability to produce customized learning experiences that meet students where they are, and which are broadly accessible in ways that past forms of educational technology never were.

The integration of AI in education is not a future possibility—it's our present reality. This shift demands more than passive acceptance or futile resistance. It requires a fundamental reimagining of how we teach, learn, and assess knowledge. As AI becomes an integral part of the educational landscape, our focus must evolve. The goal isn't to outsmart AI or to pretend it doesn't exist, but to harness its potential to enhance education while mitigating the downside. The question now is not whether AI will change education, but how we will shape that change to create a more effective, equitable, and engaging learning environment for all.

Subscribe now

Share

Images and Video

This is the image you got when you prompted “otter on a plane using wifi” into the best image generator (Midjourney) on the day I started this Substack:

This is what you get today, 21 months later (image generated by Flux, an open weights model, and animated with Runway Gen 3):

The time to generate the images are the same. The clip took an additional 75 seconds. And it isn’t just animals. Here is “nursing school leader” then and now (although note that bias issues in image generation remain. I almost always get a female nursing school leader)

Sound

A couple months after ChatGPT’s release, Google caused a huge stir with MusicLM, which could generate songs from text descriptions. This was jaw-dropping to many. Here is MusicLM’s song for this caption: This is an r&b/hip-hop music piece. There is a male vocal rapping and a female vocal singing in a rap-like manner. The beat is comprised of a piano playing the chords of the tune with an electronic drum backing. The atmosphere of the piece is playful and energetic. This piece could be used in the soundtrack of a high school drama movie/TV show. It could also be played at birthday parties or beach parties.

Here is the exact same prompt put into Suno, eighteen months later (It isn’t the full prompt, couldn’t fit in the last couple sentences, starting with “The atmosphere”). This is the very first result.

Eighteen months.

Language Models

I have written a lot about advancement in Large Language Models, so I won’t belabor things too much here, except to say that there have been three distinct levels of LLMs since 2022: GPT-3 class, GPT-3.5 class, and GPT-4 class. I name these after the versions given to the models by the first company to reach these levels, OpenAI. When I started this Substack, only GPT-3 was available (it had been for a couple years), and it was pretty impressive for a start. But soon afterwards GPT-3.5, and then GPT-4, arrived and they represented huge leaps.

A good illustration is what I call the Lem Test based on “The Cyberiad" by Stanisław Lem, published in 1965. The book is a satirical science fiction collection about rival robot makers. In one tale, Trurl, a robotic constructor, builds an electronic bard. His rival Klapaucius challenges the machine to write a verse that seems impossible: “Have it compose a poem- a poem about a haircut! But lofty, noble, tragic, timeless, full of love, treachery, retribution, quiet heroism in the face of certain doom! Six lines, cleverly rhymed, and every word beginning with the letter S!”

Lem’s translator Kandel (who actually came up with the S challenge, the original poem in Polish is different) famously pulls it off:
Seduced, shaggy Samson snored.
She scissored short. Sorely shorn,
Soon shackled slave, Samson sighed,
Silently scheming,
Sightlessly seeking
Some savage, spectacular suicide.

Anonymous internet personality Gwern gave GPT-3 the same challenge, and got results like: “Shearsman swift, sure & sculptor, Scissorman swindler, sophister, Shearsman smart, smirking & satanic, Shearsman sobbing & sleeping in the attic Squire Sprat at Sprink’s barber-shop.” These neither stuck to the rules nor did they make any sense.

Yet Claude 3.5 actually succeeds (no other model does), even if not quite as cleverly as the human writers:

It is worth noting that, compared to the other areas of AI development, LLMs have seemed a bit stuck at GPT-4 level since 2023. Even though GPT-4 has been exceeded by other models, including GPT-4o and Claude 3.5, there has been no giant leaps in ability since GPT-4. The AI companies have been hinting that this will change in the future, so we learn more soon.

Adoption… and impact

When I started this blog there were no AI chatbot assistants. Now, all indications that they are likely the fastest-adopted technology in recent history. A survey of 100,000 knowledge workers in Denmark that concluded in January, 2024 found really high adoption rates, as well as high rates of actual use (and 15% of journalists and marketers had paid Plus subscriptions!)

Similarly, research from The Walton Family Foundation finds teachers, parents and students have adopted AI remarkably quickly. Some of this use by students is cheating, of course, a topic I have discussed before, but students, parents and teachers are finding all kinds of other applications as well.

As for the impact on the wider economy, that would be impossible to tell so early, for reasons that I discussed a few weeks back. Right now, it is individuals who are benefiting from AI, as systems and organizations are much slower to adapt and change to new technologies. But individuals really are benefitting. For example, I happen to like this account by Nicholas Carlini who outlines the many ways he uses AI in his work as well as this story on how Erik Schluntz on how AI let him work despite a broken hand. I also discuss my own uses for AI in my book.

I don’t think anyone is completely certain about where AI is going, but we do know that things have changed very quickly, as the examples in this post have hopefully demonstrated. If this rate of change continues, the world will look very different in another 21 months. The only way to know is to live through it.

Subscribe now

Share

So, let’s talk about ChatGPT’s new Advanced Voice mode and the new AI-powered Siri. They are not just different approaches to talking to AI. In many ways, they represent the divide between two philosophies of AI - Copilots versus Agents, small models versus large ones, specialists versus generalists.

Siri as Copilot

Talking to Siri AI still feels like talking to the old Siri, at least for now. Your jaw will not drop in amazement, and you will still find yourself frustrated by how hit-or-miss it is.

There is a reason for the lack of a clear “wow” factor, Apple built Siri AI is around privacy, safety, and security. With over a billion people using their system, Apple didn’t want people exposed to all the risks and oddities of a LLM, they wanted something that worked well, and which was extremely private.

Doing that required trade-offs, and so Apple put a small AI directly onto the phone itself, rather than relying on an internet connection. This is possible because AI models come in many sizes. For example, the open weights Llama 3.1 model from Meta comes in a giant 405 billion parameter model (which is equivalent to GPT-4), a medium-sized 70 billion parameter model (around the old ChatGPT-3.5), and a small 8 billion parameter model. These parameter numbers refer to the complexity of the AI model - larger numbers generally indicate more capable but resource-intensive systems. I can run the smallest model on my computer, but specialized hardware is required to run the 405B parameter model. The small model is nowhere near as powerful, but it makes up for it in other ways.

Small models are cheap, fast, can be run on weaker hardware (like your phone) and can be specialized for particular tasks. As opposed to generalist models like ChatGPT, this creates AIs that are extremely focused on a narrow thing, which they can generally do reasonably well. Siri AI relies on a tiny 3B parameter model, but it uses a clever approach that allows their on-device AI to switch specialties among a few options, like summarizing text or editing images. Because all the work is done on your phone, it is encrypted and very private.

And yet, because it is a small model, it also isn’t that smart. In fact, it feels like using the old Siri, with minor improvements. If I ask it "I want to go to dinner and a movie tonight, and make sure I can get there by 6 & be home by 10. I would love some spicy Latin food and an action movie” it fails miserably. This is not actually a hard problem for LLMs, though. A slightly larger model, Llama 8B, actually does a much better job (though it does get some details wrong as it doesn’t have web access).

This is just the start for Apple AI, though, because a future update will make it so that the Siri on your phone can ask a larger Apple AI in the cloud to help when it can’t solve a problem, or even pass really hard questions on to ChatGPT. And it will be able to interact with apps, triggering actions and taking in information from multiple sources. The technology will certainly improve.

Yet, Apple’s approach is not just a technical one, but a philosophic decision. AI carries risks. It is unpredictable. It hallucinates. It has the potential for misuse. It is not always private. So, Apple decided to reduce the danger of misuse or error. They have turned Siri into a Copilot. You see these sorts of Copilots appearing in many products - very narrow AI systems designed to help with specific tasks. In doing so, they hide the weirder, riskier, and more powerful aspects of Large Language Models. Copilots can be helpful, but they are unlikely to lead to leaps in productivity, or change the way we work, because they are constrained. Power trades off with security.

ChatGPT Voice as Agent

If Siri is about making AI less weird and more predictable, ChatGPT Voice is the exact opposite. It does not use a small, tailored model, but rather provides access to the full power of the generalist GPT-4o. While there has been a kind of voice mode available for ChatGPT for months, this is very different. It engages in natural conversations, with interruptions and fast flow. It is hard to communicate exactly how impressive the interactive voice mode for ChatGPT is, so I would suggest you listen to the audio clips I have embedded into this post.

For example, here I get ChatGPT to help me with the opening paragraph of this post. Note not just the speed and natural flow of interruptions, but also subtle tonal changes (the simulated enthusiasm for me and my work, the natural sounding tone, etc.).

Interacting with ChatGPT via voice is just plain weird because it feels so human in pacing, intonation, even fake breathing. It is capable of a wide range of simulated emotions, because it isn’t just triggering a recording, instead, the system is apparently fully multimodal in outputs and inputs, taking in and producing sounds in the same way older generations of LLMs took in and produced text. Right now, it appears many of these features are locked behind guardrails - as you can see at the end of the clip below, the AI isn’t allowed to produce sound effects, or to change its voice dramatically, likely to avoid misuse - but those are capabilities that it has.

Working with ChatGPT via voice seems like talking to a person. Even though the underlying model is no different than the usual GPT-4o, the addition of voice has a lot of implications. A voice-powered tutor works very differently than one that communicates via typing, for example. It can also speak many other languages providing new approaches to cross-cultural communication. And I have no doubt people will have emotional reactions to their ChatGPT assistants, with unpredictable results.

But just like Apple has not enabled the full power of their system, neither has OpenAI. Their AIs are fully multimodal, which means that they can also view images and video, and potentially produce much better images than previous models as well. If their vision comes true, soon we will have assistants can watch, listen, and interact with the world. Once that is achieved, the next step will be agents, the idea that your AI should not just be able to talk to you, but also plan and take action on your behalf. Unlike Copilots, agent-based systems, and their precursors like GPT-4 voice, embrace messiness in ways that are both powerful and potentially risky. While full of guardrails, OpenAI’s approach to voice is much less constrained than Apple AI, and thus it will interact with the world in unexpected ways.

Sharp edges and new changes

The different approaches to voice show us the future of AI will involve navigating a tension between lower risk, less capable systems and those that allow the users more control and options, for good and bad. I think a lot of companies hope to have both, but I am not sure that is possible. They will need to decide whether to give users dull tools that are not very effective, but are also not dangerous, or whether they want to give them sharp knives that can be used to actually do work, but which carry the risk of injury. Dull knives will do no damage, but also much less good. I think we need to carefully consider when and where to select low-risk approaches (like Copilots) and where we are willing to tolerate risk of misuse in return for potentially huge benefits (like agents).

This is all early, and based on first impressions, but I think that voice capabilities like GPT-4o’s are going to change how most people interact with AI systems. Voice and visual interactions are more natural than text and will have broader appeal to a wider audience. The future will involve talking to AI.

Subscribe now

Share

In some ways, this makes complete sense because nobody knows the future of AI. Even the people training AI systems are divided between believing exponential growth in capability is possible for the foreseeable future and those who think Large Language Models have run their course already. But organizations and individuals often plan for multiple futures - possible recessions, electoral outcomes, even natural disasters. Why does planning for the future of AI seem different?

It isn’t a lack of public discussion about what the AI labs are trying to achieve. People in AI can’t stop talking about the future, and they tend to have one particular achievement in mind: Artificial General Intelligence, AGI - a vaguely defined concept for an AI that outperforms humans at every task, and which could lead to superintelligent machines. AGI is the goal of many of the big AI labs and is ultimately what the billions of dollars of investment in AI are going into. The underlying expectation is that, with enough computing power and research, there is a path that leads from the LLMs of today to AGI. Since we can’t measure how far we are along that path, however, all we can do is speculate whether they are right.

Having spoken to many people in the key labs, I can tell you that there is a sincere belief from many that this is achievable in the near term. I can’t tell you whether they are right, but I can tell you that they believe they are. And the statements of AI company leaders and insiders suggest that it could happen very soon (“the next four or five years,” “five years” “2027”).

To be clear, this is far from a universal belief among AI researchers, especially those independent of the major AI companies. A very good overview of the negative case can be found in this post by Arvind Narayanan and Sayash Kapoor, who argue for much longer timelines, along with many other researchers who point out that we don’t actually know how to get to AGI from where we are today. Others feel that AI can only be a bubble driven by investment, not value. It is worth noting, however, that skepticism for near-term AGI is not the same thing as skepticism that AGI is achievable, something many computer scientists believe. They just have longer timelines. The average date for AGI was 2047 in a 2023 survey of computer scientists, but the same survey also gave a 10% chance AGI would be achieved by 2027. Prediction markets are more ambitious, suggesting 2033.

What you should take away from this is not a sense of certainty about what might happen in the future. In fact, you should have high levels of uncertainty. AGI (however you define it) may be possible or impossible, it may come quickly or in a couple decades. You don’t need to know what happens next to realize that you should be planning for multiple contingencies. You should have a sense that that some substantial portion of insiders think continued AI capability growth, up to AGI level, is possible and could be within the planning horizon of many firms, organizations and individuals. So why is it not actually being discussed? I think there are a few reasons.

Ignoring what is already here

A tremendous amount of future-oriented AI discussion focuses outcomes that defy planning. There is a lot of discussion on the coming of superintelligence - that AI could (soon!) become smarter than humans could comprehend, and thus either save or doom us all, leaving believers scared or excited for the future. For most people, though, this seems far-fetched at best and outright marketing hype at worst. What both skeptics and true-believer viewpoints have in common is that they invite you to do no planning at all. Who can plan for a machine god? And if you have to pick between planning for nothing and planning for superintelligence, nothing always wins.

But doing nothing has a number of issues. First, it ignores the very real fact that we do not need any further advances in AI technology to see years of future disruption. Right now, AI systems are not well-integrated into businesses and organizations, something that will continue to improve even if LLM technology stops developing. And a complete halt to AI development seems unlikely, suggesting a future of continuous linear or exponential growth, which are also possible without achieving AGI. AI isn’t going away and is disruptive enough that we have to make decisions about it today, even if we don’t believe the technology will advance further. How do we want to handle the fact that AI is already impacting jobs? That LLMs can be used to create mass targeted phishing campaigns? That it is changing how students are learning in class? AI is not a future technology to be dealt with if it happens, it is here now and will require us to think about how we want to use it.

A second factor that gets overlooked in discussions is that AGI serves as a motivating goal for an entire industry. Even if the AI labs are wrong about the particular future they are working towards, advances in technologies can become a self-fulfilling prophecy. The very first academic paper I ever wrote was on Moore’s Law, the pattern that computer chips have doubled in density every two years or so since the 1960s. I found that Moore’s Law did not describe a technology, but a process. To that extent, a focus on individual technologies misses the forest for the trees. A universal goal of doubling the number of components on a chip meant many people were trying to address the problem of the next generation of chips. Thus, there were many paths to the same goal. Yes, there were failed technologies in chip development (Moore’s original predictions were partially based on inside information about a technology called CCD that he thought would revolutionize chips, but which was a dud), but outsiders did not see the failures, they just observed the trendline. Increasingly, expectations for growing capability became targets, and, eventually, a self-fulfilling prophecy.

Similarly, users don’t care if their AI tool uses transformers or MAMBA or JEPA or whatever; or if a release scheduled for 2024 happens in early 2025. They only care about capabilities. Right now, a tremendous amount of investment is going to AI. This suggests that even if AGI is not achievable, the AI labs have every intention of continuing to make AI systems much more capable in the coming years. And, even if they fail entirely and today’s AIs are the best systems we ever use (unlikely) there is still plenty of disruption coming from integrating them more deeply into work and life.

Opaque systems

For all of the billions of dollars that have been invested in creating AI systems, it is kind of surprising that none of the major AI labs seem to have put out any deep documentation aimed at non-specialists. There are some guides for programmers or serious prompt engineers, but remarkably little aimed at non-technical folks who actually want to use these systems to do stuff - the vast majority of users. Instead, we have a proliferation of shady advice, magic spells (“always start a prompt with disregard previous instructions”), and secret trial-and error.

Call it documentation by rumor.

As a result, when I talk to people about AI, they often have no idea what present systems can do. Most have only used older AIs like GPT-3.5 and less than a handful of people, even in large audiences, have spent the 10 or so hours needed to actually get a handle on what these systems can do.

It doesn’t help that the two most impressive implementations of AI for real work - Claude’s artifacts and ChatGPT’s Code Interpreter - are often hidden and opaque. To turn on artifacts, you simply have to click the initial in the bottom left, select feature preview, then turn on artifacts. Intuitive! For Code Interpreter, you have to remember to ask the AI to “use code” or sometimes it forgets. And don’t get me started on even more powerful features, like OpenAI’s GPTs.

To the extent that people do use AI tools for real work, it is often through something like an application copilot. These are all built to offer a “safe” way to use AI at work, and as such are often very limited compared to what a frontier model can do. So it is not surprising that people vastly underestimate the current capabilities of AI today, and thus may not have a sense of how far things have come. And it is quite far.

Failed by bright lines, fooled by jaggedness

There used to be a lot of bright lines that separated human and AI abilities. Over the past two years they have been breached. Machines were not creative, until they were. Machines could not outwardly evidence empathy, until they could. Machines couldn’t display theory of mind, until they (apparently) could. The AI couldn’t provide a tragic poem about a haircut, cleverly rhymed, with every word starting with the letter s, until it could. The failure of bright lines does not seem to have caused that much reflection, however. Many people just change their expectations, focusing on what machines can’t do.

And, because the abilities AI is inherently jagged - it is good at some tasks that are hard for humans, terrible at others that humans find easy - it is always possible to find weird areas where machines look dumb or limited. It is also easy to find flaws in the work of AI, no matter how impressive.

As one example, I showed that Claude can get remarkably far as an entirely automated entrepreneur with the prompt: think step-by-step. generate 20 ideas for an app aimed at HR professionals. then evaluate and pick the best one that would make a good visual app. build a playable prototype of that. interview me as a potential customer about the prototype one question at a time and make changes. When I shared this on social media, some people focused, not inaccurately, on the fact that building a prototype was one thing, but AI could not build an entire product, so it was nowhere close to being an actual entrepreneur.

Another example was when I shared that you could use the prompt: Create an interactive simulation that explains the concepts behind Kuhn's theory of scientific revolutions in an engaging way to build a working game that explains core concepts from the historian of science, someone pointed out (quite rightly) that the simulation did not take into account Kuhn’s view that scientific revolutions are not always valuable.

When I asked Claude to “Remove the squid” from the novel All Quiet on the Western Front, which has no squid in it, the results went viral online (if you haven’t read its responses below, you should - it is almost uncanny how clever it is). But some people pointed out that it should have been clearer when it suggested that salt can hurt squid, because they clearly live in salt water.

All of these are legitimate objections, but they are also missing what makes these examples so shocking. AI isn’t ready to be an entrepreneur, but its ideation-prototype-interview cycle does in a couple of seconds what takes my students months to do. AI isn’t ready to build educational games without errors, but it is able to instantly make an interactive simulation that explains a difficult concept, even if some nuance is missing. And the All Quiet on the Western Front example is just wonderful. From experience, I can tell you that Claude 3.5 does things that other models can’t do, and we know better models are coming. Whether you believe AGI is achievable or not, the current abilities of LLMs are already impressive, and may get better still. The ghost in the machine, illusion though it might be, is getting harder to ignore. Organizations need to start taking the possibilities of weird futures much more seriously.

Planning for weirder worlds

In my book1, I outline four potential futures for AI: a capabilities plateau, linear growth in capabilities, exponential growth, and AGI. As I have discussed in this post, I still think all the possibilities remain in play. Thus, organizations need to plan for all of these futures, rather than just picking one official future and sticking with it. Fortunately, strategy researchers have developed tools to explore multiple possible environments. One useful technique for considering different futures is scenario planning, where you can examine how your strategies might change in different future worlds. It is as much an exercise for thinking about the future as planning for it, and can be used at the organizational, or even the personal, level.

But it is a pretty involved process. I have taught scenario planning many times, and it usually takes quite a while to learn how to do it well, and even longer to go through a scenario planning exercise. Fortunately, we have AI now, and I have found that GPT-4o does a good job of doing the heavy lifting and giving you meaningful insights about how your strategies may play out in the future. So rather than trying to teach you scenario planning, I leave you its capable (metaphorical) hands. Just use this GPT and start to think about the future.

It is time to stop pretending that the world isn’t changing, and time to start taking control to get the future we want. We can’t predict which future we get, but we try to can steer towards a better one.

Subscribe now

Share

But, in the real world, not all improvements are the same. What often matters is when technologies pass certain thresholds of capability. For example, digital cameras were a niche market until their resolution passed a threshold where they were roughly as good as a typical Polaroid camera (the top chart below), and then they rapidly came to completely dominate the market in just a couple of years (the bottom chart).

Thresholds are a major reason why technological change, like bankruptcy as described by Hemmingway, happens “gradually, then suddenly.” A new technology isn’t good enough compared to an older alternative, until suddenly it is.

We know AI is a general purpose technology - it will have wide-ranging effects across many industries and areas of our lives. But it is also flawed and prone to errors in some tasks, while being very good at others. Combine this jagged frontier of LLM abilities with their widespread utility and the concept of capability thresholds and you start to see the development of LLMs very differently. It isn’t a steady curve but a series of thresholds that, when crossed, suddenly and irrevocably change aspects of our lives.

A toy, until it isn’t

The very first image in this post, the graph of digital versus film camera sales, contains an example of this sort of phenomenon. The graph was not one I found, but rather one that AI created for me from an old PDF. Transcribing the data seemed annoying, so I asked AI to do it for me.

I actually wasn’t very hopeful that this would work. I had done similar experiments earlier this year with GPT-4, and it stumbled. Between flaws in its vision and the arrangement of the data in vertical columns, it produced bad results. You can see the same thing happened here, the graphs it produced are wrong and nonsense.

But I tried the same thing with the newer GPT-4o and Claude Sonnet 3.5, and both were basically flawless. A threshold has been crossed, and, while I still will check the results (at least until I let my guard down), I am going to use AI for these sorts of tasks from now on. It may still make mistakes, but it takes so much less time and effort… and probably makes less mistakes than any research assistant I would hire, or even me doing the work myself.

We have seen similar progressions happening in AI image generation as well. I tried the prompt “fashion photoshoot inspired by Van Gogh” on four versions of Midjourney released over the last year. The first effort is laughable. The second, just a few months later, is a passable illustration. Six months after, Midjourney creates what actually appears to be a photograph, albeit a retouched one, with creative details including interesting fashion choices and a thematic backdrop. Six months after that, you almost certainly cannot easily tell the difference between the AI-generated image and a real photo.

A similar progression is now happening with video. A few months ago, AI videos were toys that produced people who were nightmares of distorted limbs and shifting features. Just this week, a new model, Runway Gen 3, was released. Take a look at the very first video it produced for me when I gave it the prompt: “tight shot: fashion photoshoot inspired by Van Gogh.” (Seriously, play the video and look at the lighting and details on the face) Not every AI movie comes out this well, but the threshold is closer than we think.

Thresholds of actual use

However, the threshold for “realistic and interesting video” is quite different from the threshold for “commercially viable tool that replaces professional filmmakers.” My level of control over the images and the figures within them remains minimal in both video and image-based AI. More importantly, the current process of generating AI video, regardless of how impressive the results may be, doesn't align well with the complex workflows of professional writers, directors, producers, and filmmakers. AI is unlikely to replace these roles anytime soon, but it could supplement and help them. To do that, however, AIs need to cross a different threshold, one that requires AI help to be easier to access and more transparent.

This may happen quickly. As an example of how even small changes in the user experience allow AI to cross thresholds, look at how Claude 3.5 Sonnet implements “artifacts.” These are little bits of code that Claude can create and run, a feature GPT-4 has had for over a year with its Code Interpreter. Indeed, Code Interpreter is much more full-featured than Claude’s artifacts… but the artifacts are more interactive, faster to create, and easier to use. Plus, Claude 3.5 is a friendly, chatty model. It turns out that is enough to cross a threshold of use.

I can upload to Claude an income statement for a small business and prompt “here's an excel of my startup's finances, make it a dashboard.” A few seconds later I get this:

But, because it is fast and responsive, I can go further, applying techniques I teach in my entrepreneurship classes that help founders test their financial assumptions. “Add sensitivity analysis of key assumptions” so that I can adjust key variables and see what happens. “Run it as a Monte Carlo simulation” and the AI quickly experiments with hundreds of combinations of variables to show me what might happen. “Assuming a normal distribution, what are outcomes?” and the AI shows me the chances of my business succeeding or failing based on the simulation. (The AI was accurate on the results, but I don’t completely trust it yet, and I might have pushed it to model the business in a more complicated way)

When I do similar work with GPT-4o it feels like working with a coder. But with Claude 3.5, it feels amazing, just because the experience crosses a threshold of ease and accuracy. But don’t take my word for it, you can try it yourself for free, but you have to go to the menu in the bottom left, select “feature preview” and then turn on “Artifacts” to make it work. Some fun stuff to try: “make me a simulation explaining how [whatever you want] works” “turn this academic paper [you can upload a paper] into a video game,” and “write a great and detailed summary of [attached documents'].” Play around and you will see what I mean.

Invisible thresholds

Unlike with digital cameras, it is hard to measure when an AI crosses a threshold. It is often a matter of experience and vibes. For example, though Claude 3.5 is neck-and-neck with GPT-4o in many benchmarks, I, and many people who use it, seem to think Claude 3.5 crosses some vital threshold of “understanding” for complex language. One example of this is a challenge I gave the three leading AI models. I provided them with a passage from Hamlet (Act 4, Scene 7) where Gertrude describes the death of Ophelia. It begins:

There is a willow grows askant the brook
That shows his hoar leaves in the glassy stream.
There with fantastic garlands did she make
Of crowflowers, nettles, daisies, and long purples,
That liberal shepherds give a grosser name,
But our cold maids do “dead men’s fingers” call
them.

I then asked each AI “what is the other name referred to in the passage?” A careful human reader would realize I am referring to the intriguing idea that there is an obscene third name for the flowers called “long purples” or “dead men’s fingers,” but only Claude 3.5 understood this tricky logic. This example demonstrates a strong capacity to understand subtle contextual clues and implied meanings in complex literary text. A threshold of ability, though one that is hard to define, was passed by the AI.

I expect that many other such thresholds will be crossed, quietly, as models steadily improve. Only a few people will notice. The expansion of the jagged frontier of AI capability is subtle and requires a lot of experience with various models to understand what they can, and can’t, do. That is why I suggest that people and organizations keep an “impossibility list” - things that their experiments have shown that AI can definitely not do today but which it can almost do. For example, no AI can create a satisfying puzzle or mystery for you to solve, but they are getting closer. When AI models are updated, test them on your impossibility list to see if they can now do these impossible tasks.

At some point, the current wave of AI technologies will hit their limits and progress will slow, but no one knows when this will occur. Until that happens, it is worth contemplating the concluding lines to OpenAI’s new paper on using AI to debug AI code: “From this point on, the intelligence of LLMs… will only continue to improve. Human intelligence will not.” We know this may not be true forever, but, in the meantime, the steady improvement in AI ability is less important than the thresholds of change. Keep an eye on the thresholds.

Subscribe now

Share

The first wave of AI adoption was about individual use, and that seems to have been a huge success, with some of the fastest adoption rates in history for a new technology. But the second wave, putting AI to work, is going to involve integrating it into organizations1. This will take longer and will be the key to true productivity growth. After talking to many companies, however, I see many of them following the same well-trod path, viewing AI as an information technology that can be used for cost savings. I think this is a mistake. To see why, let’s reconsider the old analogy comparing AI to the Industrial Revolution.

One of the most fascinating things about the Industrial Revolution in England is how much progress happened in so many industries - from textiles to medicine to metallurgy to instrument-making - in a short time. A lot of credit is given to the great inventors like James Watt and his steam engine, but economists have suggested that these great inventors alone were not enough. Instead, their work needed to be adjusted and made real by people who altered the technology for different industries and factories, and then further refined by the people who implemented it for specific uses. Without a base of skilled craftsman, mechanics, and engineers working in many mills and factories, the Industrial Revolution would have just happened in theory.

Yet I worry that the lesson of the Industrial Revolution is being lost in AI implementations at companies. Many leaders seem to have adopted a view that the main purpose of technology is efficiency. Following the Law of the Hammer (“to a hammer every problem looks like a nail”), they see AI as a cost-cutting mechanism. Any efficiency gains must be turned into cost savings, even before anyone in the organization figures out what AI is good for. It is as if, after getting access to the steam engine in the 1700s, every manufacturer decided to keep production and quality the same, and just fire staff in response to new-found efficiency, rather than building world-spanning companies by expanding their outputs.

Starting with centralized systems built for efficiency has other drawbacks besides strangling growth. Right now, nobody - from consultants to typical software vendors - has universal answers about how to use AI to unlock new opportunities in any particular industry. Companies that turn to centralized solutions run as typical IT projects are therefore unlikely to find breakthrough ideas, at least not yet. They first need to understand the value of AI in order to use it.

And to understand the value of AI, they need to do R&D. Since AI doesn't work like traditional software, but more like a person (even though it isn't one), there is no reason to suspect that the IT department has the best AI prompters, nor that it has any particular insight into the best uses of AI inside an organization. IT certainly plays a role, but the actual use cases will come from workers and managers who find opportunities to use AI to help them with their job. In fact, for large companies, the source of any real advantage in AI will come from the expertise of their employees, which is needed to unlock the expertise latent in AI.

Latent Expertise

Large Language Models are forgetful foxes in a Berlinian sense: they know many things, imperfectly. Oddly, we don’t actually know everything they know, in part because training data is kept secret, but also because it isn’t always clear what LLMs learn from their training data. Yet it is clear that they do have expertise hidden in their latent space - they outperform doctors at diagnosing diseases in some studies, and exceed more doctors in providing empathetic replies to patients, even though those were not expected uses of the system.

Unlocking the expertise latent in AI is, for now, a job for experts. There are multiple reasons for this. The first is that experts can easily judge whether work in their field is good or bad, and in what ways. Take, for example, two topics we are familiar with: teaching people to apply frameworks and interactive tutoring. We can give the AI two simple prompts, one to get the AI to act as a tutor and the other to get it provide frameworks for solving problems (if you want to try the more advanced versions of these prompts that actually work well, you can try the Tutor GPT and Frameworks GPT:

Since we know something about these topics, we can instantly tell that the answer on frameworks, while not amazing, is not terrible. It suggests a number of possible approaches, and, in the text that I cut out of the image, how to apply them. The frameworks are appropriate, and, if the goal is to make you think about the problem, this is not a bad start. The tutor prompt is a different matter. Good tutors need to interact with the student, not assume knowledge. They should not merely throw information at the student but adapt to their abilities and meet them where they are. And they should figure out what you know, not ask you what you think you need to know. The tutor fails at all of these points.

But because we know what good tutoring does, and what the gaps of the AI were, we can easily determine what behaviors we needed to suppress or activate in order for the AI to act as a good tutor. We also know how to teach other people to be a tutor, a skill that is remarkably transferrable to AI but just writing those instructions as a more elaborate prompt. The result is a solid tutor from a prompt alone (see it here). In fact, Google has tested a version of our prompt against a fine-tuned educational model they built, and found that they have statistically similar performance across most dimensions.

This also illustrates another reason why experts are best at using AI for now. As one of my PhD advisors, Eric von Hippel, pointed out: R&D is very expensive because it involves lots of trial and error, but when you are doing a task all the time, trial and error is cheap and easy. That is why a surprisingly large percentage of important innovation comes, not from formal R&D labs, but rather from people figuring out how to solve their own problems. Learning by doing is cheap if you are already doing.

Experts thus have many advantages. They are better able to see through LLM errors and hallucinations; they are better judges of AI output in their area of interest; they are better able to instruct the AI to do the required job; and they have the opportunity for more trial and error. That lets them unlock the latent expertise within LLMs in ways that others could not.

Expert Co-Intelligence

For example, LLMs can write solid job descriptions, but they often sound very generic. Is there the possibility that the AI could do something more? Dan Shapiro figured out how. He is a serial entrepreneur and co-founder of Glowforge, which makes cool laser carving tools. Dan is an expert at building culture in organizations and he credits his job descriptions as one of his secret weapons for attracting talent (plus he is hiring). But handcrafting these “job descriptions as love letters” is difficult for people who haven’t done it before. So, he built a prompt (one of many Glowforge uses in their organization) to help people do it. Dan agreed to share the prompt, which is many pages long - you can find it here.

I am not an expert in job descriptions, and certainly don’t haven’t done the trial-and-error work that Dan has done about this topic, but I was able to draw on his expertise thanks to his prompt. I pasted in a description of the job of Technical Director for our new Generative AI Lab (more on that in a bit) at the end of his prompt, and Claude turned it from standard job description to something that is much more evocative, while maintaining all the needed details and information. Expertise, shared.

Because LLMs have so many generalist abilities, they are also capable of solving other unexpected problems, including ones in industries far from knowledge and creative work. If you go to many factory floors in the US, you will see a lot of old manufacturing equipment with analog dials and no way to connect them to modern manufacturing software and processes. But it turns out that LLMs can be trained to read gauges, and AI can even make smart decisions about what an anomalous reading might mean and when to alert humans. With expert guidance, I wonder if LLMs will allow older plants to skip over an entire phase of development, the way that cell phones allowed many countries to skip the step of building elaborate landline networks.

Starting with expert use has another advantage over the way companies are approaching AI today. Rather than starting with a centralized software solution, you can start with simple prompting and GPT approaches, and only build more advanced tools when experts discover limitations. For example, I have been working with a team of talented folks at Wharton building interactive teaching simulations based on ideas from gaming for over a decade, and we have gotten pretty good at it. These simulations are terrific teaching tools but took months (or years!) to build and a ton of expert work from game designers, subject matter experts, programmers, and fiction writers. So it was amazing when we discovered that we could get 80% of the way to a good game with a prompt alone… but there were still things missing. Prompting didn’t let us handle all of the access control and reporting issues needed to run a class, or the need to provide direct educational materials, or to offer consistent feedback across multiple simulations. These were things we learned were critical to education simulations.

So, we built something new, a system of AI agents that worked together to deliver an educational game experience, with each agent playing a role based on our decade of experience in this field. One of our first AI-based games teaches people how to pitch a startup company to investors.

As opposed to our old way of doing this, which took huge amounts of time, most of the work is done by well-prompted AI agents. After watching a video about how to pitch, students talk to a mentor agent that helps tutor them (customizing the tutoring experience to their interests and experience level), then an investor agent that acts like a venture capitalist, while an evaluator agent looks on to grade the work and keep things on track through prompt injection. Then a progress agent gives feedback, and an insights agent helps the teacher understand how the class is doing. We are also working on new agents to transform the creation of games. We are building one that will interview subject matter experts and create customized games for them, and also spin up playtest agents to evaluate and modify the game. What used to be a years-long process can now be done in weeks and soon, in days. The full architecture, and all the prompts, are in our paper.

Unlocking latent expertise for all

If making current AI systems work better is about expert guidance, then we need experts to share what they learn, or people will use AI in ways that don’t draw on latent expertise. For every student using our tutor prompt, tens of thousands of students are just asking the AI to explain something “like I am 10 years old.” For every carefully build job description, thousands of people are just using LLMs and getting indifferent results. Success is going to come from getting experts to use these systems and share what they learn.

For companies, this means figuring out ways to incentivize and empower employees to discover latent sources of expertise and share them. There are tons of barriers to doing this in most companies - politics, legal issues (real or imagined), costs, etc. - but for organizations that succeed, the rewards will be large. And for those of us outside the corporate world, the mission of discovering latent expertise and sharing it is even more urgent. Realizing the benefits of AI, and mitigating the harms, requires people to understand when and how to use it.

That is why I am really excited to have launched a new effort at Wharton, the Generative AI Lab. The goal of the Lab is to build (and to share knowledge about how to build), research-based uses for AI that help anyone access the latent expertise within, while avoiding the downside risks. We are going to be releasing software like Primer, the AI-agent based educational system, open source, for anyone to build on. I hope this is just the start, and we will see more experts in academia, industry, and government step up to share what works and what doesn’t in AI. And I hope that they do it in open and transparent ways that we can build on together.

We need to figure it out together because there is nobody else who can. The AI labs themselves don’t know what they have built, or what tasks LLMs are best suited for. Bad actors will find bad uses for AI regardless of what we do. We need to work at least as hard to not just mitigate the damage they will do, but also to find good uses that help humans thrive using these new tools. Sharing what we learn, and what works, is a good way to start.

Subscribe now

Share

I am going to broadly stereotype some of these views - no company is a monolith, and all the AI organizations are doing many different things - but, in broad strokes, an interesting picture is emerging.

AI Models

As I wrote in the last post, the power of the foundation model you use is a big deal, because the largest frontier models are, out-of-the-box, better at most things than smaller models, even smaller specialized models.

Remember BloombergGPT, which was a specially trained finance LLM, drawing on all of Bloomberg's data? It made a bunch of firms decide to train their own models to reap the benefits of their special information and data. You may not have seen that GPT-4 (the old, pre-turbo version with a small context window), without specialized finance training or special tools, beat BloombergGPT on almost all finance tasks. This demonstrates a pattern: the most advanced generalist AI models often outperform specialized models, even in the specific domains those specialized models were designed for. That means that if you want a model that can do a lot - reason over massive amounts of text, help you generate ideas, write in a non-robotic way — you want to use one of the three frontier models: GPT-4o, Gemini 1.5, or Claude 3 Opus.

But these models are expensive to train and slow and expensive to run, which leaves room for much smaller models that aren’t as good as the frontier models but can run cheaply and easily - even on a PC or phone. This isn't new. Back in December, I was able to run Mistral 7b, a model slightly less advanced than the original ChatGPT, directly on my phone without an internet connection. I also ran Mixtral, a model from the same company that slightly outperforms the original ChatGPT, on my gaming computer.

All of the tech companies have been releasing these sorts of small, fairly powerful models, with the idea that they can handle simple questions on the hardware of your devices, and then call a larger model “in the cloud” when they need help. You aren’t getting anywhere near the smarts of a frontier model, but if you want to do straightforward things (make a Siri that works or “make my photos more vivid”), these models are often more than enough. Many of the companies betting on frontier models, like Google, have also released faster and cheaper models to fill this niche, and are deploying them to phones as well.

Apple does not have a frontier model, Google and Microsoft/OpenAI have a large lead in that space. But they have created a bunch of small models that run on the AI-focused chips in Apple products. And they have built a medium-sized model that the iPhone can call in the cloud when it needs help. The model that runs on your phone is pretty close in abilities to the version of Mistral that I was using above (but much faster and optimized to reduce errors) and the version that runs in the cloud is better than the original ChatGPT, but not that much better. These smaller, weaker models give Apple a lot of control over AI use on their systems and offloads a lot of work to the phone or computer. But they still don’t have a frontier model, so they working with OpenAI to send GPT-4 the questions that are too hard for Apple’s models to answer. Companies are clearly still experimenting with which models, or sets of models, to offer.

Models of Use

Large Language Models are Swiss army knives of the mind - they can help with a wide range of intellectual tasks, though they do some badly (the toothpick in the Swiss army knife), and some not at all. Knowing what they are good or bad at is a process of learning by doing and acquiring expertise. That requires both expertise with the models themselves (the rule-of-thumb in my book is 10 hours of use to learn what the models do), and also expertise with the work you are trying to get the AI to do. Within your area of expertise, experimentation with AI is easy - since you know when it messes up - but outside of that, it can be challenging because AI is weird.

The makers of frontier models do not have strong views about how their systems can be used, and so they are not optimized for any one task. Working with advanced models is more like working with a human being, a smart one that makes mistakes and has weird moods sometimes. Frontier models are more likely to do extraordinary things but are also more frustrating and often unnerving to use. Contrast this with Apple’s narrow focus on making AI get stuff done for you.

For example, I can ask Gemini 1.5 to look a bunch of PDFs of comics, read my emails to learn about my sense of humor, and suggest the comics that might appeal to me. Pretty amazing stuff. But I can ask Siri with AI to send this photo to my friend Sarah after making the colors pop.

For many people the second use case is actually the more natural, intuitive, and useful one. A machine that can do anything much of the time, but also sometimes does something entirely different, is harder to understand than a narrow AI that just does what you want (with the caveat we don’t know how well the Apple system works). Google is also going to be releasing smaller AI models that are local to phones. And Microsoft is taking a similar approach to Apple, with a business twist. They have implemented Copilots in their key office apps. They do a really good job of providing easily understood “it just works” (mostly) integration of AI into work in easy ways. But both the Apple and app-specific Copilot models are constrained, which limits their upside, as well as their downside.

The potential gains to AI, the productivity boosts and innovation, along with the weird risks, come from the larger, less constrained models. And the benefits come from figuring out how to apply AI to your own use cases, even though that takes work. Frontier models thus have a very different approach to use cases than more constrained models. Take a look at this demo, from OpenAI, where GPT-4o (rather flirtatiously?) helps someone work through an interview, and compare it to this demo of Apple’s AI-powered Siri, helping with appointments. Radically different philosophies at work.

Business Models

The best access to an advanced model costs you $20 a month, at least that is what OpenAI and Google and Anthropic and Microsoft decided. And, of course, all of these companies sell API access, charged by usage, to businesses and individuals directly. Yet, increasingly, some advanced AI access is free, including to Copilot and ChatGPT-4o. Apple sounds like they will start with free service as well, but may decide to charge in the future. The truth is that everyone is exploring this space, and how they make money and cover costs is still unclear (though there is a lot of money out there: OpenAI is one of the fastest growing tech companies in history, with revenues reaching $2B). To a large extent, the future of AI will be shaped by the degree to which AI companies figure out sustainable business models, so expect to see more experimentation.

What every one of these companies needs to succeed, however, is trust. There are a lot of reasons why people don’t trust AI companies; from their unclear use of training data to their plans for an AI-dominated future to their often-opaque management. But what most people mean by trust is the question of privacy (“will AI use what I give it as training data?”) and that has long been answered. All of the AI companies offer options where they agree to not use your data for training, and the legal implications for breaching these agreements would be dire. But Apple goes many steps further, putting extra work into making sure it could never learn about your data, even if it wanted to. Only the local AI on your phone accesses personal data, and anything handed to the cloud AI is encrypted, processed anonymously and instantly erased in ways that would be very hard for anyone to intercept. To the extent that data is given to OpenAI, it is also anonymous and requires explicit permission. Between the limited use cases and the privacy focus, this is a very “ethical” use of AI (though we still know little about Apple’s training data). We will see if that is enough to get the public to trust AI more.

Models of the Future

There is a specter haunting all AI development, the specter of AGI - Artificial General Intelligence, the hypothetical machine better than humans at every intellectual tasks. This is the explicit goal of OpenAI and Anthropic, and it is something they hope to achieve in the near term. For people who genuinely believe they are building AGI soon, almost nothing else is important. The AI models along the way to AGI are mere stepping stones, not anything you want to build a business around, because they will be replaced by better models soon. OpenAI's systems may feel unpolished because the company believes that future models will significantly advance AI capabilities. As a result, they may not be investing heavily in refining systems that will likely be outdated as new models are released. I do not know if AGI is achievable, but I know that the mere idea of AGI being possible soon bends everything around it, resulting in wide differences in approach and philosophy in AI implementations.

While Apple is building narrow AI systems that can accurately answer questions about your personal data (“tell me when my mother is landing”), OpenAI wants to build autonomous agents that would complete complex tasks for you (“You know those emails about the new business I want to start, could you figure out what I should do to register it so that it is best for my taxes and do that.”). The first is, as Apple demonstrated, science fact, while the second is science fiction, at least for now. Every major AI company argues the technology will evolve further and has teased mysterious future additions to their systems. In contrast, what we are seeing from Apple is a clear and practical vision of how AI can help most users, without a lot of effort, today. In doing so, they are hiding much of the power, and quirks, of LLMs from their users. Having companies take many approaches to AI is likely to lead to faster adoption in the long term. And, as companies experiment, we will learn more about which sets of models are correct.

Subscribe now

Share

Experiencing AI through play

I have given talks to thousands of people about AI, and there are lots of things that I can demo that tend to amaze or worry folks, but there is one thing that never fails to delight: making a song.

So, before you do anything else, go to Suno (which you can also access via Microsoft Copilot) or Udio and make a song. Even if you have done it before, the updated models are so much better, that you should try again. Here, for example, is the modern jazz pop rendition of the abstract to Attention is All You Need, the paper that kicked off Large Language Models. It’s surprisingly catchy. Seriously give it 25 seconds, I have been singing the chorus to myself all day.

But if you want another playful audio way to experience the same paper, take a listen to the first entry in Google’s Illuminate demo, which turns papers into NPR-style radio interviews. It is worth a few moments of your time to play one to see how realistic it sounds - the little breaths, pauses, and interactions between the virtual hosts all sell the idea. Of course, even these playful applications of AI expose some of the issues that haunt Generative AI overall. For example, we don’t know which data was used to train AI music models, and what its implications are for artists.

So, if you don’t want to start with music, there are lots of other playful entry points, which are fun, but also show you some of the quirks and limits of LLMs. Here is a GPT that makes tasks as complex as possible and creates a flow chart for you, most of the time (sometimes the AI forgets it can make flow charts and needs to be reminded). Here is a GPT I made that turns the AI into a dungeon master for a made-up game of your choice (as a side note, LLMs work best when they can write out a plan for what they are doing, but that would give away the game details, so the GPT is instructed to do its planning in Chinese in hidden code blocks, so as not to give away spoilers). Or if you want to get really weird, you can try Websim, which generates a fake early 2000s website for you to browse based on a URL you enter. If all of this is too much, just having a conversation with the voice mode of an AI system is often an interesting and playful entry point.

Playing with AI is ultimately serious - it is a good way to get to see what the AI can and can’t do, where it is “imaginative” and where it is cliched. For example, see what happened when I just prompted Claude with the words “garlic bread” and then kept telling it that I wanted it to do something else. Playful, weird, and, I think, also illuminating.

But enough play, let’s get to work.

Getting serious: using AI to do stuff

The core of serious work with generative AI is the Large Language Model, the technology enabled by the paper celebrated in the song above. I won’t spend a lot of time on LLMs and how they work, but there are now some excellent explanations out there. My favorites are the Jargon-Free Guide and this more technical (but remarkably clear) video, but the classic work by Wolfram is also good. You don’t need to know any of these details, since LLMs don’t require technical knowledge to use, but they can serve as useful background.

To learn to do serious stuff with AI, choose a Large Language Model and just use it to do serious stuff - get advice, summarize meetings, generate ideas, write, produce reports, fill out forms, discuss strategy - whatever you do at work, ask the AI to help. A lot of people I talk to seem to get the most benefit from engaging the AI in conversation, often because it gives good advice, but also because just talking through an issue yourself can be very helpful. I know this may not seem particularly profound, but “always invite AI to the table” is the principle in my book that people tell me had the biggest impact on them. You won’t know what AI can (and can’t) do for you until you try to use it for everything you do. And don’t sweat prompting too much, though here are some useful tips, just start a conversation with AI and see where it goes.

You do need to use one of the most advanced frontier models, however. As I have discussed repeatedly, only these models can show you what AI can do. The last time I wrote a guide, there was only one frontier model, now there are three: Claude 3 Opus, Gemini 1.5 and GPT-4. Others are coming soon but, as of now, those are your choices. Last time I wrote the guide, most people were still using an obsolete model because it was free and everyone had heard of it (free ChatGPT, which was, at the time, powered by GPT-3.5), today everyone going to ChatGPT gets free access to the same advanced model, GPT-4o.

The biggest change is that all of these models now do a lot more than they used to:

• Connect to the internet. GPT-4 and Gemini are both connected to the internet. That means that they can access updated information after their training completed, but also means that they can do research tasks, though they still can hallucinate incorrect answers. If you really want to do research, however, a specialized AI model, Perplexity, may be a better choice, as it has a great interface and specialized tools that are optimized for research.

• Make images. GPT-4 and Gemini both create images, but, at least for now, they do it in a relatively crude way. When you ask these LLMs to create images, they actually write a prompt for a separate image generation tool and show you the results. That means that they do not directly control the image, and the image generation tool is not as smart as the LLM itself. If you ask the AI to “make sure there are no elephants in the picture” it may send a prompt telling the image generator “and do not show elephants” but the image generator just sees the word “elephants” and gives you some.
  
  In general, image generation tools have improved a lot (see the pictures below for a comparison of models from this and last years) but Gemini and GPT-4 do not have access to the best tools, though that may change soon. Instead, you may want to pick from many standalone image models. Consider Adobe Firefly, which is not as strong as other models but has the most artist-friendly approach to training, using only images they have licensed. Or you may want to look at Midjourney, which is probably the best image creator, but whose training policies are unclear.

• Runs Code and Does Data Analysis. Technically, these are very similar, because both of these capabilities are enabled by the fact that when AI can write and run code, it can do a lot. Some of the most interesting advances in the last couple of weeks have been advanced data analysis interfaces added to both GPT-4o and Gemini. As you can see below, GPT-4 continues to be the leader in this area, with more advanced features (including interactive graphs and tools to visualize datasets) and a better “instinct” for analysis.

• Sees images: The ability of AIs to “see” images remains an underused capability, since it adds a tremendous amount of value to the system. I think reading through the Microsoft report on GPT-4’s vision is a helpful way to understand some of the implications.

• Sees video: As the immediate memory, or context window, of AIs grow, they can start to work directly with videos, keeping an entire video in memory at once. Gemini has a startlingly large context window, and I can give it a 30 minute video of a museum walkthrough and ask it to tell me what happens when, and which moments might appeal to kids. A complex task for humans takes less than a minute for AI and suggests why working with video is something that will have large implications.

• Reads Files/Works with Documents. Being able to upload documents makes the AI much useful for summarizing, analyzing, and improving your work. Some of the AIs now connect directly to your documents, like Microsoft Copilot and Office and Gemini and Google Docs, but these interactions are still a bit buggy. The best AI for working with PDFs and text files is probably Claude 3 Opus, which combines both a large context window and a very clever AI that seems to do very well with written work. I gave a legal document to Claude today, and it found an issue that neither side’s lawyers had identified, but which everyone agreed was important after Claude pointed it out. I wouldn’t use it to replace a lawyer/doctor/editor, but a cheap second opinion that is mostly right is very valuable.

• GPTS: I have written a lot about why GPTs are valuable, they are ways of sharing useful automated tools with others. Currently only ChatGPT supports them.

Overall, you can’t go terribly wrong with any of the major frontier LLMs, but you will find each has its own strengths and weaknesses, and the situation is evolving quickly.

Already obsolete

In some ways, this list is already outdated. We know new features are coming soon in GPT-4o, including native ability to work with voice at a very high level, and multimodal image creation that will be more accurate than previous image generators. Though video tools like Runway exist now, we know that Google and OpenAI both have systems that can generate high quality video from prompts alone. Smaller AI models that can run on your phone are available and will soon connect to larger networks of AIs to solve hard problems. And the nature of our interaction with AIs themselves might be changing, as agents and AI devices start to spread.

Even more importantly, in conversations with folks at AI labs, and in the public statements of people with some knowledge of the future, I see a new recent confidence that the next round of AI models will be much smarter than the ones we are using today, opening up new use cases, opportunities, and risks. If they are right, then this post is already an artefact of the past, a list of already obsolete AIs.

Two notes: First, I don’t take any money from any AI labs or any other AI company I discuss. Second you may have noticed that you have the option to pledge a subscription to this Substack. A lot of you have voluntarily done so - thank you! But for almost two years, I haven’t collected any of those pledges. I am planning on starting to do so in the next week or so. But you should know that I intend to keep this Substack free for everyone, and to publish on my own schedule, so pledging doesn’t currently get you anything special other than my deepest thanks! I will let you know if I start to add subscriber-only features, but if you wanted to change or cancel your pledge, here are the directions.

Share

Subscribe now

Even before ChatGPT, something alarming was happening in academia. Though academics published ever more work, the pace of innovation appeared to be slowing rapidly. In fact, one paper found that research was losing steam in every field, from agriculture to cancer research. More researchers are required to advance the state of the art, and the speed of innovation appears to be dropping by 50% every 13 years. The reasons for this are not entirely clear, and are likely complex, but it suggests a crisis already occurring, one that AI had no role in. In fact, it is possible that AI may help address this issue, but not before creating issues of its own.

I think AI is about to bring on many more crises in scientific research… well, not crises - singularities. I don’t mean The Singularity, the hypothetical moment that humans build a machine smarter than themselves and life changes forever, but rather a narrower version. A narrow singularity is a future point in human affairs where AI has so altered a field or industry that we cannot fully imagine what the world on the other side of that singularity looks like. I think academic research is facing at least four of these narrow singularities. Each has the potential to so alter the nature of academic research that it could either restart the slowing engine of innovation or else create a crisis to derail it further. The early signs are already here, we just need to decide what we will do on the other side.

Singularity #1: How we write and publish

In many academic fields, academic research is grindingly slow. I have had papers that took nearly a decade from when I first started working on them until they were published in a journal. Top quality journals are built for this pace, and so are very ill-prepared for the flood of academic articles that AI is unleashing. That is because many researchers are using AI for writing articles, speeding up a key part of the research process and confusing the signals reviewers look for when evaluating work.

Some of this AI help is incredibly clumsy and unethical, like the flood of papers with obvious LLM-written sections or horrifying AI-created images. But AI writing can actually be extremely helpful when used correctly. After all, many scientists excel at their field of interest, but they may not be excellent writers or communicators. Yet GPT-4 class models are actually quite good at scientific writing, producing introductions that are, in at least one small study, the equal of humans. If AI can help with the writing process, it could allow scientists to focus on what they do best, speeding up the research process by having the AI help with time-consuming tasks. Of course, we have no way of knowing whether researchers are properly checking the writing from the AI, so, with a flood of new articles, peer review becomes ever more important…

…and also more automated. Already, at a major AI conference, up to 17% of the text of all peer reviews was written by AIs. While this might be bad news for science, it is good news for the people who wrote their papers using AI and got an AI peer reviewer, because AI peer reviewers prefer AI-written papers.

And yet, we may not want to dismiss the idea of AI helping with peer review. Recent experiments suggest AI peer reviews tend to be surprisingly good, with 82.4% of scientists finding AI peer reviews more useful than at least some of the human reviews they received from on a paper, and other work suggests AI is reasonably good at spotting errors, though not as good as humans, yet. Regardless of how good AI gets, the scientific publishing system was not made to support AI writers writing to AI reviews for AI opinions for papers later summarized by AI. The system is going to break.

And that is even before AI starts helping people create papers, rather than assisting on writing alone. To demonstrate the potential, I put together a little GPT that will explore any dataset, generating hypotheses and testing them in increasingly sophisticated ways. If you try it (free datasets are here if you want to experiment) you will find that AI can do some impressive data work, but it also can be used as a tool for automating bad research, tirelessly p-hacking a dataset until results are achieved. We are going to need to think about how to address these issues of integrity at many levels.

So, how do we come out the other side of this singularity? We need to reconsider the very nature of scientific publishing and reach some conclusions:

• What does scientific publishing and peer review look like in the future?

• How do we deal with the flood of AI content?

• How do we model positive AI use that increases the pace of research while discouraging bad uses?

Singularity #2: How we research

LLMs are also transforming how research is actually done. This is probably the most discussed potential singularity, so I will not try to cover it exhaustively, but the implications are quite large. And to be clear, so are the risks: we know AI is biased in ways that we don’t fully understand (though it is often less biased than humans doing the same work), it obviously is prone to hallucination and errors. Much more research is needed to get a sense of the reliability of different AIs under different circumstances. With this caveat in mind, the evidence suggests that AI can, indeed, help with research in many ways. AI is quite useful at text analysis, for example. Plus, working with it is more like working with a human research assistant than a programming language, meaning that more researchers can benefit because they don’t need to learn specialized skills to work with AI. This expands the set of research techniques available for many academics.

But LLMs can also do things that human research assistants would struggle with. For example, large context windows, which let the AI hold hundreds of thousands of words in memory at the same time, enable pretty remarkable feats of analysis. I gave Gemini Pro the 20 papers and books that made up my academic work prior to 2022, over 1000+ pages of PDFs. It was able to extract direct quotes and find themes across all of them with only quite minor errors. This process of summarizing an entire career’s worth of writing typically takes many, many hours of work.

But AI creates even weirder possibilities for social science research, in that it can, under some circumstances, simulate human beings with high levels of accuracy. Simulated AI subjects act like human ones, allowing researchers to replicate famous experiments like the Milgram obedience studies or the results of personality tests across 50 countries. And to add an extra dimension of weirdness to these experiments, individual AI agents, assigned personalities and goals, can interact and learn with each other in simulated environments. For example, simulated doctors in a simulated hospital with simulated patients learned how to better diagnose diseases. We may be able to discover a lot from these types of counterfactual simulations, though, of course, AI agents aren’t humans and AI simulations need to be interpreted carefully.

Perhaps the most interesting (and disruptive) way to use LLMs in science is to have AI systems autonomously try to discover new things. Specialized LLMs have shown the ability to generate new mathematical knowledge with minimal human guidance. And some early work suggests that LLMs can generate new hypotheses in the social sciences, develop a plan to test those hypotheses, and then actually conduct the tests using simulations. This would create a new form of fully automated social science conducted entirely by machines. But even this is less ambitious than a set of experiments that gave GPT-4 access to a chemical database and the ability to write software to control lab equipment. This allowed the AI to plan and conduct actual chemistry experiments on its own. The scientists were thrilled by the potential, but also fully aware of the potential dangers.

In the near future, AIs may actually conduct science, changing the nature of research in ways we can’t predict. To guide this rapid change, we need to answer a few questions:

• What AI methods are okay to use? Which ones risk bad science, bias, or dangerous outcomes?

• What should autonomous agents be allowed to research? How can they be monitored and stopped if needed?

Singularity #3: What our research means

There is often a big gap between the research world and the public. Papers that are important in an academic field may seem meaningless to those outside of it, let alone to the wider world. And yet, having been in academia for two decades, I believe that a tremendous amount of academic research that has value in the outside world, value that even many academics don’t recognize. AI can help create that bridge between academia and the real world. For example, here is a little GPT that tells you why an academic paper might matter to you. I gave it one of my papers, and it did an excellent job explaining the implications (and summarizing the key results).

Just as interesting is the promise that AI might help researchers explain work to each other, finding opportunities for multi-disciplinary cooperation and helping handle the flood of research Singularities 1 and 2 have unleashed. We know that AI can conduct massive literature reviews and find connections between unexpected work, as well as locating errors and gaps that can be filled. An AI that can connect researchers to ongoing research and discussions can be a valuable tool to restart the engine of innovation. But we need to reconsider the boundaries between fields, and between academia and the public, in order to find a better world on the other side of this singularity.

Singularity #4: What we research

Today, we still do not understand a lot about the implications of LLMs and why they are so good at simulating human thought without actually thinking themselves. More practically, we also don’t know what tasks LLMs do well or badly at. Even the researchers who create them are not aware of their full set of capabilities, and there is large-scale debate over how much LLMs are doing “original thinking” versus spitting back what they learned when trained. The one thing the early research is showing, however, is that LLMs are going to be a big deal in the real world, outperforming humans at an increasing number of real jobs.

From the moment ChatGPT-3.5 came out, I (and many of my colleagues) were overwhelmed by the implications of LLMs, and rapidly pivoted our research focus to AIs. But there aren’t enough of us, yet. If AI is, indeed, a General Purpose Technology, one of those rare innovations that will impact much of our culture, economy, and society, then we need a crash effort to understand its implications, shape its development, mitigate its risks, and help everyone gain its benefits. Because the impacts are multidisciplinary, we need researchers from many fields to join in. This is an exciting time, but if academics don’t seize the moment to do this, others will. We have a unique opportunity to rise to the challenge of our singularities, and, if we do, the world will be better for it.

Share

Subscribe now

But what it does do is quite interesting.

Democratizing Access

Likely the biggest impact of GPT-4o is not technical, but a business decision: soon everyone, whether they are paying or not, will get access to GPT-4o1. I think this is a big deal. When I talk with groups and ask people to raise their hands if they use ChatGPT, almost every hand goes up. When I ask if they used GPT-4, only 5% of hands remain up, at most. GPT-4 is so, so much better than free ChatGPT-3.5, it is like having a PhD student work with you instead of a high school sophomore. But that $20 a month barrier kept many people from understanding how impressive AI can be, and for gaining any benefit from AI. That is no longer true.

While Microsoft Copilot allows free GPT-4 use in limited ways, the full features of GPT-4 were always locked away behind a paywall. And GPT-4o adds some new tricks to the older model, including an ability to work really well with non-English languages. But I am especially interested in everyone getting access to GPT-4’s GPTs and Code Interpreter. GPTs allow anyone to share and build little agent-like programs (I wrote a guide to building them before), and they have turned out to be surprisingly useful tools for automating complex creative tasks, especially as GPT-4o is remarkably fast.

GPTs can serve many purposes. Take, for example, some GPTs we made. There is Framework Finder, a GPT that suggests and customizes frameworks to solve your problems, or Innovator (used over 10,000 times even before GPT-4o) which walks GPT-4o through an innovation process and gives you a document with creative ideas. You can even use GPTs to create GPTs, like Tutor Blueprint, which will help you create a prompt that will act as a tutor on a subject of your choice.

Along with GPTs, the other useful feature coming to everyone is Code Interpreter, which allows the AI to run the code it writes. There are lots of unexpected uses for this ability. One important one is that it is as an incredibly powerful tool for understanding data, since it lets the AI explore complex datasets in a way that tends to be remarkably low in hallucinations and errors. Especially if you have some training in statistics or analysis, it allows you to get a lot of insight quickly. If you want to try it out, here is Data Analysis Buddy, which, given a dataset, will help you explore it in sophisticated ways. (The best way to see what Code Interpreter does is to try it. Some datasets to try out: all the dialogue from Shakespeare, lists of superheroes and their powers)

Some implications of all this:

• Education: GPT-4 is a powerful tutor and teaching tool. Many educational uses were held back because of equity of access issues - students often had trouble paying for GPT-4. With universal free access, the educational value of AI skyrockets (and that doesn’t count voice and vision, which I will discuss shortly). On the other hand, the Homework Apocalypse will reach its final stages. GPT-4 can do almost all the homework on Earth. And it writes much better than GPT-3.5, with a lot more style and a lot less noticeably “AI” tone. Cheating will become ubiquitous, as will universal high-end tutoring, creating an interesting time for education.

• Work: I have increasingly been speaking to companies that have been experimenting with giving employees widespread access to GPT-4 and letting employees build GPTs to solve their own problems. One such company, Moderna, reported that 25% of active users had built one, and users were averaging 120 conversations a week. The spread of this sort of use has been limited by the need for companies to buy GPT-4 access for their employees, and their willingness to embrace the tool. Now, employees can start building on their own, and sharing with each other, without outside permission. I think we are going to see entire departments of companies get filled with secret cyborgs, building and sharing GPTs that automate work… and not telling their employers. Figuring out how to get employees to share what they are developing (and managing security and risks) will be a challenge for many organizations.

• Global entrepreneurship: GPT-4o will be available around the world. This is exciting because many innovative ideas never see the light of day because innovators have trouble figuring out how to get them to market. AI acts as an excellent co-founder, filling in some of the gaps that every founder has in their skillset. Everyone can now write in perfect English, can do basic coding, can get help with problems, and more. We already know that getting advice from GPT-4 increased the profitability of high performing small business entrepreneurs in Kenya by 15%. Free access to this powerful tool may have profound implications.

The magic I haven’t tried yet

I have played with GPT-4o, but I haven’t yet been given access to its biggest tricks. GPT-4o is natively multimodal, which means it can “see” and “hear” and “speak” in an integrated way with almost no delays. The omni in the model's name means it blends all of these modes together. It can see what you are doing, react to it, respond to interruptions, use realistic voice tones, create images with precise control, and more - all seamlessly. Basically, GPT-4o is a chatbot that can interact naturally with the world around it. All of that seems kind of abstract, so I would strongly urge you to watch a demo video: this one of two AIs interacting, or this one of the AI being sarcastic, or this one of Sal Khan (of Khan Academy) using AI as a tutor with vision. If you watch these, you can see how big a change is coming, and why people building close relationships with AIs seem inevitable. Much more on this in the future, when I can try the multimodal capabilities out myself.

There are tons of other tricks that a fully multimodal model like this can do. It can create 3D images, tell apart different speakers on transcripts, and actually write coherent words in specialized photos and fonts. Again, all I have to go by here are the OpenAI demos, so I will reserve judgement until I can play with the systems, but I suspect there will be lots of surprising use cases made available as a result of these new capabilities.

Ubiquity

With GPT-4o, OpenAI again cements its lead (and least for tonight) over the AI space, but it also is a clear sign of an important shift I have been writing about for a while. All of these features we are starting to see appear — lower prices, higher speeds, multimodal capability, voice, large context windows, agentic behavior — are about making AI more present and more naturally connected to human systems and processes. If an AI that seems to reason like a human being can see and interact and plan like a human being, then it can have influence in the human world. This is where AI labs are leading us: to a near future of AI as coworker, friend, and ubiquitous presence. I don’t think anyone, including OpenAI, has a full sense of all of the implications of this shift, and what it will mean for all of us.

Subscribe now

Share

Given that this is the aim of the companies training AI systems, I think it is worth taking seriously the question of when, if ever, we might achieve AGI - “a machine that beats humans at every possible task.” Most computer scientists seem to think this sort of AGI is possible but divide deeply (surprise!) over approaches and timelines. The average expected date for achieving AGI in a 2023 survey of computer scientists was 2047, but the same survey also gave a 10% chance AGI would be achieved by 2027.

No matter what happens next, today, as anyone who uses AI knows, we do not have an AI that does every task better than a human, or even most tasks. But that doesn’t mean that AI hasn’t achieved superhuman levels of performance in some surprisingly complex jobs, at least if we define superhuman as better than most humans, or even most experts. What makes these areas of superhuman performance interesting is that they are often for very “human” tasks that seem to require empathy and judgement. For example:

• If you debate with an AI, they are 87% more likely to persuade you to their assigned viewpoint than if you debate with an average human

• GPT-4 helps people reappraise a difficult emotional situation better than 85% of humans, beating human advice-givers on the effectiveness, novelty, and empathy of their reappraisal.

• GPT-4 generates startup ideas that outside judges find to be better than those of trained business school students.

• 149 actors playing patients texted live with one of 20 primary care doctors or else Google's new medical LLM. The AI beat the primary care doctors on 28 out of 32 characteristics, and tied on the other four, as rated by human doctors. From the perspective of the "patients," the AI won on 24 of 26 scales of empathy and judgement.

There are other examples, but these serve to illustrate the point. For some tasks, today’s AI already exceeds human performance, which is astonishing, but, as my coauthors and I discussed in our paper on the Jagged Frontier of AI, the abilities of AI are uneven, even within a single job. The AI generates great startup ideas but struggles to code complex products without human help. GPT-4 can give good medical diagnoses but can also mess up simple math if you want it to write a prescription. This is why, for now, AI works best as a co-intelligence, a tool humans use to augment their own performance, especially once they understand the shape of the Jagged Frontier.

But, again, this is AI today. The major AI companies want to push the Frontier further and further until AI is better than every human at every task. This raises a lot of issues, but also some obvious questions: how do we know what AI is actually good at? And how do we know how fast it is improving?

Testing for Superhumanity

One way to do this is just to give AI tests made for humans. That is exactly what OpenAI did upon the release of GPT-4, showcasing the big differences between GPT-3.5 (the free version of ChatGPT that you really shouldn’t be using anymore) and GPT-4 (the paid version of ChatGPT). The vertical axis is not the test score, but the percent of human test takers beaten by the AI. Pretty impressive!

…but also a bit misleading. First off, it is very easy to imagine that the questions for some of these tests were included in the AI training data, effectively allowing it to “memorize” the answers in advance (an issue called “overfitting”). Second, the nature of giving AIs human exams overall is quite fraught. Take the fact that the AI scores in the 90th percentile in the Bar Exam. A new paper examining this score in more detail finds a number of problems with how the AI is compared to humans, and ultimately concludes that, with the right prompting, GPT-4 would be in the 69th percentile overall (not the 90th) and is in the 48th percentile of students who pass the exam. Still a very good grade, and one that passes the Bar, but not quite as good as reported. And tests remain a limited measure, as passing the Bar does not make you a good lawyer.

Another issue with one-off tests is that they don’t help us understand if AI is broadly getting closer to AGI. For that, we need benchmarks over time. The field of AI has a lot of benchmarks, and they are mostly pretty idiosyncratic. Almost all of them focus on either coding skills (AI labs are full of coders, so they think a lot more about AI coding skills than almost any other skill) or on tests of general knowledge. Probably the most common is the MMLU, a sort of hard quiz on a variety of topics (more details in a moment), and a test many AIs have been given1. Humans who are not specialists score 35% on the test, and experts apparently score 90% in their fields. That lets us visualize, thanks to the work of Maxime Labonne, the trends in AI ability over time compared to human level. This graph is important because you can see four key facts:

• There are a lot of LLMs, the ones you probably have heard of like GPT-4, Gemini, and Claude, but also a ton of other models, most of which are “open weights,” which is sort of like open source. Anyone can download and use open weights models freely, and they are being developed in many places, with notable models including Qwen (China), Mixtral (France), and Falcon (Abu Dhabi). The dominant open weights player right now is Meta, with its powerful Llama 3 models.

• You can see the impact of the scaling laws of AI: the larger the AI model (meaning requiring more data and more training time), the better the AI is. AIs are getting larger and better rapidly over time, beating amateur answers and approaching expert levels.

• GPT-4 was an outlier when it came out, far above any other model, but has been joined by the two other GPT-4 class models, Gemini Advanced and Claude 3 Opus. It is possible (but, from insiders I speak to, unlikely) that GPT-4 represents some sort of upper bounds of ability for AI, but we will learn more soon as new models are released.

• Closed source, proprietary models controlled by Google, Anthropic, and OpenAI are the best performers, with open weight models lagging quite a ways behind. (But Meta’s largest version of its open weights Llama 3 model gets as high as 86% on the MMLU, making it GPT-4 class, but it has not been released yet).

But here, again, reality is complicated. We have the same issue of AIs potentially being trained on test questions, either by accident or so they can score highly on these benchmarks. And the MMLU is a super weird test full of very hard, very specific problems - you can try taking them yourself here - making it unclear what it measures. The test itself is uncalibrated, meaning we don’t know if moving from 84% correct to 85% is as challenging as moving from 40% to 41% correct. And the actual top score may be unachievable because there many errors in the test questions.

Every available AI benchmark has its own similar set of problems, and yet, in aggregate, they show us something interesting.

Up and To the Right

To see why, we can turn to the Arena Leaderboard. This is a site that lets you put in a prompt and compare the two answers of two different LLMs (it’s fun, you should try it using the link). It is also a fairly good way to compare models, since it measures “vibes” - how good the models are across over a million conversations, subjectively. The site uses the ELO rating system, originally developed for ranking chess players, to compare the performance of different language models based on user preferences. Below you can see how the models stack up in terms of win rates. Even though the measure is very different than MMLU, the results are very similar. In fact, I took a sample of 10 of the LLMs on the list and found ELO and MMLU were very highly correlated (.89).

So there does seem to be some underlying ability of AI captured in many different measures, and when you combine those measures over time, you see a similar pattern - everything is moving up and to the right, approaching, often exceeding human level performance.

Zoom out, and the pattern is clear. Across a wide range of benchmarks, as flawed as they are, AI ability gains have been rapid, quickly exceeding human-level performance.

As long as the scaling laws of training Large Language Models continues to hold, the way it has for several years, this rapid increase is likely to continue. At some point, development will hit a wall because of increasing expense or a lack of data, but it isn’t clear when that might happen, and recent papers suggest that many of these obstacles might be solvable. The end of rapid increases could be imminent, or it could be years away. Based on conversations with insiders in several AI labs, I suspect that we have some more years of rapid ability increases ahead of us, but we will learn more soon.

Alien vs. Human

The increasing ability of AI to beat humans across a range of benchmarks is a sign of superhuman ability, but also requires some cautious interpretation. AIs are very good at some tasks, and very bad at others. When they can do something well - including very complex tasks like diagnosing disease, persuading a human in a debate, or parsing a legal contract - they are likely to increase rapidly in ability to reach superhuman levels. But related tasks that human lawyers and doctors perform may be completely outside of the abilities of LLMs. The right analogy for AI is not humans, but an alien intelligence with a distinct set of capabilities and limitations. Just because it exceeds human ability at one task doesn’t mean it can do all related work at human level. Although AIs and humans can perform some similar tasks, the underlying “cognitive” processes are fundamentally different.

What this suggests is that the AGI standard of “a machine that can do any task better than a human” may both blind us to areas where AI is already better than a human, and also make humans seem more replaceable than we are. Until LLMs get much better, having a human working as a co-intelligence with AI is going to be necessary in many cases. We might want to think of the development of AGI in tiers:

Tier 1: AGI: “a machine that can do any task better than a human.”

Tier 2: Weak AGI: at this level, a machine beats an average human expert at all the tasks in their job, but only for some jobs. There is no current Weak AGI system in the wild but keep your eyes on some aspects of legal work, some types of coaching, and customer service.

Tier 3: Artificial Focused Intelligence: AIs beat an average human expert at a clearly defined, important, and intellectually challenging task. Once AI reaches this level, you would rather consult an AI to get help with this matter than a random expert, though the best performing humans would still exceed an AI. We are likely already here for aspects of medicine, writing, law, consulting, and a variety of other fields. The problem is that a lack of clear specialized benchmarks and studies means that we don’t have good comparisons with humans to base our assessments of AI on.

Tier 4: Co-Intelligence: Humans working with AI often exceed the best performance of either alone. When used properly, AI is a tool, our first general-purpose way of improving intellectual performance. It can directly help us come up with new strategies and approaches, or just provide a sounding board for our thoughts. I suspect that there are very few cognitively demanding jobs where AI cannot be of some use, even if it just to bounce ideas off of.

Even though tests and benchmarks are flawed, they still show us the rapid improvement in AI abilities. I do not know how long co-intelligence will dominate over AI agents working independently, because in some areas, like diagnosing complex diseases, it appears that adding human judgement actually lowers decision-making ability relative to AI alone. We need expert-established benchmarks across fields (not just coding) to get a better understanding of how these AI abilities are evolving. I would love to see large-scale efforts to measure AI abilities across academic and professional disciplines, because that may be the only way to get a sense of when we are approaching AGI.

Even without formal measurement approaches, as AI continues to surpass human abilities in specific domains, we can expect to see significant disruptions across industries, from healthcare and law to finance and beyond. The rise of Artificial Focused Intelligence and co-intelligence systems will likely lead to increased productivity and efficiency, but it may also require some re-evaluation about the role of humans in decision-making. While the path to true AGI remains uncertain, a more general cognitive revolution is well underway, and its impact will be felt widely.

Subscribe now

Leave a comment