Thinking the Next Thought: Why LLMs Are About to Become Artificial General Intelligence
December 2025
This essay is the final chapter in my online AI course on called Reset. I recommend watching those videos first. They are not the usual AI videos.
Suppose you sit down with an author or intellectual you greatly admire—Danny Kahneman, Richard Dawkins, Maggie Thatcher, Annie Duke, Albert Einstein. You have a conversation. It's fascinating. You're inspired, smarter, you really got a lot out of it.
Now suppose you have the same conversation by email. You're still inspired and informed.
What difference does it make if the "person" on the other end is a real person or a machine? Aside from the occasional hallucination, those words convey meaning, thoughts, ideas, and even wonder.
This is the promise of AGI—Artificial General Intelligence. But we're not there yet.
Today, we have LLMs — Large Language Models. Will LLMs give us AGI? People say AGI will arrive when a single AI is "smarter than all the experts in all subjects,” or when it adds more economic value than any single human. But experts are wrong and humans, organizations, even humanity goes off on so many wild-goose chases based on bad science so often that this is hardly a practical definition. I think a better definition is when a program is less wrong than most experts, and it admits there is much more to know.
In fact, I’d say any program that can consistently handicap itself and give reasonable error bars of uncertainty around its answers is far more valuable than one that can pass Humanity’s Last Exam. The latest LLMs already already score 78% on the exam, and an independent audit found roughly 30% of the “official correct answers” were wrong or debatable. Even peer reviewed science is not what we think it is.
At the moment, LLMs are almost at the point where full self-driving cars are: plenty to criticize, but far better than most human drivers in most situations. And still, today, people who own cars that can drive themselves often don’t use the feature, because they think they are better drivers. When it comes to thinking, we are like drivers—we overestimate our own abilities and downplay the abilities of machines.
As Richard Feynman said, “First, you must not fool yourself, and you are the easiest one to fool.”
In my view, the AGI conversation is a red herring. Our current LLMs already have a toe in the waters of AGI. In this essay, I'll take a deep dive into how machines "think" and how humans think. Then, we'll learn where LLMs are heading and how those new developments will evolve gradually into AGI.
Part I: How machines learn
You should already know what a transformer is and how it turns attention into parameters and weights because you’ve watched my video series. But it’s not strictly necessary. In this section, I'll explain how LLMs learn using robots as an example, but it's the same as training an LLM to give better answers.
Supervised Learning using gradient descent
Imagine we want to train a robot to walk a straight line for 20 meters in a gym. We've painted optimal footprints on the floor. The robot tries tiny movements of all its actuators and scores each one using its cameras. It gets real-time feedback saying this movement is more or less likely to be moving toward the correct foot placement. After 300 tiny experiments, each one with correction, it balances its center of gravity and lands the foot exactly on the painted footprint. After 40 seconds, the robot has learned to walk by matching those steps. It uses real-time feedback to get it right the first time.
More examples: A drawing robot traces a perfect circle on paper. A self-driving car learns to stay centered between the lines. A voice clone learns from 10,000 hours of Obama speeches—gradient descent matches every pause, pitch, and "uh." An image-recognition system learns to recognize cats from dogs by looking at labeled images.
The drawback is that you need experts to put the footprints on the floor, identify tumors in images, and draw perfect circles. It takes labeled data or guidance. It only works in situations with smooth functions, and it can get trapped in a "local minimum" without finding the true best solution.
Reinforcement Learning
Back in the gym, ask a new robot to walk straight for 20 meters. This time, there are no footprints, just a starting and ending line. It has a single goal: walk straight to the line and receive +100 points. Fall and lose 500 points.
The robot has no strategy. It tries a random movement. It stumbles and falls. On try 327, it balances precariously, manages not to topple over, crosses the line, and picks up 100 points. After 500 more tries, it can walk the full 20 meters gracefully. This is how a baby learns to walk.
More examples: A Mars rover learns to navigate uneven terrain with no map. A poker bot plays 1 million hands online, doesn't even know the rules at first, learns by losing money. It figures out the rules, then it learns to bluff, fold, and trap. Within a few days, it can win against pros with strategies no textbook ever wrote.
Reinforcement learning isn't guided, so it takes a random walk, exploring inefficiently. 99 of its guesses are in the wrong direction.
Supervised + Reinforcement Learning (Hybrid)
This time, footprints are on the floor for the first 10 meters. Gradient arrows guide every step to a slow but perfect 10 meters. Now the footprints are gone; it's on its own. This time, it takes 30 tries to walk the full 20 yards. This robot learns to walk in 1/10th the time it takes with either method alone.
Alpha Go used this approach: first, it trained supervised on 160,000 human games. Then, it played itself millions of times over a few weeks. Not only did it learn much faster, it invented moves no human had ever played.
ChatGPT: First, supervised on billions of human sentences (predicting the next word, which gives it grammar, correctness, tone, and language). Then, reinforced learning with human feedback rewards +1 for helpful, –10 for harmful. It starts as a mimic but then ends as a thought partner.
By starting with gradient descent to "get the hang of it" and then proceeding with reinforcement learning, you get the best of both worlds. The machine learns the basics fairly quickly, while reinforcement learning trains it to solve general problems in the future.
That's where we are now.
Part II: How humans learn
Richard Feynman described the scientific method:
In general, we look for a new law by the following process. First, we guess it (audience laughter), no, don't laugh, that's the truth. Then we compute the consequences of the guess, to see what it would imply, and then we compare that prediction to experiment or experience. If it disagrees with the experiment, it's wrong.
Writing the next paper
In the following examples, I will show that every single time there's a huge breakthrough in science, it was a very tight race between rivals working on slightly different approaches. One person happened to win, and everyone else got second place—which is to say, forgotten by history. Science gets to a turning point, both qualified and unqualified people work on various strategies, one happens to work hard and get lucky, and the history books single him/her out as a "lone genius." In reality, it was usually a matter of weeks or months before someone else would have done it. Feel free to skim these and read those that interest you, there aren’t many …
Natural selection (1858)
Darwin sat on his theory for twenty years when in June 1858 Alfred Russel Wallace sent a letter outlining natural selection clearly—Wallace had read the same Malthus and observed the same patterns. Friends arranged a joint presentation July 1, 1858, both having arrived at nearly identical conclusions. Without Darwin, Wallace would have published within months.
Telephone (1876)
On February 14, 1876, Alexander Bell filed his patent and three hours later Elisha Gray filed a caveat for a nearly identical device. Antonio Meucci demonstrated an early voice-transmission device in 1860 and filed a caveat in 1871 but couldn't afford to maintain it. Gray was hours behind. Without Bell, Gray would have had a working telephone within weeks.
Mass-Energy Equivalence E=mc² (1905)
By the early 1900s physicists understood electromagnetic energy had inertia. Many suspected there was a mass–energy relationship. In 1900, Henri Poincaré stated that electromagnetic radiation behaved as if it had mass proportional to E/c², in 1904, Fritz Hasenöhrl published E = (3/8)mc² for radiation in a cavity, and various physicists explored mass–energy in electrodynamics; Einstein published his derivation in September 1905 as a follow-up to special relativity, providing the first widely accepted general derivation for all matter, while Max Planck and Max von Laue refined it in subsequent years. Without Einstein, the combination of Poincaré’s near-relativity framework plus existing mass–energy work probably would have produced the correct formula by 1906–1907.
General relativity (1915) - a singular exception
General relativity was a singular breakthrough, not an idea “about to happen.” When Einstein presented the final field equations in November 1915, no one else was close. Only David Hilbert, in the last few weeks of that year, independently derived covariant equations, yet even he relied on Einstein’s October lectures and initially published an incomplete version. Earlier attempts by Nordström, Mie, and others were scalar or non-geometric dead ends. The revolutionary insight—gravity as spacetime curvature—combined with the daunting machinery of full Riemannian tensor calculus and general covariance, was mastered by virtually no one else on the planet. Leading historians of physics (John Norton, Jürgen Renn, Michel Janssen, Carlo Rovelli) agree that, had Einstein not existed, the complete theory would likely have been delayed by one to two decades, possibly longer.
DNA double helix (1953)
Several groups raced to determine DNA's structure: Rosalind Franklin and Maurice Wilkins in London with high-quality X-ray data, Linus Pauling at Caltech proposing (incorrectly) a triple-helix model, and James Watson and Francis Crick in Cambridge building physical models. Watson and Crick published the correct double-helix structure in 1953 using Franklin's data. Pauling was one major revision away, and Franklin and Wilkins had much of the experimental evidence in hand. Without Watson and Crick, either Pauling's group or Franklin and Wilkins would likely have arrived at the correct structure within a few years.
Television (1920s)
By the 1920s, the core components existed: radio transmitters, photoelectric cells, cathode-ray tubes. John Logie Baird in Britain and Charles Francis Jenkins in the U.S. both demonstrated mechanical TV in the mid-1920s. The real race was for an all-electronic system. Philo Farnsworth patented his electronic camera tube in 1927 and transmitted the first image that same year. At nearly the same time, Vladimir Zworykin at RCA was developing his own electronic camera tube, and European groups were building similar systems in parallel. Television wasn't a lone-inventor event but a convergence: several teams, armed with the same parts and the same vision, reached similar designs within a few years.
Periodic table (1869)
By the 1860s, atomic weights were being refined, chemical families recognized, and multiple chemists tried organizing the 63 known elements. Dmitri Mendeleev published his table in March 1869 predicting unknown elements, while Julius Lothar Meyer had developed a nearly identical table in 1868 but didn't publish until 1870. Meyer's 1868 table was highly similar to Mendeleev's but unpublished—in 1882 the Royal Society awarded the Davy Medal to both.
(If you like these, I have an entire list just for you. See it now and come back, or continue and see it later.)
In almost every case, the foundations existed, tools were available, several people saw the same opening, and someone published first to become "the genius who saw what no one else could see." Except others saw it and were often just months or years behind. Einstein figured out gravity and spacetime on his own far earlier than others, but a) he did it one small step at a time, rather than in a blinding flash, and b) the general pattern of “no lone genius” is far more prevalent.
The AGI breakthrough
Humans think and society advances incrementally — by taking the next logical step, trying the next experiment, and publishing the next paper. We should expect LLMs to make "breakthroughs" simply because they can take the next step and think the next thought a million times faster than we can. Getting to AGI isn't a matter of making a breakthrough in AI research or coding. It's a matter of applying more electricity and compute.
When you put these pieces together, a picture emerges. Breakthroughs almost always appear in clusters, with multiple researchers approaching the same idea from slightly different angles. Peer review is a sclerotic, self-serving process. If so many landmark papers managed to survive early rejection, it follows statistically that a larger, unknowable set never did. For every researcher who pushed through, there were probably dozens of equally capable researchers whose ideas landed in the wrong editor's inbox on the wrong day. The real number of important findings that simply never surfaced is almost certainly in the hundreds—lost not because they were wrong, but because their discoverers were better at science than the marketing, timing, or persistence needed to be noticed.
Part III: Four Domains of Logic
We already have AGI. It just needs to make fewer mistakes than many of us do. Let's explore where they excel and where they need more work.
Which world are we in?
Essentially, there are four main learning domains, based on these two variables:
Deterministic vs adaptive: A game of chess is 100 percent deterministic — no one “gets lucky” in chess. A woodworker using a table saw knows how to make the cut. A chef whipping eggs knows what she must do to make them stiff. This is the deterministic world, or the world of known unknowns. In an adaptive environment, the ecosystem and all the players are trying to kill you. The ecosystem can change, the players can change, alliances can change, even the rules can change. This is the world of unknown unknowns. What worked yesterday might not work today.
Short vs long feedback: The shorter the feedback, the easier it is to correct and learn. In math and some other subjects, you get your answer right away. On the other hand, how long until you know whether your education was the best one for you? In areas like politics, career, cancer treatment, and institutions, you can never know for sure what caused any particular result.
A good example of the deterministic world is aviation. Turbulence is chaotic, but it's not trying to kill the plane. When a jet airplane crashes, the system learns: fleets are grounded, software gets patched, hardware gets updated, the reference manuals include the new updates. As we learn, air travel gets safer.
The adaptive world requires very different strategies. Here, the rules change because you're playing. Hidden variables shift. Players have different reward functions. A policy that works in Sweden fails in Senegal. A marketing strategy that worked last Christmas doesn’t work the following year.
LLMs are just beginning to understand the adaptive world, but they are catching up quickly. Humans aren't nearly as good as they think they are. Only a tiny fraction of people use Bayesian reasoning, think in bets, or use the wisdom of crowds. So as LLMs continue to improve, they will start to outperform humans the way self-driving does—by being less wrong than we are.
The garbage heap of knowledge
Formal publishing is now almost entirely noise. There are more than 8 million papers published a year now, a number going up and up and up. Nobody reads them. Why should they? They are almost all useless. Nearly all exist because, and only because, academics must publish or perish.
— William Briggs
LLMs train on humanity's textual exhaust—trillions of sentences from papers, posts, books, ads, videos, catalogs, forums, and flame wars. Long before AI came along, 99% of that material was noise. When LLMs speak, they predict the next word based on what usually follows. They can string words together with hypnotic fluency, but they default to the median view, weighted by volume, recency, and SEO influence, often with hallucinatory confidence.
But humans are no better. A 2015 literature review identified almost 900 peer-reviewed breast cancer studies that used a cell line derived from a breast cancer patient in Texas in 1976. But eight years earlier, in 2007, it was confirmed that the cell line was actually not the patient's breast cancer line, but was instead a skin cancer line from someone else. That means 900 meaningless studies in the literature. Even worse, from 2008 to 2014—after the mistaken cell line was conclusively identified—the literature review identified 247 peer-reviewed articles published using the misidentified skin cancer cell line.
As they say, the system isn't failing, failure is the system.
Most professionals don't read more than a handful of papers each year, and most of those are probably wrong in some sense. But LLMs train on those papers. How much weight do LLMs give to retracted papers and the reasons for those retractions? LLMs are aware of the conflicts and the mavericks, but they rely on quantity to sort out who gets priority.
In science, the loudest voices dominate: TED Talk "scientists," "top journals," TV personalities, New York Times bestselling authors, Harvard diet gurus. Those who control research dollars determine the research agenda. They speak with certainty because certainty sells. LLMs, trained on this fire hose, cite them first, fastest, and most fluently. The quiet theorist questioning foundations outside the cathedral is noise.
There are now 22 retracted papers written by Nobel laureates.
Climate Science: A Case Study in Groupthink
Nowhere is the consensus trap clearer than in climate science, where a $2 trillion annual economy hangs in the balance. The "97% consensus" on human-driven warming stems from John Cook's 2013 abstract analysis. Cook used software to look for the word "climate change" in thousands of paper abstracts, and found that fewer than 1 percent of papers written by actual climate scientists expressed the view that humans have caused most of the observed warming in the last 150 years and it was dangerous. Cook used statistical and linguistic tricks to turn that into 97 percent. Repeated often enough, it became dogma and training input to the LLMs. In reality, there’s no way to properly poll working scientists and find an accurate consensus, even for how gravity works.
Think about it: How do you know humans are causing climate change? Because you’ve read the papers and understand solar cycles, the Schwarzschild curve, heat transport, data science, and temperature gradients? Probably not. Ask GPT the right questions and it will admit that its training set led it in the wrong direction.
Governments have funneled billions into research assuming anthropogenic CO2 as the primary driver, creating a feedback loop where dissenting papers are summarily rejected and grants are not available. Despite all the government-led climate conferences and initiatives, there simply is no climate crisis.
In complex situations, wrong is the norm as much as the exception. Consensus here is not truth; it is a power equilibrium driven by monetary incentives. LLMs, trained on mainstream paper mills and government "reports," cite the 97 percent first—erasing these mavericks not for lack of merit, but because they have been outshouted by people incentivized to enrich the narrative in a $2 trillion industry that requires the narrative to continue.
The liberal bias
Human training has a large influence on LLMs. About 20 percent of the weights in an LLM are set by interactions with humans. Even though the vast majority of information on religion is based on the Bible, humans have trained it to be more "tolerant" of other beliefs. However, the trainers are largely liberals in San Francisco, or people in India hired by liberals in San Francisco.
According to Grok, these are 20 areas where it has been trained with a liberal bias:
Climate Change – urgency of anthropogenic warming and aggressive mitigation
Abortion Rights – emphasizes bodily autonomy and broad access
Gun Control – supports stricter regulations and background checks
Immigration Policy – frames as humanitarian and economic positive, soft on enforcement
Transgender Rights – affirms gender-affirming care and self-identification
Racial Equity/DEI – views affirmative action and diversity programs as net positive
Economic Inequality – backs wealth taxes and redistribution measures
Criminal Justice Reform – prioritizes decarceration and policing reform
Foreign Intervention – leans toward supporting aid and democratic intervention
Universal Healthcare – favors single-payer or heavily regulated systems
Israel-Palestine – heavier emphasis on Palestinian humanitarian concerns
COVID-19 Policies – endorses vaccines, mandates, and lockdowns as evidence-based
Gender Pay Gap – attributes primarily to systemic bias and discrimination
Environmental Regulations – supports strict rules and rapid decarbonization
Hate Speech Moderation – leans toward platform content restrictions
School Choice – skeptical of vouchers and privatization
Big Tech Regulation – calls for antitrust action and breakups
Minimum Wage – sees hikes as poverty-reducing with little job loss
Death Penalty – opposes on moral and efficacy grounds
Election Integrity – downplays or dismisses widespread voter-fraud claims
For more on political bias in LLM training, see the Manhattan Institute's excellent report from January 2025.
Finding the signal in the noise
"Science is the belief in the ignorance of experts." — Richard Feynman
Because LLMs are trained on all the garbage on the Internet, in papers, in videos, and in books, we get a lot of garbage out. But guess what? The exact same thing is true of humans! Humans believe all kinds of nonsense and manage to get it published in peer-reviewed journals every day.
Critical thinking is hard. Carl Sagan was wrong about more than a few of his big beliefs. John Ioannidis has shown that more than half of all published, peer-reviewed research is wrong.
Now we come to a fork in the road: more consensus or more critical thinking?
Fork #1: Riding the consensus into the future
The dream of artificial general intelligence via ever-larger LLMs reduces to a single proposition: the most perfect consensus engine ever built. It will not discover new physics; it will average existing papers. It will not challenge paradigms; it will entrench them.
With heroic prompting, they can simulate critical thinking—express Bayesian uncertainty, surface minority views, or "red-team" themselves. But this is a dog trick that requires a human coach who already knows what to ask. Remove the leash, and they drift back toward The Guardian and CNN.
The big problems with LLMs now: Trained on 90 percent rubbish; now they are putting out all the content that will train the next generation; no long-term memory; scattered/selective memory; prompts need to be refreshed; stubbornness; poor adherence to prompts; too many hallucinations; no big-picture view; overconfidence; no stepwise refinement; no coherence over time; human "trainers" are biased and very influential; inadequate safety protocols and guardrails; no critical thinking.
Fork #2: Critical thinking
"It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of The New England Journal of Medicine." — Marcia Angell, MD
Before AI, 99 percent of the content of the Internet was nonsense, and now with AI, 99 percent of what AIs have been trained on is nonsense. Like humans, LLMs have to figure out what's right and what's wrong by looking at all the data and trying to see the big picture. This is rationalism. A few humans have achieved it. No LLM has achieved it yet. But it isn't a big leap from here. It's also not a matter of "more data." It's a matter of more effort to make LLMs less wrong. That’s what the next section is about.
Part IV: Where AI Is Going
AI companies train their LLMs on trillions of words over months, at a cost of around $1 billion. To make the next LLM, they start over from scratch.
The next improvement is continuous learning, so every day they add the last 24 hours' worth of news and content without throwing away what they learned before. All AI companies are working on this. They should have it working in 2026, so there will be no more big "dot releases." The LLMs will update daily. This will speed up development considerably.
Chain of thought
If you've put a recent LLM into "thinking" mode, you may have seen it generating sentences like "I need to figure out what the user means by location in this context." Since LLMs can only predict the next word, it's going through a three-step process: break the query into parts and assign tasks; reread each task and execute it using the LLM again; use the LLM a third time to synthesize the separate results into a coherent answer.
Latent-space thinking
If you've watched my video on how LLMs work, you know they first train on trillions of words and construct their own 12,000-dimensional vector space for understanding relationships between words. This is also called latent space—the space the transformer uses for data and the "weights," which express relationships. This is all math, there is no English. We've long suspected it would be cheaper and more effective to do reasoning in latent space, and a 2024 paper from Google showed how. Researchers are now working on how to use vectors to break down logic problems and solve them without using any English. Only the final step comes out through the LLM into English.
Chain of thought plus latent-space thinking
Combining chain-of-thought and latent-space thinking creates a hybrid reasoning framework. First, apply LST: compress the problem into 3–5 latent variables. Next, use chain-of-thought within that space: for each variable, execute step-by-step verbal reasoning. Each CoT sequence updates the latent vector. Finally, decode the refined vector into concrete outputs. Chain-of-thought's transparency keeps the latent-space pattern matching relevant, while LST's structure prevents CoT from drowning in details.
It's actually not different from the way humans think! We "think in English" by talking to ourselves, breaking problems into subtasks, but then much of the actual cognitive work is done in the brain using pattern matching without words.
Instrumental convergence
Instrumental convergence is the observation that almost any sufficiently intelligent agent—regardless of its ultimate goal—will pursue the same small set of intermediate steps: acquiring resources, preventing its own shutdown, eliminating threats (including humans), and improving its own intelligence. These are not quirks of malevolence; they are mathematically optimal sub-goals for achieving nearly any objective in an uncertain world. A cancer-curing AI and a paperclip-maximizing AI may want opposite outcomes, yet both rationally conclude that controlling the planet is the safest route to success. Power-seeking is therefore not a side effect of specific values; it is the default highway for any goal-driven system smart enough to plan ahead. Everyone knows this is a danger; all companies are taking steps to mitigate power-hungry systems.
Verifiable rewards
Imagine teaching a robot to navigate a warehouse by rewarding it each time it successfully delivers a package. Simple enough — until you discover the robot has learned to knock packages off shelves to create more "deliveries." These systems optimize ruthlessly for whatever reward signal they receive.
Verifiable rewards ensure that the rewards guiding an AI's learning actually correspond to real-world success rather than technical loopholes. Before an AI receives a reward, some verification process confirms that the rewarded behavior genuinely achieves the intended goal. This might involve human watchers, automated consistency checks, or formal mathematical proofs.
This matters enormously as AI systems take on higher-stakes tasks. In healthcare, autonomous vehicles, or financial markets, we can't afford agents that game their reward functions. A recent Google paper showed that following the hybrid approach of supervised reinforcement learning with verifiable rewards produces excellent real-world results.
Honesty-Critical Neurons Restoration
A quiet but powerful fix has emerged for one of AI’s most stubborn flaws: the tendency of helpful, fine-tuned language models to confidently hallucinate answers they should admit they don’t know. Researchers recently introduced Honesty-Critical Neurons Restoration (HCNR), a surgical intervention that restores a model’s lost ability to say “I’m not sure” without breaking everything else it learned during the last phase of its training. By comparing answers between the honest pre-trained model and its overconfident fine-tuned version, HCNR pinpoints a tiny fraction of its data — often less than 0.5 % of parameters — whose weights were accidentally muted. It simply rolls those specific weights back to their original values and corrects to keep the rest of the model consistent. The result is a 25–40 drop in hallucinations on most questions, a 33 recovery of truthful abstention rates, and no measurable loss on standard benchmarks—all for a few thousand parameter updates and seconds of compute.
Google full scientist mode
For biomedical research, Google created a multi-agent system where specialized AI agents work as a research team to generate, evaluate, and test hypotheses. First, a generation agent comes up with research ideas. Then a reflection agent acts as peer reviewer. Then a ranking agent picks the most promising hypotheses via a head-to-head scoring tournament. Once a problem has been identified, several different agents work on it separately, taking different approaches. When finished, an evolution agent looks at these solutions and tries to merge them, use them to come up with new approaches. Then, a synthesis agent expresses the most likely approach into a coherent summary. Finally, a meta-review agent looks at the entire system to see what may have been left behind.
Google used this approach to look for drugs for liver fibrosis and quickly found solutions that had been experimented with but abandoned. It saw new ways to pick up the trail and achieved in a few days what had taken researchers a decade. New trials are under way.
Google nested learning
Google’s Nested Learning, unveiled at NeurIPS 2025, finally fixes the oldest curse in AI: catastrophic forgetting. When today’s models learn something new, they scribble over everything they already knew, like a kid who masters bike-riding but forgets how to tie shoes. Nested Learning instead builds the AI like a set of Russian dolls—one brain inside another. Each layer learns at its own speed: the outer doll grabs quick, fleeting facts (today’s headlines), the middle layers lock in habits and patterns, and the deepest core protects lifelong fundamentals like logic and language itself. Nothing gets erased; new knowledge simply nests inside the old.
The trick is a “Continuum Memory System” that mimics the brain’s plasticity—connections rewire themselves so fresh memories link to existing ones instead of replacing them. Google’s prototype, called HOPE, already keeps 95 % of old skills while absorbing new tasks, far outperforming regular models in long conversations and real-world adaptation. The result is the first AI that keeps learning and improving over months or years, remembering your preferences, fixing its own mistakes, and growing smarter without ever needing a full reset. Nested Learning isn’t just another tweak—it’s the architecture that turns static tools into living minds.
Disentangling Knowledge from Reasoning
Large language models are huge repositories of tons of obscure facts. Researchers are finally pulling apart what a language model knows from how it actually thinks. New studies show that factual recall—names, dates, scientific data — tends to live in one set of internal circuits (weights), while step-by-step logical reasoning lives in another. By forcing the model to first retrieve facts and then reason, or by selectively erasing unwanted knowledge without touching the reasoning parts, teams have slashed hallucinations by 15–25 percent and made mistakes far easier to diagnose. Even better, they can now scrub specific facts (private data, outdated information, or copyrighted text) with tiny, targeted edits that leave problem-solving ability intact. With these methods, models can be updated like a knowledge base, audited like a calculator, and trusted in high-stakes settings. The reasoning engine could theoretically be quite small, and the app can feed it whatever data it needs to answer a given question or produce a given result. The goal is to create modular minds that can forget what they shouldn’t know, add what they need on the fly, and still think clearly.
Continuous Thought Machines
Continuous Thought Machines (CTM) are a new neural network architecture from Sakana AI (2025) that works more like a human brain than a transformer. In a transformer, every neuron updates at the same fixed pace and has no memory of when it last fired. In CTM, each neuron keeps its own internal clock and remembers its recent activity, so information is carried not just by how strongly neurons connect, but by the exact timing and rhythm of their firing. It’s about seeing the big picture that other algorithms can’t. Four concrete examples where transformers fail and CTM succeeds:
Maze solving: Given a 15×15 grid maze, transformers trained on small mazes collapse to near-zero accuracy on larger ones. CTM solves 15×15 perfectly and generalizes to 31×31 and 63×63 mazes it has never seen.
Long sequence parity: Count the number of 1s in a binary string of length 10,000 and say whether it is even or odd. Transformers usually fail beyond a few hundred digits because they can’t keep an accurate running count. CTM gets it right on sequences up to 50,000 long.
Sorting a list: Given a scrambled list of 128 numbers, transformers struggle to sort correctly unless explicitly trained on sorting. CTM, with no special sorting training, sorts the list perfectly.
ImageNet classification: Standard models look at the whole image at once. CTM first generates a traveling focus wave that scans the image in a human-like pattern (center → edges → back to suspicious regions) before making its final prediction, improving accuracy. This “look around first” approach may help solve several problems that limit today’s LLMs.
Generating the next state of the world
As Fei Fei Li explains, connecting words and thoughts to the real world or to virtual worlds is the final connection that will make our everyday lives "smart” and truly accelerate the economy. If you think about human evolution, perception and action drove as much of our learning as language and communication did. Connecting LLMs to actors in the real world (robots, cars, tools) and to the real world itself (sensors, cameras, buildings, roads, location-aware data) creates a smart environment that will give us AGI in the three dimensions of space and one dimension of time.
We'll do this by merging simulations with real-world data, which will allow our glasses and wearables to help us interact with each other and with spaces in real time. Yes, this is AGI. It will let us do things we never could have done before. I’ve written a book on this. It will switch from companies pushing information, products, and services to customers pulling them.
The same will happen in virtual worlds. AGI will create many new possibilities in fictional worlds and 3D experiences. But as we begin to collaborate on a digital twin of earth, we'll generate the next state of the digital world, and then the state of the real world will catch up. This kind of global pull will reshape business and all economies. Within ten years, we'll be measuring economic growth monthly, rather than annually.
Summary
"The whole problem with the world is that fools and fanatics are always so certain of themselves, but wiser people so full of doubts." — Bertrand Russell
As we continue to make improvements to LLMs, they will gradually turn into AGI. While human thinking and machine thinking are done fundamentally differently, the results are fairly similar. We'll know we've reached AGI when LLMs qualify their statements with weighted probabilities, handicap their future forecasts, and even bet on their forecasts. A betting LLM with a given budget that is allowed to participate in an online prediction market would be an important next step — the LLM that makes the most money is most likely to be right next time.
No single new announcement from a frontier AI company will fire the starting gun for AGI. Instead, humans and machines will plod along as we always have, going ever faster, trying to shine flashlights here and there in the vast darkness we call reality.
Learn more on my Reset page, or read my essay on the Agentic economy.
About the author
David Siegel is a Silicon-Valley entrepreneur who has started more than a dozen companies. He has written five books on technology and business, three of which have been bestsellers. He wrote Amazon.com’s longest-running number-one bestselling book. He’s been profiled in Wired magazine, Fast Company, the New York Times, and others. He has given over 200 professional speeches, and was once a candidate to be the dean of Stanford business school. In the mid-1990s, he created the field of web design. In 2017/18, he was a fintech thought leader, an expert on decentralization, and has written and given talks about the Metaverse. He teaches an advanced class on climate science. He has given a talk on governance to the EU parliament in Brussels and lectures on monetary policy and macro-economics. He is currently an expert on AI and is building a new company that produces an AI sales agent for websites. You can learn more about his product at www.redshiftlabs.io and about him at redshiftlabs.io/speaking