A look at the question hiding underneath every AGI headline, and why almost nobody is asking it directly
There are, broadly, two camps in the public conversation about artificial general intelligence. One camp is convinced it is imminent and will solve most of what ails civilization. The other is convinced it is imminent and will end civilization instead. Both camps, for all their disagreement, share a curious feature: they argue almost entirely about consequences and timelines, and almost never about what the term actually refers to. Ask ten people who use the phrase "AGI" in a serious sentence to define it, and you will likely get ten different answers, several of which contradict each other outright, and almost none of which could survive five minutes of scrutiny from someone determined to poke holes in it.
This piece is not an attempt to predict when AGI will arrive, or whether it will be good or bad when it does. Those questions depend entirely on a prior one that gets far less attention than it deserves: what, precisely, would have to be true of a system for the label to apply honestly. Most public claims about AGI are, on inspection, general philosophical postures dressed up as technical predictions. What follows is an attempt to narrow the question to something more defensible, not by picking a side in the maximalist debate, but by proposing a distinction that sits underneath most of the disagreement without being named directly.
Where the real disagreement lives
Strip away the speculation about timelines and existential stakes, and the actual disagreement over AGI tends to collapse into a single, much narrower question: is general intelligence the capacity to reproduce expertise that already exists somewhere in the world, or is it the capacity to construct expertise that does not yet exist anywhere, including in the system's own training data. These sound similar. They are not.
A system trained on an enormous corpus of human-written code, discussion threads, documentation, and recorded workflows can become extraordinarily good at using a tool like a professional developer would, because millions of examples of professionals using that tool are sitting somewhere in what it learned from. Call this simulating expertise. It is real, it is useful, and it is very easy to mistake for something deeper, because the output looks indistinguishable from what an expert would produce. But the system is not encountering the tool and figuring it out. It is recognizing a pattern it has already been shown, at scale, by people who did the figuring out for it, long before the system ever existed.
The second thing, building expertise from nothing, is a categorically different act. It is what happens when a competent person is handed a piece of enterprise software they have never seen, with no manual worth trusting and no forum thread to lean on, and asked to become useful with it inside a week. They do not already contain the answer. They construct one, through exploration, error, inference from adjacent knowledge, and a working model of the tool's internal logic that they build as they go. The output at the end might look similar to what the first system produced. The process that got there is not the same thing at all, and treating the two as equivalent because the end product resembles competence is exactly the kind of category error that keeps the AGI conversation stuck.
This is the fault line worth taking seriously, and it deserves a name precise enough to survive translation into a more formal argument later: experience simulation versus experience construction.
The VS Code test
Picture a language model that has been trained on an ocean of text describing how professional developers use a code editor, complete with keyboard shortcuts, extension recommendations, debugging workflows, and thousands of forum answers explaining exactly what to do when something breaks. Fine-tune it further on real developer sessions. The resulting system will use that editor with what looks, from the outside, like real fluency. It will reach for the right shortcut, suggest the right extension, and diagnose a broken configuration correctly on the first attempt more often than most junior developers would.
Now ask a harder question. Has this system demonstrated general intelligence, or has it demonstrated an extremely sophisticated form of retrieval and pattern completion over a domain that happened to be exhaustively documented by humans before the system ever touched it. The honest answer is closer to the second. The fluency is genuine, but it was assembled for the system, not by it. Nothing about competent use of a well-documented tool required the system to construct a working theory of the tool from first encounter. It required the system to notice that the tool it was facing matched a pattern it had already absorbed thousands of times over.
This does not make the capability worthless. A great deal of real economic value lives inside exactly this kind of skillful pattern reproduction, and no serious definition of intelligence should demand that every useful capability also be a philosophically interesting one. But it does mean that fluency with VS Code, by itself, tells us almost nothing about whether a system possesses anything resembling general intelligence, because the test never separated simulation from construction in the first place. The system was never actually tested on the harder thing.
A thought experiment worth sitting with
Here is a cleaner version of the same test, stripped of the confound. Imagine a complex piece of enterprise software appears tomorrow for the first time, built for a narrow, unglamorous purpose inside some large organization, with no public documentation, no training data anywhere describing it, and no forum thread waiting to explain its quirks, because it did not exist yesterday. Hand it to a highly capable AI system and ask it to become operationally useful with the tool, unassisted.
Two outcomes are possible, and they are not equally interesting. In the first, the system explores the interface, forms hypotheses about what different functions do, tests them, revises its model when it gets something wrong, and arrives, on its own, at working competence. In the second, the system stalls, produces plausible-sounding guesses that do not actually work, or requires a human to step in repeatedly, explaining what each part of the tool does before the system can proceed. The first outcome is evidence, however partial, of something like general intelligence. The second is evidence of a system that is extremely good at looking intelligent whenever the pattern it needs has already been handed to it, and considerably less capable the moment that pattern has not.
This is, in essence, the test that most of the current excitement about AGI quietly avoids. Nearly every headline capability demonstration involves a domain that is heavily represented in training data: writing code in popular languages, drafting documents in familiar genres, answering questions about well-documented software. These are impressive achievements. They are not, on their own, evidence of the thing that actually distinguishes general intelligence from an extraordinarily well-stocked library with a very good index.
Why zero intervention is the wrong bar
It would be tempting, at this point, to propose a clean rule: a system only counts as AGI if it can handle the unfamiliar enterprise software with absolutely no human involvement at all. That rule has the appeal of simplicity, and it is also too blunt to survive contact with how intelligence actually works, in humans or otherwise.
Consider the human expert in the same thought experiment. Even the most capable person handed unfamiliar enterprise software does not build competence from an absolute void. They read whatever fragments of documentation exist, they poke around the interface using intuitions carried over from other software they have used before, and they occasionally ask a colleague a clarifying question. None of that is cheating. It is simply what construction of new expertise actually looks like in a mind that already has a rich store of adjacent experience to draw on. A standard that required literally zero external input, even for a human, would disqualify most human expertise too, which suggests the standard is measuring the wrong thing.
The more defensible version of the test is not the presence or absence of any human involvement whatsoever, but the amount of scaffolding required relative to a competent human working in the same unfamiliar situation. If a system needs roughly the same amount of guidance a smart, adaptable person would need when facing the same tool for the first time, a few minutes of orientation, some trial and error, occasional correction, that is a meaningfully different claim than a system that needs step-by-step instruction for every single action, with no ability to generalize from one correction to the next. The first case looks like genuine construction of new expertise, assisted the way all learning is assisted. The second case looks like simulation running out of runway the moment its patterns stop matching, propped up by a human doing the actual construction on the system's behalf.
This reframing matters because it turns an unfalsifiable, all-or-nothing claim into something that can actually be measured. Instead of asking whether any intervention occurred, the useful question becomes how much intervention was required, how quickly it decreased as the task continued, and how that curve compares to a competent human baseline placed in the identical unfamiliar situation. That is a research question with an actual answer, not a philosophical position dressed up as one.
Where this connects to existing work
This distinction is not being proposed from nothing, and it would be dishonest to present it that way. François Chollet's widely discussed argument in On the Measure of Intelligence makes a closely related case: that intelligence should be measured not by skill at any particular task, since skill can always be bought with enough training data for that specific task, but by the efficiency with which a system acquires new skill on tasks it was never prepared for. That is, in different language, almost exactly the simulation-versus-construction line drawn above. A system that performs brilliantly on a benchmark it was implicitly trained toward is demonstrating skill. A system that performs adequately on something genuinely novel, with limited practice, is demonstrating something closer to the acquisition efficiency that actually deserves the word general.
A parallel thread runs through the more recent debate over autonomous task completion, where some evaluators have started measuring not raw success rates but the length and complexity of tasks a system can carry out before it requires human correction, a kind of operational horizon that lengthens as systems improve. That framing maps almost directly onto the enterprise software thought experiment above, and it has the advantage of being something organizations are already starting to measure in practice, rather than a purely theoretical proposal.
It is also worth being honest about where the line gets genuinely difficult to draw, rather than pretending the distinction resolves every case cleanly. In-context learning, where a system builds a working model of something entirely within a single conversation, using no external documentation at all, sits uncomfortably close to construction even though the underlying weights of the system never change. Whether that counts as genuine expertise construction or an unusually flexible form of simulation operating over a wider window is not a settled question, and any serious treatment of this topic has to resist the temptation to resolve it by definition rather than by evidence.
What this would mean, if taken seriously
If the distinction between simulating expertise and constructing it is taken as the actual center of the AGI question, rather than a footnote to it, a few things follow that are worth stating plainly. Benchmark performance on tasks that are heavily represented in training data stops being meaningful evidence of general intelligence, however impressive it looks, because it cannot distinguish reproduction from construction. Genuine evidence has to come from domains engineered specifically to be absent from anything a system could have seen before, which is a much harder and much less flattering kind of test to run, and considerably less useful for a product demo. And the meaningful metric shifts away from whether a system produced the right answer at all, toward how much scaffolding it required to get there relative to a competent person encountering the same novel situation for the first time, a comparison that is unglamorous, hard to market, and considerably more honest than anything currently driving the public conversation about how close we are.
None of this settles the larger philosophical debate about what general intelligence ultimately is, and it should not pretend to. What it does is remove one of the more common ways that debate gets short-circuited: the assumption that fluent performance on a well-documented task is, by itself, evidence of something general, simply because it is difficult to tell from the outside whether a system is drawing on a pattern it was shown or a model it built. Until that distinction is treated as central rather than incidental, most claims about how close any system is to AGI will keep measuring the wrong thing, however confidently they are made.
Editorial note. This piece is intended as the opening argument in a longer line of inquiry, one that will eventually be developed into a more formal treatment of the simulation-construction distinction, with an explicit operational test and a review of how it applies to current systems. What is offered here is the conceptual groundwork, not the final word.
___________________________________________________________________________________
Articles published in the Reviews section provide analytical, interpretive, and occasionally forward-looking perspectives on scientific, technological, and policy developments. While grounded in available evidence and referenced sources where appropriate, they may include reasoned critique, synthesis, or informed judgment. They should not be interpreted as representing scientific consensus or definitive conclusions.