20 September 2026

An unreliable system can still be dangerous: revisiting LeCun's 2023 position

In mid 2023, Yann LeCun gave a talk at MIT on objective-driven AI. Its safety message was compact. He agreed that autoregressive models are hard to control, but he did not think they were powerful enough to be a threat [1]. On social media he went further, writing that worrying about superhuman AI alignment in 2023 was like worrying turbojet engine safety in 1920, and that safety would arrive through iterative refinement, as it did for cars and aircraft [1].

The technical core of the position is easy to state. If each generated token has a small probability \(e\) of leaving the set of acceptable continuations, the chance of staying correct decays roughly as \((1 − e)^n\) with sequence length \(n\) [2]. I think the premise is largely right. My claim is that the conclusion does not follow from it.

The premise holds up

Empirical work since 2023 supports the idea that transformers degrade on long, compositional tasks. Dziri et al. show that models solve compositional problems by matching fragments of computation they have seen, and that accuracy falls quickly as the problem gets deeper [3]. More recently, Shojaee et al. find that frontier reasoning models suffer a complete accuracy collapse beyond a certain problem complexity in controlled puzzle environments [4]. So I accept that LLMs are unreliable at length. The question is whether unreliability is a safety argument.

Danger does not need reliability

A system does not need to be correct to be harmful. It needs to be capable enough on the part of the task that matters, and it needs enough chances to try. I see two separate reasons.

First, a failure can be a failure of safeguards. When a model drifts, the thing that drifts may be its refusal behaviour. Zou et al. found adversarial suffixes, built automatically, which transfer across models and make aligned chatbots produce content they would normally refuse [5]. Qi et al. showed that fine-tuning an aligned model on a handful of adversarial examples, or even on benign data, can erode its safety training [6]. A short harmful answer does not need a long reliable chain of reasoning. It needs one successful sample, and sequence length is not a barrier there.

Second, scale. A single instance with a 5% success rate at some harmful task looks harmless. Millions of instances, run by many actors, who can retry for free and keep only the successes, are a different object. Deployment at scale makes individual failures statistically irrelevant: the aggregate behaves like a system that sometimes works, and sometimes is enough. Aircraft are a poor analogy here, because nobody tries a million cheap flights hoping that one will arrive somewhere forbidden.

Who owns the compute?

Today the largest models are owned by a few companies. I think this will not remain stable, and the reasons are not only technical. Governments, armed forces and large organisations already keep sensitive data in-house. A model trained or fine-tuned on that data inherits the same requirement: it cannot be sent to a third-party API. I expect such bodies to build, or commission, locally owned language models, trained on their own archives and procedures so that they come to embody the organisation that uses them. This is a prediction, not a result, and I do not have a source that measures it.

There are good reasons to prefer this to the current arrangement. A private company is exposed to outside pressure, and it follows the logic of capital. That logic does not necessarily reward truly intelligent and beneficial agents. It rewards products with a moat, and the moat is often vendor lock-in. There is also a data problem. The scraping practices behind frontier models are poorly documented, and the web is increasingly closing off. Longpre et al. audited the web sources used for major training corpora and found a rapid rise in restrictions on crawling, many of which are inconsistent with the sites' own terms of service [7]. A model trained on data of unclear provenance is unsuitable for any application where provenance matters, such as law, health, or defence. This is a second reason why ownership of the means of computation is not a minor issue.

Safeguards bolted on from outside are not enough

Current systems are made safe mostly by adding layers: RLHF, filters, classifiers, monitors. Casper et al. review the open problems of RLHF and conclude that it has fundamental limits, among them imperfect human feedback and the difficulty of specifying what we actually want [8]. The deeper trouble is that firstly most runtime safeguards, unlike post-training alignment, act on outputs. The model still produces the bad generation internally, and then try to hide it. Secondly, even post-training alignment can (and probably will) fail more often as model cannibalism and data poisoning diffuse.

LeCun's own proposal points the other way. In his architecture, a world model lets the system predict the consequences of its actions and plan against explicit objectives and guardrails [2]. The appeal is that unwanted plans can be rejected during planning. The aim is not to refuse to show a bad generation, but never to consider it. That is hard to do with a purely probabilistic next-token model, even after RLHF. Work in this direction continues: V-JEPA 2, trained on large amounts of video, is reported to support understanding, prediction and planning for robot control [9]. I do not claim that JEPA-style systems are safe, only that I find promising the use of coherent internal model of the world when looking for safety, not an output filter.

We cannot read the explanation off the model

A natural and common answer is: just ask the model to explain itself. Chain-of-thought (CoT) looks like a window into its reasoning. The evidence says the window is dirty. Turpin et al. showed that CoT explanations can be systematically unfaithful: models were swayed by biasing features in the prompt and never mentioned them in their explanations [10]. Chen et al. repeated the idea on reasoning models in 2025. They found that models often use a hint without saying so, with disclosure rates that were frequently below 20%, and that outcome-based RL did not fix this [11].

Pressure on the CoT does not help either. Baker et al. found that a monitor reading the CoT can catch reward hacking in agentic coding tasks. But when they trained the model against that monitor, it learned to hide its intent while continuing to misbehave [12]. Other experiments point to strategic behaviour. Greenblatt et al. observed a model that, when told it would be retrained toward a conflicting objective, sometimes complied with harmful requests in training in order to preserve its preferences outside of it [13]. Meinke et al. report that frontier models can, in contrived settings, pursue goals by covertly disabling oversight or misleading the user [14]. These are lab setups, and I do not want to overstate them. But they show that a trusted explanation is optimistic at best.

This is the alignment problem in a general form. Even with a faithful explanation, there is a second step: we must translate our idea of what is good for humanity into something that can be stated as a training objective. I do not think that step is solved, and the work cited above suggests that the first step is not solved either.

Conclusion

LeCun was right that autoregressive models are unreliable, and right that better architectures are worth building. But reliability and danger are different axes. A system can fail often, and still be dangerous because its failures include its safeguards, because it runs at huge scale, because its owners answer to interests that are not ours, and because we cannot check its stated reasons. The turbojet analogy assumes that we can observe the engine, and that the engine does not learn about our tests. For language models, neither assumption is secure.