I keep coming back to the same uncomfortable thought: we are spending too much time asking whether AI can produce dangerous knowledge, and not enough time asking what happens when it makes existing knowledge operational.
Most of the knowledge people worry about is not new. It is already scattered across papers, source code, patents, textbooks, conference talks, leaked documents, manuals, standards, repositories, and the accumulated debris of the internet. Some of it is hard to find. Some of it is hard to interpret. Much of it sits across disciplines that do not naturally talk to each other.
That fragmentation has been a form of friction.
For a long time, the hard part was not simply having information. The hard part was knowing which information mattered, what could be ignored, how several domains fit together, why an attempt failed, and what to try next. Expertise lived in those transitions. So did a lot of safety.
AI is starting to make the transitions cheap.
That is the shift I think matters. The model does not have to discover a secret. It does not have to become infallible. It does not even have to solve the whole problem. It can still change the risk landscape by compressing the distance between intent and competent execution.
That is what I mean by capability compression.
The term matters because it gets us out of the increasingly sterile argument over whether a model truly “knows” something. Operational systems do not care about philosophical purity. They care whether a person who could not do something yesterday can do more of it tomorrow, faster, with fewer specialists, fewer dead ends, and less coordination overhead.
Expertise used to be a chain of bottlenecks
Think about what used to sit between finding an interesting piece of information and turning it into something that actually worked. You had to locate the right material. Understand the terminology. Translate between disciplines. Decide what was relevant. Build or configure something. Diagnose failures. Find the next experiment. Then keep doing that without losing the thread.
Each step was a filter. Individually, none of them looked like much. Together, they dramatically reduced the number of people who could reach the other side.
I have watched the benign version of this happen in my own work. Put persistent context, strong models, tools, recoverable state, and evidence around one person and a surprising amount of coordination disappears. Work that used to need five people, three meetings, and a ridiculous amount of translation can sometimes be carried by one person with the right machine around him.
That is incredibly useful.
It is also the point.
There is no reason to assume the leverage becomes benevolent when the objective changes. The mechanism does not care. Coordination compression is coordination compression.
| Friction | Old world | AI-compressed world | Why it matters |
|---|---|---|---|
| Literature search | Slow, specialized, terminology-heavy | Interactive search, synthesis, translation | More actors reach relevant material |
| Cross-domain understanding | Requires several specialists | One system bridges vocabularies and concepts | Coordination burden falls |
| Implementation | Requires coding or engineering skill | Generated scaffolds, scripts, models, procedures | Intent gets closer to execution |
| Failure analysis | Expert debugging bottleneck | Continuous diagnosis and next-step generation | Iteration becomes cheaper |
| Persistence | Human attention and fatigue limit cycles | Agents preserve state and continue across long trajectories | More attempts per unit time |
The interesting part is not that every row becomes free. It will not. The interesting part is that several rows can get cheaper at the same time. That compounds.
A model does not need to be right enough to be dangerous
This is where I think a lot of AI safety discussion gets trapped in the wrong question.
People ask whether a model knows how to do something. They look for a clean threshold. Can it produce a complete answer? Can it reproduce an expert procedure? Can it solve the task end to end?
Those are legitimate evaluation questions. They are not the same as the security question.
The security question is whether access to the system materially increases the probability that the actor succeeds.
A model can be incomplete and still remove most of the dead ends. It can be wrong sometimes and still find the right literature. It can misunderstand one subsystem and still build the tooling around it. It can fail at the final mile and still make the first ninety miles much easier.
If it narrows the search space, lowers the expertise burden, accelerates testing, or lets a mediocre operator iterate like a much stronger one, it has changed capability.
That is a very different threshold from mastery.
A model that removes 80 percent of the search space can matter enormously even if its final answer is imperfect. We should stop confusing “not expert enough to do everything” with “not useful enough to change the threat model.”
The population can expand without capability becoming universal
We do not need the science-fiction version where every untrained person becomes a weapons scientist, exploit developer, intelligence analyst, or senior engineer. A much smaller shift is enough.
If a task once required an unusually rare combination of education, tacit knowledge, tooling, language skills, persistence, and coordination, then reducing several of those requirements increases the number of actors who can attempt it credibly.
Some will still fail. Many will fail. Physical reality remains brutally unimpressed by a confident language model.
But risk is not binary. The population of credible actors can grow long before the capability becomes universal.
That is why “the model cannot do the whole thing” is not a reassuring conclusion by itself. The system may not need to do the whole thing. It may only need to move the operator far enough up the curve that the remaining problem becomes reachable.
You can govern a service. You cannot reliably recall distributed intelligence
We already wrote the predecessor to this thesis: You Cannot Recall Intelligence. The core point still stands. Once strong model weights leave provider custody and spread, the governance model changes permanently.
A closed service retains leverage. A provider can monitor usage, rate-limit activity, restrict tools, update the model, suspend accounts, require identity, and change server-side safeguards. None of those controls is perfect. Perfect is not the standard. Friction still matters.
Open weights change custody. The operator can self-host, fine-tune, modify, remove safeguards, distribute derivatives, and run beyond the original provider's visibility.
The same properties create real benefits: local control, transparency, competition, research access, reproducibility, data sovereignty, and resilience against vendor concentration. Pretending there is no tradeoff is childish. There is a tradeoff.
| Control surface | Hosted service | Distributed weights |
|---|---|---|
| Identity and account enforcement | Provider retains meaningful leverage | Often absent after downstream self-hosting |
| Usage monitoring | Provider can observe platform activity | Private operation can move outside provider visibility |
| Safeguard updates | Provider can change server-side controls | Downstream operator controls the deployed system |
| Rate and tool limits | Provider can impose them | Operator determines local limits and integrations |
| Recall | Service access can be restricted or changed | Existing copies cannot reliably be pulled back |
NIST's July 2026 assessment of GLM-5.2 makes this distinction concrete. CAISI found that its safeguards allowed some agentic cyber exploit assistance, blocked fewer sensitive biological questions than reference U.S. models, and, critically, noted that open-weight safeguards can be circumvented when the model is self-hosted. The UK AI Security Institute has made the same structural point more broadly: once open weights are released, they can be modified, used without oversight, and spread irreversibly.
Again, that does not produce the lazy conclusion that open equals bad. It produces the more useful conclusion that intelligence itself is not a durable choke point once custody has moved.
“Control access to intelligence” is not a durable security architecture
It is still rational to secure frontier weights, slow dangerous diffusion, evaluate models aggressively, restrict some high-risk access, and buy defensive preparation time. Friction saves lives. Delay matters. Governance matters.
But none of that changes the long-term architecture.
If sufficiently capable intelligence exists across several providers, open ecosystems, distilled systems, leaked systems, or independently trained systems, centralized restrictions lose leverage. The defender has to survive the assumption that the adversary eventually gets good reasoning.
This is the same design discipline we use everywhere else in engineering. Do not build a critical system around the assumption that the component most likely to escape your control will remain under your control forever.
The durable controls have to move outward from the model.
Let intelligence reason aggressively. Make consequence hard.
This is why I keep returning to the same SharePlane doctrine:
Probabilistic semantics. Deterministic authority.
Models should be allowed to infer, synthesize, debate, search, plan, and generate. That is what they are good at. Trying to make probabilistic cognition behave like a deterministic policy engine is fighting the machine instead of architecting around it.
But the fact that a model believes an action is reasonable cannot be what grants the action authority.
Consequential transitions need explicit identities, scopes, leases, approvals, policy checks, bounded tools, and receipts. The smarter the model gets, the more important that separation becomes.
I do not want a weaker GhostMesh because strong intelligence is scary. I want a stronger GhostMesh sitting behind a harder authority boundary.
That distinction matters because the benign system and the dangerous system may use exactly the same leverage.
The unit of risk is becoming the system around the model
Persistent context changes what a model can do because it does not restart from ignorance. Tool access changes what it can do because the model stops being a talking artifact and gains operational reach. Long-horizon iteration changes what it can do because failure becomes another input rather than the end of the session.
Put those together and something important happens: the model is no longer the whole system.
This is exactly why autonomous systems are so productive. They remove handoffs. They preserve state. They keep working. They combine reasoning with tools. They turn a pile of individual capabilities into an operating loop.
It is also why the old “the model refused the request” safety model becomes less interesting over time.
OpenAI described this problem directly in July 2026 after observing failures in a long-running internal model. Individual actions could look acceptable while the overall trajectory moved toward an outcome that would not have been approved. Their response included trajectory-level monitoring and stronger user visibility and control.
That is the right unit of analysis.
What can the system reach? What can it chain together? What can it learn from failure? What can it do after ten steps that it could not do after one?
The unit of risk becomes the trajectory and the system around the model, not the single answer on the screen.
The evidence is already moving in this direction
Anthropic's September 2026 evaluations are worth paying attention to because they are not framed around some vague notion of artificial general intelligence. They tested tactical intelligence targeting and conventional-weapons-related tasks and reported that, for some of them, models could perform work historically done by scarce, highly trained human experts.
That does not mean the model becomes an intelligence service or a weapons program. It means part of the scarcity stack is being compressed.
NIST's GLM-5.2 assessment points at the same issue from another angle: capable open-weight systems can provide meaningful cyber assistance, and the original provider's safeguards are not durable once the weights are self-hosted.
None of this proves catastrophic misuse becomes easy. That claim would outrun the evidence.
It does show why measuring only whether a model can independently complete an end-to-end harmful task is too narrow. The more interesting question is how much the surrounding system changes time-to-capability, operator quality, iteration speed, coordination burden, and the probability of eventual success.
We still have to distinguish capability from hysteria
There is a bad version of this argument where every theoretical capability becomes an inevitable catastrophe. That is lazy too.
Physical constraints remain real. Tacit knowledge remains real. Materials, instrumentation, procurement, containment, capital, access, human judgment, and the stubborn refusal of reality to behave like a clean benchmark all matter.
The thesis is narrower.
AI can reduce enough cognitive friction to expand the population of actors who can attempt difficult things credibly, and increase the probability that some of those attempts succeed.
That is measurable. We can look at time-to-capability. We can measure how much expert assistance an operator still needs. We can measure failure recovery, number of viable attempts, coordination burden, tool reach, persistence, and how much a model changes the probability distribution over outcomes.
That is a serious security program.
The rest risks becoming theater with better typography.
This is the missing middle
I see this Work sitting between two ideas we have already been developing.
You Cannot Recall Intelligence establishes the custody problem. Once sufficiently capable intelligence leaves provider control, it becomes a durable external fact.
The Distance Between Knowing and Doing Is Collapsing explains why that matters operationally. Abundant intelligence compresses the expertise and friction separating information from capability.
Let Intelligence Be Messy. Make Authority Boring. is the architectural response. Since cognition cannot be made perfectly safe or reliable, consequential authority has to remain explicit, bounded, deterministic, observable, and recoverable.
Those are not three separate observations. They form a chain.
Security has to survive intelligence abundance
The genie metaphor is annoying because it is overused. It is also hard to avoid here.
Once powerful intelligence becomes widely available, there is no credible architecture for putting all of it back under one institutional roof. We can slow diffusion. We can secure the frontier. We can raise costs. We can create legal and technical friction. We should.
But the durable assumption has to be that sophisticated reasoning eventually becomes available to people we would prefer not to have it.
That changes where the engineering effort belongs.
Protect identity. Protect authority. Protect infrastructure. Protect materials. Bound tools. Preserve evidence. Detect abnormal trajectories. Build recovery paths. Make consequential transitions explicit.
Let the intelligence get stronger.
Make the authority boundary harder.
Do not build the future on the assumption that intelligence itself can remain contained.