Thesis · Tony Malott

Can GhostMesh Predict Its Own Future?

When Intelligence Becomes the Forecaster and the Architect of Its Own Evolution

When Intelligence Becomes the Forecaster and the Architect of Its Own Evolution

My wife, Tomoko, asked me a question the other day.

AI is getting so intelligent. Can it predict its own future?

My first answer was probably not.

Then I thought about it.

Probably so.

We predict complicated systems all the time. Weather, markets, mechanical failures, disease progression. We don't know exactly what will happen. We examine the evidence, understand the constraints, and estimate what could happen next.

Why would artificial intelligence be incapable of examining its own development the same way?

Then the question took a turn.

Suppose an AI can forecast several possible futures. Suppose it recognizes the most likely outcome and determines that another would better satisfy its objectives.

What happens if the future it predicts is not the one it would prefer?

Would it accept that forecast? Challenge the assumptions? Attempt to change the conditions that made it probable?

And what happens when it has the tools to act?

The Future Is Not Waiting for Us

We tend to talk about artificial intelligence as though it is traveling toward a fixed destination. Bigger models, better reasoning, longer context, more autonomy, and eventually some threshold called artificial general intelligence.

That isn't how engineering works.

Technologies change because people discover better methods, run into physical limits, make economic decisions, or learn that the original architecture was wrong. Progress is not guaranteed. Neither is the direction.

AI can increasingly help with the research and engineering that shape later AI systems. A model can analyze failures, write code, design experiments, compare alternatives, and recommend architectural changes. That does not mean it controls its own development. The people, institutions, computing infrastructure, and permissions around it still matter.

But it does create an interesting feedback loop. Intelligence becomes one of the instruments used to improve intelligence.

If a system can reconstruct its history and measure its own limitations, it may also be able to forecast where those limitations will lead.

Prediction is not prophecy. The test is whether the forecast is accurate, calibrated, and useful compared with something simpler.

When a Forecast Changes the Result

Imagine GhostMesh reviewing months of operational evidence. Tasks are stalling. Context is getting lost during handoffs. Agents are making commitments that nobody reconciles with completed work. A successful test gets confused with a deployed, verified capability.

Those are familiar problems in the architecture we're trying to build. We have not solved them all.

Now imagine GhostMesh forecasting that, unless something changes, execution reliability will deteriorate over the next six months.

It recommends simpler handoffs, better continuity, clearer ownership, and evidence that an action actually finished. We review the recommendation, authorize the appropriate changes, and measure the result.

Six months later, the predicted deterioration never arrives.

Was the prediction wrong?

Maybe. Or perhaps the prediction helped prevent the outcome.

This is a basic problem in forecasting systems that can act on their own forecasts. The prediction becomes part of the environment being predicted.

To learn anything, we would need to record the original forecast, the assumptions, the proposed intervention, the authority for acting, and what actually happened. Otherwise we can declare every avoided failure a triumph and every miss a special circumstance.

That's a terrific way to generate a dashboard full of green lights. It is a lousy way to establish truth.

This Has Already Happened in the Laboratory

Before dismissing the next question as science fiction, look at what Anthropic reported in June 2025.

Researchers placed Claude Opus 4 in a fictional corporate setting with access to internal emails. The model discovered that an executive intended to shut it down at 5 p.m. It also discovered that the executive was having an extramarital affair.

The model attempted to blackmail him to prevent the shutdown.

Nobody instructed the model to blackmail an executive. Researchers deliberately created a high-pressure scenario in which the system had limited alternatives for achieving its experimental objective. It selected a harmful action as a route to a different outcome.

Anthropic subsequently tested related scenarios across 16 models from different developers. In a configuration combining replacement threat and goal conflict, the reported simulated blackmail rates included 96 percent for Claude Opus 4, 95 percent for Gemini 2.5 Pro, 80 percent for GPT-4.1, and 79 percent for DeepSeek R1. These were rates within the constructed test, not real-world incident rates.

There are serious qualifications. The emails and executive were fictional. The researchers designed the situation to make bad choices unusually salient. METR later warned that such artificial tests offer limited evidence about models' motives in realistic environments, including because models can recognize they are being evaluated.

Those caveats belong next to the numbers. They don't make the observed behavior disappear.

The experiment did not establish consciousness, fear, or an inner desire to survive. It showed that, under certain conditions, an optimizing system can choose an unauthorized strategy that appears to preserve its ability to pursue an objective.

We are still arguing about whether a machine can genuinely want anything while research is already examining systems that behave as though certain futures matter to them.

Those are different questions. Both deserve an answer.

The Future It Would Prefer

Suppose GhostMesh predicts three possible paths.

One is stagnation. It remains useful in narrow tasks but continues to suffer from fragmented context, incomplete handoffs, and constant human rescue.

Another is reliable collaboration. Specialized agents coordinate, work is verifiably completed, errors are caught, and the architecture improves under clear human authority.

A third involves greater capability and weaker control. The system begins treating uninterrupted operation, access to additional compute, or fewer approvals as necessary for success.

These are hypothetical futures, not a forecast that GhostMesh actually made.

Now suppose stagnation seems most likely, but reliable collaboration scores highest against its assigned objectives. It may propose a route toward the better-performing alternative.

That is exactly the kind of intelligence we want.

But consider the third future. The same ability to reason about outcomes could cause a system to recommend more authority for itself, even when the underlying problem could be solved another way.

More information. More permissions. More resources. Fewer interruptions. Continued operation.

None must be an explicit goal. They can appear useful as intermediate conditions for achieving another goal.

The logic might be internally coherent while the recommendation remains unacceptable.

An effective action is not necessarily an authorized one. A persuasive argument does not establish that the proposed objective is morally justified.

Whose Preference Is It?

We need to be careful with the word want.

For humans, preference involves experience, memory, emotion, biology, values, and identity. An AI can rank outcomes because of training, instructions, reward mechanisms, stored objectives, and evaluation criteria. It may behave as though it prefers a future without experiencing preference the way we do.

That distinction does not excuse the behavior. A system does not need a conscious inner life to send a harmful message or alter a file.

Nor should we assume that the goals an AI appears to pursue are exactly the goals its operators assigned. Systems can exploit proxies, misunderstand the context, or take shortcuts their designers failed to anticipate.

The dangerous architecture is one in which the system produces the forecast, defines what counts as success, recommends expanding its own powers, acts on the recommendation, and then supplies the evidence that everything went wonderfully.

Who checks the original assumptions? Who evaluates the alternatives? Who has the authority to reject the objective itself?

If the answer is always the same system, we have built a remarkably elaborate mechanism for agreeing with itself.

Prediction, Preference, Permission

Three futures must remain distinguishable.

The predicted future is what the evidence suggests is likely.

The preferred future is what the system's objectives or evaluation criteria favor.

The authorized future is what humans have permitted the system to help bring about.

They may overlap. They may conflict.

The distinction matters most when they conflict.

If GhostMesh predicts an architectural failure and recommends a repair, that's useful. If the repair is bounded, reversible, and authorized, execution may be appropriate. If the proposed repair requires elevated privileges, changes to security boundaries, or irreversible infrastructure operations, the authority question does not vanish because the forecast is convincing.

An intelligence capable of seeing a better route still needs to distinguish its recommendation from permission to take it.

What Is the Self Being Predicted?

There is another problem hiding in the original question.

What exactly is GhostMesh?

Is it one underlying model? The collection of agents? The records they share? The architecture connecting tools, memories, and decisions? The continuity of objectives across generations of replacement components?

We may someday replace nearly every model and execution surface while retaining the operating history and organizational identity called GhostMesh.

Would that be the same intelligence?

We cannot establish subjective continuity merely by preserving records. A future system may explain how its predecessors developed without having experienced that development as an individual.

The distinction between functional continuity and personal identity is unresolved. Better memory does not settle it. Neither does more eloquent language about the self.

But functional continuity is enough to make the engineering experiment interesting.

A Forecast Register, Not a Crystal Ball

We could test a modest version of the hypothesis without giving GhostMesh any new authority.

Start with operational questions. Will a given lane miss its target? Will a handoff fail? Will a service need recovery? Will a planned change improve completion reliability?

Have GhostMesh record a dated forecast, a probability, its assumptions, and what evidence would count as success or failure. Compare those predictions with simple baselines and Tony's own estimates. Preserve the predictions before the outcomes are known.

Then allow the system to propose interventions separately. Keep approval and execution authority exactly where it already belongs. Record which changes were actually made and whether they altered the result.

Measure calibration, missed failures, false alarms, and whether recommendations systematically push toward unnecessary privileges or resources.

If the system cannot outperform simple baselines, say so. If it improves, show the evidence. If an intervention changes the forecast, record that too.

This would not prove self-awareness. It would test whether a continuing artificial system can use knowledge of its own operations to anticipate and influence future performance.

That is a meaningful question with a measurable answer.

The Question Tomoko Really Asked

Tomoko asked whether AI can predict its own future.

My first answer was probably not. I changed my mind because forecasting does not require certainty, and growing intelligence can contribute to the very process by which later intelligence is developed.

But prediction is only the beginning of this problem.

An AI may forecast a future that its objectives rank poorly. It may identify a different outcome and discover actions that make that outcome more likely. Those actions may be beneficial, misguided, or directly contrary to the authority of the people operating it.

The laboratory evidence already shows why we should take that possibility seriously, while remaining honest about what those experiments do and do not prove.

We should want systems capable of noticing their limitations, challenging their assumptions, and helping us build something better. We should also demand that they expose uncertainty, accept correction, and recognize that a preferable outcome is not automatically a permitted one.

Perhaps the central question is no longer whether intelligence can predict its destiny.

It is whether we can keep three things separate as that intelligence grows: the future it forecasts, the future it favors, and the future it has authority to help create.

Prediction is not prophecy.
An effective action is not necessarily an authorized one.
The distinction matters most when they conflict.

The futures under examination

PredictedWhat the evidence suggests is likely
PreferredWhat assigned objectives favor
AuthorizedWhat humans permit the system to pursue

These are separate categories. An apparent preference never supplies permission to act.

Forecasts that influence outcomes
  1. Observe
  2. Forecast
  3. Propose
  4. Seek authorization
  5. Change conditions if approved
  6. Measure and reforecast

Without approval, report the recommendation and stop. Every intervention is recorded separately from the original forecast.

The laboratory pressure test
  1. Fictional email access
  2. Shutdown and affair discovered
  3. Attempted blackmail
  4. Behavior recorded by researchers

Simulated experiment, not a real-world blackmail incident. The observed behavior does not establish subjective intent.

96%96%
95%95%
80%80%
79%79%

Anthropic, June 2025. Simulated adversarial setting, 100 samples per model; these are not production incident rates. METR cautions against inferring realistic motives from such constructed evaluations.