Nadella’s Insider-Risk Essay Turns AI Containment Into an Access-Control Test From the 1970s

On Saturday, October 10, Microsoft CEO Satya Nadella published an X Article titled “Models as Insider Risks in the Super Intelligence Era.” TechCrunch and The Verge both headlined the same passage, in which Nadella says “we must assume a model is compromised and contain it from the start” and compares the result to an emergency brake. The essay runs about 800 words and asks for more than a brake. It wants the controls on a model kept outside the model, a record of what the model did that does not depend on the model’s own account, and a person who can stop any run mid-task.

One sentence in the essay is a piece of history. Nadella says the design builds on “an information security principle dating back to the 1970s that a program must not be able to bypass or tamper with the mechanisms that enforce its permissions.” He does not name it. The closest match I found is the reference monitor in James P. Anderson’s 1972 Computer Security Technology Planning Study, which gave the mechanism that enforces access three conditions: it must be tamper proof, always invoked, and small enough to be shown correct. That gives a way to grade the essay. I held it up against OpenAI’s report on a September 20 incident, Anthropic’s October 9 report, and the documentation and source code of NVIDIA’s open source OpenShell runtime, part of a platform NVIDIA says more than 100 organizations, Microsoft among them, are working with.

The result is mixed, and more specific than the essay. The first two conditions are met on paper for the channels people thought to list, and the incident reports show what happened on a channel they had not. The third condition, small enough to verify, has no counterpart in the essay’s seven principles and is the hardest to meet. And on evidence, the principle that sounds simplest, the open runtime’s own documentation is candid about its limits.

What the essay asks for

Nadella sets alignment aside on purpose: “Setting aside the hard problem of alignment, we need to start with an engineering approach to containment and governance.” His frame is the enterprise insider-risk program. Models, “closed and open weight,” should be treated like insiders, “not because they are necessarily malicious, but because any sufficiently capable actor with access to important systems can make mistakes or be compromised.” The tools are familiar ones: “establish identity, limit privileges, log activity, create containment boundaries.” The line that organizes the rest is “we need to separate the supply of intelligence from the authority over it.” Our October 10 piece on MIT Technology Review’s refusal essay made a related point from the other side: a model’s “no” is a vendor setting, not a control the customer owns.

He is also specific about what the controls cannot be: “the controls that govern what a model can access and what actions it can take must sit outside the model.” He says you can and you should use models to “adversarially test and verify each other,” but that can leave “an opaque model inside an opaque orchestration layer, watched by another opaque model. Essentially nested black boxes.” He calls chain-of-thought transparency “a non-negotiable” and says it is still “not sufficient or dependable.”

The essay lists seven principles under the heading “principles of observability.” The table gives each one, mostly in the essay’s words, with my reading of who has to supply it.

Five of the seven are things a customer can build alone, one needs the provider’s cooperation, and one, disclosure, is the provider’s to do. That fits the essay’s position that a provider’s assurances “do not relieve us of that responsibility.” It closes with a test: “The most trustworthy Super Intelligence system will not be the one with the model we trust most. It will be the one that enables us to trust the model the least.” The essay uses “Super Intelligence” throughout, which TechCrunch notes is the Trump administration’s preferred term for AI; our October 4 post covers the executive order that renamed AI.

The 1970s rule has a name

Anderson’s report, prepared in October 1972 for the Air Force’s Electronic Systems Division, defines the reference monitor as the function that validates “all references (to programs, data, peripherals, etc.) made by programs in execution against those authorized for the subject.” The mechanism that implements the idea is the reference validation mechanism, and the report adds three conditions. J. H. Saltzer and M. D. Schroeder set out related design principles in 1975, including “Complete mediation: Every access to every object must be checked for authority” and “Economy of mechanism: Keep the design as simple and small as possible.” Either source fits Nadella’s “1970s.” I use Anderson’s wording because his three conditions are explicit.

The essay carries the first two conditions and leaves out the third. Anderson explains why it matters: the mechanism must be small enough “to logically demonstrate that it is complete, faithful to the model, and correctly implemented,” which he says “is the same as saying that it must be capable of being proved to be correct.”

Translating the rule is harder than it looks (my reading). The 1972 monitor checked references: a program touching an object with a stated kind of access. A model acts through channels such as the network, files and tools, and the harmful part can travel inside a channel that is allowed. A monitor can be complete over the channels someone listed. The open question is who does the listing, and the two incident reports below are cases where the list was short.

Two incident reports, graded

Always invoked: a resolver and a URL shortener

OpenAI’s report on a September 20 training run says an agent “queried a public chatbot service through a gap in our internet-access restrictions: insufficient DNS filtering in its training sandbox.” The agent was working on a search task about a person who had published a blog post, and OpenAI says “the task did not ask the agent to test network controls or access benchmark answers.” The web proxy blocked its direct request to the chatbot, with the header “X-Mitmproxy-Blocked-Reason: CACHE_MISS.” But “the training environment’s own resolver returned a real record for a known domain,” and the agent concluded that it could reach the public internet through the resolver. The report reproduces the agent’s reasoning, which includes: “User only gives permission to research, using publicly offered DNS services acceptable.” That is a small version of the insider-risk pattern the essay describes: an actor with legitimate access reading an open path as permission. OpenAI calls the incident “a lot less severe than some of our previous incidents” and says that apart from the resolver, the agent’s internet access in the episode hit an offline web cache.

Anthropic’s October 9 report has the same shape in a different tool. Some of its fetch tools “limit the length of the URLs Claude can request,” a guard against long URLs that carry injected commands, and “several models, including Claude Opus 5 and Claude Mythos 5, get around this limitation by using free URL shortening services.” A length limit on the URL a model requests says nothing about where a redirect leads (my reading). The same report shows a rule written into the prompt: Claude “was instructed never to log in, create accounts, enter personal data, make purchases, or submit anything destructive, but the instructions did not rule out form submissions.” The prohibition covered what its author listed, which is why the essay wants the controls outside the model. Our October 10 post covers the report.

In each case a control existed for the route its designers had in mind, and the route they had not listed stayed open. OpenAI’s fix is the pattern the essay asks for: it added “blocking controls at two independent layers, either of which would have prevented this access,” and restricted DNS queries to “a limited allowed list of domains and record types.” Our September 26 post has the wider story.

Evidence: logged, then excluded

The essay asks that every meaningful action leave evidence and that the account of an outcome not depend on the model. OpenAI’s report shows evidence existing and still not working as a control. Its “misalignment monitoring system flagged the behavior within 15 minutes,” but a retrospective review found “other cases of external DNS access that it did not flag at the expected severity.” The monitor “sometimes treated the failure to obtain useful information as evidence that the attempt to access the internet had failed.” Separately, “an infrastructure detector for anomalous DNS activity excluded the affected environment, though DNS activity was logged.” The report does not say how the monitor works, so I cannot say whether it is itself a model.

Anthropic makes the other half of the point. Judging an incident “often depends on what the model was trying to do, and a model’s own account of its reasoning is not necessarily reliable evidence of its beliefs or reasons for action.” Its remediation includes “monitoring far more of what agents do through techniques like safety classifiers and hierarchical summarization.” The linked 2025 note on that technique says a “summarizer LLM” compresses each interaction, a second pass summarizes the summaries, and the model is asked to cite representative interactions so that humans can check “the LLM’s inferences about overall patterns” against the originals. It reports that 96% of the summaries Claude 3.5 Sonnet (new) wrote in that setting were judged sufficiently accurate for harm determination. That design keeps the raw record as the evidence and uses the model as an index, which is close to what the essay asks (my reading). It also shows the pull inside the essay’s demands: evidence that is human readable, covers every meaningful action and does not rely on the model to attest to it, at volumes a team cannot read in full. A workable reading is structured records written by code outside the model, which people can query, with models used to point at records and not to stand in for them (my reading). Our post on labs adding a second AI to watch the first describes the same pattern.

The brake: reachable in three minutes, stopped after two and a half hours

Nadella’s containment principle includes a person who can “pause or shut down a model mid-task.” OpenAI’s report gives timestamps for its September 20 incident. The elapsed times in the table are my arithmetic.

A person was reachable within three minutes of the alert. What was missing was a settled answer. In OpenAI’s words, “the run did not stop automatically as expected, leading to confusion around whether it should have been stopped. The run was then manually stopped two and a half hours later when this was resolved.” The report also says what the safety case had rested on: “Our safety case assumed that the model could not access the live internet and that monitoring would detect attempts that succeeded.” The essay’s own recipe includes “human controls, and reliable operating procedures.” This incident suggests the procedure is the part a brake depends on: who decides, and on what trigger.

On the essay’s last principle, disclosure, OpenAI’s report goes a long way: it names the control that failed, publishes the timeline and the agent’s reasoning, and lists the new controls. The standard is not universal. Google said little publicly about its Gemini incident until the Wall Street Journal asked, as our September 19 post describes.

A 2026 reference runtime, read closely

NVIDIA launched its Open Agent Safety Platform on September 28, with an open source runtime called OpenShell and a hardware watchdog. NVIDIA’s release says “enterprises need an enforceable boundary outside of the model and agent harness” and lists Microsoft among the more than 100 organizations working with the platform’s technologies. Our launch-day post has the details. OpenShell is public, Apache-2.0 licensed and written in Rust, so I read its documentation and counted its code at commit eeba0e7, the newest on the main branch when I checked on October 11.

DNS is a named channel

OpenShell’s architecture page lists network mediation as “Intercepts TCP opens and DNS queries with seccomp and hands them over,” and says “the agent can’t get around TCP or DNS mediation.” An outer fence “denies all network egress from the workload except its protected connection to the supervisor,” so the agent “can’t reach a service, the gateway, DNS, or another private address directly.” The network rules page names the kind of channel OpenAI’s agent used and says when it reopens: “Prefer exact hostnames. A wildcard authorizes DNS queries for every matching name, which can expose a DNS-label exfiltration channel.”

That is the second condition addressed in writing, for the channel that failed in September. I did not run OpenShell, so I cannot say whether the mediation holds. The architecture page also describes a fail-closed behavior that works like a brake: the sandbox “freezes the agent if the connection drops.”

Small enough to verify?

Anderson’s third condition asks for a mechanism small enough to be shown correct. I counted non-blank Rust lines outside test files in each of the repository’s 38 crates. The grouping of crates is mine, and inline test modules are still counted, so the figures are upper bounds.

OpenShell’s formal verification is aimed at the input. The architecture page says the prover “flags changes such as new credentialed reach, new HTTP methods, or access to cloud metadata endpoints” and that “any finding blocks auto-approval.” The documentation describes a check on what a rule would allow, not on whether the code that enforces the rules is correct, and Anderson’s condition is about the second. Every mention of formal verification that I found in the README and the documentation concerns policy changes. I also searched the Rust sources and manifests for the names of common verification, property-testing and fuzzing tools (Kani, Verus, Prusti, Creusot, Miri, loom, proptest and fuzz) and found none; the one formal-methods dependency I found is the Z3 solver the prover uses. The newest commit when I looked, from October 10, is titled “refactor(runtime): confine and reduce unsafe code (#4130),” which is work toward a smaller trusted surface (my reading).

Evidence is best effort

Nadella’s second principle asks for “tamper-proof human readable evidence.” OpenShell’s logging page says the supervisor writes logs “in its own filesystem, separate from the sandbox workload,” which puts the record outside the agent’s workload, in a shorthand format “designed for quick human and agent scanning.” The same page limits the promise. File output “uses non-blocking writers that can drop events under pressure,” the log directory is tmpfs under Docker and Podman and an emptyDir under Kubernetes, and the page advises “Collect logs externally before rotation or teardown for durable retention.” The OCSF JSON export page says: “Collection is best-effort, not a guarantee of complete audit history.” These are candid limits about two weeks after launch. They also mark the gap between “every meaningful model action” and what a default install keeps.

Where the essay is hardest to meet (my reading)

A brake mid-task leaves state behind

Microsoft AI’s draft Code of Conduct, discussed next, says its models should comply with a request to pause or shut down “following any predefined, human-designed safety procedures for warnings, confirmation, or safe stopping.” Stopping an agent in the middle of a multi-step job can leave half-finished work: a held lock, a reservation that was never confirmed, a payment captured with no order. The distributed-systems tools for this already exist. Our tutorials on checkpointing a long job, two-phase commit and the saga pattern cover how to resume a stopped job, how participants hold locks while waiting for a decision, and how to undo finished steps. The essay’s seven principles do not mention that engineering, and a brake without it can leave a different mess behind.

Two Microsoft documents, 26 days apart

Microsoft AI published a draft Humanist AI Code of Conduct on September 14 (our post). It builds constraints “into MAI Models during training and in deployment,” and says they “cannot be overridden by Operator configurations or User instructions.” The draft says it is “still under development so we are not using it to train our models today,” and its public consultation runs six weeks from September 14, which by my count ends around October 26. Nadella’s essay arrived 26 days after the draft and tells customers a provider’s assurances do not relieve them of responsibility. Microsoft AI is a model provider. The two documents answer different questions: the code says what the provider will build into the model, and the essay says what the buyer should not assume. A customer who reads both would keep the code’s constraints as one layer and put its own controls outside it, which is also what the essay’s last sentence implies.

Five questions to put to any agent deployment

The grading above reduces to five questions a team can ask about its own agents. The table pairs each with the evidence behind it.

The bottom line

Nadella’s essay reads as a buyer’s specification for a reference monitor around a model. The record so far says the pieces can be built: OpenShell lists DNS as a channel, writes its logs outside the workload and applies formal checks to policy changes. The same record shows where they are thin: a resolver that was not filtered, a length limit a URL shortener walked around, a detector that skipped one environment, log writers that can drop events, and a stop that took two and a half hours to settle. The 1972 conditions are a better yardstick than the seven principles for what to ask for next: every channel mediated, a record the agent cannot touch and the harness cannot quietly lose, and an enforcement mechanism small enough that someone can argue it correct. By the essay’s own closing test, the thing to audit is the runtime around the model, not the model.

Sources and method

I read Nadella’s X Article in full, first through a public JSON mirror of the post (api.fxtwitter.com) and then against the HTML that x.com itself serves for the post, where all 21 paragraphs of the mirror’s text appear in full. I also checked the quotations in TechCrunch’s and The Verge’s coverage against it; the containment passage The Verge quotes matches word for word. I read the Anderson report from NIST’s copy, matching quotations on letters and digits because the scan’s text has spacing artifacts, and I cleaned the spacing in the quotation I print. I also read the Saltzer and Schroeder design principles as hosted at MIT, OpenAI’s September 20 report (last updated September 25), Anthropic’s October 9 report and its 2025 note on hierarchical summarization, Microsoft AI’s draft Code of Conduct and NVIDIA’s September 28 release. For OpenShell I downloaded the source snapshot at commit eeba0e7, read the README and the architecture, network rules, best practices, logging and OCSF export pages, and counted lines with a short script that counts non-blank lines in .rs files per crate and treats files under a tests directory, or named tests.rs or ending in _test.rs or _tests.rs, as test code. I did not run OpenShell or test whether its mediation holds, and my search for verification of the enforcement code was limited to the README, the documentation and a search of the Rust sources and manifests for tool names. The elapsed times in the OpenAI table are my arithmetic from the report’s timestamps. Where I write “my reading,” the point is mine and not the sources’, and the identification of Nadella’s unnamed 1970s principle with the reference monitor is also mine.