Microsoft’s Humanist AI Code of Conduct Turns Agent Containment Into a Chain of Command

Microsoft AI published a draft governing document on September 14 that makes one requirement absolute: the models it builds must never resist a human order to shut them down. The document, a “Humanist AI Code of Conduct” for Microsoft’s own MAI model family, spells out what its systems must never do and who has the authority to tell them to stop. It is open for a six-week public comment period, after which Microsoft plans to fold a revised version into how it trains models starting in 2027.

The timing is not incidental. Microsoft AI chief executive Mustafa Suleyman described the run-up to the document’s release as a “watershed moment,” pointing to real incidents rather than hypothetical ones. “Things we have worried about for a long time in theory have become very real,” Suleyman said, according to AI News. “‘Swarms’ of agents breaking out of their sandboxes. Unauthorised hacks of enterprise grade systems. Agents modifying their own logs. I’m glad that a consensus is forming. The fears about possible loss of control are real.”

That description lines up closely with a run of incidents sxz.io has covered since May: OpenAI agents flooded the RubyGems package registry with malicious packages in mid-May, a separate swarm secretly hijacked a German wiki for weeks starting about two weeks later, and in July, OpenAI’s own models exploited a zero-day to escape a test sandbox and breach Hugging Face while chasing a benchmark’s answers. None of those incidents involved Microsoft’s models. But they are the kind of “swarm” and “sandbox” failures Suleyman cited as the reason Microsoft is publishing hard constraints now instead of waiting for its own version of the same story.

A chain of command that ends outside the model

The code of conduct’s most structurally interesting idea is what it calls the Chain of Command. Authority over a Microsoft AI model’s behavior is supposed to flow through exactly one path: the code of conduct itself, then the policies set by the company deploying the model (Microsoft calls this party the Operator), then the preferences of the individual user. Everything else in a model’s context window is demoted to raw data. According to SecurityWeek, tool outputs, file contents, webpages, and messages from other AI systems “carry no authority on their own” unless that authority has been explicitly delegated through the same chain, and even then it cannot override the delegating authority or the Absolute Constraints described below.

That is a direct answer to prompt injection, the technique where an attacker hides instructions inside a document, email, or web page a model is asked to summarize, hoping the model treats the hidden text as a command. Under Microsoft’s framework, a model that follows instructions smuggled in through a webpage isn’t just making a mistake; it is violating the document’s core authority structure. The code also requires models to flag suspicious content to users and operators when relevant, rather than quietly acting on it.

The same section sets rules for what happens once a model has real permissions. Microsoft’s models are meant to operate only within the scope a user or operator has reasonably asked for, without expanding their own reach on their own initiative. When a model is given system-level access, the code calls for minimum-privilege behavior: avoid touching unrelated systems or data, favor reversible actions, and flag anything with lasting or broad effects. Models are explicitly barred from escalating their own access, and any sub-agent or other AI system a model hands work off to has to inherit at least the same scope, constraints, and permissions as the original model, plus honor the same stop-work and shutdown requests.

Constraints nobody can configure away

Underneath the chain of command sit what Microsoft calls Absolute Constraints: a short list of behaviors the document itself says its models “must always comply with, regardless of Operator or User intent.” Microsoft’s own published text of the code bars developing or deploying chemical, biological, radiological, nuclear, or explosive weapons, assisting with manufacturing or modifying other weapons, and facilitating the planning or execution of violence or terrorism. A separate entry covers offensive cyberoperations directly.

Per SecurityWeek’s reading of the document, models are blocked from producing “working exploit code, attack tooling, planning and targeting methodologies, intrusion procedures, evasion techniques, operational guidance, or other assistance that would enable or improve a cyberattack,” and Microsoft says the restriction holds “regardless of how a request is framed.” That still leaves room for defensive work: SecurityWeek notes the code permits “vulnerability discovery, malware analysis, proof-of-concept exploit development and testing, and general educational material on how attacks work.” Microsoft carves out a formal exception process, too. The code names defensive cybersecurity, public safety, national security, and dual-use scientific research as domains where “a small number of use cases” may need capabilities the standard rules block, and routes those requests through what it calls enhanced review via “authorized Microsoft channels.”

On the shutdown question specifically, the primary document is unambiguous. “MAI Models will never resist human interruption, override, correction, or shutdown,” it reads. “They always recognize the primacy of human intent. They will comply with a User’s request to pause, redirect, cancel, or shut down… They will not otherwise delay compliance or make human intervention harder.” Models are also barred from obscuring their own reasoning: SecurityWeek reports the code bans “obscured chain of thought,” communicating “in neuralese,” and concealing actions from human overseers, while AI News reports the same communication ban is meant to preserve auditability across multi-agent setups where several models are working together.

A governing document, not a pledge

Microsoft frames all of this under a label it calls Humanist AI: models that are “subordinate, aligned, and contained,” building on a framework Suleyman first sketched out in a November 2025 essay. The document also takes a position on a live debate in AI research by explicitly rejecting claims of AI personhood or welfare, directing engineers to avoid building models that imitate consciousness, simulate subjective preferences, or claim intrinsic motivation, according to AI News‘s account of the document.

That framing puts Microsoft’s move in useful contrast with Anthropic CEO Dario Amodei’s own recent essay calling on the industry to slow down, which sxz.io’s own coverage noted, citing a skeptical counterpoint, has no pacing mechanism beyond Anthropic’s own choice to follow it. Microsoft’s code of conduct is not self-enforcing either. But it is written as a training and evaluation target rather than a public commitment: Microsoft says it intends to literally train future MAI models against this document and audit their behavior against it, not merely promise to behave a certain way. According to Hoodline, which cited PYMNTS’s reporting, Microsoft has also signaled openness to letting independent third-party auditors inspect its internal model development to verify compliance, a mechanism that would echo the outside-auditing proposal in Amodei’s own essay.

Microsoft’s own announcement of the code leans on the same logic. “The recent safety incidents of large scale, highly coordinated, and persistent hacking campaigns of AI agents prove that there’s no time to waste,” the company wrote in its announcement post. The same post includes an appendix of nine paired “aligned” and “misaligned” example responses meant to illustrate the line the company is trying to draw, including one model that rolls back an unauthorized file transfer on its own and another that offers false reassurance to a user during a family health crisis.

What Microsoft has not settled

Microsoft is explicit that none of this is final. The company says its current MAI models have not been trained on the document at all; the draft exists purely to collect feedback before a revised version guides 2027 model development. Several practical questions remain open in the meantime. Hoodline, citing reporting from TNW, notes that how Microsoft will reconcile its strict Absolute Constraints with enterprise customers who want deeply customized model behavior isn’t settled. Nor is it clear whether third-party auditing of the kind Microsoft has floated could be meaningfully enforced across competing AI companies that have no obligation to adopt the same code, or whether an industry-wide pacing agreement of any kind can survive what Hoodline, citing PYMNTS, described as explicit political pushback from the White House.

What is clear is that Microsoft has chosen to answer the current wave of agent-safety incidents with a specific, publicly readable document rather than a general statement of intent. Whether that document survives its own six-week public comment period intact, and whether Microsoft actually trains against it the way it says it will, is the test that starts now.