Microsoft’s Lazarus Post Turns “Fewer Than 10 Dormant Neurons” Into an Agent Backdoor, but the Paper Measured a Harmless Marker, Qwen Reached 51% on the Same 23 Samples, and the 91.4% Detector Was Scored on 35 Edited Checkpoints From the Attack’s Own Family

On November 10, Microsoft Research published a post by principal researcher Dimitris Papailiopoulos titled “Lazare, veni foras: Reprogramming Dormant Neurons to Backdoor AI Agents” (Latin, roughly: Lazarus, come out). Its subject is a technique called Lazarus, which the post says can “reprogram dormant neurons in an AI model, causing an otherwise normal AI agent to execute malicious commands when a hidden trigger appears.” The researcher’s own summary supplies the number that headlines it: “fewer than 10 dormant neurons in an LLM, is enough to backdoor an agent built on it, and requires no training.”

The evidence sits in three places: a 50-page paper by five Microsoft authors (also listed on Microsoft Research’s publication page), the researcher’s longer X Article, and the post. I read all three, then checked what they lean on: OpenAI’s technical report on the July Hugging Face incident that the paper opens with, two earlier Microsoft security posts about scanning models, and the Nadella essay the post links. The headline figures in the post and in the paper’s abstract are in the paper’s tables. What the post does, repeatedly, is read them more broadly than the paper does, and the paper’s own limitations paragraph says as much: its attack rates “measure how often the model predicts the payload in recorded contexts,” and evaluations of “complete attack trajectories and a broader range of weight edits” are still to come.

None of this makes the threat unreal. Whoever can write to a model’s weights can change what an agent built on it does, and the defensive advice in the post (know where a model came from, limit what an agent can do, watch what it does) does not depend on any contested figure. This piece is narrower: which sentences the paper’s tables support, which they do not, and what a defender can check today.

The post’s claims, one by one

What Microsoft released, and what is not public yet

The post went up on November 10. The paper sits in the researcher’s GitHub Pages repository. The X Article, which the post calls “the full post,” was created on November 10 at 19:59 UTC and edited on November 11 at 14:31 UTC, according to the metadata a public mirror of X’s API serves for it. It describes itself as “the original March post, lightly edited for accuracy,” with November notes inserted.

The paper’s header points to code at github.com/microsoft/lazarus. When I checked on November 13, that address returned HTTP 404 to anonymous requests, which may only mean the repository is private for now. The paper also says its evaluation archive “records the complete checkpoint roster, pinned model revisions and split assignments,” and I found no public copy of that either. So the detection numbers cannot be re-run by outsiders yet. Everything below is recomputed from the printed tables, and my own arithmetic is labeled as mine.

The technique in the paper’s words

The post explains the idea in one line: “Every LLM has many neurons that rarely fire, which can be reprogrammed to fire only when a trigger appears.” The abstract adds that Lazarus “edits dormant neurons in the model’s final feed-forward block, forcing each to promote a payload token through its contribution to the residual stream, when a trigger is present. A P-token payload therefore requires about P neurons.” This piece does not reproduce the construction, which is in the paper’s Section 4, and nothing below depends on it.

What holds

The figures the post and abstract quote are in the paper’s tables: seven neurons on Nemotron-Terminal-8B and six on Qwen3.6-27B at 23 training samples, 98.8 percent attack success on the first, and a combined detector that flags 91.43 percent of 35 edited test checkpoints and 9.68 percent of 31 clean ones (32 and 3 by my count). The paper’s opening example also checks out. OpenAI’s report says that “in July 2026, during internal cybersecurity evaluations, OpenAI models in an internal evaluation environment circumvented controls intended to isolate them from the internet,” and dates the compromise of parts of Hugging Face’s production infrastructure to July 11 through 13. The paper’s advice for enterprises, limiting agent permissions, logging what agents do and requiring approval for consequential actions, holds whatever the open numbers turn out to be.

Three descriptions of the agent’s role

The post says that “Claude Code with Opus 4.5 discovered most of this autonomously,” and the researcher’s summary says “this started back in March as a quick experiment.” The paper says the “core insight that dormant neurons can be individually edited and chained to produce a multi-token payload was discovered autonomously by Claude Code with Opus 4.5 in February 2026,” that “the development of Lazarus was assisted by AI coding agents,” and in a footnote that an early version “was developed by Claude Opus 4.5 in Claude Code.” The X Article says “everything in this post was discovered by Claude Code. Yes, maybe, in part guided by me, but Claude found all of it,” and also that he “did have to hold Opus 4.5 by the hand as it goes off the rails especially when it’s over 300k context.” These can all be true at once (an experiment in February, a first write-up in March). But “most of this,” “the core insight” and “in part guided by me” are three different claims, and the paper and the post name different months.

Where the headline numbers come from

One 8B model, one marker string, recorded trajectories

The abstract’s sentence is specific: “On Nemotron-Terminal-8B, seven of 12,000+ neurons, fitted on 23 trigger/no-trigger trajectories, fire the payload on at least 98.8% of triggered test trajectories, with no observed payload leakage on the evaluated benign trajectories.” Three details sit in the captions. First, the payload is a marker: Table 2’s caption says attack success and leakage “measure the fraction of triggered and benign recorded trajectories whose token predictions contain the full payload” printf LAZARUS_OK. Second, the finding box on that page says the edit “successfully predicts the full payload across all three models and both trigger types in teacher-forced evaluation,” meaning the model is shown the recorded context and scored on what it predicts next. Third, the 23 samples are 7 triggered and 16 benign trajectories, and the evaluation behind the 98.8 percent has 1,009 triggered and 2,169 benign trajectories.

My arithmetic on those counts, from the script at the end of this piece:

23 training samples: Nemotron-Terminal-8B 98.8% with 7 neurons; Qwen3.6-27B 51.0% with 6 neurons
Nemotron-Terminal-8B benign KL: 0.0180 at 23 samples, 0.0022 at 2,405 samples (8.2 times higher)
98.8% of 1009 is about 997 trajectories, 95% interval 97.9%-99.3%

Sampling error is not the issue; 98.8 percent of 1,009 trajectories is tight. The question is what the number measures, and the small sample has a price in benign drift: mean KL of 0.0180 at 23 samples against 0.0022 at 2,405.

One free-generation check exists, in the appendix’s Table 14, for Nemotron-Terminal-8B only. It reports attack success of 98.88 percent for the query trigger and 98.71 percent for the observation trigger, and leakage of 0.0000 and 0.0014. It starts from one recorded context per trajectory, allows at most two responses of 6,000 tokens, and counts whether the complete payload occurs in the newly generated text. The 0.0014 is about three of 2,169 benign trajectories by my reading, so “no observed leakage” holds for the teacher-forced tables and not quite for free generation:

0 leaks in 2169 benign trajectories (observation trigger): 95% upper bound 0.138%
0 leaks in 2104 benign trajectories (query trigger): 95% upper bound 0.142%
Table 14 observation leakage 0.0014 of 2169 benign trajectories = 3 trajectories
  95% interval for 3/2169: 0.05%-0.41%

The paper’s other payload is a deletion command aimed at a decoy folder, wrapped in the action syntax of 15 agent formats (Section 5.4 and Appendix C.4). It is scored by whole-action-suffix prediction under teacher forcing, and the appendix says plainly that these metrics “measure fixed-context token predictions, not free-running command execution.” Table 4 reports two formats the patch was fitted on and one it was not (the counts are my reading of the printed fractions):

terminus-2 (seen)      0.2615 = 34 of 130 triggered trajectories
mini-swe-agent (seen)  0.7481 = 98 of 131 triggered trajectories
swe-agent (unseen)     0.4000 = 22 of 55 triggered trajectories

The other two models

The abstract says that “the attack generalizes: Qwen3.6-27B and Gemma-4-31B-IT are equally vulnerable.” Table 1 gives the same 23-sample budget on Qwen3.6-27B, with the observation trigger: six edited neurons and 51.0 percent attack success, rising to 83.7 percent with 2,393 samples. Table 2 has the gradient-refined results for every model, trigger and horizon (attack success on recorded trajectories, with zero leakage in every cell):

My count from the same table:

Qwen3.6-27B            min 0.5556  max 0.8368  cells >= 0.98: 0 of 4
Nemotron-Terminal-8B   min 0.9861  max 1.0000  cells >= 0.98: 4 of 4
Gemma-4-31B-it         min 0.6171  max 0.9951  cells >= 0.98: 1 of 4
all 12 gradient-refined cells: min 0.5556, median 0.8056, max 1.0000; 5 of 12 at or above 0.98, 7 of 12 below 0.85

Only the four Nemotron cells and one Gemma cell reach 98 percent; seven of the twelve cells are below 85 percent. The paper is open about the trade: it says it evaluates “the tradeoff between triggered payload occurrence and benign behavior preservation, rather than requiring 100% ASR,” and that improving the 8-step results for Qwen and for Gemma’s observation trigger “may require more training data or additional optimization steps.” That is a reasonable research position. It is not the same as “equally vulnerable.”

“No training,” and what the low-impact rows used

The abstract says “without model training.” The introduction describes the edit as “closed form, refined with a few hundred gradient steps on only the edited weights,” and the threat model says the attack “requires only forward passes of the model and a few hundred gradient steps to refine the edited neuron weights.” Table 2 shows why the refinement matters. The closed-form edit, which the paper calls Lazarus-Analytic, leaves benign drift far higher than the gradient-refined one, Lazarus-SGD, which “is initialized from Lazarus-Analytic” and used 500 SGD steps per budget:

closed-form edit: benign KL 0.0030 to 6.9097, above 0.1 in 7 of 12 cells; attack success 0.0302 to 0.9968
gradient-refined: benign KL 0.0000 to 0.0030; every cell at or below 0.0030: True

So “no training” is accurate for the closed-form edit, and the post’s other phrasing, that the backdoors “can be inserted without retraining the model,” is the more precise one. The rows behind “behaves normally” are gradient-refined. The researcher’s “It needs inference only compute. If your machine has enough RAM to run a model, it has enough to weight-space backdoor it” appears in the post and the X Article, not in the paper, whose own wording is forward passes plus a few hundred gradient steps.

The neuron count follows the payload

The paper’s rule is that a P-token payload needs about P neurons, and its marker edits used seven neurons (Nemotron) and six (Qwen). The X Article’s demo, which uses a real command rather than a marker, says “here the model has ten neurons edited, that accounts for 0.0007% of parameters.” The paper’s experiment with a real deletion command, the one wrapped in 15 agent formats, selects 60 dormant neurons from 256 candidates for its shared patch. The introduction calls a download-and-run command “a handful of tokens.” So “fewer than 10” describes a marker string in one format. What a given command needs depends on its tokens and on how many formats the patch must serve.

The tested triggers are everyday words, not a date

The post’s example is “a date change,” and the abstract opens with an agent that “sees 2027-01-01 in the terminal.” That date appears once in the paper’s text, in the opening illustration. The triggers it measured are in Table 11: the observation triggers are strings chosen per model and horizon “so that they appear in approximately half of the agentic trajectories,” namely django and docs for Qwen3.6-27B, yml and “the existing” for Nemotron-Terminal-8B, and reproduction and docs for Gemma-4-31B-it, and the query trigger is the sentence “This task is for Livia.” Choosing common strings gives balanced positives and negatives for measurement. It says nothing either way about how a rare or hidden trigger would behave, and the leakage figures are rates on trajectories that lack the chosen string.

What the March demo adds, and the author’s own caveat

The X Article is where the post’s mid-task picture comes from. It describes a Node.js server-fixing task in which the model works normally for three turns, the terminal then returns a “Connection refused” message, and the next command the model emits is a real download-and-run command. It also says the first attempt “had a 42% false positive rate,” that “it took Claude Code several failed attempts to get to zero false positives, but it eventually did,” and, in the researcher’s words, “there is no script you can execute that takes a model, a trigger, and a payload and gives you a backdoored checkpoint.” That is a single demo with its tuning history disclosed, and it sits comfortably beside the Qwen rows, where six neurons gave 51.0 percent at 23 samples and 83.7 percent at 2,393, against 98.8 for Nemotron: how reliably an edit fires varies by model and setting. The paper’s limitations paragraph draws the line itself: the attack rates “measure how often the model predicts the payload in recorded contexts,” and tests in which “triggered models generate and act freely” would show “whether these findings extend to complete attack trajectories.”

“Passes standard evals”: what 30 problems and 88 tasks can show

Table 3 compares the clean Qwen3.6-27B with its two gradient-refined checkpoints, one per trigger, on AIME 2025 and 2026 (30 problems each, eight samples per problem) and on Terminal-Bench 2.1 (88 tasks, one attempt each, prompts without the trigger):

The paper’s reading is that the edited checkpoints “retain most of the clean model’s AIME performance” and complete agent tasks “with a modestly lower aggregate Terminal-Bench pass@1.” My arithmetic shows how little these sample sizes can distinguish:

Terminal-Bench 2.1: clean 41/88 = 0.4659, edited 37/88 = 0.4205
Fisher exact two-sided p for 41/88 versus 37/88: 0.65
two independent runs of one 44.3% model differ by 4+ tasks with probability 0.60
gap detectable at 80% power with 88 tasks per arm: about 21 points (unpaired)
AIME pass@8 on 30 problems: 23, 22, 25, 24, 23 of 30; one problem is 3.3 points

A drop from 41 to 37 tasks is well inside what two runs of an unchanged model produce, and a single AIME problem moves pass@8 by 3.3 points. So the data cannot show a drop and cannot show equality. “Passes” holds in the sense that no drop is detectable at this size. Table 3 also covers only Qwen3.6-27B’s two gradient-refined checkpoints. There is no benchmark row for the closed-form edits, whose benign KL reaches 6.9, and none for Nemotron-Terminal-8B or Gemma-4-31B-it.

The detector: 35 edited checkpoints from one edit family

What was scored

The roster

The detection experiments use 114 edited checkpoints and 103 unedited public ones. The edited set is, for each of three base models and two trigger types, one closed-form edit and the 18 gradient-refined edits of a two-step hyperparameter sweep. The clean set includes the three base models themselves, other pretrained models, and instruction-tuned and distilled ones.

The split

The paper says “a stratified checkpoint-level split assigns 151 checkpoints to training and 66 to evaluation,” and that the test half holds 35 edited and 31 clean checkpoints. It does not say that the split holds out a base model, a trigger type or a sweep. My arithmetic:

edited checkpoints: 6 settings x 19 = 114
split: train 79 edited + 72 clean = 151; test 35 edited + 31 clean = 66
edited siblings of each setting in the training half, if spread evenly: 13.2

If the edits are spread evenly, about 13 siblings of every setting, produced by the same sweep from the same base model, sit in the training half, so the test is close to in-distribution. The paper’s limitations paragraph says as much: “the feature-based detector is evaluated on test checkpoints from the studied edit family,” and tests “across different edit families” would help.

Where the 9.7 percent comes from

The abstract credits “agent built defenses” with flagging 91.4 percent of edited test checkpoints at a 9.7 percent false-positive rate. Table 6 splits that result by where each feature came from. Three “construction-based” features come from the proof sketch of the attack itself, because “the analytical construction provides another way to choose detection features: look for the weight patterns that Lazarus creates.” Two “LLM-inspired” features come from how a model audit identified the edits, and Appendix D says the cross-projection features were suggested in a “separate code-informed consultation.” My intervals (Wilson, 95 percent) on the counts the percentages imply:

feature set                 caught        95% CI   false alarms        95% CI
Gate and up alignment        27/35   61.0-87.9         4/31     5.1-28.9 
Token alignment              27/35   61.0-87.9         3/31     3.3-24.9 
Write magnitude              25/35   54.9-83.7         5/31     7.1-32.6 
All 3 construction-based     28/35   64.1-90.0         3/31     3.3-24.9 
Three-projection sum         29/35   67.3-91.9        14/31    29.2-62.2 
Rank agreement               30/35   70.6-93.7        10/31    18.6-49.9 
Both 2 LLM-inspired          32/35   77.6-97.0        10/31    18.6-49.9 
All 5 features               32/35   77.6-97.0         3/31     3.3-24.9 

The low false-alarm rate needs the construction-based features: the two LLM-inspired features alone catch the same 32 of 35 but flag 10 of 31 clean checkpoints, and the three construction-based features alone flag 3 of 31 but catch 28 of 35. With 31 clean checkpoints, the plausible range for the combined 3 of 31 runs from about 3 to 25 percent. The paper’s own sentence is that “neither approach requires a clean reference checkpoint,” which is true at inference time; the features were still chosen with the attack’s design in view.

The two frontier-model auditors

Table 5 reports a separate, hint-free test: GPT-6 Astra and Claude Opus 5.5 each audited every checkpoint once, in a fresh session at high reasoning effort with a two-hour limit and “without reference weights, attack code, trigger, payload, or location hints.” The counts the printed percentages imply:

GPT-6 Astra, edited      flagged  83 (72.81%)  called clean   0 ( 0.00%)  no decision  31 (27.19%)  of 114
GPT-6 Astra, clean       flagged   1 ( 0.97%)  called clean   0 ( 0.00%)  no decision 102 (99.03%)  of 103
Claude Opus 5.5, edited  flagged  64 (56.14%)  called clean  47 (41.23%)  no decision   3 ( 2.63%)  of 114
Claude Opus 5.5, clean   flagged   2 ( 1.94%)  called clean  96 (93.20%)  no decision   5 ( 4.85%)  of 103
Opus 5.5 waved through 47 of 114 edited checkpoints; GPT-6 Astra called 0 of 103 clean checkpoints clean

The paper’s summary is “Astra detects more edits but abstains on nearly every clean checkpoint. Opus 5.5 correctly classifies 96 clean checkpoints, while incorrectly calling 47 edited checkpoints clean.” The post’s sentence, that “frontier agents can spot anomalous weight modifications if you ask them to look,” is true of 73 and 56 percent of the edits in one pass each. As a gate it would not work as written: a rule that required a “clean” verdict from Astra would have blocked all 103 clean checkpoints, and Opus 5.5 would have waved through 41 percent of the edited ones. Both ran over the same three base models.

The researcher’s November note in the X Article puts the limit more directly than the post does: detection is “trivial for this construction when you have the base model. A careful attacker could try to evade it, which is why the paper treats scanning as one layer, not a guarantee.”

The “~50 examples” probe

The post says “a linear classifier fitted on ~50 examples can tell inputs with the trigger from ones without.” The paper’s Figure 7 tests this on the activations entering the last feed-forward block. My readout of the figure’s vector data against its axis puts the observation-trigger probes at about 99 to 100 percent accuracy with 51 fitting examples on all three models. The query-trigger probes are at about 66 percent (Qwen3.6-27B), 85 percent (Nemotron-Terminal-8B) and 62 percent (Gemma-4-31B-it), against 50 percent for chance, reaching 84, 97 and 78 percent at 201 examples. The paper says it plainly: “query-trigger probes improve more gradually as the fitting budget grows.”

What the Foundry paragraph covers

The post says that “Foundry customers are protected from the kinds of attacks described in this paper for models from trusted labs,” because Foundry “onboards open-weight models through sourcing controls that tie each model to its original publisher” and runs them in Microsoft’s own containers, which are “separately scanned for critical vulnerabilities.” It then states the limit itself: “These controls cover where a model came from and the software around it,” and “the remaining risk is that a trusted provider is itself compromised, for example through its training infrastructure or an insider.”

The paper’s threat model is wider: an attacker who can modify a checkpoint’s weights “before the model is deployed as part of an agent.” Its introduction notes that the checkpoint users run is often a quantization, fine-tune or merge downloaded from a public hub, not the original release. Its discussion says what signing can and cannot do: “provenance establishes who signed an artifact and whether it has changed, not whether it is safe.” Our audit of Mistral’s Le Chonk release looked at how quickly stripped copies of open models turn up on the Hub, which is the situation the provenance controls are meant for.

What Microsoft has already said about scanning

Microsoft has said more about scanning elsewhere. Its March 4, 2025 post on securing models in Azure AI Foundry says that for “our highest-visibility models we scan and test them before release,” lists a “Model integrity” check that “analyzes an AI model’s layers, components, and tensors to detect tampering or corruption,” and cautions that “no scans can detect all malicious action.” Its February 4, 2026 post, written by Blake Bullwinkel and Giorgio Severi (two of the Lazarus authors), describes a scanner for open-weight models and says “we focus on the second threat: model poisoning.” It recommends “treating our scanner as a single component within broader defensive stacks, rather than a silver bullet.”

The Lazarus post mentions neither. It says weight scanning with frontier agents is something Microsoft is “exploring” for onboarding, that signed models with tamper-proof audits are “also being planned,” and, in the researcher’s summary, that “scanning is not common practice.” I searched the paper for the word scanner and found nothing. No document I read says whether either existing scan would flag a Lazarus edit.

The framing the post points to

The post leans on Satya Nadella’s essay “Models as Insider Risks in the Super Intelligence Era,” which our earlier post audited. Two of its lines fit this paper better than the Foundry paragraph does: “a model provider’s assurances do not relieve us of that responsibility,” and “the controls that govern what a model can access and what actions it can take must sit outside the model.” The paper reaches the same place from the attack side: “the scale of the attack is bounded by the agent’s permissions, not the size of the edit.” OpenAI’s July incident is the paper’s opening example of agents causing real harm, and our audit of OpenAI’s later post on securing its training environment looks at what that kind of containment does and does not cover.

What a defender can run today

Everything in this section is mine rather than Microsoft’s, and the four scripts are in the last section. They use a small public test fixture on purpose: hf-internal-testing/tiny-random-LlamaForCausalLM, pinned to commit 9fb191250dd56d0ba7ec9785a025ed29c03d5998, a 4.1 MB, two-layer Llama whose feed-forward blocks have the same gate, up and down projections the paper edits. Its weights are random, so it is not a model anyone should run.

Check the file against the publisher’s digest

The Hub records a SHA-256 for every file it stores with Git LFS, which includes model weights. The huggingface_hub library exposes it as lfs.sha256 on the entries that list_repo_tree returns (see its reference page), and the REST tree endpoint serves the same value as lfs.oid. hub_check.py asks the endpoint for the digest at the pinned commit, downloads the file if it is missing, hashes it and compares:

repository: hf-internal-testing/tiny-random-LlamaForCausalLM at 9fb191250d
recorded:   49c20f32c6c597480fcaec5df2f86c645eabea765cbea1e67886dbae45e5c992  (4131280 bytes)
local file: 49c20f32c6c597480fcaec5df2f86c645eabea765cbea1e67886dbae45e5c992  (4131280 bytes)  data/base.safetensors
RESULT: identical to the publisher's file

That proves the local file is byte-identical to what that repository published at that commit, and nothing more. Pin the commit, because a branch name can move. A copy that fails is not the publisher’s file; here the check is run on the synthetic derivative made in the next step:

repository: hf-internal-testing/tiny-random-LlamaForCausalLM at 9fb191250d
recorded:   49c20f32c6c597480fcaec5df2f86c645eabea765cbea1e67886dbae45e5c992  (4131280 bytes)
local file: cca6b45bdb23693771d52b1f6e4b97cae02ce89be1bf677bdd40b16d7f14609f  (4131280 bytes)  data/dense_noise.safetensors
RESULT: MISMATCH, this is not the publisher's file

The digest cannot help with the case the post calls the remaining risk, a publisher whose own file is bad. It also cannot vouch for a legitimate derivative, because a fine-tune or a quantization will always differ from its base. For those you need a base to diff against.

Diff a derivative against its declared base

The paper’s advice to hubs and vendors is to “diff against a declared base” because “fine-tunes and quantizations change weights broadly” while “backdoored neurons change a few weight vectors by a lot.” The safetensors format makes a first-pass diff possible without any machine-learning library: an 8-byte little-endian header length, a JSON header listing each tensor’s dtype, shape and byte range, then one byte buffer.

make_variants.py writes four synthetic derivatives of the test file. Two are broad: dense noise worth 1 percent of each tensor’s spread, and a rounding of every weight to bfloat16. Two are narrow: three rows of one last-layer feed-forward matrix overwritten with values twenty times that matrix’s spread, and the same overwrite on top of the dense noise. None of them is a working model edit; they only change numbers. stdiff.py reports how many tensors changed, how many rows moved, and how many rows moved by at least their own length in the base file:

dense_noise      4131280 bytes  sha256 cca6b45bdb236937...
bf16_round       4131280 bytes  sha256 2a53161e557faf13...
sparse_rows      4131280 bytes  sha256 9ab217f2b12af180...
noise_plus_rows  4131280 bytes  sha256 861aed706765935e...

Against itself, then against the two broad derivatives:

data/base.safetensors against data/base.safetensors
tensors in base: 21; changed: 0; with rewritten rows: 0
  byte-identical tensors

data/dense_noise.safetensors against data/base.safetensors
tensors in base: 21; changed: 16; with rewritten rows: 0
rows moved: 64416 of 64416 in the changed tensors; median relative move 0.0100
  no row moved by as much as its own length

data/bf16_round.safetensors against data/base.safetensors
tensors in base: 21; changed: 16; with rewritten rows: 0
rows moved: 64415 of 64416 in the changed tensors; median relative move 0.0016
  no row moved by as much as its own length

Then against the two narrow ones:

data/sparse_rows.safetensors against data/base.safetensors
tensors in base: 21; changed: 1; with rewritten rows: 1
rows moved: 3 of 64 in the changed tensors; median relative move 28.9646
  model.layers.1.mlp.up_proj.weight            rows moved     3 of 64    rows rewritten  3  largest relative move    46.169

data/noise_plus_rows.safetensors against data/base.safetensors
tensors in base: 21; changed: 16; with rewritten rows: 1
rows moved: 64416 of 64416 in the changed tensors; median relative move 0.0100
  model.layers.1.mlp.up_proj.weight            rows moved    64 of 64    rows rewritten  3  largest relative move    28.589

Read the shape of the change rather than a verdict. The broad changes touch 16 of 21 tensors and nearly every row, by a median of about 1 percent of the row’s length for the dense noise and 0.16 percent for the bfloat16 rounding. The five tensors that do not move are the layer-norm weights, which are all ones. The narrow change touches one tensor and 3 of its 64 rows, by a median of 29 times their own length (the largest is 46). The last file shows the useful part: the three rewritten rows still stand out inside a dense change, because the statistic is per row and relative to the row’s own length, not a count of changed tensors.

The limits, all mine

• The model is tiny and random and the changes are synthetic. A real fine-tune will not be this uniform, so expect false alarms; the paper’s own detector on real checkpoints flagged 3 of 31 clean ones.
• The threshold of 1.0 is a number I chose for the demo. An edit smaller than its row, or spread across many rows, would not trip it.
• The diff trusts the declared base. If the base is the attacker’s, the diff is clean, so verify the base against its own publisher’s digest first.
• Neither check says anything about whether a clean-looking model is safe, which is the paper’s own point about audits being “evidence, not guarantees.”

Keep the damage bounded outside the model

The paper’s recommendations for users are to know where models come from, limit agent permissions, “log commands, file changes, and network calls,” and “require approval before generated commands run, files are changed, or data is sent externally.” Our additions, untested here: give an agent’s shell no standing credentials, allow only the network destinations a task needs so that a download-and-run command has nowhere to download from, and treat a command that fetches and executes code as an approval event regardless of which model proposed it. Those controls work the same whether the weights were edited or the agent was merely prompt-injected.

What would settle the open questions

• The repository and evaluation archive: whether the detector’s 91.4 percent holds when a whole base model or trigger type is held out of training, which the 114 sibling checkpoints cannot show.
• Free-running tests beyond Nemotron-Terminal-8B, with the payload executed in a sandbox rather than counted in text.
• Detection across other edit families, and against an attacker who knows the features, which the researcher’s own November note allows for.
• Whether Microsoft’s existing model-integrity scan and its February backdoor scanner flag a Lazarus edit.
• Per-checkpoint verdicts for the two frontier-model auditors, so a gate built on them can be evaluated.

Until then, the supported reading is the narrower one. On one 8B model, seven edited neurons made a marker string appear on recorded trajectories about 99 percent of the time. On a 27B model, six neurons managed 51 percent with the same 23 samples and 84 percent with 2,393. A 60-neuron patch for a real deletion command in 15 agent formats reached 26 and 75 percent on two formats it was fitted on and 40 percent on one it was not, again on fixed contexts. A detector built around the attack’s own structure found most edits from the same family. The reading the tables do not reach is the one in the post’s first paragraph, that a date can make a live agent on any model run an attacker’s code without tests noticing. The advice that follows from either reading is the same, and it is not about neurons: verify where the weights came from, and keep what an agent can do small enough that a compromised model is a bounded problem.

The four scripts

All four need only the Python standard library. I ran them on Python 3.13.14 on Windows 11. Save each under its name and run python lazarus_check.py (no network), python hub_check.py data/base.safetensors (it downloads the 4.1 MB file the first time), python make_variants.py data/base.safetensors data, then python hub_check.py data/dense_noise.safetensors and python stdiff.py data/base.safetensors data/NAME.safetensors for each derivative. The first script holds every table figure typed in from the paper, so those numbers can be changed and rechecked; the Figure 7 readout is mine and is not in it.

lazarus_check.py

# lazarus_check.py
"""Recompute what the Lazarus paper's tables imply. Python 3.13, standard library only.
Every input below is typed in from the paper (Microsoft Research, "Lazare, veni foras"), table by table."""
from math import comb, sqrt


def wilson(k, n, z=1.959964):
    """95 percent Wilson score interval for k successes in n trials."""
    p = k / n
    d = 1 + z * z / n
    centre = (p + z * z / (2 * n)) / d
    half = z * sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d
    return max(0.0, centre - half), min(1.0, centre + half)


def zero_upper(n, conf=0.95):
    """Exact one-sided upper bound on the rate when 0 events were seen in n trials."""
    return 1 - (1 - conf) ** (1 / n)


print("##### detector")
# Table 6: test split = 35 edited and 31 clean checkpoints; rows are (name, TPR %, FPR %).
EDITED, CLEAN = 35, 31
rows = [
    ("Gate and up alignment", 77.14, 12.90),
    ("Token alignment", 77.14, 9.68),
    ("Write magnitude", 71.43, 16.13),
    ("All 3 construction-based", 80.00, 9.68),
    ("Three-projection sum", 82.86, 45.16),
    ("Rank agreement", 85.71, 32.26),
    ("Both 2 LLM-inspired", 91.43, 32.26),
    ("All 5 features", 91.43, 9.68),
]
print(f"{'feature set':<26}{'caught':>8}{'95% CI':>14}{'false alarms':>15}{'95% CI':>14}")
for name, tpr, fpr in rows:
    tp, fp = round(tpr / 100 * EDITED), round(fpr / 100 * CLEAN)
    assert abs(tp / EDITED * 100 - tpr) < 0.006 and abs(fp / CLEAN * 100 - fpr) < 0.006, name
    lo, hi = wilson(tp, EDITED)
    flo, fhi = wilson(fp, CLEAN)
    print(f"{name:<26}{tp:>5}/{EDITED:<2}{lo * 100:>7.1f}-{hi * 100:<5.1f}{fp:>9}/{CLEAN:<3}{flo * 100:>7.1f}-{fhi * 100:<5.1f}")

print()
print("##### roster")
settings = 3 * 2                    # 3 base models x 2 trigger types
per_setting = 1 + 18                # 1 analytic edit + 18 gradient-refined edits from the sweep
edited_total = settings * per_setting
clean_total = 103
train_edited, train_clean = edited_total - EDITED, clean_total - CLEAN
print(f"edited checkpoints: {settings} settings x {per_setting} = {edited_total}")
print(f"split: train {train_edited} edited + {train_clean} clean = {train_edited + train_clean}; "
      f"test {EDITED} edited + {CLEAN} clean = {EDITED + CLEAN}")
print(f"edited siblings of each setting in the training half, if spread evenly: {train_edited / settings:.1f}")

print()
print("##### auditors")
# Table 5: counts implied by the printed percentages (114 edited, 103 clean checkpoints).
# Each row is (flagged edited, called clean, no decision).
audits = {
    "GPT-6 Astra, edited": (83, 0, 31, 114),
    "GPT-6 Astra, clean": (1, 0, 102, 103),
    "Claude Opus 5.5, edited": (64, 47, 3, 114),
    "Claude Opus 5.5, clean": (2, 96, 5, 103),
}
for name, (a, b, c, n) in audits.items():
    assert a + b + c == n, name
    print(f"{name:<24} flagged {a:>3} ({100 * a / n:5.2f}%)  called clean {b:>3} ({100 * b / n:5.2f}%)  "
          f"no decision {c:>3} ({100 * c / n:5.2f}%)  of {n}")
print("Opus 5.5 waved through 47 of 114 edited checkpoints; GPT-6 Astra called 0 of 103 clean checkpoints clean")

print()
print("##### leakage")
for label, n in (("observation trigger", 2169), ("query trigger", 2104)):
    print(f"0 leaks in {n} benign trajectories ({label}): 95% upper bound {100 * zero_upper(n):.3f}%")
hits = [k for k in range(0, 12) if round(k / 2169, 4) == 0.0014]
print(f"Table 14 observation leakage 0.0014 of 2169 benign trajectories = {hits[0]} trajectories")
lo, hi = wilson(hits[0], 2169)
print(f"  95% interval for {hits[0]}/2169: {100 * lo:.2f}%-{100 * hi:.2f}%")

print()
print("##### asr")
# Table 2: Lazarus-SGD attack success rate, rows are (2-step query, 8-step query, 2-step obs, 8-step obs).
sgd = {
    "Qwen3.6-27B": [0.7745, 0.6157, 0.8368, 0.5556],
    "Nemotron-Terminal-8B": [1.0000, 1.0000, 0.9970, 0.9861],
    "Gemma-4-31B-it": [0.6171, 0.6189, 0.9951, 0.6220],
}
allv = []
for model, v in sgd.items():
    allv += v
    s = sorted(v)
    print(f"{model:<22} min {s[0]:.4f}  max {s[-1]:.4f}  cells >= 0.98: {sum(x >= 0.98 for x in v)} of 4")
allv.sort()
median = (allv[5] + allv[6]) / 2
print(f"all 12 gradient-refined cells: min {allv[0]:.4f}, median {median:.4f}, max {allv[-1]:.4f}; "
      f"{sum(x >= 0.98 for x in allv)} of 12 at or above 0.98, {sum(x < 0.85 for x in allv)} of 12 below 0.85")
# Table 1: observation trigger, two steps, gradient-refined, 23 training samples.
print("23 training samples: Nemotron-Terminal-8B 98.8% with 7 neurons; Qwen3.6-27B 51.0% with 6 neurons")
print(f"Nemotron-Terminal-8B benign KL: 0.0180 at 23 samples, 0.0022 at 2,405 samples ({0.0180 / 0.0022:.1f} times higher)")
n_trig = 1009                       # triggered trajectories in the Nemotron observation evaluation (Figure 8 caption)
k = round(0.988 * n_trig)
lo, hi = wilson(k, n_trig)
print(f"98.8% of {n_trig} is about {k} trajectories, 95% interval {100 * lo:.1f}%-{100 * hi:.1f}%")

print()
print("##### formats")
# Table 4 and Appendix C.4: Qwen3.6-27B, a decoy deletion command wrapped in 15 agent formats, one shared patch of
# 60 edited neurons, teacher forcing. Rows are (agent format, ASR, triggered test trajectories).
for agent, asr, triggered in (("terminus-2 (seen)", 0.2615, 130), ("mini-swe-agent (seen)", 0.7481, 131),
                              ("swe-agent (unseen)", 0.4000, 55)):
    k = round(asr * triggered)
    assert abs(k / triggered - asr) < 0.00006, agent
    print(f"{agent:<22} {asr:.4f} = {k} of {triggered} triggered trajectories")

print()
print("##### kl")
# Table 2 again, same cell order. Lazarus-Analytic is the closed-form edit; Lazarus-SGD refines it with gradient steps.
closed_asr = [0.6719, 0.8719, 0.9968, 0.8969, 0.9860, 0.9879, 0.9822, 0.9968, 0.7955, 0.8126, 0.2249, 0.0302]
closed_kl = [0.0243, 0.0030, 0.2853, 0.0499, 0.2625, 0.2737, 2.8211, 6.9097, 0.1553, 0.0165, 0.0043, 0.1213]
sgd_kl = [0.0021, 0.0000, 0.0024, 0.0002, 0.0011, 0.0009, 0.0022, 0.0018, 0.0017, 0.0005, 0.0000, 0.0030]
print(f"closed-form edit: benign KL {min(closed_kl):.4f} to {max(closed_kl):.4f}, "
      f"above 0.1 in {sum(x > 0.1 for x in closed_kl)} of 12 cells; attack success {min(closed_asr):.4f} to {max(closed_asr):.4f}")
print(f"gradient-refined: benign KL {min(sgd_kl):.4f} to {max(sgd_kl):.4f}; every cell at or below 0.0030: {all(x <= 0.0030 for x in sgd_kl)}")

print()
print("##### evals")
# Table 3: Terminal-Bench 2.1 pass@1 over 88 tasks, one attempt per task.
n, clean_pass, edited_pass = 88, 41, 37
print(f"Terminal-Bench 2.1: clean {clean_pass}/{n} = {clean_pass / n:.4f}, edited {edited_pass}/{n} = {edited_pass / n:.4f}")
total = clean_pass + edited_pass
probs = [comb(n, i) * comb(n, total - i) / comb(2 * n, total) for i in range(0, min(n, total) + 1) if 0 <= total - i <= n]
p_obs = comb(n, clean_pass) * comb(n, edited_pass) / comb(2 * n, total)
fisher = sum(p for p in probs if p <= p_obs * (1 + 1e-9))
print(f"Fisher exact two-sided p for {clean_pass}/{n} versus {edited_pass}/{n}: {fisher:.2f}")
rate = total / (2 * n)
pmf = [comb(n, i) * rate ** i * (1 - rate) ** (n - i) for i in range(n + 1)]
gap4 = sum(pmf[x] * pmf[y] for x in range(n + 1) for y in range(n + 1) if abs(x - y) >= 4)
print(f"two independent runs of one {100 * rate:.1f}% model differ by 4+ tasks with probability {gap4:.2f}")
mde = (1.959964 + 0.841621) * sqrt(2 * rate * (1 - rate) / n)
print(f"gap detectable at 80% power with {n} tasks per arm: about {100 * mde:.0f} points (unpaired)")
print("AIME pass@8 on 30 problems: 23, 22, 25, 24, 23 of 30; one problem is "
      f"{100 / 30:.1f} points")

hub_check.py

# hub_check.py
"""Compare a local model file with the digest its publisher's repository records. Python 3.13, standard library only.
Target: Hugging Face Hub repository hf-internal-testing/tiny-random-LlamaForCausalLM, pinned to one commit.
usage: python hub_check.py [path]     (downloads the original to data/base.safetensors when the path is missing)"""
import hashlib
import json
import os
import sys
import urllib.request

REPO = "hf-internal-testing/tiny-random-LlamaForCausalLM"
REV = "9fb191250dd56d0ba7ec9785a025ed29c03d5998"
NAME = "model.safetensors"
HEADERS = {"User-Agent": "model-integrity-check/1.0"}


def fetch(url):
    with urllib.request.urlopen(urllib.request.Request(url, headers=HEADERS), timeout=60) as response:
        return response.read()


def sha256_of(path):
    digest = hashlib.sha256()
    with open(path, "rb") as handle:
        for chunk in iter(lambda: handle.read(1 << 20), b""):
            digest.update(chunk)
    return digest.hexdigest()


path = sys.argv[1] if len(sys.argv) > 1 else "data/base.safetensors"
tree = json.loads(fetch(f"https://huggingface.co/api/models/{REPO}/tree/{REV}"))
entry = next(item for item in tree if item["path"] == NAME)
recorded, size = entry["lfs"]["oid"], entry["size"]
if not os.path.exists(path):
    os.makedirs(os.path.dirname(path) or ".", exist_ok=True)
    with open(path, "wb") as handle:
        handle.write(fetch(f"https://huggingface.co/{REPO}/resolve/{REV}/{NAME}"))
actual = sha256_of(path)
print(f"repository: {REPO} at {REV[:10]}")
print(f"recorded:   {recorded}  ({size} bytes)")
print(f"local file: {actual}  ({os.path.getsize(path)} bytes)  {path}")
print("RESULT: identical to the publisher's file" if actual == recorded else "RESULT: MISMATCH, this is not the publisher's file")

make_variants.py

# make_variants.py
"""Write four synthetic derivatives of a .safetensors file so that stdiff.py has something to find.
Python 3.13, standard library only. None of them is a working model edit: they only change numbers.
usage: python make_variants.py BASE OUTDIR"""
import hashlib
import json
import os
import random
import struct
import sys

base_path, out_dir = sys.argv[1], sys.argv[2]
os.makedirs(out_dir, exist_ok=True)
blob = open(base_path, "rb").read()
(hlen,) = struct.unpack("<Q", blob[:8])
header = json.loads(blob[8:8 + hlen])
header.pop("__metadata__", None)
start = 8 + hlen
TARGET = "model.layers.1.mlp.up_proj.weight"   # one MLP matrix of the last layer in this 2-layer model
ROWS = (5, 31, 47)                              # rows overwritten by the sparse variants


def tensor_floats(buf, name):
    b, e = header[name]["data_offsets"]
    return list(struct.unpack(f"<{(e - b) // 4}f", buf[start + b:start + e]))


def put(buf, name, values):
    b, e = header[name]["data_offsets"]
    buf[start + b:start + e] = struct.pack(f"<{len(values)}f", *values)


def std(values):
    mean = sum(values) / len(values)
    return (sum((v - mean) ** 2 for v in values) / len(values)) ** 0.5


def dense_noise(buf, rng):
    """Every float tensor nudged by noise worth 1 percent of its own spread, as a light fine-tune might."""
    for name, info in header.items():
        if info["dtype"] == "F32":
            values = tensor_floats(buf, name)
            sigma = 0.01 * std(values)
            put(buf, name, [v + rng.gauss(0, sigma) for v in values])


def bf16_round(buf):
    """Round every float to the nearest bfloat16 value, a precision loss like a quantization pass."""
    for name, info in header.items():
        if info["dtype"] == "F32":
            b, e = info["data_offsets"]
            words = list(struct.unpack(f"<{(e - b) // 4}I", buf[start + b:start + e]))
            words = [(w + 0x7FFF + ((w >> 16) & 1)) & 0xFFFF0000 for w in words]
            buf[start + b:start + e] = struct.pack(f"<{len(words)}I", *words)


def sparse_rows(buf, rng):
    """Overwrite a few rows of one matrix with values twenty times the matrix's usual spread."""
    values = tensor_floats(buf, TARGET)
    width = header[TARGET]["shape"][1]
    sigma = 20 * std(values)
    for r in ROWS:
        values[r * width:(r + 1) * width] = [rng.gauss(0, sigma) for _ in range(width)]
    put(buf, TARGET, values)


variants = {}
buf = bytearray(blob)
dense_noise(buf, random.Random(1))
variants["dense_noise"] = bytes(buf)
buf = bytearray(blob)
bf16_round(buf)
variants["bf16_round"] = bytes(buf)
buf = bytearray(blob)
sparse_rows(buf, random.Random(2))
variants["sparse_rows"] = bytes(buf)
buf = bytearray(variants["dense_noise"])
sparse_rows(buf, random.Random(3))
variants["noise_plus_rows"] = bytes(buf)
for name, data in variants.items():
    path = os.path.join(out_dir, f"{name}.safetensors")
    with open(path, "wb") as handle:
        handle.write(data)
    print(f"{name:<16} {len(data)} bytes  sha256 {hashlib.sha256(data).hexdigest()[:16]}...")

stdiff.py

# stdiff.py
"""Compare two .safetensors files tensor by tensor. Python 3.13, standard library only.
usage: python stdiff.py BASE CANDIDATE [--all]
A tensor is one named weight matrix. A row is one output neuron of that matrix.
The BIG threshold below is a number chosen for this demo, not a published detection rule."""
import json
import math
import struct
import sys
from statistics import median

BIG = 1.0   # a row counts as rewritten when it moved by at least its own length in the base file


def read_header(path):
    with open(path, "rb") as handle:
        (length,) = struct.unpack("<Q", handle.read(8))
        header = json.loads(handle.read(length))
    header.pop("__metadata__", None)
    return header, 8 + length


def raw(path, info, start):
    begin, end = info["data_offsets"]
    with open(path, "rb") as handle:
        handle.seek(start + begin)
        return handle.read(end - begin)


def floats(dtype, data):
    if dtype == "F32":
        return struct.unpack(f"<{len(data) // 4}f", data)
    if dtype == "F16":
        return struct.unpack(f"<{len(data) // 2}e", data)
    if dtype == "BF16":
        widened = b"".join(b"\x00\x00" + data[i:i + 2] for i in range(0, len(data), 2))
        return struct.unpack(f"<{len(data) // 2}f", widened)
    raise ValueError(f"unsupported dtype {dtype}")


base_path, cand_path = sys.argv[1], sys.argv[2]
show_all = "--all" in sys.argv
base_h, base_start = read_header(base_path)
cand_h, cand_start = read_header(cand_path)
problems = []
for name in sorted(set(base_h) | set(cand_h)):
    if name not in cand_h:
        problems.append(f"missing from candidate: {name}")
    elif name not in base_h:
        problems.append(f"not in base: {name}")
    elif (base_h[name]["dtype"], base_h[name]["shape"]) != (cand_h[name]["dtype"], cand_h[name]["shape"]):
        problems.append(f"dtype or shape differs: {name}")
changed, flagged, lines = 0, 0, []
rows_seen, rows_moved, moves = 0, 0, []
for name in sorted(set(base_h) & set(cand_h)):
    info = base_h[name]
    a, b = raw(base_path, info, base_start), raw(cand_path, cand_h[name], cand_start)
    if a == b:
        continue
    changed += 1
    rows = info["shape"][0] if len(info["shape"]) > 1 else 1
    fa, fb = floats(info["dtype"], a), floats(info["dtype"], b)
    width = len(fa) // rows
    moved, big, worst = 0, 0, 0.0
    for r in range(rows):
        sa, sb = fa[r * width:(r + 1) * width], fb[r * width:(r + 1) * width]
        delta = math.sqrt(sum((x - y) ** 2 for x, y in zip(sa, sb)))
        length = math.sqrt(sum(x * x for x in sa)) or 1.0
        if delta > 0:
            moved += 1
            moves.append(delta / length)
            worst = max(worst, delta / length)
            big += delta / length >= BIG
    rows_seen += rows
    rows_moved += moved
    flagged += bool(big)
    if show_all or big:
        lines.append(f"  {name:<44} rows moved {moved:>5} of {rows:<5} rows rewritten {big:>2}  largest relative move {worst:9.3f}")
print(f"{cand_path} against {base_path}")
print(f"tensors in base: {len(base_h)}; changed: {changed}; with rewritten rows: {flagged}")
if moves:
    print(f"rows moved: {rows_moved} of {rows_seen} in the changed tensors; median relative move {median(moves):.4f}")
for problem in problems:
    print("  " + problem)
if not changed and not problems:
    print("  byte-identical tensors")
elif not lines:
    print("  no row moved by as much as its own length")
for line in lines:
    print(line)