TRENDING
A black manual telephone switchboard with rows of orange jacks and red patch cords plugged into the lower row, above a rotary dial and a handset
November 16, 2026
How to Route Alerts in Alertmanager So Pages Reach the On-Call and Tickets Reach the Queue, and Test Grouping, Inhibition, and Silences Before an Outage Does
A brass nail with a threaded screw section lying on graph paper, a fastener that is both nail and screw
November 16, 2026
Ai2’s XYBench Turns the XY Problem Into a Benchmark, but the Model That Drafted Every Rubric Also Judges the Answers, the Human Baseline Is Graded on Rubrics Read Off Its Own Answers, and the Emphasis Judge Disagrees With the Annotators on 20 to 26 Percent of Validation Responses
An amber warning beacon on a white surface, a flashing-light stand-in for the FBI’s FLASH alert
November 16, 2026
The FBI’s ARTEX Warning Says a Supply Chain Compromise “Enabled Access” to 2,250 Tenants Where Microsoft Counted Sign-In Attempts and Cites “Over 1,600” Consoles Where Censys Counted 1,465, and a Microsoft Post It Cites Mistypes One Indicator in Its Hunting Query
Two eggs and a colour-changing egg timer in a saucepan of water on an induction hob
November 16, 2026
How to Set gRPC Deadlines and Retries in Python, Pass Them Down a Call Chain, and Stop Retries From Charging a Card Twice
A whole lemon and a halved lemon side by side on a black background
November 16, 2026
IBM’s LLMON Turns Prompt-Injection Defense Into a Token Layout, but the “Nearly Zero” Chart Is One Benchmark, the Paper’s Own Red Team Lets 15% to 20% of Indirect Attacks Through, and the Shipped Template Lets a Document Type the Delimiters
16 Nov 2026
SXZ.io SXZ.io
  • Home
Search the Site
Popular Searches:
Technology Amazon AI
Recent Posts
A manual paper cutter with a wooden grid board, a clear plastic guard and a long cutting arm, a picture of text being trimmed to fit
How to Stop Ollama From Silently Cutting Your Prompt, Find the Real Context Window, and Set num_ctx Without Running Out of Memory
November 16, 2026
A brown llama and her gray cria walking across a salt flat beside a red lake in Bolivia
llama.cpp’s v1.0.0 Promises Major Bumps for API and ABI Breaks, but Its Rule Table Lists Only the C API, libmtmd Shares the Version While Its Header Warns of Breaking Changes, and the Release Page Carries a 7-Byte Pointer to a Nightly Build Instead of Binaries
November 16, 2026
A stack of chocolate chip cookies on a white plate beside a cup of tea
NoisyTCP Can Crash a Linux Server Over the Network, but the Fix Has Reached Only Three of the Seven Kernel Lines Still Being Released, Red Hat and Ubuntu Have No Update, and the Researcher’s Own Timings Run From 5 Seconds to 14 Hours
November 16, 2026
SXZ.io SXZ.io
  • Home

Categories

Articles 302 Posts
News 307 Posts
Learning Hub 274 Posts
Home/Articles/A Report Turns Model Weights Into a GPU Registry, and It Reproduces in Four Seconds, but the Fingerprint Dates the Starting Weights, Not the Training, and the X Post’s “Conclusive” Covers Six Models Where the Report Claims It for One
Articles

A Report Turns Model Weights Into a GPU Registry, and It Reproduces in Four Seconds, but the Fingerprint Dates the Starting Weights, Not the Training, and the X Post’s “Conclusive” Covers Six Models Where the Report Claims It for One

A report says published weights carry the fingerprint of the GPU that created them; its public code reproduces in seconds and an RTX A400 confirms the mechanism, but the evidence dates the starting...

November 14, 2026 19 Min Read
36

On November 12 Will DePue, a former OpenAI researcher who now leads Unemployment Labs, published a report titled “Top Chinese Frontier AI Models Carry the Fingerprint of Export-Controlled NVIDIA Blackwell GPUs” and announced it on X at 21:29 UTC. The Register and TechCrunch ran it on November 13. The idea is unusual and testable: a model’s published weights keep a trace of how many streaming multiprocessors (SMs) the GPU had when PyTorch generated the starting values, so the weights can name the GPU class that created them. DePue reads six Chinese models released since April as Blackwell-initialized and earlier ones as Hopper-initialized.

Table Of Content

  • What the report claims, and how each retelling words it
  • How the fingerprint works
  • Reproducing the report
  • The shipped rows
  • Scanning rows the report did not ship
  • A control that fooled me first
  • A GPU the report never used
  • What 8 sigma and 1,385 sigma measure
  • What the evidence does not settle
  • Initialization is not training
  • Two of the six Blackwell models are outside the public package
  • AMD, and the cost table
  • What I did not test
  • What would settle it
  • Sources and method

The report ships its code, so this post does not have to take the method on trust. I ran that code at its only commit, scanned hundreds of rows it did not ship, ran the method’s core assumption on an NVIDIA RTX A400 that appears nowhere in the report, and recomputed the statistics and the cost table. The mechanism holds up. The headline claims travel further than the evidence does, and the A400 shows one constant in the public code that is not universal.

  • The public code reproduces all five models it covers in about four seconds on a CPU. A scan of 1,280 rows of DeepSeek-V3-Base found 476 untouched rows that match the Hopper layout where the report shipped 8, and no row matches under a wrong layout or a shifted seed.
  • On the A400, the report’s CPU model reproduces the real GPU output (r = 1.000000) only with 1,536 threads per SM, the value PyTorch’s source computes for that card, not the 2,048 the public code hard-codes.
  • The fingerprint dates where the starting weights were generated. The report says it supports, without proving, training on the same hardware. The X post says “trained”, and The Register’s headline says “training”.
  • The post calls the evidence conclusive for six models. The report calls it strong for six and conclusive for one, DeepSeek-V4-Flash. Two of the six, DeepSeek-V4-Pro and Kimi K3.1-Pro, are not in the public package.
  • The sigma figures are the correlation times the square root of the number of values compared. The 8 sigma chance benchmark is sound. The run-size table rests on assumptions, and the X post still carries the first version’s GPU counts.

What the report claims, and how each retelling words it

The report’s Table 1 is headed “Chip the model was started on”, and its summary says the fingerprint “identifies how the weights were created and supports, without proving, the inference that Blackwell also performed the training.” The retellings move that line in small steps.

Question The report (Nov 12) The X post (Nov 12, 21:29 UTC) The Register (Nov 13) TechCrunch (Nov 13)
What the fingerprint dates The chip the model “was started on”; training on it is an inference “a hidden signature of the GPU that trained it” Headline: “Chinese AI labs are training models on Nvidia’s Blackwell despite export controls” Lede: models “were trained on Nvidia chips that the U.S. government restricted”; later, “That doesn’t mean that these models were trained entirely on Nvidia chips”
How strong “strong evidence that six Chinese AI models were initialized on export-controlled NVIDIA hardware, and that for DeepSeek-V4-Flash the evidence is conclusive” “conclusive evidence that six Chinese AI models” were initialized on Blackwell systems Quotes the post: “conclusive evidence that six Chinese AI models” “compelling evidence”
Who checked it No reviewer named “independently verified by senior researchers at OpenAI and Anthropic” Passes the claim on as DePue’s; no reviewer named No reviewer named

I could not check the reviewer claim, and none of the three texts names a reviewer. The code arrived later than the post: GitHub’s API dates the repository’s creation to 05:39 UTC on November 13 and its single commit, “initial release”, to 05:52 UTC, about eight hours after the X post. The report page I read today links the code.

How the fingerprint works

PyTorch fills a large CUDA tensor in normal_ with a grid of threads. Each thread gets its own stream of the Philox random number generator, keyed by the seed and the thread index, and writes four values per call, spaced one grid apart. The grid size comes from calc_execution_policy in DistributionTemplates.h, which I read on November 14: a block is block_size_bound = 256 threads, blocks_per_sm is maxThreadsPerMultiProcessor / block_size, and grid.x is the smaller of multiProcessorCount * blocks_per_sm and the number of blocks the tensor needs. For a large tensor the thread count G is therefore the SM count times the maximum threads per SM, and element i comes from thread i mod G.

The same seed gives the same numbers on any GPU, but they land in different positions when G differs. The first G elements of a tensor are identical under every layout, so only positions beyond that can tell layouts apart. The report’s table of layouts, with the A100 row from the repository’s README:

GPU class SMs Threads (G = 2,048 x SMs)
A100 / A800 108 221,184
H100 / H800 / H200 132 270,336
B200 / B300 148 303,104
GB200 / GB300 152 311,296

The trace survives in the weights that training barely touched. Embedding rows for tokens that never appear in training get no gradient, so they keep their starting values. Trained weights keep a weaker trace, because a final weight is its starting value plus the updates. The report’s code, gpuid.py, is a NumPy model of Philox4x32-10 (checked against the Random123 known-answer vectors on import) and of curand_normal4. It rebuilds a row for a candidate seed and layout and correlates it with the published row.

Reproducing the report

The shipped rows

The repository carries the embedding rows it uses, hash-checked against models.json. On this Windows 11 machine, with Python 3.13.14, NumPy 2.5.4 and SciPy 1.18.1, the replay took about four seconds:

git clone https://github.com/macrogrants/checkpoint-forensics
cd checkpoint-forensics
git checkout b633870c4a8b0f85d70026b7f77f7d0858358f89
pip install numpy scipy
python replay.py
model                     seed   SMs      rows  median r   seed+1     next best  gpu
DeepSeek-V3-Base     953049583   132       8/8  0.999999  -0.0038   183 (0.016)  Hopper
DeepSeek-V4-Flash    953049583   148   461/469  0.999999   0.0016   209 (0.023)  Blackwell B200/B300
Kimi-K3               20256586   152     13/13  0.999996   0.0042    69 (0.018)  Blackwell GB200/GB300
MiniMax-M3                2760   148     17/17  0.999990   0.0017    87 (0.018)  Blackwell B200/B300
Ling-3.1-flash-base       3952   152   384/384  0.983436   0.0004   192 (0.025)  Blackwell GB200/GB300

All five rows of the README’s table reproduce: 8 of 8, 461 of 469, 13 of 13, 17 of 17 and 384 of 384 rows, with seed + 1 near zero and the runner-up layout at 0.016 to 0.025. Ling-3.1-flash is the soft one: its median correlation is 0.983 because the repository’s model of the bf16 path for trunc_normal_ is approximate, as its README says.

Scanning rows the report did not ship

The report lists 8 rows for DeepSeek-V3-Base. That could be the tip of a block or a hand-picked few, so I read a contiguous block from the pinned Hugging Face revision (HTTP range requests on the safetensors shard, one of them for the whole block) and tested every row against the report’s layout and seed. The script runs from the repository folder and reuses its draw, corr and get functions.

# scan.py MODEL LO HI  (run from the checkpoint-forensics folder, after pip install numpy scipy)
# Reads embedding rows LO..HI-1 of one model from its pinned Hugging Face revision (one HTTP range request),
# tests every row against the report's CPU model of PyTorch's normal_ at the report's seed and SM count,
# then repeats with the other two layouts and with seed + 1 as controls.
import json, struct, sys, time
import numpy as np
from gpuid import draw, corr
from fetch import get

name, LO, HI = sys.argv[1], int(sys.argv[2]), int(sys.argv[3])
m = [x for x in json.load(open("models.json")) if x["name"] == name][0]
d = m["shape"][1]
base = f"https://huggingface.co/{m['repo']}/resolve/{m['revision']}/"
shard = base + json.loads(get(base + "model.safetensors.index.json"))["weight_map"][m["tensor"]]
n = struct.unpack("<Q", get(shard, 0, 7))[0]
info = json.loads(get(shard, 8, 7 + n))[m["tensor"]]
assert info["dtype"] == "BF16" and info["shape"] == m["shape"]
at = 8 + n + info["data_offsets"][0]
raw = get(shard, at + LO * d * 2, at + HI * d * 2 - 1)
Y = (np.frombuffer(raw, "<u2").reshape(HI - LO, d).astype(np.uint32) << 16).view(np.float32).astype(np.float64)


def rvec(sms, seed):
    out = np.empty(HI - LO)
    for i, r in enumerate(range(LO, HI)):
        pos = np.uint64(r * d) + np.arange(d, dtype=np.uint64)
        out[i] = corr(Y[i], draw(pos, sms * 2048, seed, m.get("cpu_offset", 0), m["dist"]))
    return out


def ranges(ids):
    out, s, p = [], None, None
    for x in ids:
        if s is None:
            s = p = x
        elif x == p + 1:
            p = x
        else:
            out.append((s, p)); s = p = x
    if s is not None:
        out.append((s, p))
    return ", ".join(f"{a}-{b}" if a != b else str(a) for a, b in out)


sms, seed = m["sms"], m["seed"]
r = rvec(sms, seed)
hit = np.where(r > 0.99)[0]
rest = np.delete(r, hit)
print(f"{name}: rows {LO}..{HI - 1} read from {m['repo']}@{m['revision'][:8]} ({len(raw) / 1e6:.1f} MB)")
print(f"  layout {sms} SMs, seed {seed}: {len(hit)} rows with r > 0.99; ids {ranges((hit + LO).tolist())}")
print(f"  lowest r among those {len(hit)} rows: {r[hit].min():.6f}")
print(f"  report's rows inside my set: {sum(1 for x in m['rows'] if LO <= x < HI and (x - LO) in set(hit.tolist()))} of {sum(1 for x in m['rows'] if LO <= x < HI)} in range")
print(f"  other {len(rest)} rows: median r {np.median(rest):.3f}, min {rest.min():.3f}, max {rest.max():.3f}; median z = r*sqrt(d) = {np.median(rest) * np.sqrt(d):.1f}")
others = [s for s in (132, 148, 152) if s != sms]
# the first G elements of a tensor are identical under every layout, so controls only use rows that start
# beyond the largest of the three G values (152 * 2048 = 311,296 elements)
keep = (np.arange(LO, HI) * d >= 152 * 2048)
print(f"  controls use {keep.sum()} of {HI - LO} rows (rows starting beyond element 311,296; earlier ones are layout-independent)")
for s2, sd2, label in [(o, seed, f"{o} SMs") for o in others] + [(sms, seed + 1, "seed + 1")]:
    rr = rvec(s2, sd2)[keep]
    z = rr * np.sqrt(d)
    print(f"  control {label}: max |r| {np.abs(rr).max():.4f} (= {np.abs(z).max():.1f} standard errors over {len(rr)} rows), rows above 0.5: {(rr > 0.5).sum()}; sd of z {z.std():.2f}")

For DeepSeek-V3-Base, run with python scan.py DeepSeek-V3-Base 128000 129280:

DeepSeek-V3-Base: rows 128000..129279 read from deepseek-ai/DeepSeek-V3-Base@afb92e1f (18.4 MB)
  layout 132 SMs, seed 953049583: 476 rows with r > 0.99; ids 128803-128804, 128806-129279
  lowest r among those 476 rows: 0.999999
  report's rows inside my set: 8 of 8 in range
  other 804 rows: median r 0.168, min 0.132, max 0.810; median z = r*sqrt(d) = 14.2
  controls use 1280 of 1280 rows (rows starting beyond element 311,296; earlier ones are layout-independent)
  control 148 SMs: max |r| 0.0427 (= 3.6 standard errors over 1280 rows), rows above 0.5: 0; sd of z 1.00
  control 152 SMs: max |r| 0.0474 (= 4.0 standard errors over 1280 rows), rows above 0.5: 0; sd of z 1.01
  control seed + 1: max |r| 0.0377 (= 3.2 standard errors over 1280 rows), rows above 0.5: 0; sd of z 0.97

The untouched block is the end of the embedding matrix: 476 of the 477 rows from ID 128,803 up are copies of the Hopper-layout values (the lowest r among them is 0.999999), 3.4 million values in all against the 57,344 the report shipped. The other 804 rows include trained tokens and still correlate at a median r of 0.168. The init standard deviation in models.json is 0.006 and a trained row I checked has 0.029, so independent updates would leave r near 0.2 (my back-of-envelope figure), close to what the scan shows. The table adds the other scans I ran with the same script, and its first row is the run printed above. In the last column the first figure is the spread of z (r times the square root of the row length) across the three controls, two wrong layouts and seed + 1, which is 1.0 for a pure chance match, and the second is the largest control score in standard errors:

Model, rows read Rows with r above 0.99 Report’s rows in range found Median r of the other rows Controls: sd of z, largest
DeepSeek-V3-Base, IDs 128,000 to 129,279 476 8 of 8 0.168 0.97 to 1.01; 4.0 standard errors
DeepSeek-V4-Flash, IDs 128,000 to 129,279 448 448 of 455 0.034 0.98 to 1.01; 3.5
DeepSeek-V4-Flash, IDs 0 to 1,279 13 13 of 13 0.113 0.98 to 1.16; 6.3
Kimi-K3, IDs 0 to 1,279 13 13 of 13 0.222 0.98 to 1.06; 5.0
Kimi-K3, IDs 162,560 to 163,839 0 none in range 0.000 0.98 to 0.99; 3.8

The report’s V4-Flash count holds: its 469 rows include 13 low IDs and 455 near the end of the matrix, 448 of which my scan finds to match, and the 7 misses are consistent with its note that 8 rows were partly changed by training. Kimi K3’s untouched rows are the low IDs 124 to 125 and 177 to 187, and the last 1,280 rows of its matrix hold no trace under the report’s seed and layout.

A control that fooled me first

My first controls on the low-ID ranges looked alarming: wrong layouts scored up to 32 standard errors, with a spread of z between 2.6 and 4.7 instead of 1. The cause is the rule above: the first G elements are identical under every layout, so rows that start below element 311,296 (IDs under 76 at 4,096 values per row, under 44 at 7,168) cannot discriminate layouts. The script now keeps only rows beyond that point. With that fix the controls on the end-of-matrix blocks behave as a standard normal (spread of z 0.97 to 1.01). On the low-ID ranges the wrong-layout controls stayed a little wide (1.04 to 1.16, up to 6.3 standard errors over about 1,200 rows) and I did not find out why. Against exact-copy rows it changes nothing, but it is the one place where the normal null the report assumes looks slightly optimistic. Any model whose untouched rows sit below that point could not be fingerprinted at all.

A GPU the report never used

The public code hard-codes 2,048 threads per SM (sms * 2048 in replay.py). PyTorch’s source uses the device’s maximum threads per SM, which differs by architecture. This machine has an RTX A400: 6 SMs, compute capability 8.6, driver 595.79. With CUDA PyTorch 2.14.1 I seeded the generator with DeepSeek’s seed, filled a float32 tensor of 2^24 elements with normal_(0, 0.006), and compared 8,192 elements from position 4,000,000 with the report’s CPU model under every layout from 2 to 300 SMs at both 1,536 and 2,048 threads per SM.

# gpu_premise.py: does the report's CPU model of PyTorch's normal_ reproduce a GPU the report never mentions?
# Needs a CUDA build of PyTorch, plus numpy. Run from the checkpoint-forensics folder.
import numpy as np, torch
from gpuid import draw, corr

p = torch.cuda.get_device_properties(0)
sms, per_sm = p.multi_processor_count, p.max_threads_per_multi_processor
print(f"{p.name}: {sms} SMs, compute capability {p.major}.{p.minor}, max threads per SM {per_sm}, torch {torch.__version__}")
seed, N, start, L = 953049583, 1 << 24, 4_000_000, 8192
torch.cuda.manual_seed(seed)
x = torch.empty(N, dtype=torch.float32, device="cuda").normal_(0, 0.006).cpu().numpy().astype(np.float64)
pos = np.arange(start, start + L, dtype=np.uint64)
res = {}
for mult in (1536, 2048):
    for s in range(2, 301):
        res[(mult, s)] = corr(x[start:start + L], draw(pos, s * mult, seed, 0, "normal"))
print("best layouts by correlation on elements 4,000,000 to 4,008,191 (seed 953049583, float32):")
for (mult, s) in sorted(res, key=res.get, reverse=True)[:4]:
    print(f"  {s:>3} SMs x {mult} threads = G {s * mult:>7}: r = {res[(mult, s)]:.6f}")
print(f"report's rule (2,048 x {sms} SMs = {2048 * sms}): r = {res[(2048, sms)]:.4f}")
print(f"max threads per SM x SMs ({per_sm} x {sms} = {per_sm * sms}): r = {res[(per_sm, sms)]:.6f}")
best_2048 = max((k for k in res if k[0] == 2048), key=res.get)
print(f"best layout among 2,048 x 2..300 SMs on this GPU (the report sweeps 16 to 256): {best_2048[1]} SMs, r = {res[best_2048]:.4f}")
# the same seed on a Hopper or Blackwell layout, for the same 8,192 elements
for name, g in (("H100 (132 SMs)", 132 * 2048), ("B200 (148 SMs)", 148 * 2048), ("GB200 (152 SMs)", 152 * 2048)):
    print(f"  A400 output vs {name} layout: r = {corr(x[start:start + L], draw(pos, g, seed, 0, 'normal')):+.4f}")
# size of the disagreement at the matching layout (the GPU uses fast sin and cos intrinsics, the CPU model uses NumPy)
y = 0.006 * draw(pos, per_sm * sms, seed, 0, "normal")
print("largest absolute difference between the GPU and the CPU model at G = %d: %.2e (values have a standard deviation of 0.006)" % (per_sm * sms, np.abs(x[start:start + L] - y).max()))
# first elements are layout independent
print("first 8,192 elements vs the 2,048 x 132 layout: r = %.6f" % corr(x[:L], draw(np.arange(L, dtype=np.uint64), 132 * 2048, seed, 0, "normal")))
NVIDIA RTX A400: 6 SMs, compute capability 8.6, max threads per SM 1536, torch 2.14.1+cu130
best layouts by correlation on elements 4,000,000 to 4,008,191 (seed 953049583, float32):
    6 SMs x 1536 threads = G    9216: r = 1.000000
   63 SMs x 1536 threads = G   96768: r = 0.033202
   10 SMs x 1536 threads = G   15360: r = 0.032113
  298 SMs x 1536 threads = G  457728: r = 0.029929
report's rule (2,048 x 6 SMs = 12288): r = 0.0119
max threads per SM x SMs (1536 x 6 = 9216): r = 1.000000
best layout among 2,048 x 2..300 SMs on this GPU (the report sweeps 16 to 256): 145 SMs, r = 0.0280
  A400 output vs H100 (132 SMs) layout: r = -0.0062
  A400 output vs B200 (148 SMs) layout: r = +0.0119
  A400 output vs GB200 (152 SMs) layout: r = -0.0118
largest absolute difference between the GPU and the CPU model at G = 9216: 1.14e-08 (values have a standard deviation of 0.006)
first 8,192 elements vs the 2,048 x 132 layout: r = 1.000000

The report’s model reproduces the real GPU when G is 6 x 1,536 = 9,216, which is what calc_execution_policy computes for this card: the largest difference over the 8,192 elements is 1.14e-08 against values with a standard deviation of 0.006, which is float32 rounding. The report’s 2,048 would give 12,288 and r = 0.0119. Under the report’s own sweep, 2,048 threads per SM across 16 to 256 SMs, this GPU would show no peak (the best, 145 SMs, scores 0.028), so replay.py would report a miss, not a wrong answer. The report’s own GPU script agrees on this card:

NVIDIA RTX A400, 6 SMs, torch 2.14.1+cu130
model                 init on      rows  median r    seed+1
DeepSeek-V3-Base      132 SMs       0/8  0.012113       0/8
DeepSeek-V4-Flash     148 SMs     0/469  0.000334     0/469
Kimi-K3               152 SMs      0/13 -0.000655      0/13
MiniMax-M3            148 SMs      0/17  0.002913      0/17

Two consequences. The CPU model is right on a GPU class the report never tested, so its other results are less likely to be an artifact of the emulation. And the layout is the maximum threads per SM times the SM count, not 2,048 times the SM count. Because 303,104 / 1,536 = 197.33 and 311,296 / 1,536 = 202.67, no GPU with 1,536 threads per SM can reproduce the 148 or 152 layout exactly, which fits the report’s statement that China-market Blackwell cards “use a different number of threads per SM and do not produce either total”. I could only check the arithmetic, since I do not know what those cards report. I have no 132-, 148- or 152-SM GPU, so the H100, B300 and GB200 runs behind the Blackwell claims remain the author’s.

What 8 sigma and 1,385 sigma measure

The report searched about six trillion seed and layout combinations and sets a benchmark: the best accidental match should reach about 8 sigma, while its six Blackwell results run from 49 to 1,385 sigma. I recomputed that and the other arithmetic.

# stats_check.py: recompute the report's arithmetic (needs numpy, scipy; run from the checkpoint-forensics folder)
import json
import numpy as np
from scipy.stats import norm

N = 6e12
print("trials in the search (report): %.0e" % N)
print("z whose expected number of chance hits is 1: %.2f" % norm.isf(1 / N))
for z in (5, 7.3, 8, 20):
    print("z = %-4g one-sided p = %.2e, expected chance hits in 6e12 trials = %.3g" % (z, norm.sf(z), N * norm.sf(z)))
print("5 sigma is 1 in %.2f million" % (1 / norm.sf(5) / 1e6))
for extra in (1e3, 1e6):
    print("search x %.0e (more free parameters): z at expected count 1 = %.2f" % (extra, norm.isf(1 / (N * extra))))

# the report's sigma = r * sqrt(n) for exact-copy models? n = rows * row length
print()
print("%-22s %6s %6s %12s %9s %12s" % ("model", "rows", "d", "n values", "median r", "r*sqrt(n)"))
R = {"DeepSeek-V3-Base": 0.999999, "DeepSeek-V4-Flash": 0.999999, "Kimi-K3": 0.999996, "MiniMax-M3": 0.999990, "Ling-3.1-flash-base": 0.983436}
for m in json.load(open("models.json")):
    rows, d = len(m["rows"]), m["shape"][1]
    n = rows * d
    print("%-22s %6d %6d %12d %9.6f %12.1f" % (m["name"], rows, d, n, R[m["name"]], R[m["name"]] * np.sqrt(n)))
print("Figure 5: r = 0.066 over 745,472 weights per layer -> %.1f sigma" % (0.066 * np.sqrt(745472)))

# Table 2 arithmetic (GPU counts, H100 equivalents, cost)
print()
gpus = {"DeepSeek-V4-Flash": (1500, 2.3), "DeepSeek-V4-Pro": (3000, 2.5), "Kimi K3": (4500, 2.5), "MiniMax-M3": (1500, 2.3), "Ling-3.1-flash": (1500, 2.5)}
tot = 0
for k, (g, f) in gpus.items():
    tot += g
    print("%-18s %5d GPUs x %.1f = %6.0f H100e; at $70,000 = $%.0fM" % (k, g, f, g * f, g * 70000 / 1e6))
print("five runs: %d GPUs, $%.0fM; $/GPU implied by WSJ: 2.5e9/36000 = $%.0f" % (tot, tot * 70000 / 1e6, 2.5e9 / 36000))
print("earlier version: 2,000 to 3,000 GPUs -> $%.0fM to $%.0fM" % (2000 * 70000 / 1e6, 3000 * 70000 / 1e6))
trials in the search (report): 6e+12
z whose expected number of chance hits is 1: 7.28
z = 5    one-sided p = 2.87e-07, expected chance hits in 6e12 trials = 1.72e+06
z = 7.3  one-sided p = 1.44e-13, expected chance hits in 6e12 trials = 0.863
z = 8    one-sided p = 6.22e-16, expected chance hits in 6e12 trials = 0.00373
z = 20   one-sided p = 2.75e-89, expected chance hits in 6e12 trials = 1.65e-76
5 sigma is 1 in 3.49 million
search x 1e+03 (more free parameters): z at expected count 1 = 8.16
search x 1e+06 (more free parameters): z at expected count 1 = 8.96

model                    rows      d     n values  median r    r*sqrt(n)
DeepSeek-V3-Base            8   7168        57344  0.999999        239.5
DeepSeek-V4-Flash         469   4096      1921024  0.999999       1386.0
Kimi-K3                    13   7168        93184  0.999996        305.3
MiniMax-M3                 17   6144       104448  0.999990        323.2
Ling-3.1-flash-base       384   4096      1572864  0.983436       1233.4
Figure 5: r = 0.066 over 745,472 weights per layer -> 57.0 sigma

DeepSeek-V4-Flash   1500 GPUs x 2.3 =   3450 H100e; at $70,000 = $105M
DeepSeek-V4-Pro     3000 GPUs x 2.5 =   7500 H100e; at $70,000 = $210M
Kimi K3             4500 GPUs x 2.5 =  11250 H100e; at $70,000 = $315M
MiniMax-M3          1500 GPUs x 2.3 =   3450 H100e; at $70,000 = $105M
Ling-3.1-flash      1500 GPUs x 2.5 =   3750 H100e; at $70,000 = $105M
five runs: 12000 GPUs, $840M; $/GPU implied by WSJ: 2.5e9/36000 = $69444
earlier version: 2,000 to 3,000 GPUs -> $140M to $210M

The benchmark is fair. With six trillion trials the z that yields one expected chance hit is 7.28, and 8 sigma leaves an expected 0.004 chance hits. The models carry extra free parameters (models.json has a Philox offset of 3,776 for Kimi K3, four tensor-parallel shards with seed + rank for MiniMax and a different distribution for Ling), but a thousand times more trials moves the threshold only to 8.16 and a million times to 8.96. Even the weakest reported result, 20 sigma, is far above that.

The sigma values themselves are the correlation times the square root of the number of values compared. That reproduces 239.5, 1,386.0, 305.3 and 323.2 for V3, V4-Flash, K3 and M3 (the report gives 239, 1,385, 305 and 323) and 57.0 for Figure 5. Ling’s 244 does not come from the repository’s 384 rows, which would give 1,233, so it must use a smaller held-out set (the report says grid-scan figures use cells held out of the search). For exact copies the figure counts how many values were compared, so 49 against 1,385 says little about which model is more convincing. The difference is the kind of evidence: V4-Pro’s 49 comes from trained weights, V4-Flash’s from untouched rows.

What the evidence does not settle

Initialization is not training

The report is explicit that a lab “could generate the weights on one system and move them to another, or continue training an existing model on new hardware”, and that the weights “cannot reveal who owned the chips or where they were located.” The report’s detail on Kimi K3 is that “at least 64 separately seeded processes each left the same 152-SM layout”, and it still says a Blackwell-compatible pattern “does not establish that the entire training run used Blackwell”. TechCrunch carries the same caveat (“the chips could have been in an offshore data center”) beneath a headline that says the chips are “behind China’s most powerful AI models”. The report also lists Alibaba’s Qwen, Zhipu’s GLM, Tencent, Xiaomi and StepFun as reading Hopper, and lists Qwen3-235B, Qwen3-32B and Qwen3.5-9B as initialized on the CPU, which “leaves no GPU layout”. That last point is practical for anyone who publishes weights: generating the initial values on the CPU erases the fingerprint.

Two of the six Blackwell models are outside the public package

Model Layout Evidence in the report In the public code? My result
DeepSeek-V4-Flash 148 SMs Exact copy, 1,385 sigma Yes 461 of 469 rows; 448 match in a fresh scan
DeepSeek-V4-Pro 152 SMs Partial survival, 49 sigma No Not tested
Kimi K3 152 SMs Exact copy, 305 sigma Yes 13 of 13 rows
Kimi K3.1-Pro 152 SMs Exact copy, 305 sigma; same seed as K3 No Not tested
MiniMax-M3 148 SMs Exact copy, 323 sigma Yes 17 of 17 rows
Ling-3.1-flash 152 SMs Untrained rows, r 0.98, 244 sigma Yes, as the base checkpoint 384 of 384 rows, median r 0.983

The report says K3.1-Pro “may be a checkpoint of K3 with further post-training”, so the six readings are five independent initializations. DeepSeek-V4-Pro is the one The Register names in its quote and the one in the cost table’s second-largest run, and its strongest evidence, 49 sigma, is the lowest of the six. Like K3.1-Pro’s, it cannot be rerun from the repository.

AMD, and the cost table

A physical MI300X limited to 152 or 148 of its units reproduces the Blackwell layouts, so the layout alone does not settle the question. The report rules AMD out for DeepSeek-V4-Flash with a vendor-specific arithmetic difference (the published weight matches NVIDIA’s in 363 of 377 differing values and AMD’s in none) and calls it “very unlikely” for the other five. That is a judgment, not a test, and I did not test AMD.

The cost table is the least settled part, and the report says so: “We are not confident in these estimates.” It was revised on November 12 from 2,000 to 3,000 GPUs and $150 million to $200 million for the largest runs to about 4,500 and 3,000 GPUs and “roughly $200 million to $300 million”, because the first figures assumed benchmark-level efficiency. The X post still carried the first version when I read it. My arithmetic on the report’s own inputs: 4,500 GPUs at $70,000 is $315 million, just above the stated range, 3,000 is $210 million, and the five runs total 12,000 GPUs and $840 million. The Kimi K3 range in Table 2 runs from 1,000 to 9,500 GPUs.

What I did not test

I did not run any 132-, 148- or 152-SM GPU, repeat the search over all 2^32 seeds, check the 24 known-hardware cases (the report notes that eleven are step-zero checkpoints and related models are not independent, so it is “a consistency check rather than a measured accuracy rate”), test DeepSeek-V4-Pro or Kimi K3.1-Pro, test AMD hardware, or verify the reviewer claim. I found no response from the four labs in the report or in either article.

What would settle it

Three things would move this from a reproducible mechanism to a settled finding: a third party running the seeds on a physical 148- or 152-SM GPU, the V4-Pro and K3.1-Pro rows added to the public package, and the labs saying where their starting weights were generated. For related reading, see our audits of Microsoft’s Lazarus post, which also treated published weights as evidence, and of WIRED’s story on Chinese open-weight models and hacking.

Sources and method

The report and the X post (read through the api.fxtwitter.com mirror) were read in full on November 14. The code is macrogrants/checkpoint-forensics at commit b633870. The weights were read from the pinned Hugging Face revisions the repository names: DeepSeek-V3-Base, DeepSeek-V4-Flash and Kimi-K3. The lab ran on Windows 11 with Python 3.13.14, NumPy 2.5.4 and SciPy 1.18.1, and PyTorch 2.14.1 (CUDA 13.0 build) on the RTX A400. The scripts above are the ones I ran, and the outputs are copied from those runs.

Tags:

BlackwellDeepSeekExport ControlsModel WeightsNVIDIAPyTorch

Share

A car windshield wiper arm and blade resting against a dark windshield above a white hood
Previous Post

Check Point Says a Gaia Root Flaw Was Exploited Against “a Couple of Dozen” Customers, Its Researchers Tie It to PHP2Shell and an ESXi Wiper, and the Fix Is Listed Only in Jumbo Takes, Not in the October PHP Hotfix

A moss-covered wooden buffer stop with rusty end plates at the end of a disused railway line, with a birch tree growing through the undergrowth beside it
Next Post

How to Build a Buffer Pool in Python and Stop One Table Scan From Flushing Your Hot Pages With Clock Sweep, a Scan Ring, and Midpoint Insertion

No Comment! Be the first one.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Latest
16 Nov
How to Stop Ollama From Silently Cutting Your Prompt, Find the Real Context Window, and Set num_ctx Without Running Out of Memory
16 Nov
llama.cpp’s v1.0.0 Promises Major Bumps for API and ABI Breaks, but Its Rule Table Lists Only the C API, libmtmd Shares the Version While Its Header Warns of Breaking Changes, and the Release Page Carries a 7-Byte Pointer to a Nightly Build Instead of Binaries
Trending
November 16, 2026
How to Stop Ollama From Silently Cutting Your Prompt, Find the Real Context Window, and Set num_ctx Without Running Out of Memory
November 16, 2026
llama.cpp’s v1.0.0 Promises Major Bumps for API and ABI Breaks, but Its Rule Table Lists Only the C API, libmtmd Shares the Version While Its Header Warns of Breaking Changes, and the Release Page Carries a 7-Byte Pointer to a Nightly Build Instead of Binaries
November 16, 2026
NoisyTCP Can Crash a Linux Server Over the Network, but the Fix Has Reached Only Three of the Seven Kernel Lines Still Being Released, Red Hat and Ubuntu Have No Update, and the Researcher’s Own Timings Run From 5 Seconds to 14 Hours
November 15, 2026
How to Route Alerts in Alertmanager So Pages Reach the On-Call and Tickets Reach the Queue, and Test Grouping, Inhibition, and Silences Before an Outage Does
November 15, 2026
Ai2’s XYBench Turns the XY Problem Into a Benchmark, but the Model That Drafted Every Rubric Also Judges the Answers, the Human Baseline Is Graded on Rubrics Read Off Its Own Answers, and the Emphasis Judge Disagrees With the Annotators on 20 to 26 Percent of Validation Responses
November 15, 2026
The FBI’s ARTEX Warning Says a Supply Chain Compromise “Enabled Access” to 2,250 Tenants Where Microsoft Counted Sign-In Attempts and Cites “Over 1,600” Consoles Where Censys Counted 1,465, and a Microsoft Post It Cites Mistypes One Indicator in Its Hunting Query

Related Posts

Blue-lit server racks in a modern data center, illustrating the compute infrastructure behind the AI boom.
Articles

The AI Boom Is Spending Real Money Before Proving Real Returns

June 7, 2026
Technician working with a laptop beside server racks, representing enterprise AI retrieval infrastructure
Articles

Google’s Agentic RAG Push Makes Enterprise AI Less of a One-Shot Guess

June 7, 2026
A person with a laptop and smartphone, representing digital attention and AI-assisted work
Articles

AI Chatbots Are Making Attention a Design Problem

June 7, 2026
A customer-support representative wearing a headset against a dark studio background.
Articles

The Meta AI Support Hack Was a Plain Old Authorization Failure

June 7, 2026
SXZ.io SXZ.io
  • [email protected]

Categories

Articles
Learning Hub
News

All Rights Reserved by SXZ.io ©2026