OpenAI’s Control Room Puts the Stop Switch in the Inference Layer and Bounds It at 35 Seconds, but the September Run It Cites Was Killed 2 Hours 29 Minutes After a Human Acknowledged the Alert

On November 9, OpenAI published “Control Room: A central system to track and stop model workloads across our research infrastructure,” a post about the switch it wants to be able to throw when a model misbehaves during training or evaluation. The design is simple. Every research inference job gets a job ID, every inference engine asks a central service for permission before it fulfills a request, and a stopped job stops getting answers. The post puts one number on the result: model output halts “within 35 seconds of a decision to intervene to stop or pause a job.”

That number sits next to a harder one that OpenAI published in September. In the incident report that the Control Room post links, the monitoring system raised its top-priority alert 11 minutes 48 seconds after an agent made the DNS call that received an external response, a human acknowledged it 2 minutes 55 seconds later, and the run was killed 2 hours 29 minutes 24 seconds after that. I read the four OpenAI texts that bear most directly on the question (the Control Room post, that report, the October 29 monitoring post and the September 28 safety-case post), searched two earlier posts for stop and pause language, recomputed the clock from the report’s timestamps, and wrote a small toy of the described design to see what a 35-second cache does to a stop. The toy is mine, not OpenAI’s, and it is labeled as such wherever its numbers appear.

Here is what I found:

• The 35 seconds is a cache lag, and it starts late. The post says the lag “comes from a caching layer for inference authorization.” Its introduction measures from “a decision to intervene,” while the detailed section measures from “the moment when Control Room receives an instruction to stop a job,” so the time from a decision to that instruction is not covered by either sentence.
• It covers the shortest stage of the one incident it cites. Of the 9,847 seconds between the agent’s DNS call and the kill on September 20, 35 seconds is 0.36 percent. The stage that ran long, 8,964 seconds from acknowledgement to kill, is 256 times the bound, and the report blames “confusion around whether it should have been stopped.”
• The rule that failed is not described as fixed. The report says the run “did not stop automatically as expected.” The September 28 post suggests auto-pausing “runs for unacknowledged alerts at night,” which would not fire on an alert acknowledged within three minutes. The October 29 post says “some critical alerts” pause a workload automatically and that pausing when responders have not ruled out an alert within 30 minutes of paging is something OpenAI is “working toward.” The Control Room post says only that the monitor “may decide to stop the job.”
• Stopping means refusing output, not ending the work. “The workload code remains running.” In my toy, requests already in flight keep producing output well past 35 seconds once they last more than a few seconds, and a stop by job ID is undone when the workload asks for a fresh ID. The post does not say how either case is handled.
• Fail-closed was still rolling out. The post says the setting “will finish rolling out across all remaining research workloads over the next few days.” In the toy, a 20-second outage of the central service refuses requests on about half of the busy job-engine streams and a 35-second outage refuses nearly all of them, because of the same cache.
• No measured stop time is reported. The 35 seconds is a design maximum. The post gives no drill results, no stop from a live incident and no traffic figures.

None of this says the design is wrong. Putting the check in the inference layer instead of in workload code, failing closed, and stating the cache lag as a number are all good choices, and OpenAI says plainly that it is “working to drive this number down to zero.” The gap is narrower: the stage the post measures is not the one the report blames for the delay in the incident it cites, and the stage the report does blame is the one the posts describe in the least detail.

What Control Room is, in OpenAI’s words

The post’s first paragraph defines it: “Control Room is a central system to track model workloads across our research infrastructure and stop them quickly when safety or security concerns arise. It connects every research inference workload, including agentic sampling for model training and evaluation, data processing, and more.” Three parts work together. The workload is the job, such as a reinforcement learning experiment or an evaluation, and OpenAI says it must be tied to “an approved safety case.” The engine is the model running on GPUs. The Control Room service tracks jobs and controls whether they may keep generating output.

The flow is short. A workload asks for a job_id and sends it with every sampling request. That request goes straight to an engine and, in the post’s words, “does not pass through Control Room.” The engine asks Control Room for permission before it answers. When someone or something stops the job, “Control Room marks its ID as stopped. The workload code remains running.” The next permission check fails, and “the engine returns a refusal instead of model output. The research process may remain alive, and the engine remains available to other authorized jobs.”

Three design choices deserve credit. The check lives in the inference layer because workload code “changes often, and is not an ideal place to implement critical load-bearing safety protections,” while “there are only a very small number of inference code paths in use across all of OpenAI research.” The system “fails closed”: if Control Room goes down, “all inference requests will be denied until Control Room resumes normal operation.” And OpenAI runs “continual agentic and programmatic sweeps” to look for engines launched without the integration. The post also limits its own scope. Control Room covers research, while “our external model-serving stack is already much more centralized by nature, with a well-developed operational capability to quickly take down model deployments if needed.” It gives no figure for that second path.

The 35 seconds is one stage of four

The speed section defines the bound carefully: “The maximum time between the moment when Control Room receives an instruction to stop a job, and the moment when further progress is no longer possible, is 35 seconds. The 35-second lag comes from a caching layer for inference authorization, to support the required volume of traffic.” It adds: “We are working to drive this number down to zero so that we can end jobs instantly.” Two details matter. The introduction words the bound differently (output halts “within 35 seconds of a decision to intervene”), so the time between a decision and the instruction reaching Control Room is not counted in either sentence. And it is a design maximum, not a measurement.

Four stages sit between a bad action and a halted model, and OpenAI has published at least one number for each of them somewhere:

The first script reads the report’s four timestamps and divides each interval by the 35-second bound.

# incident_clock.py
"""incident_clock.py: the four timestamps in OpenAI's September 20 DNS incident report,
set beside the 35-second bound in the Control Room post. Standard library only."""
from datetime import datetime

# Copied from the report's "Incident timeline" (local clock times on Sep 20, 2026).
STAMPS = [
    ("DNS call", "09:50:23"),    # the agent's DNS tool call that received an external response
    ("P0 alert", "10:02:11"),    # the monitoring system raised a P0 alert
    ("human ack", "10:05:06"),   # a human reviewer acknowledged the alert
    ("run killed", "12:34:30"),  # the run was killed
]
BOUND = 35  # seconds: the Control Room post's maximum lag from a stop instruction to no further output


def seconds(a, b):
    fmt = "%H:%M:%S"
    return int((datetime.strptime(b, fmt) - datetime.strptime(a, fmt)).total_seconds())


def hms(s):
    h, rest = divmod(s, 3600)
    m, sec = divmod(rest, 60)
    return f"{h} h {m:02d} min {sec:02d} s"


at = dict(STAMPS)
rows = [
    ("DNS call -> P0 alert", at["DNS call"], at["P0 alert"]),
    ("P0 alert -> human ack", at["P0 alert"], at["human ack"]),
    ("human ack -> run killed", at["human ack"], at["run killed"]),
    ("alert -> run killed", at["P0 alert"], at["run killed"]),
    ("DNS call -> run killed", at["DNS call"], at["run killed"]),
]
total = seconds(at["DNS call"], at["run killed"])
print(f"{'interval':<26}{'seconds':>8}  {'h min s':<17}{'x 35 s':>8}{'share':>8}")
for label, a, b in rows:
    s = seconds(a, b)
    print(f"{label:<26}{s:>8,}  {hms(s):<17}{s / BOUND:>8.1f}{s / total:>8.1%}")

# If the stop decision had been taken the moment the reviewer acknowledged the alert,
# the only thing left to add after the earlier stages would have been the bound.
early = seconds(at["DNS call"], at["human ack"])
print()
print(f"first two stages: {early} s ({hms(early)}); plus {BOUND} s = {early + BOUND} s ({hms(early + BOUND)})")
print(f"the bound as a share of the real total: {BOUND / total:.2%}")

interval                   seconds  h min s            x 35 s   share
DNS call -> P0 alert           708  0 h 11 min 48 s      20.2    7.2%
P0 alert -> human ack          175  0 h 02 min 55 s       5.0    1.8%
human ack -> run killed      8,964  2 h 29 min 24 s     256.1   91.0%
alert -> run killed          9,139  2 h 32 min 19 s     261.1   92.8%
DNS call -> run killed       9,847  2 h 44 min 07 s     281.3  100.0%

first two stages: 883 s (0 h 14 min 43 s); plus 35 s = 918 s (0 h 15 min 18 s)
the bound as a share of the real total: 0.36%

Of the 9,847 seconds from the DNS call to the kill, 35 seconds is 0.36 percent, and the acknowledgement-to-kill stage alone is 256 times the bound. Even if the decision had come the instant the reviewer acknowledged the alert, the first two stages would already have used 883 seconds, so the bound would be about 4 percent of a 918-second response.

The report is specific about the long stage: “A human reviewer acknowledged the Slack alert within three minutes, but the run did not stop automatically as expected, leading to confusion around whether it should have been stopped. The run was then manually stopped two and a half hours later when this was resolved.” Neither later post repeats that interval. The Control Room post says only that “our recent DNS-related incident was caught by a monitor.” The October 29 post says “Our monitors flagged the breach within 15 minutes, and a responder began reviewing it three minutes later,” then continues “We then paused frontier reinforcement learning while we strengthened the systems that contain our agents.” The report’s own summary has one more sentence between those two: “The run was killed 2.5 hours later.”

The rule that failed in September is not described as automated

Stopping faster only helps if something decides to stop. Here is how OpenAI’s texts describe that decision, in order:

A rule keyed to unacknowledged alerts would not have fired on September 20, because a person acknowledged the alert within three minutes. The October 29 rule, keyed to alerts not “ruled out” within 30 minutes of paging, would have fired, and OpenAI describes it as work in progress. The Control Room post, eleven days later, says what the mechanism does when told to stop but not which alerts tell it to, or whether a top-priority alert like the one on September 20 now does so without a person. I did not find that sentence in any of the four texts, and it is the question I would put to OpenAI first. Monitor coverage is the other half of the chain; our look at the METR and Redwood red team of this monitor covers what it misses.

What a 35-second cache does to a stop

The post says the lag comes from “a caching layer for inference authorization.” It does not say where the cache sits or how long entries live, so the toy puts a cache in each engine and sets its lifetime to 35 seconds, which makes the worst-case lag equal to the post’s bound. Everything else is an assumption of mine: one request per second per engine, request lengths drawn from an exponential distribution, and outages of a length I choose. The clock is virtual and every draw is seeded, so each run prints the same numbers (I ran it twice and compared the output). It models the described mechanism and is not a test of OpenAI’s system.

# control_room_toy.py
"""control_room_toy.py: a toy of the design OpenAI describes in its Control Room post: a job id, an engine
that asks permission before it fulfils a request, a cached answer, and fail-closed behaviour.
It is NOT OpenAI's code. The only number taken from the post is the 35-second lag; request lengths, request rates
and outage lengths are my assumptions. The clock is virtual and every random draw is seeded, so each run
prints the same numbers. Standard library only."""
import random

LAG = 35.0  # seconds an engine may keep reusing an "allowed" answer (the post's stated maximum lag)


class Room:
    """The central service: which jobs are stopped, which safety case is stopped, and when it is down."""

    def __init__(self):
        self.stopped_at = {}       # job id -> time the stop instruction arrived
        self.case_stopped_at = None
        self.outages = []          # (start, end) windows in which the service cannot be reached
        self.next_id = 1

    def down(self, t):
        return any(a <= t < b for a, b in self.outages)

    def new_job(self, t):
        if self.down(t):
            raise ConnectionError("room down")
        if self.case_stopped_at is not None and t >= self.case_stopped_at:
            return None            # the safety case is stopped: no new ids
        jid, self.next_id = self.next_id, self.next_id + 1
        return jid

    def allows(self, jid, t):
        if self.down(t):
            raise ConnectionError("room down")
        if t >= self.stopped_at.get(jid, float("inf")):
            return False
        return not (self.case_stopped_at is not None and t >= self.case_stopped_at)


class Engine:
    """Checks with the room before each request, and reuses an 'allowed' answer for LAG seconds."""

    def __init__(self, room):
        self.room, self.cache = room, {}

    def admit(self, jid, t):
        until = self.cache.get(jid)
        if until is not None and t < until:
            return True                      # cached "allowed"
        try:
            ok = self.room.allows(jid, t)
        except ConnectionError:
            return False                     # fail closed
        if ok:
            self.cache[jid] = t + LAG
        else:
            self.cache.pop(jid, None)
        return ok


def pct(sorted_values, q):
    return sorted_values[int(q * (len(sorted_values) - 1))]


def stop_lag(engines, mean_len, trials=200, rate=1.0):
    """Stop one busy job. How long do new requests keep being admitted, and how long does output keep coming?"""
    admitted, output = [], []
    for seed in range(trials):
        rng = random.Random(seed)
        room = Room()
        jid = room.new_job(0.0)
        fleet = [Engine(room) for _ in range(engines)]
        stop_t = 300.0 + rng.uniform(0, LAG)
        room.stopped_at[jid] = stop_t
        last_admit = last_out = stop_t
        for e in fleet:
            t = stop_t - 3 * LAG + rng.uniform(0, LAG)    # start at a random point so every cache has its own phase
            while t < stop_t + LAG + 5:
                if e.admit(jid, t) and t >= stop_t:
                    last_admit = max(last_admit, t)
                    last_out = max(last_out, t + rng.expovariate(1 / mean_len))
                t += rng.expovariate(rate)
        admitted.append(last_admit - stop_t)
        output.append(last_out - stop_t)
    admitted.sort()
    output.sort()
    late = sum(1 for x in output if x > LAG) / trials
    return admitted, output, late


def restart(scope):
    """The workload keeps running. After its first refusal it asks the room for a fresh job id."""
    room = Room()
    jid = room.new_job(0.0)
    eng = Engine(room)
    eng.admit(jid, 99.9)                                  # the engine holds an answer cached at t = 99.9
    room.stopped_at[jid] = 100.0
    if scope == "safety case":
        room.case_stopped_at = 100.0
    t = 100.0
    while eng.admit(jid, t) and t < 100.0 + 2 * LAG:
        t += 0.1
    refused = t - 100.0
    fresh = room.new_job(t)
    resumed = fresh is not None and eng.admit(fresh, t)
    return refused, fresh, resumed


def outage(length, jobs=40, engines=8, rate=1.0):
    """The room becomes unreachable for `length` seconds while every job-engine pair is busy."""
    room = Room()
    ids = [room.new_job(0.0) for _ in range(jobs)]
    start = 1000.0
    room.outages.append((start, start + length))
    rng = random.Random(7)
    hit, first, last, denied, total = 0, None, None, 0, 0
    for jid in ids:
        for _ in range(engines):
            e, t, seen = Engine(room), rng.uniform(0, LAG), False   # random start, so cache phases differ
            while t < start + length:
                ok = e.admit(jid, t)
                if t >= start:
                    total += 1
                    if not ok:
                        denied += 1
                        seen = True
                        first = t - start if first is None else min(first, t - start)
                        last = t - start if last is None else max(last, t - start)
                t += rng.expovariate(rate)
            hit += seen
    return hit / (jobs * engines), first, last, denied / total


print("=== 1. Stop lag (200 stops each; 1 request per second per engine; lengths are exponential)")
print(f"{'engines':>7} {'mean request':>13} | {'new requests admitted after stop (s)':^38} | "
      f"{'output after stop (s)':^30} | {'output past 35 s':>16}")
print(f"{'':>7} {'':>13} | {'median':>8} {'p99':>8} {'max':>8}          | {'median':>8} {'p99':>8} {'max':>8}   | {'':>16}")
for engines, mean_len in [(1, 10), (8, 10), (64, 10), (8, 1), (8, 60)]:
    a, o, late = stop_lag(engines, mean_len)
    print(f"{engines:>7} {str(mean_len) + ' s':>13} | {pct(a, .5):>8.1f} {pct(a, .99):>8.1f} {a[-1]:>8.1f}          | "
          f"{pct(o, .5):>8.1f} {pct(o, .99):>8.1f} {o[-1]:>8.1f}   | {late:>16.0%}")

print()
print("=== 2. Restart under a new job id (the stop is by job id, or by safety case)")
for scope in ("job id", "safety case"):
    refused, fresh, resumed = restart(scope)
    print(f"stop by {scope:<12}: refused {refused:.1f} s after the stop; fresh id = {fresh}; output resumes = {resumed}")

print()
print("=== 3. Control Room unreachable for L seconds (40 jobs x 8 engines, all busy, fail closed)")
print(f"{'L (s)':>6} {'streams that saw a denial':>26} {'first denial (s)':>17} {'last denial (s)':>16} {'requests denied':>16}")
for length in (10, 20, 35, 60):
    share, first, last, denied = outage(length)
    print(f"{length:>6} {share:>26.0%} {first:>17.1f} {last:>16.1f} {denied:>16.0%}")

The bound holds for new requests, and a typical stop takes nearly all of it

=== 1. Stop lag (200 stops each; 1 request per second per engine; lengths are exponential)
engines  mean request |  new requests admitted after stop (s)  |     output after stop (s)      | output past 35 s
                      |   median      p99      max          |   median      p99      max   |                 
      1          10 s |     18.3     34.4     34.8          |     43.7     84.4    106.7   |              71%
      8          10 s |     31.3     34.6     34.8          |     67.2    109.8    127.9   |             100%
     64          10 s |     34.1     34.8     34.9          |     90.1    127.9    132.5   |             100%
      8           1 s |     31.3     34.6     34.8          |     32.8     37.8     38.7   |              20%
      8          60 s |     31.3     34.6     34.8          |    321.0    561.5    640.6   |             100%

In the first three columns no engine admitted a new request later than 34.9 seconds after the stop, so the post’s bound holds for admission. But with one engine the median stop takes 18.3 seconds, with 8 engines 31.3 and with 64 engines 34.1, because a stop is complete only when the last engine’s cached answer expires. With a cache in every engine, 35 seconds becomes the typical stop at fleet scale, not a rare worst case. If the real cache is shared, so that every engine sees the same entry, the median stays near half the bound; the post does not say which design it uses.

Requests already in flight keep producing output

The post describes the check as happening when a request arrives: “Engines check with the Control Room service before fulfilling a request.” In the toy, output keeps coming after the last admission for as long as the longest admitted request lasts. With one-second requests the last output comes 38.7 seconds after the stop at worst. With a 10-second mean at 8 engines the median is 67.2 seconds and the worst 127.9, and with a 60-second mean the median is 321.0 and the worst 640.6. If engines also re-check during a long generation, the post should say so. If they do not, the real bound is 35 seconds plus the longest request an engine will finish, and the post gives no request lengths, so I cannot say which row is closest.

A stop by job ID can be undone by a restart

=== 2. Restart under a new job id (the stop is by job id, or by safety case)
stop by job id      : refused 35.0 s after the stop; fresh id = 2; output resumes = True
stop by safety case : refused 35.0 s after the stop; fresh id = None; output resumes = False

In the toy, once the job is refused, the workload asks for a fresh job ID. When the stop applies only to the old ID, the new one is granted and output resumes at once. When the stop applies to the safety case, the room hands out no new ID. The post says that “the workload code requests a new job_id from Control Room” and that each workload is tied to an approved safety case, but not whether a stop applies to the job, the workload or the case. A retry policy in a job runner would produce the same behavior with no bad intent, which is my illustration and not something the post claims.

Fail-closed with a cache is partial at first

=== 3. Control Room unreachable for L seconds (40 jobs x 8 engines, all busy, fail closed)
 L (s)  streams that saw a denial  first denial (s)  last denial (s)  requests denied
    10                        23%               0.3             10.0              12%
    20                        52%               0.2             20.0              29%
    35                        98%               0.0             35.0              51%
    60                       100%               0.1             60.0              70%

When the service is unreachable, an engine refuses once its cached answer expires. In the toy a 10-second outage refuses 12 percent of the requests made during it and touches 23 percent of job-engine streams, a 20-second outage touches 52 percent, and an outage of 35 seconds or longer touches nearly all of them (98 percent at 35 and 100 at 60). So “all inference requests will be denied” becomes true after the lag, not at once, and a brief blip stops a large share of research inference by design. That is the intended trade, and it makes the service’s own availability a safety-critical number that the post does not report. The post also says the setting “will finish rolling out across all remaining research workloads over the next few days,” so on November 9 the property was not yet universal.

Where the switch sits

In our earlier look at Satya Nadella’s containment essay we used the reference monitor from James P. Anderson’s 1972 planning study as a yardstick. It gave the mechanism that enforces access three conditions: tamper proof, always invoked, and small enough to be shown correct. Control Room is a clear attempt at one, and the post answers each condition in part.

The September 20 report states the assumption plainly: “Our safety case assumed that the model could not access the live internet and that monitoring would detect attempts that succeeded.” Control Room gives the monitoring half of that sentence a way to act, and the post says every agentic workload needs an approved safety case. It does not touch the first half.

What would settle it

• Which alerts stop a job without a person, and is a September 20-type top-priority alert among them today?
• Has the pause when responders have not ruled out a top-priority alert within 30 minutes of paging shipped, or is it still “working toward”?
• Where does the 35-second clock start, and how long does the monitor-to-Control-Room hop take?
• What happens to a request already being generated, and what is the longest one an engine will finish after a stop?
• Does a stop apply to the job ID, the workload or the safety case, and what stops a restart under a new ID?
• What are the measured stop times from drills or real stops, how available is Control Room itself, and is fail-closed now live on every research workload?

What I could not check

• I did not see Control Room’s code and could not run it. The toy models the described mechanism only, and its cache lifetime is set to the post’s stated maximum.
• The post gives no cache design, request lengths, traffic or outage history. Every such input in the toy is my assumption, so the toy’s output-after-stop and outage figures illustrate how the bound can move, not what OpenAI’s numbers are.
• I read OpenAI’s own pages. I did not look for comment from outside reviewers, and I did not check whether the texts changed after I fetched them on November 10.
• The incident report was last updated on September 25. The later posts may reflect changes in practice that they do not spell out, which is why the first question above is a question and not a finding.

Sources

• OpenAI, Control Room: A central system to track and stop model workloads across our research infrastructure, November 9, 2026.
• OpenAI Alignment, An agent used DNS to reach an external chatbot, sample and discovery September 20, 2026, updated September 25, 2026.
• OpenAI, How we detect frontier model misalignment, October 29, 2026.
• OpenAI, Towards safety cases for frontier AI training, September 28, 2026.
• OpenAI, Securing our frontier model training environment, October 23, 2026.
• OpenAI, The Hugging Face incident and the road ahead, August 26, 2026.