MIT Technology Review’s “The Agentic Web Is Here” Turns a Partial-Credit Benchmark Score Into a 73% Success Rate, and DoorDash’s CEO Counts Outside AI Agent Orders in the Hundreds

MIT Technology Review published Grace Huckins’s feature “The agentic web is here” on November 2, 2026. It opens with Tim Berners-Lee’s 2001 Semantic Web article, argues that language-model agents now do what tagged web pages were meant to make possible, and then follows the consequences: agents that drive browsers, connectors that let them act inside services, ads written for bots, and an attention economy that may not survive a mostly artificial audience. It carries exactly three percentages, a 73% benchmark result and two figures of about 40%, and it ties its other factual claims to papers, posts and reports. I traced all three percentages and most of the rest.

The short version: the essay’s direction is defensible, but each of its three percentages reads differently at the source, and the one adoption count I could find, from a chief executive on an earnings call four days before publication, points the other way. The “73% success rate” is a partial-credit score on an offline subset of a benchmark. The Pew drop is nearer 47% than 40%, and the Digiday 40% counts ad requests rather than revenue. The study of sponsored tags that the essay cites without a percentage found a penalty, not indifference. And DoorDash’s chief executive told analysts that outside AI agents have placed “several hundred orders.”

What I found

• The 73% is a partial-credit score on an offline subset. OpenAI’s launch table labels the 72.6% figure “OSWorld 2.0 (v2026.08.08, offline set, partial score)”, and its footnote defines that set as “a subset of the original OSWorld V2 that works without internet access”. The benchmark’s primary metric is strict completion. On the public leaderboard’s 500-step table, an entry with a 73% partial score completes 33% of tasks, and the best entry completes 44%. I found no GPT-6 Astra row there, so I cannot report Astra’s completion rate.
• The sponsored-tag study found a penalty, not indifference. The preprint says agents “consistently penalize sponsored tags”. A product that would be chosen 10% of the time was chosen 7.9% to 8.9% of the time with the tag on an app-style page, and 1.8% to 8.9% on a headless list. The authors add that sponsored ads “secure premium placement”, so the tag’s cost is not the whole story for sellers.
• The Pew drop is nearly half. Pew counted a click on a traditional result link in 8% of Google visits with an AI summary and 15% without, “nearly twice as often”. That is 47% fewer clicks, not “about 40%”.
• The Digiday 40% is ad requests, and spend moved differently. Ozone’s data show Q2 request volumes down 32% to 41% depending on market. First-half spend fell 44% in the U.S. and 14.3% in the U.K. The cohort is Ozone’s premium publisher members, not the web at large.
• DoorDash counts outside agent orders in the hundreds. Its chief executive twice told analysts “several hundred orders” on the third-quarter call, against 993 million total orders for the quarter. For Muse and Dots the essay cites “record downloads” and gives no count of completed tasks.
• TIME’s bot-only ads are real and still running. On November 2, requests carrying five different AI bot names got a markdown page with a labeled Ally ad, while requests claiming to be Chrome got HTTP 406.
• Two smaller points. For merchants that do not accept Link, Stripe says Link issues Muse a single-use virtual card, which the essay’s “the agent never sees it” leaves out. And the Verge article linked at “businesses will shutter” reports no closure.

The essay’s claims and the sources behind them

The table pairs each claim, in the essay’s words, with the source it links or rests on and what that source says. Later sections give the detail for the ones that matter most.

The 73 percent: partial credit on a subset built to run offline

OSWorld is a family of benchmarks in which an AI agent operates a computer to finish tasks. The essay says GPT-6 Astra, which OpenAI released in September, “achieved a 73% success rate on OSWorld 2.0”, and concludes that “agents are close to being able to navigate the human web just as readily as we can”. Three documents bear on that sentence.

What OpenAI reported

OpenAI’s GPT-6 Astra launch post shows OSWorld 2.0 in a computer-use table under the label “OSWorld 2.0 (v2026.08.08, offline set, partial score)”. The row reads 72.6% for Astra, 65.7% for GPT-5.6 Sol and 70.2% for Claude Opus 5. Footnote 3 defines the set: “OSWorld V2-Offline is a subset of the original OSWorld V2 that works without internet access”. The text beside the table reports the same 72.6% from a latency simulation, “at roughly 40 minutes per task, compared with 65.7% at roughly 75 minutes”. The only places the page says “success rate” are its cyber benchmarks, not this one.

What the benchmark measures

The OSWorld 2.0 paper (submitted June 28, 2026 and revised July 13) describes 108 long workflows that take human users “a median of about 1.6 hours to complete”. Its primary metric is binary completion, and its abstract reports that the best system at the time, Claude Opus 4.8 with maximum thinking, “still completes only 20.6% of tasks at a 54.8% partial score”. In that version the authors conclude that “current agents are still far from professional-level computer use”. The leaderboard page explains the two scores: “Binary completion (strict success) and Score (partial credit averaged across an average of 27.25 scoring checkpoints per task)”.

What the leaderboard shows

The page’s first table covers the 500-step budget; its Opus 4.8 and GPT-5.5 rows match the figures in the paper’s abstract. Six of its rows, with the ratio of partial score to completion, are below. The ratios come from the script at the end of this post.

A partial score in the low 70s therefore does not mean seven tasks in ten finished: the entry with a 73.11% score completes a third of them. That entry is a different model, and the page’s model filter lists no GPT-6 model, so the table cannot say what Astra’s completion rate is. The point is narrower and does not depend on it. The number the essay calls a success rate is a score that gives credit for finished checkpoints.

What the figure can and cannot support

The essay uses the number to say agents are close to navigating the human web as readily as people do. The leaderboard says the full benchmark’s self-hosted setup “Preserves open-web access while ensuring reproducibility across runs”, while the offline subset is, by OpenAI’s definition, the part that works without internet access. So the subset is not a measure of how agents cope with the live web. The leaderboard’s own summary points the other way: “On the longest workflows in the corpus, top frontier agents approach near-zero end-to-end completion regardless of step budget”.

The trend in OpenAI’s own table is upward. Astra scores 6.9 points above GPT-5.6 Sol on the same measure and, per OpenAI, gets there “in about 47% less time per task”. That supports the claim that agents are getting better at long desktop workflows. It does not support an arrival date.

Sponsored tags: a penalty, not indifference

The essay says a team led by an assistant professor of operations management at the University of Toronto “investigated whether and how agents are influenced by the ways that shopping websites present search results” and found that agents “entirely ignore items” with a sponsored tag, which “suggests that selling favorable positions on results pages might not be a great way for sites to make money in an agentic economy”. The preprint is What Is Your AI Agent Buying? by Amine Allouah and four co-authors (version 1 of August 4, 2025; version 3 of December 17, 2025).

What the experiment did

The authors built a mock shopping app that shows eight products per page. For each product category they generated 500 scenarios in which positions were shuffled and tags such as “Sponsored”, “Overall Pick” and “Only X Remaining” were assigned at random, along with random changes to price, rating and review count. They then measured how often each model chose a product. Because the badges are randomized, a drop in selection can be read as an effect of the badge.

What they found

The paper says “A sponsored tag reduces the likelihood of selection”. In the app-style layout a product with a 10% baseline falls to 8.9% under Claude Sonnet 4, 8.0% under GPT-4.1 and 7.9% under Gemini 2.5 Flash. In the paper’s headless list, a text-only interface, the same product falls to 5.4%, 1.8% and 8.9%. For newer models, the paper’s Table 11 (Claude Opus 4.5, GPT-5.1, Gemini 3.0 Pro Preview) puts the price cut a seller would need to offset a sponsored tag at 16.5%, 12.4% and 24.2%. Every result I read shows a penalty. The largest drop, from 10% to 1.8% for GPT-4.1 on the headless list, is still not zero.

Why the essay’s inference does not follow

The authors caution against that reading. They write that their estimates “do not imply that sponsored ads are harmful for sellers” because “sponsored ads secure premium placement”, and that “Our findings should therefore be interpreted conditionally: given a fixed position, the mere presence of the sponsored tag lowers an agent’s selection probability”. They also found strong position effects: for Claude Sonnet 4, moving a product from the bottom-right corner to the top row in the second or third column raised its selection rate five-fold. A paid position that buys top-row exposure and a tag that costs somewhere between 11% and 82% of a product’s selection rate can both be true, and whether the trade pays depends on the numbers. The study covers product listings in a mock app and 2025 models; it is not about prose.

Two economics figures that read differently in their sources

Pew: nearly half, not about 40 percent

The essay cites “a study from Pew last year” for people being “about 40% less likely to click through to websites from the Google search results page” when a summary shows. Pew’s short read says users “clicked on a traditional search result link in 8% of all visits” with a summary and “nearly twice as often (15% of visits)” without one. The relative drop is (15 minus 8) / 15 = 46.7%. A 40% drop from 15% would be 9.0%. The sample is 900 U.S. adults whose browsing was tracked in March 2025, and the dataset holds 68,879 unique Google searches, 12,593 of which produced a summary. The essay’s figure points the right way and understates the gap by about seven points.

Digiday: the 40 percent counts ad requests

The essay says “one analysis found that some publishers saw ad revenue decrease by about 40% between 2025 and 2026”. The linked Digiday article is headlined “Publisher ad supply fell by up to 40% in Q2 as AI search choked the open web”. Its data come from Ozone’s publisher members, which include the Guardian, News UK and the Wall Street Journal, across roughly 20 billion impressions. Request volumes fell 32% to 37% year on year in the U.S. and 39% to 41% in the U.K. between April and June. Programmatic spend for the first half fell 30.6% for the combined cohort: 14.3% in the U.K., where higher yields bridged the gap, and 44% in the U.S. The article also notes, citing INMA’s Gabe Dorosz, that “some of the supply shrinkage is deliberate”, as publishers cut ad load and low-value bid requests to protect prices. So the headline number is real, but it is a count of ad opportunities in one company’s publisher cohort, and the revenue picture depends on the country.

The adoption count: DoorDash’s several hundred orders

For the new personal agents, the essay’s usage signal is that Muse and Dots “are logging record downloads”. Downloads are not completed tasks, and none of the essay’s three percentages measures use. The nearest transaction count I found came four days before publication, on DoorDash’s third-quarter call on October 29, as transcribed by the Motley Fool. Asked by a Cantor Fitzgerald analyst about agents such as Muse, Tony Xu said: “we’re seeing maybe several hundred orders in total from some of these third-party agents”. Asked again by an analyst from Rothschild and Co Redburn, he said: “AI agents are generating several hundred orders across the time that they’ve been up and running. So it’s been a fairly low use case”. He added that “when it comes to a task that’s as simple as ordering food, we tend to see that come directly to DoorDash”.

How big that is

The same transcript page lists 993 million total orders for the quarter, up 28% from a year earlier, and Ars Technica reported the figure as “just shy of 1 billion”. Taking “several hundred” as at most 1,000, that is at most about one agent order in 993,000. The windows differ, because Xu’s count covers however long the agents have been running and the 993 million covers the quarter, so the ratio is an order of magnitude and not a measured share. Ars adds that DoorDash’s connector is less than a month old.

What the number leaves out

DoorDash announced its connector on September 30 as “a limited beta focused on companies ordering for their own employees”, with SpaceXAI, Vercel, Cognition, Tempo and Mercor as early partners. Ars notes that the count excludes DoorDash’s own AI tools such as Ask DoorDash, and on the call Xu also named Text DoorDash. The number comes from an interested party, since outside agents compete with DoorDash’s app and its own tools, and Xu did not say the picture would stay this way: “my take on all things technology is you never want to fight gravity. But at the same time, you don’t want to be delusional”. The fair reading is not that agents will not matter. It is that “here” in the headline describes capability and early plumbing, not use, and that DoorDash’s count, from a month-old beta in a single category, is the only agent transaction figure I found in the essay’s neighborhood.

TIME’s bot-only ads, tested

The essay says TIME “has pioneered” serving agent-oriented ads to crawlers “in the hopes that they will be incorporated into these databases and surface when agents search for information”. The reporting supports it. German developer Vincent Schmalbach found the practice and The Register reproduced it on August 5. The Media Copilot says TIME began on July 28, serves the ads to “roughly 70 AI agents it lets in”, sold first to Ally Bank and Project Management Institute, and that Perplexity, a TIME licensing partner, “called the units deceptive”.

What I ran

On November 2, around 10:40 UTC, I sent plain GET requests to one public TIME page, the TIME100 Creators 2026 collection, changing only the User-Agent header: ClaudeBot, OAI-SearchBot, PerplexityBot, GPTBot, ChatGPT-User and Chrome. With a few repeats and a look at the home page, that was ten requests to two pages in all, with no credentials. The five bot user agents each got a 30,873-byte text/markdown response that contained an HTML comment reading mobian-agent-ad sponsored="true" with campaign="ally-2026-q3", followed by “Sponsored content. Supplied in partnership with Ally. Ally is the sponsor and source of this material”. Two requests claiming to be Chrome, one of them with a full browser-style Accept header, got HTTP 406 and an empty body. I did not test from a real browser, so I draw no conclusion about what human visitors receive.

What it shows and does not

• The Ally campaign is still live more than three months after the July 28 start, labeled as sponsored in plain text.
• In August The Register reported that GPTBot and ChatGPT-User received HTTP 406 in Schmalbach’s tests. On November 2 both got the markdown page. Time’s list may have changed, or it may use signals I cannot see. My requests only claimed those names and did not come from OpenAI’s network, so what the result proves is that the gate answers to the User-Agent string for this page.
• The ad carries a plain-text sponsored label, the same kind of signal the sponsored-tag study tested in product listings. That study did not test prose, so I draw no conclusion about whether agents discount a sponsored FAQ. Mollick’s experience in the essay, that newer models call an agent-directed page manipulative, suggests the question is open.

Smaller points where the source says a little more

Stripe’s Link and the card the agent “never sees”

The essay says Stripe knows the card number “but the agent never sees it, and the connector requires human confirmation before a purchase goes through”. Stripe’s announcement supports both halves for the underlying card: consumers approve the transaction total in the chat, and “Muse never sees their underlying payment details”. It also says that at businesses that do not accept Link, “Link issues Muse a single-use virtual card scoped to the approved purchase”. The announcement does not describe how the card details reach the merchant, so my reading that the virtual card is visible to Muse’s checkout flow is an inference. The omission is small, but it matters for anyone modeling what an agent can leak.

The Verge link

The anchor on “businesses will shutter” goes to The Verge’s October 7 piece on DoorDash and a startup called Bites. It describes DoorDash’s August emails warning restaurants about being listed without consent and poses questions about what happens to platforms if agents bypass them. It does not report a business closing. The sentence is the essay’s inference, and the link supports the framing, not the prediction.

Crawled index or live fetch

The essay says agents “often don’t even access the websites they reference” and instead query databases built by crawler bots. That describes one path. OpenAI’s bot page says that “it may visit a web page with a ChatGPT-User agent” and that “Because these actions are initiated by a user, robots.txt rules may not apply”. Perplexity’s page says “Since a user requested the fetch, this fetcher generally ignores robots.txt rules”. An earlier sxz.io post covered four vendors’ statements on user-initiated fetchers. None of these pages gives the share of answers that come from the index versus a live visit, so “often” cannot be checked, and TIME’s gate in my test treated both kinds of agent name alike.

What to take from the essay

• Read capability numbers with their metric. A partial score on an offline subset is evidence of progress on long desktop workflows. To judge whether an agent can do the job, wait for strict completion on the full set, and ask which subset was run.
• Use the narrower economics figures. Pew’s 8% against 15% is the cleaner statement, and Digiday’s request counts are not revenue.
• Ask for transaction counts. Downloads, connectors and demos describe supply. DoorDash’s several hundred orders is a count of use, and the right question for the next earnings call is how many orders, not how many integrations.
• Treat the design choices as open. TIME’s allow list, its labeled sponsored FAQs, Amazon’s block on Muse and the vendors’ own fetcher rules are all decisions by site owners and agent makers. The essay’s hope that agents will make pay-per-article micropayments practical is also being built, and our look at Cloudflare’s agent billing betas asks who holds the meter.

Several statements held up: TIME does serve ads to bots, Stripe’s wording about the underlying card is accurate, the Pew and Digiday figures exist and point the way the essay says, and OpenAI’s number is real and rising. The essay’s hedges are also fair: it says the full-featured, tool-based design its researchers envision “doesn’t yet exist” and that it is “still tough to tell exactly what forms the agentic web will take”. The headline is where the essay gets ahead of its evidence.

What I did not check

• The expert quotations and their attributions, including those of Siva Reddy, Xing Han Lu, Ethan Mollick, David Rothschild, Margaret Mitchell and the Opera and Mozilla executives.
• Mollick’s test of an agent-directed page for his book, reports of Resy blocking agent users, the OpenAI sandbox incident and the German wiki, Claude Duet, the “record downloads”, and the Etsy and Expedia partnerships. For Dots, see our September 29 post.
• The Ozone data itself, which I read through Digiday, and the rest of the Pew report.
• DoorDash’s call audio. I read the Motley Fool transcript page, whose summary section lists the 993 million figure, and Ars quotes the same statement.
• OSWorld 2.0 beyond the abstract, the leaderboard page and OpenAI’s table. The leaderboard is a snapshot from November 2 and may change.
• Any later edits to the essay. I read it on November 2, within hours of publication.

Reproducing the arithmetic

This standard-library script recomputes the derived figures above from the numbers printed on the source pages, and restates the Ozone ranges for comparison. It ran on Python 3.13.14.

# agentic_web_checks.py
# Recomputes the figures in this audit from the numbers printed on each source page.
# Standard library only; tested on Python 3.13.14.

# 1. Pew Research Center, July 22, 2025: share of Google visits that ended in a click on a traditional result link
with_summary, without_summary = 8, 15
drop = (without_summary - with_summary) / without_summary * 100
print(f"Pew: {with_summary}% vs {without_summary}% -> {drop:.1f}% fewer result clicks when a summary shows "
      f"({without_summary / with_summary:.2f}x as often without one)")
print(f"Pew: a 40% drop from {without_summary}% would be {without_summary * 0.6:.1f}%, not {with_summary}%")

# 2. Allouah et al., arXiv 2508.02630v3: a product that starts at a 10% selection rate, then gets a Sponsored tag
cases = {
    "app-style page, Claude Sonnet 4": 8.9,
    "app-style page, GPT-4.1": 8.0,
    "app-style page, Gemini 2.5 Flash": 7.9,
    "headless list, Claude Sonnet 4": 5.4,
    "headless list, GPT-4.1": 1.8,
    "headless list, Gemini 2.5 Flash": 8.9,
}
for name, rate in cases.items():
    print(f"Sponsored tag, {name}: 10.0% -> {rate}% ({(10 - rate) / 10 * 100:.0f}% lower)")

# 3. OSWorld 2.0 leaderboard, 500-step table as listed on Snorkel AI's page: strict completion vs partial score
rows = [
    ("Claude Opus 5, max, batch tool", 44.33, 77.67),
    ("Muse Spark 1.3, max, batch tool", 39.25, 69.56),
    ("Claude Opus 5, xhigh, batch tool", 33.33, 73.11),
    ("GPT-5.6 Sol, max, batch tool", 27.34, 62.72),
    ("Claude Opus 4.8, max, batched tool", 20.6, 54.8),
    ("GPT-5.5, xhigh, batch tool", 13.0, 49.5),
]
for name, completed, partial in rows:
    print(f"OSWorld 2.0, {name}: completes {completed}% of tasks, partial score {partial}% ({partial / completed:.2f}x)")

# 4. DoorDash, third quarter 2026 (Motley Fool transcript page): total orders against "several hundred" agent orders
q3_orders = 993_000_000
generous_agent_orders = 1_000
print(f"DoorDash: {generous_agent_orders:,} / {q3_orders:,} = {generous_agent_orders / q3_orders:.2e}, "
      f"about 1 in {q3_orders / generous_agent_orders:,.0f}")

# 5. Ozone cohort as reported by Digiday, July 15, 2026: what fell by roughly 40% and what fell by less
print("Ozone Q2 ad requests, year on year: U.S. -32% to -37%, U.K. -39% to -41%")
print("Ozone first-half programmatic spend, year on year: U.S. -44%, U.K. -14.3%, combined -30.6%")

Its output:

Pew: 8% vs 15% -> 46.7% fewer result clicks when a summary shows (1.88x as often without one)
Pew: a 40% drop from 15% would be 9.0%, not 8%
Sponsored tag, app-style page, Claude Sonnet 4: 10.0% -> 8.9% (11% lower)
Sponsored tag, app-style page, GPT-4.1: 10.0% -> 8.0% (20% lower)
Sponsored tag, app-style page, Gemini 2.5 Flash: 10.0% -> 7.9% (21% lower)
Sponsored tag, headless list, Claude Sonnet 4: 10.0% -> 5.4% (46% lower)
Sponsored tag, headless list, GPT-4.1: 10.0% -> 1.8% (82% lower)
Sponsored tag, headless list, Gemini 2.5 Flash: 10.0% -> 8.9% (11% lower)
OSWorld 2.0, Claude Opus 5, max, batch tool: completes 44.33% of tasks, partial score 77.67% (1.75x)
OSWorld 2.0, Muse Spark 1.3, max, batch tool: completes 39.25% of tasks, partial score 69.56% (1.77x)
OSWorld 2.0, Claude Opus 5, xhigh, batch tool: completes 33.33% of tasks, partial score 73.11% (2.19x)
OSWorld 2.0, GPT-5.6 Sol, max, batch tool: completes 27.34% of tasks, partial score 62.72% (2.29x)
OSWorld 2.0, Claude Opus 4.8, max, batched tool: completes 20.6% of tasks, partial score 54.8% (2.66x)
OSWorld 2.0, GPT-5.5, xhigh, batch tool: completes 13.0% of tasks, partial score 49.5% (3.81x)
DoorDash: 1,000 / 993,000,000 = 1.01e-06, about 1 in 993,000
Ozone Q2 ad requests, year on year: U.S. -32% to -37%, U.K. -39% to -41%
Ozone first-half programmatic spend, year on year: U.S. -44%, U.K. -14.3%, combined -30.6%

Sources

• Huckins, MIT Technology Review, The agentic web is here, November 2, 2026.
• OpenAI, GPT-6 Astra: A new generation of intelligence; OSWorld 2.0, arXiv 2606.29537 and the leaderboard page.
• Allouah and co-authors, What Is Your AI Agent Buying?, arXiv 2508.02630v3.
• Pew Research Center, Google users are less likely to click on links when an AI summary appears in the results, July 22, 2025.
• Digiday, Publisher ad supply fell by up to 40% in Q2 as AI search choked the open web, July 15, 2026.
• DoorDash, Q3 2026 earnings call transcript (Motley Fool) and connector announcement; Ars Technica, DoorDash says outside AI agents have only led to hundreds of orders so far, October 30; The Verge, AI could upend food delivery, October 7.
• TIME’s agent ads: Schmalbach, TIME serves AI bots a different website; The Register, August 5; The Media Copilot, What TIME learned from a month of selling ads to bots.
• Stripe, Stripe helps Muse, Meta’s new personal AI agent, shop across the internet with Link, September 8, 2026.
• Bot documentation: OpenAI and Perplexity.