Google’s ToolGrad Turns Tool-Use Training Data Into an Answer-First Bet

Google Research spent September 10 publicizing a technique that inverts how large language models learn to use software tools, and the headline result is blunt: a 12-billion-parameter open-weight model fine-tuned on data built this way scored within a tenth of a point of Gemini 2.5 Pro on a widely used tool-use benchmark, while clearing GPT-5 by close to nine points. The technique is called ToolGrad, and its central idea is almost mundane to state and surprisingly hard to execute well: generate the correct answer first, then write the question that goes with it.

The result matters less as a product launch than as a data point in a bigger argument running through the AI industry right now: is the ceiling on how well an AI agent uses tools set by the size of the model, or by the quality of the data it was trained on? ToolGrad’s authors, most of whom worked on the project through an academic collaboration rather than a Google product team, come down firmly on the side of data quality.

The Old Way: Guess a Question, Then Go Hunting for an Answer

Teaching a model to call APIs reliably, whether that means running a web search, reading a local file, or hitting an internal company system, requires training examples: a user request paired with the correct sequence of tool calls that satisfies it. Building those pairs at scale has historically run in one direction. First, sample a pool of available APIs and have an LLM invent a plausible user instruction that would require some of them. Second, hand that instruction to a search agent, typically a depth-first search (DFS) process, and let it try, backtrack, and try again until it stumbles onto a tool-use chain that actually answers the question.

Google’s blog post, credited to research scientist Zhongyi Zhou and Google XR’s interactive perception and graphics lead Ruofei Du, describes the flaw in that approach plainly: it is “inherently inefficient because its core concept is to distill valuable trajectories from a complex agent exploration for training an LLM.” Much of the search agent’s exploration is wasted. Some invented questions turn out to have no clean tool-use solution at all, and the ones that do often need expensive, multi-step trial and error to find. Cost and failure rate both climb as the tool library grows and the desired chains get longer, which is exactly the direction agentic AI is heading.

Flip the Order: Build the Answer First, Then Ask the Question

ToolGrad, formally described in a paper on arXiv, reverses that sequence. Instead of starting from a guessed question and searching for a chain that answers it, the framework first constructs a verified, working sequence of API calls, then runs a separate step that writes a natural-language user query the chain would actually satisfy.

The logic, as Google’s post frames it, is that “an explicit tool-use solution provides more unambiguous information than a prompt, making the annotation, from tool usage to the use query, much easier and requiring only one LLM step.” Put another way: describing in plain language what a known-working sequence of tool calls accomplishes is far easier than guessing in advance whether some invented question is even solvable.

Four Modules in a Loop

The paper names four cooperating components that run iteratively to build each tool-use chain, drawing an explicit comparison to how neural networks improve through gradient descent:

• API Proposer: narrows a sampled pool of candidate APIs down to a few promising options that could extend the workflow built so far.
• API Executors: run those candidates in parallel against the current workflow state and generate detailed execution reports.
• API Selector: reviews the reports and picks the single best-performing call, a step Google describes as acting like “a textual gradient that provides directional feedback for improvement.”
• LLM Updater: revises the synthetic user query and the model’s draft response to stay consistent with the newly added call.

Repeating that loop produces one training sample: a verified, executable tool-use chain plus a matching user query and final response. Because the workflow is proven to work before any prompt is written, the resulting data hits what Google calls an “almost 100% pass rate,” a sharp contrast with DFS-based generation, where a meaningful share of invented questions never resolve to a working answer.

Where “Gradient” Comes From

The name borrows directly from machine learning. A standard neural network improves by computing numerical loss gradients across batches of training examples, then nudging its weights in the direction those gradients point. Google’s post traces the textual version of that idea to TextGrad, an earlier framework that used an LLM acting as a critic to produce rich, written feedback on a prompt, in effect a gradient expressed in language rather than numbers, which then guided the prompt’s next revision.

ToolGrad borrows that mechanism and repurposes it. Instead of optimizing a static prompt, its API Selector step reviews execution reports the way TextGrad’s critic reviews a draft, and the “best-performing API call” it chooses functions as the directional signal that steers the next iteration of the workflow. The training data itself, not a prompt, is what gets built through the gradient-like loop.

The Benchmark Numbers

To test the approach, the team generated a small dataset called ToolGrad-500 using ToolBench’s library of more than 16,000 real-world APIs, then fine-tuned three sizes of Google’s open Gemma 3 model (1B, 4B, and 12B parameters) on it, naming the results ToolGrad-1B, ToolGrad-4B, and ToolGrad-12B. They evaluated all three on the Berkeley Function Calling Leaderboard (BFCL), an independent benchmark from UC Berkeley’s Gorilla LLM team that tests tool-calling accuracy against a completely different tool set than ToolBench, a genuine out-of-distribution test rather than one the training data was tuned to pass. BFCL’s own documentation defines its headline number as an “Overall Accuracy” score: an unweighted average across every sub-category in the benchmark, out of 100.

Against that benchmark, ToolGrad-12B scored 83.1, a fraction behind Gemini 2.5 Pro’s 83.2, just ahead of Claude 4.5 Opus’s 82.8, and 8.7 points clear of GPT-5’s 74.4. Google’s post calls the smaller model “highly competitive with the industry’s most advanced proprietary models,” and the numbers back that up for this specific task, even if a single benchmark obviously does not capture everything those larger models can do.

A Model That Outgrew Its Own Teacher

One detail in the results deserves more attention than a single bullet point: the ToolGrad-500 dataset used to train these models was itself generated by gemini-2.5-flash-lite, a smaller, cheaper Gemini model acting as the “teacher.” Gemma-3-12B, fine-tuned on that teacher’s output, ended up outperforming the teacher itself. Google’s post labels this “self-evolving” behavior. It is a small-scale example of a pattern researchers elsewhere call weak-to-strong generalization: a less capable model can still generate training signal that lets a student model exceed it, provided the generation process filters out the failures before they ever reach the student.

Beating the Other Open Tool-Use Specialists

ToolGrad-12B was also compared against purpose-built open tool-use models rather than just general-purpose frontier LLMs, specifically ToolACE and Hammer-2.1-7B. Google reports that ToolGrad-12B “establishes a clear lead” over both, including ToolACE, which the post notes was fine-tuned on a more advanced API database than the one ToolGrad-500 was built from. If accurate, that suggests the gain is coming from how the data was generated rather than from a richer source of APIs to draw on.

Who Actually Built This

It is worth being precise about the provenance here, because a Google-branded blog post can read like a Google product announcement when it is closer to Google amplifying outside academic research. The post’s acknowledgements section states plainly that “this research was primarily conducted by Zhongyi Zhou during his Visiting Researcher tenure at Google.” Co-author Tatsuya Harada, per his own institutional page at the University of Tokyo’s Research Center for Advanced Science and Technology, is a professor there and a group director at RIKEN’s Center for Advanced Intelligence Project, not a Google employee. The paper’s author list mixes a Google-based co-author, Ruofei Du, with what reads as Harada’s University of Tokyo lab, a genuine academic-industry collaboration rather than an internal Google product effort.

The timeline is also longer than the blog post alone suggests. The paper first went up on arXiv in August 2025, went through two further revisions, and was formally published in July 2026 as part of the Findings track at ACL, the Association for Computational Linguistics’ flagship annual conference, held that year in San Diego. Google’s blog post arrived roughly two months after that publication, which makes it a wider public re-introduction of already peer-reviewed work rather than a fresh research announcement. None of that undercuts the result, but it does mean readers should treat this as Google spotlighting a completed, reviewed paper, not as breaking news from an active lab.

Why It Matters Beyond the Benchmark

The practical argument buried in these numbers is aimed squarely at enterprise AI teams: the model doing the comparing and the model being compared to are not neutral third parties here. Gemma 3 is Google’s own open-weight family, and the “teacher” that generated ToolGrad-500’s training data, gemini-2.5-flash-lite, is also a Google model. What the results actually show is that Google’s own smaller, freely downloadable model, trained on synthetic data from Google’s own mid-tier model, lands within striking distance of Google’s own flagship proprietary model, Gemini 2.5 Pro, on this specific narrow task. That is a more precise claim than “small beats big,” and arguably a more useful one for a team deciding whether to keep paying per-token for a frontier API or fine-tune something they can run themselves.

Google’s own closing line makes the enterprise pitch explicit: “As agentic workflows become increasingly embedded in enterprise and everyday tasks, frameworks like ToolGrad lay the essential groundwork for training digital agents that are both highly capable and economically scalable to deploy.” The company’s stated next steps point the same direction: scaling the framework to handle larger, more dynamic API ecosystems, and extending the self-evolving property toward continuous, on-the-fly personalization rather than a one-time fine-tuning run.

It also lands in the middle of an active argument about who gets to build training data this way. Earlier this month, CISA, the NSA, and the FBI jointly accused six Chinese AI companies of industrial-scale model distillation, systematically extracting outputs from US frontier models to train cheaper rivals. ToolGrad is a different mechanism (its training data comes from a model’s own verified tool executions, not from querying a competitor’s API), but the underlying dynamic is the same one regulators are worried about: efficient data-generation techniques are becoming as valuable a lever as raw compute, and they travel easily once published.

Trying ToolGrad Yourself

The code, the ToolGrad-500 dataset, and the fine-tuned models are published under an Apache 2.0 license on GitHub, alongside a paper link and citation. The repository’s quick-start path uses uv venv and uv sync to set up a Python environment, then walks through a framework demo that needs no GPU: a Model Context Protocol (MCP) filesystem example that requires Node.js and a Gemini API key. Reproducing the full BFCL evaluation results requires a GPU, and generating a custom ToolGrad dataset from scratch requires a ToolBench API key. At the time of writing, the repository had 71 stars and 9 forks, a modest footprint for a paper with these benchmark numbers, which fits the fact that this is a research release rather than a marketed product.

For developers who want a hands-on comparison point, sxz.io’s own walkthrough of building a local tool-calling agent with a 2.6-billion-parameter model shows what an efficient, self-hosted tool-use setup looks like at the smaller end of the scale ToolGrad is arguing for. And for a look at how AI agent tooling is evolving from a different angle entirely, the harness layer that sits between a model and the tools it calls, see this week’s coverage of DeepSeek’s viral harness release and the three competing bets on agent architecture it has sparked. Training data, benchmarks, and harness design are turning out to be three separate fronts in the same race to make AI agents both capable and cheap enough to actually deploy.