LLM Groupthink Makes AI Creativity an Evaluation Problem

MIT Technology Review’s July 2 edition of The Download points to a deceptively small failure mode in modern chatbots: ask several leading models for a random number from one to ten and, surprisingly often, the answer is seven. The linked MIT Technology Review feature by Will Douglas Heaven argues that large language models can be far more predictable than users expect, especially when the request is open-ended rather than factual.

The point is not that a chatbot choosing seven is dangerous by itself. The larger issue is that many AI products are now being asked to brainstorm, design options, summarize strategy, write copy, plan trips, recommend architectures, and act as creative collaborators. In those settings, sameness is not a harmless quirk. It is a product-quality problem that ordinary accuracy benchmarks can miss.

Why the “random seven” demo matters

In the MIT Technology Review feature, Springboards cofounder and CEO Pip Bingemann uses the random-number prompt as a sales trick for the company’s model, Flint. The story describes mainstream chatbots repeatedly choosing seven, then compares that behavior with Flint producing a less conventional value in a restarted session. MIT Technology Review also cites examples in which generic models converged on familiar car names and campaign taglines while Flint produced a different, if not necessarily better, answer.

That distinction is important. A model can be correct and still be unhelpful for ideation. If every tool returns the first plausible answer from the same statistical neighborhood, a team gets speed without much exploration. For coding, compliance summaries, and factual research, convergence can be useful. For brand strategy, design review, scenario planning, product naming, or incident-response brainstorming, convergence can silently narrow the option set before humans have done any real thinking.

Springboards is a product signal, not a universal fix

Springboards presents itself as an AI platform for advertising and marketing teams that want “better ideas, not just faster answers.” Its own site says most AI models narrow toward the most likely answer and that Springboards is designed to widen the field. The company says it measures output diversity by embedding responses and visualizing them in two-dimensional space, and claims that for common advertising and marketing tasks its responses covered 10 to 30 times more creative territory than leading general-purpose LLMs.

Those are vendor claims and should be treated as claims, not settled science. But the product direction is revealing: the next AI quality contest is not only about more tokens, larger context windows, or higher benchmark scores. It is also about whether systems can deliberately preserve useful variety without collapsing into nonsense.

The research problem is broader than one startup

The MIT article connects the product story to a research thread around open-ended homogeneity. The arXiv paper “Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)” says language models often struggle to generate diverse, human-like creative content and introduces Infinity-Chat, a dataset of 26,000 real-world, open-ended user queries. The paper says those prompts admit many plausible answers rather than one ground truth, making them a better test bed for diversity than narrow random-number experiments.

The paper’s abstract reports both intra-model repetition and inter-model homogeneity, meaning one model can repeat itself and different models can converge on strikingly similar outputs. It also says Infinity-Chat includes 31,250 human annotations across absolute ratings and pairwise preferences, with 25 independent annotations per example. That matters because the paper frames diversity not merely as “more random text,” but as a mismatch between model preferences, reward models, model judges, and individual human preferences in open-ended tasks.

Open-ended prompts need different gates

A factual benchmark usually asks whether a model reached the right answer. An ideation benchmark needs to ask a different set of questions: Did the system explore distinct directions? Did it surface minority-but-plausible options? Did it avoid repeating the same metaphor, architecture, or slogan across users? Did it preserve enough surprise without sacrificing usefulness, safety, or domain constraints?

That is why “groupthink” is a stronger product label than “low creativity.” The failure is not only aesthetic. A support agent that always suggests the same escalation path, a planning assistant that always proposes the same vendor pattern, or an AI coding assistant that always reaches for the same architecture can reduce organizational resilience. The risk is subtle because the answers look polished.

Temperature is a blunt instrument

Most model stacks already expose sampling controls. The Hugging Face Transformers generation documentation describes settings such as do_sample, temperature, top_k, and top_p. Temperature changes next-token probabilities; top-p keeps the smallest set of probable tokens whose probabilities add up to the configured threshold. These are useful controls, but they operate at the decoding layer.

The MIT feature says Springboards tried ordinary randomness settings and found them too blunt for its goal. The problem is easy to see: if the task is “Where should I go in Europe?”, the model may need more variety at the point where it names a destination, not across every connective phrase in the sentence. Raising randomness everywhere can make output less coherent without making the important choice more interesting.

Task-level diversity is different from token-level noise

For enterprise teams, the practical lesson is that diversity has to be scoped. A customer-support response should not invent policy. A security runbook should not become “creative” with evidence. A brainstorming assistant, however, may need to produce options that differ by audience, price point, geography, risk tolerance, architecture, or narrative frame. The control should apply to the part of the task where variety is valuable.

Treat “more random” as a requirements smell

If a product requirement says only “make the model more random,” it is probably underspecified. A better requirement names the dimension of variation, the safety boundaries, the evaluation method, and the human decision point. For example: “generate ten cloud migration plans that differ by cost profile, operational risk, and lock-in assumptions, while keeping all compliance constraints fixed.” That is a diversity requirement. It is also testable.

Qwen3 shows why open weights matter here

MIT Technology Review reports that Springboards built Flint on top of Qwen 3. The Qwen3 announcement says the Qwen team open-weighted two mixture-of-experts models and six dense models, with model sizes ranging from Qwen3-0.6B to Qwen3-235B-A22B. That open-weight base is relevant because many diversity experiments will happen above or around the base model: fine-tuning, preference data, custom decoding, domain-specific evaluations, and product-specific guardrails.

Open weights do not solve homogeneity by themselves. MIT Technology Review says the researchers behind the artificial-hivemind work speculate that similar training methods, similar data, and similar objectives may help explain convergence across models. But open weights give smaller teams a place to test countermeasures without training a frontier model from scratch.

A practical evaluation checklist

Teams adding AI to creative, strategy, or planning workflows should not wait for a universal benchmark. They can build a local evaluation gate that treats sameness as a measurable failure mode.

• Separate factual tasks from ideation tasks. Do not reward novelty where correctness is the priority, and do not reward generic correctness where variety is the product.
• Run repeated trials across models and prompts. Sample multiple outputs from the same model and compare them with outputs from competing models.
• Cluster outputs by meaning, not just wording. Embeddings, topic labels, or human review can reveal when ten polished answers are really one answer in different clothes.
• Track user-specific preference differences. The arXiv paper’s emphasis on idiosyncratic human preferences is a reminder that one aggregate score can erase valuable disagreement.
• Keep human judgment in the loop. Variety is useful only when people can compare options, reject weak ones, and explain why a less obvious answer is worth pursuing.

What this changes for AI product teams

The next release gate for many AI tools should include a diversity report alongside accuracy, latency, cost, and safety checks. That report does not have to be complicated at first. It should document the open-ended prompt set, the number of samples per prompt, the clustering method, examples of repeated patterns, examples of useful divergence, and the human-review rubric. Without that evidence, teams may ship models that feel fast and intelligent while quietly standardizing everyone’s ideas.

The bottom line

Springboards and Flint are interesting because they turn a research concern into a product requirement: sometimes the AI system should not race to the center of the distribution. But the broader lesson is bigger than one marketing tool. As generative AI becomes infrastructure for work, teams need to decide where sameness is a feature, where it is a risk, and how they will measure the difference.

For factual answers, “average” may be exactly what users want. For creative exploration, strategy, and planning, average can become a hidden bottleneck. LLM groupthink should therefore be treated as an evaluation problem before it becomes an organizational habit.

Sources

• MIT Technology Review: The Download: a startup has a solution for AI’s groupthink problem
• MIT Technology Review: LLMs are stuck in a groupthink groove. This startup is trying to get them out.
• Springboards: The Creative AI Tool for Agencies & Strategists
• arXiv: Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)
• Hugging Face Transformers: GenerationConfig and text generation parameters
• Qwen: Qwen3: Think Deeper, Act Faster
• Featured image source: NOAA diver and bigeye scad school on Wikimedia Commons
• Wikimedia Commons: PD-USGov-NOAA license template

Image credit: NOAA Fisheries / Jeff Milisen, public domain U.S. NOAA work via Wikimedia Commons; cropped, resized, and converted to WebP for sxz.io.