Reflection’s Beam Turns an Open 501B Model Into a Memory Budget, and Its Own Serving Guides Stop at 256K Tokens

On October 16, Reflection put the weights of Beam, its first open model, on Hugging Face under an Apache 2.0 license, eleven days after announcing it as a preview. Hugging Face’s launch post, bylined to Ben Burtenshaw, Merve and Pedro Cuenca, pitches Beam to “teams that want a frontier-scale model on their own hardware, infrastructure, or cloud.” It is a mixture-of-experts model with 501.3 billion parameters, 23.4 billion of them active for each token, and a 1M-token context window.

I checked that pitch against the files and the vendor’s own documentation. I recomputed the parameter counts from config.json, summed the checkpoint sizes from Hugging Face’s file listing, rebuilt the key-value (KV) cache table, and read Reflection’s serving guides, the vLLM recipe and the SGLang cookbook side by side. The arithmetic backs the Hugging Face post’s hardware advice: the weights set the entry price, and the cache for even a 1M-token session is a small line beside them. It leaves the 1M headline as a window length rather than a result the vendor has published. Reflection’s own vLLM and SGLang guides run 256K tokens and eight requests on four B300 GPUs that could hold, by my arithmetic, roughly 20 to 40 million tokens of cache, and none of the pages I read breaks out a result by context length beyond 128K.

What the three checkpoints weigh

Table 1 gives the sizes. I summed the .safetensors files in each Hugging Face repository through the Hub’s file-listing API. Reflection’s download page quotes the same figures in GiB, binary gigabytes, where one GiB is 1.074 decimal GB.

The files agree with the config. From config.json I counted 49 mixture-of-experts layers with 256 experts each, and each expert has 39.3 million parameters. The experts add up to 493.25 billion parameters, 98.40% of the total, which is the Hugging Face post’s 98.4%. Everything else (attention, three dense layers, the router, the embeddings) is 8.02 billion parameters, or 16.04 GB in BF16, matching the post’s “8B parameters, or 16 GB in BF16.” The total comes to 501.27 billion and the active count, with eight experts per layer, to 23.43 billion, against the model card’s 501.3 billion and 23.4 billion. The BF16 index file’s byte total, 1,002,663,700,352, is within 634 KB of my count when the router weights and their bias are counted at four bytes instead of two.

The NVFP4 size follows from Reflection’s quantization page. NVFP4 is NVIDIA’s 4-bit floating-point format for Blackwell GPUs, and only the routed experts are quantized with it, as 4-bit values where “each block of 16 values shares one FP8 E4M3 scale,” which is 4.5 bits per parameter, or 277.5 GB for the experts. Attention, the dense layers, the router and the embeddings stay in BF16, another 16.2 GB. My prediction is 293.6 GB and the download is 293.6 GB. The Hugging Face post’s “~280 GB” is close to the experts alone; the whole download is 13.6 GB larger.

A note on units, because this arithmetic mixes sources. Reflection’s pages count in GiB, while NVIDIA lists GPU memory in “GB”: 141 for an H200, 1,440 GB for the eight B200s in a DGX B200 (180 each, the figure the Hugging Face post uses) and 2.1 TB for the eight B300s in a DGX B300, where the post uses 288 GB per B300. Reflection’s FP8 page reports 135.08 GiB in use per H200, which cannot fit in 141 decimal gigabytes (131.3 GiB), so NVIDIA’s unit is binary in practice. I read every GPU figure as GiB, which is an assumption for the B200 and the B300, and I use the lower B300 figure.

The cache is the small line

The Hugging Face post’s cache table holds up. Nine of Beam’s 52 layers attend to the whole context and the other 43 look back 512 tokens. With 8 key-value heads of 128 dimensions, the cache grows by 36 KiB per token in BF16, which is 36.08 GiB at 1M tokens. Table 2 adds the 256K length that Reflection’s guides use and an FP8 column, because the SGLang guide and the vLLM recipe turn on an FP8 cache.

The rows reproduce in about a dozen lines of Python that read the same config file:

# kv.py
import json
import urllib.request

URL = "https://huggingface.co/reflection/Beam-1.0-501B-A23B/raw/main/config.json"
cfg = json.load(urllib.request.urlopen(URL))
kinds = cfg["layer_types"]
full, window = kinds.count("full_attention"), kinds.count("sliding_attention")
values = 2 * cfg["num_key_value_heads"] * cfg["head_dim"]  # K and V values per token per layer


def kv_gib(tokens, bytes_per_value):
    kept = min(tokens, cfg["sliding_window"])
    return (full * tokens + window * kept) * values * bytes_per_value / 2**30


for tokens in (262_144, 1_048_576):
    print(f"{tokens:>9,} tokens: BF16 {kv_gib(tokens, 2):.2f} GiB, FP8 {kv_gib(tokens, 1):.2f} GiB")

  262,144 tokens: BF16 9.08 GiB, FP8 4.54 GiB
1,048,576 tokens: BF16 36.08 GiB, FP8 18.04 GiB

Reflection’s NVFP4 page adds a caveat that a cache table cannot show: the checkpoint “does not include calibrated KV-cache scales,” so an FP8 cache runs at a scale of 1.0. The FP8 page says its FP8 scores used an FP8 cache. The model card and the NVFP4 page do not say which cache type the BF16 and NVFP4 scores used, so the FP8 column is a memory figure, not a setting with a published quality result.

Table 3 asks how many full-length sessions the memory left after the weights could hold. It is an upper bound: it ignores activations, CUDA graphs, allocator overhead and the drafter. It uses 90% of GPU memory, because the SGLang guide sets that fraction, Reflection’s vLLM guide sets 95% and vLLM’s main branch defaults to 92%.

The post’s hardware advice survives. Two B200s hold the weights “with little room left for activations and the KV cache,” and the table agrees: room for one full-length session at most with a BF16 cache, and none if NVIDIA’s B200 “GB” were decimal. “Plan for four B200s (720 GB), which leaves plenty of room for long contexts and concurrent requests” also agrees, at 10 to 20 sessions of the full 1M tokens. For the Hopper route, the vLLM recipe says the FP8 checkpoint on eight H200s holds “about 14M to 28M tokens” of cache. My arithmetic gives 15.9 million at BF16 and 31.8 million at FP8 before overhead, about 14% above the recipe’s range at both ends, which is the direction overhead would push it.

Reflection’s own recipes run a smaller Beam

Table 4 sets out the context and concurrency in each place the numbers appear.

Reflection calls each of its guides “one setup, for development and evaluation.” The vLLM guide caps a request at 262,144 tokens and the server at eight requests, so at most 2.1 million tokens of cache are ever live, which is 5 to 10% of what four B300s could hold (21 million tokens at BF16 and 42 million at FP8, on my arithmetic at the guide’s 95% memory use). The SGLang guide caps the whole cache pool at 262,144 tokens, under 1% of the FP8 figure, with a flag that SGLang’s argument reference says is “typically used for development and debugging purposes.” Neither guide says why. The nearest number is in the launch post: the reinforcement-learning run used “a maximum context length of 256K tokens,” and midtraining “extends Beam’s effective context length to 1M tokens.”

The evidence past 256K is thin in the pages I read. The DFlash2 drafter card’s long-context table stops at the 64k to 128k prompt bucket. The model card lists LongBench v2 and AA-LCR as one number each, and Reflection’s NVFP4 page says that quantization was calibrated on 1,000 sequences of 32,768 tokens each. Beam Verifier, the vendor’s endpoint checker, ships reference scores for GPQA Diamond (90.6%) and AIME 2026 (97.7%) and lists “GSM8K and three InfiniteBench long-context sets” as optional ones “without a reference score.” The model’s technical report is marked “coming” on the Hugging Face card.

None of that says 1M does not work. The vLLM recipe and Hugging Face’s SGLang command both serve the full length, and the hybrid attention makes the cache affordable. It says that a buyer who needs 1M-token sessions has a window length and memory arithmetic, not a vendor-reported result at that length, and should test long prompts on its own tasks before sizing a fleet around them.

“The speed of a 23B one” holds for a single request

The Hugging Face post says deployment “needs the memory of a 501B model but runs at the speed of a 23B one.” That is true of compute per token. Memory traffic depends on how many tokens share a step, because each token picks 8 of a layer’s 256 experts. If routing were independent across tokens, a step carrying T tokens would touch one minus (31/32) to the power T of a layer’s experts. Table 5 applies that to the NVFP4 checkpoint, where the always-read part is 16.2 GB and the experts are 277.5 GB.

The 64-token row is the guides’ own setup: eight concurrent requests, each verifying a block of about eight tokens with the DFlash2 drafter. This is my arithmetic and an upper-end estimate. Reflection reports “almost-perfect uniform utilization” of experts on its base model, but the tokens of one request are probably correlated, which would lower the count, and expert and tensor parallelism spread the reads across GPUs. I measured nothing.

Reflection’s own measurements show the effect from the outside. On the drafter card, using the NVFP4 checkpoint on four B300s with greedy decoding on GSM8K, one request decodes at 288 tokens a second without the drafter. Sixteen concurrent requests produce 2,515 tokens a second in total (about 157 each) and thirty-two produce 3,931 (about 123 each): 13.7 times the throughput for 32 times the requests. Expert reads, communication and attention all plausibly contribute, and I cannot separate them. The point for planning is that a capacity estimate built from the single-request figure will be too generous, and the concurrency rows are the ones to plan from.

What the benchmark tables settle

Beam’s scores appear in two places, and they do not match. The launch post’s table, updated on October 8, and the model card’s table, which Reflection’s availability post says “includes final benchmarks,” share 21 benchmarks. The launch post does not say which precision its Beam numbers are for. Against the card’s BF16 column they differ by 0.94 points on average and by as much as 2.8 (SWE Atlas Codebase QnA, 34.6 against 31.8); five rows differ by 2 points or more. The card says its runs used temperature 0.7 and top-p 0.99. The launch post gives no sampling settings, and neither page says how many runs were averaged.

The same noise limits what the card’s precision columns can show. FP8 scores above BF16 on 9 of the card’s 23 rows and NVFP4 on 4; on SWE Bench Pro v2-Hard the BF16 score is 76.5% and the FP8 and NVFP4 scores are 78.4% and 78.6%. The largest drop is NVFP4 on SciCode, 2.3 points. SWE-bench Verified has 500 tasks, so its 1.4-point BF16-to-FP8 gap is the equivalent of seven tasks. The post’s statement that NVFP4 “retains most of the quality of the bf16 checkpoint” can be neither confirmed nor refuted at this resolution; the practical check is Beam Verifier plus your own tasks.

Against peers, Reflection’s own table is candid. Counting the rows where both models report a score, Beam is higher than GLM 5.2 on 8 of 13, than GLM 5.3 on 0 of 12, than Kimi K3 on 1 of 14, than Qwen 3.8 Max on 0 of 13 and than DeepSeek V4.1 Flash on 2 of 9, and it beats Inkling, the Thinking Machines model covered here in July, on 13 of 15. The launch post says Beam is “competitive with larger open models like GLM 5.2” and that “where frontier open models like Kimi K3 remain ahead on raw capability, Beam’s advantage is efficiency at inference time.”

The efficiency claim appears in two forms. The launch post says Beam reaches scores comparable to GLM 5.2 on advanced reasoning benchmarks while using three to four times less inference compute. Its Figure 2 caption says Beam’s efficiency holds “both when measured in terms of FLOPS and token count,” with FLOPs estimated as twice the active parameter count times the mean generated tokens, an estimate the caption calls “an approximate compute comparison rather than measured inference cost.” The model card says Beam uses “approximately 3x fewer inference-time reasoning tokens.” FLOPs and tokens are different quantities, and the FLOP estimate depends on active parameter counts as well as tokens. The data sits in the figure, not in a table I could read, so I could check neither number.

What “open” covers here

The license is Apache 2.0: the repository’s LICENSE file matches the text Apache publishes once whitespace is normalized, and the model card says it “does not modify or restrict the model’s license.” The card adds that Reflection “does not prohibit distillation,” use of outputs to train other models, fine-tuning, quantization or merging. What is not yet public, on the pages I read, is the technical report, still marked “coming,” and the training data, which the card describes by category rather than listing.

Self-hosting is one of several routes. Reflection’s provider page lists Amazon SageMaker and Google Cloud Model Garden as available, Azure and Oracle as coming soon, Dell’s Enterprise Hub for on-premises systems and 17 inference providers, among them Baseten, Fireworks AI, Together AI, Cloudflare and CoreWeave. Its docs also carry Kubernetes and NVIDIA Dynamo guides for scaling past one server, which I did not test; this site’s piece on prefill and decode disaggregation explains why such stacks split a request’s two phases.

Where the Hugging Face post and Reflection’s docs differ

These are differences I found on October 18, and either side may change.

• The vLLM install. The post says “Beam has day-0 support in vLLM” and gives a plain vllm serve command. The vLLM recipe says “Beam support has not shipped in a vLLM release yet” and tells readers to use the vllm/vllm-openai:reflection-beam image. I found no Beam entry in vLLM’s model registry at v0.32.0, the newest release on PyPI (published October 18), or on main.
• The drafter. The post’s vLLM command names reflection/dflash2-drafter-preview and 15 speculative tokens. That repository answers anonymous requests with HTTP 401, the same code Hugging Face returns for a made-up name, so it is private or missing; Reflection’s guides use reflection/Beam-1.0-501B-A23B-DFlash2, which is public. The vLLM recipe says the drafter supports up to 15 tokens and “7 is recommended,” and Reflection’s speculative decoding page says “blocks of 8 are recommended for serving.”
• Which Blackwell GPU. The post says to plan four B200s for NVFP4. Reflection’s NVFP4 page says native NVFP4 “requires NVIDIA Blackwell GPUs” and lists one validated configuration, four B300s, adding that the count “is per copy of the model and is not a minimum requirement.” The SGLang cookbook says “NVFP4 requires B300 or GB300,” while the vLLM recipe lists four B200s or B300s.

What I would do with this

• Size from the checkpoint. NVFP4 is 273.5 GiB and needs Blackwell, with four B300s as the validated unit; FP8 is 468.7 GiB and fits eight H200s. The vLLM recipe’s minimums, weights plus 20% headroom, are 353 GB for NVFP4 and 604 GB for FP8.
• Price sessions, not the window. At 256K a session’s cache is 9.08 GiB in BF16, so every row of Table 3 has room; at 1M it is 36.08 GiB, and Table 3 says how many fit.
• Plan throughput from concurrency. The vendor’s figure is 123 tokens a second per request at 32 concurrent requests, not 288, and that is on short GSM8K prompts.
• Verify before trusting. Run Beam Verifier on the endpoint, then add your own long-prompt and tool-calling tests, since the verifier has no long-context reference score.
• Compare with a hosted endpoint if your load is bursty, because the vendor’s validated unit is four B300s whether one session or thirty are running.

What I did not check

• I ran no Beam inference, so every speed, quality and memory-use figure here is either Reflection’s or my arithmetic.
• The cache numbers are model-level bytes. They ignore allocator overhead, activations, CUDA graphs, prefix caching and the drafter’s own cache.
• I did not read the Beam Safety Technical Report, the Kubernetes or Dynamo guides, or the TokenSpeed recipe.
• The B200 and B300 memory figures are NVIDIA’s, read as GiB; the evidence for that reading comes from Reflection’s own H200 measurement.
• I could not check the figure behind the efficiency claim.

Sources and method

The sources are Hugging Face’s launch post (October 16) and the model’s repositories (BF16, FP8, NVFP4 and the drafter: config files, README, index, quantization config and file listings through the Hub API, fetched on October 18); Reflection’s launch post (October 5, updated October 8), availability post (October 16) and model card; Reflection’s developer docs (the download, vLLM, SGLang, NVFP4, FP8, speculative decoding, Beam Verifier and provider pages); the vLLM recipe (updated October 16); SGLang’s Beam cookbook and server-argument reference; vLLM’s source on GitHub and its PyPI release list; and NVIDIA’s H200, DGX B200 and DGX B300 pages.

I saved the text of each source and checked every quotation in this piece against it. Parameter counts, byte totals, cache sizes, headroom, expert coverage and the benchmark comparisons are my own arithmetic, produced by a script from the files above; the cache arithmetic is reproduced in the code block. The HTTP 401 and the registry and PyPI checks are as of the morning of October 18 and can change.