DeepSeek published DeepInfer last week, the inference engine it runs behind its own service, together with the kernels it uses. SGLang's team then took DeepInfer's own benchmarks, ran both engines on the same hardware, and published the result on 24 October. The numbers are close: up to 1.04x prefill throughput for SGLang, decode steps 1.1 to 7.4 percent shorter on 64 GPUs, and within 1 percent at the remaining measured points.
If you are choosing what to run on servers you own, the interesting part is not who won by four percent on prefill. It is why the two land that close, what the comparison does not cover, and what is left for you to operate either way.
What was measured
Prefill is measured on a single four-GPU node, first four GB300 and then four B300. Every request carries a 128,000-token context and each GPU prefills between 4,000 and 64,000 new tokens per step. SGLang is ahead at 4K, 32K and 64K new tokens per step on GB300, and at 4K, 8K and 32K on B300, with parity at the points in between.
Decode is measured at a different scale: 64 GB300 across 16 nodes, with data-parallel attention and expert parallelism on both sides (DP64/EP64), 128,000-token contexts sharing a cached prefix, and between 8 and 256 requests per GPU. SGLang's step is 1.1 to 7.4 percent shorter from 16 requests per GPU upward and on par at 8.
So the two halves of the comparison describe two different machines. That matters when you quote it: prefill evidence from four GPUs says little about a 64-GPU decode fleet, and the decode evidence is taken at a context length where memory layout dominates.
The kernels are the same on both sides
The reason the numbers are close is stated plainly in the post. SGLang now runs DeepSeek's kernels, FlashMLA, DeepGEMM, DeepEP, DeepSelect and the TileLang kernels, plus the Mega series: MegaAttention, MegaMoE, MegaGate and Mega-mHC. MegaAttention is the default attention path on B300 and GB300. SGLang also runs the checkpoint's DSpark speculative decoding and SWA bounded replay, where the late decoder layers reprocess only the last 128 prompt tokens.
When both engines call the same kernels on the same GPU model, a step-time benchmark is measuring the work around the kernels: how steps are scheduled, how memory is laid out, how much intermediate traffic is avoided. Both engines have had years to tune that layer, and the result is two implementations in the same performance class.
We run SGLang in our own inference work, so the practical reading is reassuring rather than dramatic. It means the engine choice is no longer mainly a question of who has the faster kernel for this model family. It also means the differences that remain are the ones you have to operate.
Three reasons to read the table as directional
The appendix lists where the two runs differ, and those lines matter more than the headline.
DeepInfer times prefill over about nine batches on GPU 0. SGLang averages every complete batch on every GPU between the point where 20 percent of requests have finished and the point where 90 percent have. DeepInfer repeats the same prompt across requests on a GPU; SGLang generates different new tokens for each. Both are defensible ways to measure a step and they do not time the same interval.
On decode, SGLang keeps the model's Engram tables in device memory while DeepInfer keeps them in host memory, and DeepInfer's draft model carries 64 redundant experts where SGLang's has none. Those are differences in what the hardware is doing, not just in how the clock is read.
The SGLang run also enables, in the authors' words, a few optimizations that are not on by default yet, with balanced expert routing on and expert-parallel load balancing off. The post says those settings exist for the comparison only.
None of that makes the comparison dishonest. It makes it a comparison of two specific configurations rather than of two products at their defaults. Read it as evidence that the engines are in the same class. We have not reproduced it, and a single vendor-run comparison is not a reason to change a deployment on its own.
What the table leaves out, and you would still run
The sharpest part of the post is not the results section. DeepInfer's README, as the post quotes it, lists three parts of a complete inference service that its release does not include: a key-value cache system, prefill and decode running on separate servers, and load balancing across replicas. SGLang ships all three, and the details are where an operating plan actually lives.
The cache is a unified radix tree that reuses key-value blocks across requests and conversation turns. In the configuration shown, Linker connects local GPU caches directly to shared L3 storage such as 3FS or Mooncake, and HiCache adds a host-memory tier for deployments that want one. That is the machinery that decides how much of a node's memory is available for context rather than weights, and what happens to a conversation's cache when a node is drained.
The split puts prefill and decode on separate servers and moves the cache between them. SGLang's documentation currently lists Mooncake and NIXL as the transfer backends, and the model gateway distributes requests across replicas, including routing by cached prefix.
Every one of those components is something an operator configures, monitors and restarts. A step-time benchmark of a single engine says nothing about them, and in a private deployment they are most of the work.
What to ask instead
The comparison is public: More Than DeepInfer: What SGLang Delivers, published on 24 October, together with the commits and a gist holding the launch commands and benchmark clients for both engines. Someone with the hardware can rerun it.
That is the standard to hold other engine comparisons to. When a supplier sends a chart, the useful questions are which commit, which configuration, which interval was timed, and whether the settings are the defaults a customer would receive. A number without those four answers is hard to act on, however carefully it was produced.
The next time an engine comparison arrives, read the appendix before the bar chart. It tells you whether the number describes your workload or the author's configuration.