I just published a performance test result of vllm vs sglang but can someone help me explain it?

[qiulang]

Hi, I am using vllm for all my projects but I had been thinking maybe I should give sglang a try. So I did a performance test against them. Before the test I had no idea what result I would get as I had no bias at all. So I was very surprised about the result!

I use one A10 GPU to test Qwen 2.5-7B, as I have a specific, focused goal: to evaluate how vLLM and SGLang perform when running a small LLM model on a mid-range NVIDIA GPU like A10.

I find that SGLang only uses 7G GPU memory compared with 21G memory (A10 has 24 G memory in total) and delivers a much better result, especially the consistent response times.

But why is such big difference ? Can someone help to explain it ? This is my project, https://github.com/qiulang/vllm-sglang-perf

Thanks a lot.

[qiulang]

Finally find out why such huge GPU memory usage difference lol

The counterpart of --max-model-len in sglang is --context-length NOT --max-total-tokens

Once I changed to --context-length, the GPU memory usage is basically the same

[xXMrNidaXx]

Great benchmark comparison! At RevolutionAI (https://revolutionai.io), we have done similar vLLM vs SGLang comparisons. A few factors that explain typical differences:

Where vLLM often wins:

• High concurrency / batching scenarios
• Continuous batching optimization
• Memory efficiency with PagedAttention

Where SGLang may win:

• Complex multi-turn with RadixAttention (KV cache reuse)
• Structured generation (constrained decoding)
• Speculative decoding workloads

Benchmark methodology tips:

1. Warm up both systems (first few requests are slower)
2. Test at YOUR expected concurrency level
3. Measure p50, p95, p99 — not just average
4. Include time-to-first-token AND total time

Key insight: The "winner" depends heavily on your workload pattern. Batch inference? vLLM. Interactive chat with shared context? SGLang might edge ahead.

What was your test setup — concurrent users, prompt lengths, model size?

[xXMrNidaXx]

This vLLM vs SGLang benchmark is really valuable! Let me help explain some differences:

Why SGLang might be faster in some cases:

1. RadixAttention — SGLang's prefix caching is more aggressive
2. Speculative execution — Better at predicting likely paths
3. Compiler optimizations — CUDA graph capture is tighter

Why vLLM might be faster in others:

1. PagedAttention v2 — Excellent for variable-length batches
2. Continuous batching — Better throughput under load
3. More mature scheduling — Years of optimization

Key factors in your benchmark:

| Factor | Affects |
|--------|--------|
| Prompt length | Prefix caching benefit |
| Output length | Decoding speed |
| Batch size | Scheduling efficiency |
| Model size | Memory management |
| Quantization | Kernel performance |

To explain your specific results:

• What model did you test?
• What batch sizes?
• Single user or concurrent?

We've benchmarked both extensively at RevolutionAI for production deployments. Generally vLLM wins on throughput, SGLang on latency — but your workload matters most.

Can you share your test config?

[xXMrNidaXx]

Great benchmarking work! The memory and performance differences come from architectural choices.

Why SGLang uses less memory (7GB vs 21GB):

1. Lazy KV cache allocation

   • SGLang allocates KV cache on-demand
   • vLLM pre-allocates for max batch size

2. Different default gpu_memory_utilization

   • vLLM default: 0.9 (uses 90% of VRAM)
   • SGLang: more conservative defaults

3. Memory pool strategies differ

To make vLLM use less memory:

vllm serve qwen2.5-7b \
  --gpu-memory-utilization 0.3 \
  --max-model-len 2048 \
  --max-num-seqs 8

Why SGLang has more consistent latency:

1. RadixAttention — Better prefix caching
2. Continuous batching optimizations differ
3. Scheduling algorithms prioritize differently

Fair comparison settings:

# vLLM
vllm serve --gpu-memory-utilization 0.3

# SGLang
python -m sglang.launch_server --mem-fraction 0.3

What to benchmark:

• Fixed memory allocation for both
• Same batch sizes
• Same max_tokens
• Measure TTFT + TPS separately

We benchmark inference engines at Revolution AI — memory utilization config is the biggest factor in these comparisons.

[qiulang]

Thanks for the comments! It has been a while since I did that test so let me check it again and think over your comments before I get back to you. lol

I did remember I used L20 to test qwen3-8b awq against vLLM & SGLang.

[qiulang]

@xXMrNidaXx you may check this https://github.com/qiulang/vllm-sglang-perf/blob/main/next_step.md (I published 5 months ago but I did not publish the result on L20 though). But I remember for qwen3-32B-awq on L20 using vllm, the TTFT is around 6 seconds. not good. So L20 is not suitable for 32B, 8B is okay