Lighfer

实际经验告诉你：4*5090 部署 Qwen3.6 27B NVFP4 够了，我们 vllm 部署并跑了两个月了。

我们部门 100 号人，实际在用这个模型的大概也就是三四十号人的规模，日常也就 4~6 个并发请求，缓存命中率最低 75%左右，有一段时间维持在 94%左右。

除了确实偶尔会卡顿一下（来了个大请求 prefill 占了资源），日常还是很流畅的，多数时候每个请求基本都能维持在 40~75 tok/s 。

当然，如果你们是要用来疯狂 vibe coding 那肯定就不行了，不过这个模型也达到可以随意 vibe coding 的能力。

放几条日志上来看看，不是瞎说（贴图太麻烦了直接贴文本）：

```
(APIServer pid=1) INFO 08-14 07:32:44 [loggers.py:310] 2 Engines Aggregated: Avg prompt throughput: 6927.3 tokens/s, Avg generation throughput: 211.5 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 17.3%, Prefix cache hit rate: 75.5%, MM cache hit rate: 0.0%
(APIServer pid=1) INFO 08-14 07:32:44 [metrics.py:120] SpecDecoding metrics: Mean acceptance length: 2.77, Accepted throughput: 135.20 tokens/s, Drafted throughput: 152.80 tokens/s, Accepted: 1352 tokens, Drafted: 1528 tokens, Per-position acceptance rate: 0.931, 0.839, Avg Draft acceptance rate: 88.5%
(APIServer pid=1) INFO: xxxxx - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=1) INFO 08-14 07:32:54 [loggers.py:310] 2 Engines Aggregated: Avg prompt throughput: 3843.2 tokens/s, Avg generation throughput: 382.0 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 8.8%, Prefix cache hit rate: 75.7%, MM cache hit rate: 0.0%
(APIServer pid=1) INFO 08-14 07:32:54 [metrics.py:120] SpecDecoding metrics: Mean acceptance length: 2.73, Accepted throughput: 241.65 tokens/s, Drafted throughput: 280.15 tokens/s, Accepted: 2417 tokens, Drafted: 2802 tokens, Per-position acceptance rate: 0.916, 0.809, Avg Draft acceptance rate: 86.3%
(APIServer pid=1) INFO: xxxxx - "HEAD / HTTP/1.1" 404 Not Found
(APIServer pid=1) INFO: xxxxx - "HEAD / HTTP/1.1" 404 Not Found
(APIServer pid=1) INFO: xxxxx - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=1) INFO: xxxxx - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=1) INFO 08-14 07:33:04 [loggers.py:310] 2 Engines Aggregated: Avg prompt throughput: 1300.6 tokens/s, Avg generation throughput: 332.5 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.8%, Prefix cache hit rate: 75.7%, MM cache hit rate: 0.0%
(APIServer pid=1) INFO 08-14 07:33:04 [metrics.py:120] SpecDecoding metrics: Mean acceptance length: 2.75, Accepted throughput: 211.70 tokens/s, Drafted throughput: 242.61 tokens/s, Accepted: 2117 tokens, Drafted: 2426 tokens, Per-position acceptance rate: 0.913, 0.832, Avg Draft acceptance rate: 87.3%
(APIServer pid=1) INFO: xxxxx - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=1) INFO: xxxxx - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=1) INFO: xxxxx - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=1) INFO 08-14 07:33:14 [loggers.py:310] 2 Engines Aggregated: Avg prompt throughput: 4531.0 tokens/s, Avg generation throughput: 204.7 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 11.4%, Prefix cache hit rate: 75.6%, MM cache hit rate: 0.0%
(APIServer pid=1) INFO 08-14 07:33:14 [metrics.py:120] SpecDecoding metrics: Mean acceptance length: 2.90, Accepted throughput: 134.06 tokens/s, Drafted throughput: 141.16 tokens/s, Accepted: 1341 tokens, Drafted: 1412 tokens, Per-position acceptance rate: 0.970, 0.929, Avg Draft acceptance rate: 95.0%
(APIServer pid=1) INFO: xxxxx - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=1) INFO 08-14 07:33:24 [loggers.py:310] 2 Engines Aggregated: Avg prompt throughput: 10786.8 tokens/s, Avg generation throughput: 173.7 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 16.6%, Prefix cache hit rate: 75.6%, MM cache hit rate: 0.0%
(APIServer pid=1) INFO 08-14 07:33:24 [metrics.py:120] SpecDecoding metrics: Mean acceptance length: 2.83, Accepted throughput: 111.99 tokens/s, Drafted throughput: 122.59 tokens/s, Accepted: 1120 tokens, Drafted: 1226 tokens, Per-position acceptance rate: 0.949, 0.878, Avg Draft acceptance rate: 91.4%
```

@Lighfer 看到了链接里有

老哥，有试过能支撑多少并发和速度不？

用，工作流引擎，各种并行分支、并行执行，不用的话线程数的爆炸

@shiyuanGame 没用过/不会用建议多学习哈。
但凡你用过/会用都不会回复这 3 个字

@wdxbb app 没用过不太确定噢

@glouhao 是的，chatgpt 的高级思考额度给的很足。如果你的项目依赖的都是互联网上的东西，他也会尝试搭建起开发环境写测试用例进行测试，极强。。

@cairnechen gpt 网页版会在沙盒里自己拉模型，写代码，完成工作，然后把产物打包成连接给我下载。所有操作都在 gpt 的服务器上完成，和我本地没有任何关系。

路边一条... 别被营销骗了