Repository
syv-ai/HyperQwen
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
Stars
2.1k
Today
+13
last 24 hours
This week
+13
Forks
281
Clone
git clone https://github.com/syv-ai/HyperQwen.gitStar growth
#kv-cache#llm-inference#local-llm#quantization#qwen#qwen3#rtx-3090#speculative-decoding#vllm#consumer-gpu
- Created
- 2026-08-15
- Last commit
- 2026-10-11
- Gained since tracking
- +13
Updated Sun, 18 Oct 2026 15:32:30 GMT · description from the project itself