BetaRare
zhihu/ZhiLight
A highly optimized LLM inference acceleration engine for Llama and its variants.
- Owner
- zhihu (organization)
- Stars
- Stars: 906
- Forks
- Forks: 105
- Language
- C++
- License
- Apache-2.0
- Created
- 2024
- Topics
- inference-enginellmcudagptllamallm-servingpytorchllm-inferencemodel-servingdeepseek-r1
- Tier
- Beta (700 to 1,999 ★) What this tier means
More repos by zhihu
All 13 repos by zhihu- zhihu/MatisseStars: 12,492JavaStable
- zhihu/griffithStars: 2,507TypeScriptRC
- zhihu/kidsStars: 1,213C++Beta
- zhihu/ruceneStars: 1,019RustBeta
- zhihu/cuBERTStars: 546C++Alpha
- zhihu/RxLifecycleStars: 507JavaAlpha
Nearby in C++
All C++ repos- eyalroz/cuda-api-wrappersStars: 906C++Beta
- DustinHLand/vkDOOM3Stars: 906C++Beta
- google/quicheStars: 906C++Beta
- CLD2Owners/cld2Stars: 905C++Beta