AlphaUncommon
SqueezeAILab/KVQuant
[NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
- Owner
- SqueezeAILab (organization)
- Stars
- Stars: 436
- Forks
- Forks: 49
- Language
- Python
- License
- None
- Created
- 2024
- Topics
- compressionefficient-inferenceefficient-modellarge-language-modelsllamallmlocalllamalocalllmmistralmodel-compression
- Tier
- Alpha (250 to 699 ★) What this tier means
More repos by SqueezeAILab
All 5 repos by SqueezeAILab- SqueezeAILab/LLMCompilerStars: 1,900PythonBeta
- SqueezeAILab/SqueezeLLMStars: 724PythonBeta
- SqueezeAILab/TinyAgentStars: 501PythonAlpha
- SqueezeAILab/LLM2LLMStars: 196PythonPrototype
Nearby in Python
All Python repos- Shao-Music-AI/ShaoStars: 436PythonAlpha
- joeseesun/qiaomu-downloadStars: 436PythonAlpha
- jss8649/quiet-tizenStars: 436PythonAlpha
- openid/python-openidStars: 436PythonAlpha
- guoguo12/billboard-chartsStars: 436PythonAlpha
- wmayner/pyphiStars: 436PythonAlpha