BetaRare
IST-DASLab/marlin
FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.
- Owner
- IST-DASLab (organization)
- Stars
- Stars: 1,174
- Forks
- Forks: 95
- Language
- Python
- License
- Apache-2.0
- Created
- 2024
- Topics
- 4bitkernelllmquantization
- Tier
- Beta (700 to 1,999 ★) What this tier means
More repos by IST-DASLab
All 13 repos by IST-DASLab- IST-DASLab/gptqStars: 2,389PythonRC
- IST-DASLab/sparsegptStars: 902PythonBeta
- IST-DASLab/PanzaMailStars: 305PythonAlpha
- IST-DASLab/qmoeStars: 279PythonAlpha
- IST-DASLab/llmqStars: 266C++Alpha
- IST-DASLab/qutlassStars: 204C++Prototype
Nearby in Python
All Python repos- dannymcc/bluehoodStars: 1,174PythonBeta
- TIGER-AI-Lab/ClawBenchStars: 1,174PythonBeta
- viticci/shortcuts-playground-pluginStars: 1,174PythonBeta
- muricoca/crabStars: 1,174PythonBeta
- tomchop/malcomStars: 1,174PythonBeta
- SamR1/FitTrackeeStars: 1,174PythonBeta