PrototypeCommon
kekzl/imp
Run large AI chat models privately on your own NVIDIA RTX 50 or RTX PRO graphics card. On one RTX 5090, Qwen3.8-27B writes about 70 words per second, 1.4x faster than vLLM. Even a model almost four times larger than the card's memory runs. Works with apps made for ChatGPT or Claude. Starts with one…
- Owner
- kekzl (personal account)
- Stars
- Stars: 124
- Forks
- Forks: 5
- Language
- C++
- License
- MIT
- Created
- 2026
- Topics
- blackwellcudamixture-of-expertsquantizationrtx-5090qweninference-enginenvfp4llm-inferencelocal-llm
- Tier
- Prototype (100 to 249 ★) What this tier means
Nearby in C++
All C++ repos- UnionCrax-Team/uc-online2Stars: 124C++Prototype
- TypeDreamMoon/DreamShaderStars: 124C++Prototype
- Shuqian-Tech/SQMesh-PublicStars: 124C++Prototype
- kelvie/basketStars: 124C++Prototype
- ruphy/speedcrunchStars: 124C++Prototype
- ThomasHabets/injcodeStars: 124C++Prototype