PrototypeCommon
yalun753/moe-l2
MoE expert offload for low-VRAM GPUs — run 100B+ MoE models (DeepSeek, Qwen, Mixtral) on 8 GB cards. Expert cache with LRU hot experts, OpenAI-compatible proxy, GGUF multi-shard.
- Owner
- yalun753 (personal account)
- Stars
- Stars: 145
- Forks
- Forks: 6
- Language
- Cuda
- License
- Apache-2.0
- Created
- 2026
- Topics
- consumer-gpudeepseekexpert-cachingllama-cppllm-inferencemoevram-optimization
- Tier
- Prototype (100 to 249 ★) What this tier means
Nearby in Cuda
All Cuda repos- lixincn2015/Partial-Order-PruningStars: 147CudaPrototype
- eunomia-bpf/basic-cuda-tutorialStars: 146CudaPrototype
- Infatoshi/MegaQwenStars: 145CudaPrototype
- enCRYPTON-labs/HEonGPUStars: 144CudaPrototype
- meder411/PyTorch-EMDLossStars: 143CudaPrototype
- leimao/CUTLASS-ExamplesStars: 143CudaPrototype