PrototypeCommon
azrtydxb/Fastllm-proxy
The lowest-overhead LLM router. Production-ready, highly available, one OpenAI-compatible endpoint in front of 80 providers and your own vLLM/SGLang — 0.76 µs per request, no I/O on the request path, cache-affinity routing, RBAC, budgets and a 13-screen UI in the binary.
- Owner
- azrtydxb (organization)
- Stars
- Stars: 108
- Forks
- Forks: 92
- Language
- Rust
- License
- Apache-2.0
- Created
- 2026
- Topics
- ai-gatewayinferencelitellmllmllm-gatewayload-balancingobservabilityopenai-apirbacreverse-proxy
- Tier
- Prototype (100 to 249 ★) What this tier means
More repos by azrtydxb
All 2 repos by azrtydxb- azrtydxb/procoderStars: 210GoPrototype
Nearby in Rust
All Rust repos- 2youg1/sprawling-agentsStars: 108RustPrototype
- ProjectTesca/MenSharpStars: 108RustPrototype
- zhidian-cmd/gitport-githubStars: 108RustPrototype
- ruuda/robigo-luculentaStars: 108RustPrototype
- deprecrated/net2-rsStars: 108RustPrototype
- matzipan/envoyerStars: 108RustPrototype