PrototypeCommon
ferrumox/fox
A local LLM server built for concurrent work. Drop-in replacement for Ollama and/or OpenAI and Ollama APIs on one port. Requests that share a prompt reuse each other's KV cache instead of each prefilling it. Rust, wrapping llama.cpp.
- Owner
- ferrumox (organization)
- Stars
- Stars: 190
- Forks
- Forks: 28
- Language
- Rust
- License
- None
- Created
- 2026
- Topics
- cudaggufinference-enginellama-cppllmlocal-llmmlxollamaollama-apiopenai-api
- Tier
- Prototype (100 to 249 ★) What this tier means
More repos by ferrumox
All 2 repos by ferrumox- ferrumox/rabbitStars: 191RustPrototype
Nearby in Rust
All Rust repos- MetaFARS/codex-relayStars: 190RustPrototype
- UCS-Care-Group/mailcraftStars: 190RustPrototype
- timoncool/speech-studioStars: 190RustPrototype
- georust/projStars: 190RustPrototype
- brendanzab/approxStars: 190RustPrototype
- raymanfx/libv4l-rsStars: 190RustPrototype