AlphaUncommon
microsoft/sarathi-serve
A low-latency & high-throughput serving engine for LLMs
- Owner
- microsoft (organization)
- Stars
- Stars: 535
- Forks
- Forks: 68
- Language
- Python
- License
- Apache-2.0
- Created
- 2023
- Topics
- llamallm-inferencepytorchtransformer
- Tier
- Alpha (250 to 699 ★) What this tier means
More repos by microsoft
All 1,923 repos by microsoft- microsoft/vscodeStars: 194,462TypeScriptLTS
- microsoft/markitdownStars: 193,456PythonLTS
- microsoft/PowerToysStars: 140,398CLTS
- microsoft/generative-ai-for-beginnersStars: 122,788Jupyter NotebookLTS
- microsoft/TypeScriptStars: 111,714GoLTS
- microsoft/terminalStars: 105,311C++LTS
Nearby in Python
All Python repos- InternLM/WildClawBenchStars: 535PythonAlpha
- stormzhang/token-trackerStars: 535PythonAlpha
- davidondrej/jailbreak-autoresearchStars: 535PythonAlpha
- paulnasca/paulstretch_pythonStars: 535PythonAlpha
- cgoldberg/ystockquoteStars: 535PythonAlpha
- EiNSTeiN-/decompilerStars: 535PythonAlpha