# microsoft/sarathi-serve

<https://repo.masters-masters.com/en/repos/microsoft/sarathi-serve>

[Home](https://repo.masters-masters.com/en) › [microsoft](https://repo.masters-masters.com/en/owners/microsoft) › [sarathi-serve](https://repo.masters-masters.com/en/repos/microsoft/sarathi-serve)

microsoft/sarathi-serve has 535 stars on GitHub. Its RepoMasters card: Alpha tier. A low-latency & high-throughput serving engine for LLMs Language: Python.

| Field | Value |
| --- | --- |
| Owner | [microsoft](https://repo.masters-masters.com/en/owners/microsoft) |
| Stars | 535 |
| Forks | 68 |
| Language | [Python](https://repo.masters-masters.com/en/languages/python) |
| License | Apache-2.0 |
| Created | 2023 |
| Topics | llama, llm-inference, pytorch, transformer |
| Tier | [Alpha (250 to 699 ★)](https://repo.masters-masters.com/en/ranks/b_movie) |
| View on GitHub | [microsoft/sarathi-serve](https://github.com/microsoft/sarathi-serve) |

## Repository

A low-latency & high-throughput serving engine for LLMs

The tier for repos with 250 to 699 ★. Published odds per card pulled: 20%. Rarity: Uncommon.

## More repos by microsoft

- [microsoft/vscode · ★ 194,543 · TypeScript · LTS](https://repo.masters-masters.com/en/repos/microsoft/vscode)
- [microsoft/markitdown · ★ 193,786 · Python · LTS](https://repo.masters-masters.com/en/repos/microsoft/markitdown)
- [microsoft/PowerToys · ★ 140,513 · C · LTS](https://repo.masters-masters.com/en/repos/microsoft/PowerToys)
- [microsoft/generative-ai-for-beginners · ★ 122,938 · Jupyter Notebook · LTS](https://repo.masters-masters.com/en/repos/microsoft/generative-ai-for-beginners)
- [microsoft/TypeScript · ★ 111,732 · Go · LTS](https://repo.masters-masters.com/en/repos/microsoft/TypeScript)
- [microsoft/terminal · ★ 105,321 · C++ · LTS](https://repo.masters-masters.com/en/repos/microsoft/terminal)

## Nearby in Python

- [NVlabs/GEM-X · ★ 535 · Python · Alpha](https://repo.masters-masters.com/en/repos/NVlabs/GEM-X)
- [InternLM/WildClawBench · ★ 535 · Python · Alpha](https://repo.masters-masters.com/en/repos/InternLM/WildClawBench)
- [damonwan1/AutoScholarLoop · ★ 535 · Python · Alpha](https://repo.masters-masters.com/en/repos/damonwan1/AutoScholarLoop)
- [paulnasca/paulstretch\_python · ★ 535 · Python · Alpha](https://repo.masters-masters.com/en/repos/paulnasca/paulstretch_python)
- [cgoldberg/ystockquote · ★ 535 · Python · Alpha](https://repo.masters-masters.com/en/repos/cgoldberg/ystockquote)
- [EiNSTeiN-/decompiler · ★ 535 · Python · Alpha](https://repo.masters-masters.com/en/repos/EiNSTeiN-/decompiler)

---
Data: GitHub · Illustration drawn by RepoMasters from the repo’s ID and its language color.