# psmarter/mini-infer

<https://repo.masters-masters.com/en/repos/psmarter/mini-infer>

[Home](https://repo.masters-masters.com/en) › [psmarter](https://repo.masters-masters.com/en/owners/psmarter) › [mini-infer](https://repo.masters-masters.com/en/repos/psmarter/mini-infer)

psmarter/mini-infer has 230 stars on GitHub. Its RepoMasters card: Prototype tier. LLM inference engine from scratch — paged KV cache, continuous batching…

| Field | Value |
| --- | --- |
| Owner | [psmarter](https://repo.masters-masters.com/en/owners/psmarter) |
| Stars | 230 |
| Forks | 12 |
| Language | [Python](https://repo.masters-masters.com/en/languages/python) |
| License | MIT |
| Created | 2025 |
| Topics | cuda, inference, llm, machine-learning, pagedattention, pytorch, transformer, triton, language-model, continuous-batching |
| Tier | [Prototype (100 to 249 ★)](https://repo.masters-masters.com/en/ranks/rush) |
| View on GitHub | [psmarter/mini-infer](https://github.com/psmarter/mini-infer) |

## Repository

LLM inference engine from scratch — paged KV cache, continuous batching, chunked prefill, prefix caching, speculative decoding, CUDA graph, tensor parallelism, OpenAI-compatible serving

The tier for repos with 100 to 249 ★. Published odds per card pulled: 70%. Rarity: Common.

## More repos by psmarter

- [psmarter/PMPP-Learning · ★ 192 · C++ · Prototype](https://repo.masters-masters.com/en/repos/psmarter/PMPP-Learning)
- [psmarter/CUDA-Practice · ★ 122 · Cuda · Prototype](https://repo.masters-masters.com/en/repos/psmarter/CUDA-Practice)

## Nearby in Python

- [wonderfulearth/primitive-operation-painter · ★ 230 · Python · Prototype](https://repo.masters-masters.com/en/repos/wonderfulearth/primitive-operation-painter)
- [technamooz/Panel\_Technamooz\_VPN · ★ 230 · Python · Prototype](https://repo.masters-masters.com/en/repos/technamooz/Panel_Technamooz_VPN)
- [michelle11032424-jpg/Interpark- · ★ 230 · Python · Prototype](https://repo.masters-masters.com/en/repos/michelle11032424-jpg/Interpark-)
- [eyeseast/python-tablefu · ★ 230 · Python · Prototype](https://repo.masters-masters.com/en/repos/eyeseast/python-tablefu)
- [wardi/speedometer · ★ 230 · Python · Prototype](https://repo.masters-masters.com/en/repos/wardi/speedometer)
- [craigds/django-typed-models · ★ 230 · Python · Prototype](https://repo.masters-masters.com/en/repos/craigds/django-typed-models)

---
Data: GitHub · Illustration drawn by RepoMasters from the repo’s ID and its language color.