# harleyszhang/rapid\_llm

<https://repo.masters-masters.com/en/repos/harleyszhang/rapid_llm>

[Home](https://repo.masters-masters.com/en) › [harleyszhang](https://repo.masters-masters.com/en/owners/harleyszhang) › [rapid\_llm](https://repo.masters-masters.com/en/repos/harleyszhang/rapid_llm)

harleyszhang/rapid\_llm has 196 stars on GitHub. Its RepoMasters card: Prototype tier. RapidLLM is an LLM inference framework with continuous batching, tensor…

| Field | Value |
| --- | --- |
| Owner | [harleyszhang](https://repo.masters-masters.com/en/owners/harleyszhang) |
| Stars | 196 |
| Forks | 33 |
| Language | [Python](https://repo.masters-masters.com/en/languages/python) |
| License | None |
| Created | 2024 |
| Topics | llama3, llm, llm-inference, python3, triton-kernels, attention, continues, continuous-batching, flash-attention-3, qwen3-moe |
| Tier | [Prototype (100 to 249 ★)](https://repo.masters-masters.com/en/ranks/rush) |
| View on GitHub | [harleyszhang/rapid\_llm](https://github.com/harleyszhang/rapid_llm) |

## Repository

RapidLLM is an LLM inference framework with continuous batching, tensor and data parallelism, quantized weights, and with pluggable Triton/CUDA kernels.

The tier for repos with 100 to 249 ★. Published odds per card pulled: 70%. Rarity: Common.

## More repos by harleyszhang

- [harleyszhang/cv\_note · ★ 2,651 · Python · RC](https://repo.masters-masters.com/en/repos/harleyszhang/cv_note)
- [harleyszhang/llm\_note · ★ 896 · Python · Beta](https://repo.masters-masters.com/en/repos/harleyszhang/llm_note)
- [harleyszhang/dl\_note · ★ 532 · Python · Alpha](https://repo.masters-masters.com/en/repos/harleyszhang/dl_note)
- [harleyszhang/llm\_counts · ★ 125 · Python · Prototype](https://repo.masters-masters.com/en/repos/harleyszhang/llm_counts)

## Nearby in Python

- [dacnay816y62-hub/fantasy-movie-poster-skill · ★ 196 · Python · Prototype](https://repo.masters-masters.com/en/repos/dacnay816y62-hub/fantasy-movie-poster-skill)
- [zhezh/kddcup2026\_champion · ★ 196 · Python · Prototype](https://repo.masters-masters.com/en/repos/zhezh/kddcup2026_champion)
- [Jakeschincariol/chatgpt-youtube-agent-skill · ★ 196 · Python · Prototype](https://repo.masters-masters.com/en/repos/Jakeschincariol/chatgpt-youtube-agent-skill)
- [patpadgett/corkscrew · ★ 196 · Python · Prototype](https://repo.masters-masters.com/en/repos/patpadgett/corkscrew)
- [wbond/sublime\_prefixr · ★ 196 · Python · Prototype](https://repo.masters-masters.com/en/repos/wbond/sublime_prefixr)
- [seomoz/reppy · ★ 196 · Python · Prototype](https://repo.masters-masters.com/en/repos/seomoz/reppy)

---
Data: GitHub · Illustration drawn by RepoMasters from the repo’s ID and its language color.