PrototypeCommon
microsoft/batch-inference
Dynamic batching library for Deep Learning inference. Tutorials for LLM, GPT scenarios.
- Owner
- microsoft (organization)
- Stars
- Stars: 106
- Forks
- Forks: 6
- Language
- Python
- License
- MIT
- Created
- 2023
- Topics
- deep-learninginferenceperformance-optimizationdynamic-batchingpythongptllm
- Tier
- Prototype (100 to 249 ★) What this tier means
More repos by microsoft
All 1,923 repos by microsoft- microsoft/vscodeStars: 194,428TypeScriptLTS
- microsoft/markitdownStars: 193,282PythonLTS
- microsoft/PowerToysStars: 140,362CLTS
- microsoft/generative-ai-for-beginnersStars: 122,672Jupyter NotebookLTS
- microsoft/TypeScriptStars: 111,706GoLTS
- microsoft/terminalStars: 105,300C++LTS
Nearby in Python
All Python repos- Ta0p1/ResolveLabStars: 106PythonPrototype
- PlusF666/mctaste-labStars: 106PythonPrototype
- Sac-Y/match-cutStars: 106PythonPrototype
- mkelly12/google_closure_compilerStars: 106PythonPrototype
- alex/django-admin-histogramsStars: 106PythonPrototype
- bretth/wovenStars: 106PythonPrototype