PrototypeCommon
NetEase-Media/grps
Deep Learning Deployment Framework: Supports tf/torch/trt/trtllm/vllm and other NN frameworks. Support dynamic batching, and streaming modes. It is dual-language compatible with Python and C++, offering scalability, extensibility, and high performance. It helps users quickly deploy models and provi…
- Owner
- NetEase-Media (organization)
- Stars
- Stars: 172
- Forks
- Forks: 11
- Language
- C++
- License
- Apache-2.0
- Created
- 2024
- Topics
- tensorflowtensorrttorchvllmdynamic-batchingservingtriton-inference-servertensorrt-llm
- Tier
- Prototype (100 to 249 ★) What this tier means
More repos by NetEase-Media
All 4 repos by NetEase-Media- NetEase-Media/easy-ngoStars: 300GoAlpha
- NetEase-Media/ngoStars: 183GoPrototype
- NetEase-Media/grps_trtllmStars: 161PythonPrototype
Nearby in C++
All C++ repos- Times-Z/GeekMagic-Open-FirmwareStars: 172C++Prototype
- dji-sdk/Cloud-API-2Stars: 172C++Prototype
- und3ath/xollvmStars: 172C++Prototype
- GilesBathgate/RapCADStars: 172C++Prototype
- pololu/pololu-led-strip-arduinoStars: 172C++Prototype
- racklin/moedict-desktopStars: 172C++Prototype