PrototypeCommon
NetEase-Media/grps_trtllm
Higher performance OpenAI LLM service than vLLM serve: A pure C++ high-performance OpenAI LLM service implemented with GPRS+TensorRT-LLM+Tokenizers.cpp, supporting chat and function call, AI agents, distributed multi-GPU inference, multimodal capabilities, and a Gradio chat interface.
- Owner
- NetEase-Media (organization)
- Stars
- Stars: 161
- Forks
- Forks: 12
- Language
- Python
- License
- Apache-2.0
- Created
- 2024
- Topics
- llmopenaitensorrt-llmchatglmllama3qwen2function-callai-agentllama-indexmulti-modal
- Tier
- Prototype (100 to 249 ★) What this tier means
More repos by NetEase-Media
All 4 repos by NetEase-Media- NetEase-Media/easy-ngoStars: 300GoAlpha
- NetEase-Media/ngoStars: 183GoPrototype
- NetEase-Media/grpsStars: 172C++Prototype
Nearby in Python
All Python repos- VectifyAI/jev-doc-searchStars: 161PythonPrototype
- Aoisop/mawoagentStars: 161PythonPrototype
- lemonadesaltbagel/crabStars: 161PythonPrototype
- lilspikey/django-batch-selectStars: 161PythonPrototype
- benosteen/FaceRecognitionStars: 161PythonPrototype
- defnull/multipartStars: 161PythonPrototype