A high-throughput and memory-efficient inference and serving engine for LLMs
📊 Project Info
- Language
- Python
- Stars
- ⭐ 86,350
- Forks
- 19,461
- Today
- +104
- Ranking
- #12
- Collection
- Language
- Trending Date
- July 15, 2026
- Last Push
- 7/15/2026
🏷️ Topics
amdblackwellcudadeepseekdeepseek-v3gptgpt-ossinferencekimillamallmllm-servingmodel-servingmoeopenaipytorchqwenqwen3tputransformer