PrototypeCommon
microsoft/Tokenizer
Typescript and .NET implementation of BPE tokenizer for OpenAI LLMs.
- Owner
- microsoft (organization)
- Stars
- Stars: 214
- Forks
- Forks: 36
- Language
- C#
- License
- MIT
- Created
- 2023
- Topics
- aigptllmopenaitokenizer
- Tier
- Prototype (100 to 249 ★) What this tier means
More repos by microsoft
All 1,923 repos by microsoft- microsoft/vscodeStars: 194,428TypeScriptLTS
- microsoft/markitdownStars: 193,282PythonLTS
- microsoft/PowerToysStars: 140,362CLTS
- microsoft/generative-ai-for-beginnersStars: 122,672Jupyter NotebookLTS
- microsoft/TypeScriptStars: 111,706GoLTS
- microsoft/terminalStars: 105,300C++LTS
Nearby in C#
All C# repos- DecartAI/Decart-XRStars: 214C#Prototype
- Hypabolic/HypaStars: 214C#Prototype
- idlerunner00/procedural-pixel-creaturesStars: 214C#Prototype
- PureMVC/puremvc-csharp-multicore-frameworkStars: 214C#Prototype
- riyadparvez/data-structures-csharpStars: 214C#Prototype
- alphaleonis/AlphaVSSStars: 214C#Prototype