# microsoft/Tokenizer

<https://repo.masters-masters.com/en/repos/microsoft/Tokenizer>

[Home](https://repo.masters-masters.com/en) › [microsoft](https://repo.masters-masters.com/en/owners/microsoft) › [Tokenizer](https://repo.masters-masters.com/en/repos/microsoft/Tokenizer)

microsoft/Tokenizer has 214 stars on GitHub. Its RepoMasters card: Prototype tier. Typescript and .NET implementation of BPE tokenizer for OpenAI LLMs.

| Field | Value |
| --- | --- |
| Owner | [microsoft](https://repo.masters-masters.com/en/owners/microsoft) |
| Stars | 214 |
| Forks | 36 |
| Language | [C\#](https://repo.masters-masters.com/en/languages/c-sharp) |
| License | MIT |
| Created | 2023 |
| Topics | ai, gpt, llm, openai, tokenizer |
| Tier | [Prototype (100 to 249 ★)](https://repo.masters-masters.com/en/ranks/rush) |
| View on GitHub | [microsoft/Tokenizer](https://github.com/microsoft/Tokenizer) |

## Repository

Typescript and .NET implementation of BPE tokenizer for OpenAI LLMs.

The tier for repos with 100 to 249 ★. Published odds per card pulled: 70%. Rarity: Common.

## More repos by microsoft

- [microsoft/vscode · ★ 194,514 · TypeScript · LTS](https://repo.masters-masters.com/en/repos/microsoft/vscode)
- [microsoft/markitdown · ★ 193,456 · Python · LTS](https://repo.masters-masters.com/en/repos/microsoft/markitdown)
- [microsoft/PowerToys · ★ 140,398 · C · LTS](https://repo.masters-masters.com/en/repos/microsoft/PowerToys)
- [microsoft/generative-ai-for-beginners · ★ 122,788 · Jupyter Notebook · LTS](https://repo.masters-masters.com/en/repos/microsoft/generative-ai-for-beginners)
- [microsoft/TypeScript · ★ 111,723 · Go · LTS](https://repo.masters-masters.com/en/repos/microsoft/TypeScript)
- [microsoft/terminal · ★ 105,311 · C++ · LTS](https://repo.masters-masters.com/en/repos/microsoft/terminal)

## Nearby in C\#

- [FabriBertani/Plugin.Maui.ScreenSecurity · ★ 214 · C\# · Prototype](https://repo.masters-masters.com/en/repos/FabriBertani/Plugin.Maui.ScreenSecurity)
- [microsoft/vs-solutionpersistence · ★ 214 · C\# · Prototype](https://repo.masters-masters.com/en/repos/microsoft/vs-solutionpersistence)
- [DecartAI/Decart-XR · ★ 214 · C\# · Prototype](https://repo.masters-masters.com/en/repos/DecartAI/Decart-XR)
- [PureMVC/puremvc-csharp-multicore-framework · ★ 214 · C\# · Prototype](https://repo.masters-masters.com/en/repos/PureMVC/puremvc-csharp-multicore-framework)
- [riyadparvez/data-structures-csharp · ★ 214 · C\# · Prototype](https://repo.masters-masters.com/en/repos/riyadparvez/data-structures-csharp)
- [alphaleonis/AlphaVSS · ★ 214 · C\# · Prototype](https://repo.masters-masters.com/en/repos/alphaleonis/AlphaVSS)

---
Data: GitHub · Illustration drawn by RepoMasters from the repo’s ID and its language color.