Illustration drawn by RepoMasters from the repo’s ID and its language color. Credits
AlphaUncommon

dmitry-brazhenko/SharpToken

SharpToken is a C# library for tokenizing natural language text. It's based on the tiktoken Python library and designed to be fast and accurate.

Owner
dmitry-brazhenko (personal account)
Stars
Stars: 263
Forks
Forks: 19
Language
C#
License
MIT
Created
2023
Topics
gptgpt-3openaitokenizercsharpcl100kbasegpt-4
Tier
Alpha (250 to 699 ★) What this tier means
View on GitHub

Nearby in C#

All C# repos