# tiktoken-go/tokenizer

<https://repo.masters-masters.com/en/repos/tiktoken-go/tokenizer>

[Home](https://repo.masters-masters.com/en) › [tiktoken-go](https://repo.masters-masters.com/en/owners/tiktoken-go) › [tokenizer](https://repo.masters-masters.com/en/repos/tiktoken-go/tokenizer)

tiktoken-go/tokenizer has 462 stars on GitHub. Its RepoMasters card: Alpha tier. Pure Go implementation of OpenAI's tiktoken tokenizer Language: Go.

| Field | Value |
| --- | --- |
| Owner | [tiktoken-go](https://repo.masters-masters.com/en/owners/tiktoken-go) |
| Stars | 462 |
| Forks | 39 |
| Language | [Go](https://repo.masters-masters.com/en/languages/go) |
| License | MIT |
| Created | 2023 |
| Tier | [Alpha (250 to 699 ★)](https://repo.masters-masters.com/en/ranks/b_movie) |
| View on GitHub | [tiktoken-go/tokenizer](https://github.com/tiktoken-go/tokenizer) |

## Repository

Pure Go implementation of OpenAI's tiktoken tokenizer

The tier for repos with 250 to 699 ★. Published odds per card pulled: 20%. Rarity: Uncommon.

## Nearby in Go

- [Kava-Labs/kava · ★ 462 · Go · Alpha](https://repo.masters-masters.com/en/repos/Kava-Labs/kava)
- [HugoSmits86/nativewebp · ★ 462 · Go · Alpha](https://repo.masters-masters.com/en/repos/HugoSmits86/nativewebp)
- [web-infra-dev/rslint · ★ 462 · Go · Alpha](https://repo.masters-masters.com/en/repos/web-infra-dev/rslint)
- [dustin/seriesly · ★ 462 · Go · Alpha](https://repo.masters-masters.com/en/repos/dustin/seriesly)
- [astronomer/astro-cli · ★ 462 · Go · Alpha](https://repo.masters-masters.com/en/repos/astronomer/astro-cli)

---
Data: GitHub · Illustration drawn by RepoMasters from the repo’s ID and its language color.