# alasdairforsythe/tokenmonster

<https://repo.masters-masters.com/en/repos/alasdairforsythe/tokenmonster>

[Home](https://repo.masters-masters.com/en) › [alasdairforsythe](https://repo.masters-masters.com/en/owners/alasdairforsythe) › [tokenmonster](https://repo.masters-masters.com/en/repos/alasdairforsythe/tokenmonster)

alasdairforsythe/tokenmonster has 625 stars on GitHub. Its RepoMasters card: Alpha tier. Ungreedy subword tokenizer and vocabulary trainer for Python, Go, C++…

| Field | Value |
| --- | --- |
| Owner | [alasdairforsythe](https://repo.masters-masters.com/en/owners/alasdairforsythe) |
| Stars | 625 |
| Forks | 23 |
| Language | [Go](https://repo.masters-masters.com/en/languages/go) |
| License | MIT |
| Created | 2023 |
| Topics | tokenisation, tokenization, tokenize, tokenizer, tokenizing, vocabulary, vocabulary-builder, text-tokenization, vocabulary-generator |
| Tier | [Alpha (250 to 699 ★)](https://repo.masters-masters.com/en/ranks/b_movie) |
| View on GitHub | [alasdairforsythe/tokenmonster](https://github.com/alasdairforsythe/tokenmonster) |

## Repository

Ungreedy subword tokenizer and vocabulary trainer for Python, Go, C++ & Javascript

The tier for repos with 250 to 699 ★. Published odds per card pulled: 20%. Rarity: Uncommon.

## Nearby in Go

- [Vencord/Installer · ★ 625 · Go · Alpha](https://repo.masters-masters.com/en/repos/Vencord/Installer)
- [DataDog/orchestrion · ★ 625 · Go · Alpha](https://repo.masters-masters.com/en/repos/DataDog/orchestrion)
- [reteps/dockerfmt · ★ 625 · Go · Alpha](https://repo.masters-masters.com/en/repos/reteps/dockerfmt)
- [gruntwork-io/fetch · ★ 625 · Go · Alpha](https://repo.masters-masters.com/en/repos/gruntwork-io/fetch)
- [pires/go-proxyproto · ★ 625 · Go · Alpha](https://repo.masters-masters.com/en/repos/pires/go-proxyproto)
- [prometheus-community/ipmi\_exporter · ★ 625 · Go · Alpha](https://repo.masters-masters.com/en/repos/prometheus-community/ipmi_exporter)

---
Data: GitHub · Illustration drawn by RepoMasters from the repo’s ID and its language color.