AlphaUncommon
alasdairforsythe/tokenmonster
Ungreedy subword tokenizer and vocabulary trainer for Python, Go, C++ & Javascript
- Owner
- alasdairforsythe (personal account)
- Stars
- Stars: 625
- Forks
- Forks: 23
- Language
- Go
- License
- MIT
- Created
- 2023
- Topics
- tokenisationtokenizationtokenizetokenizertokenizingvocabularyvocabulary-buildertext-tokenizationvocabulary-generator
- Tier
- Alpha (250 to 699 ★) What this tier means
Nearby in Go
All Go repos- Vencord/InstallerStars: 625GoAlpha
- DataDog/orchestrionStars: 625GoAlpha
- reteps/dockerfmtStars: 625GoAlpha
- gruntwork-io/fetchStars: 625GoAlpha
- pires/go-proxyprotoStars: 625GoAlpha
- prometheus-community/ipmi_exporterStars: 625GoAlpha