# huggingface/tokenizers

<https://repo.masters-masters.com/en/repos/huggingface/tokenizers>

[Home](https://repo.masters-masters.com/en) › [huggingface](https://repo.masters-masters.com/en/owners/huggingface) › [tokenizers](https://repo.masters-masters.com/en/repos/huggingface/tokenizers)

huggingface/tokenizers has 11,217 stars on GitHub. Its RepoMasters card: Stable tier. 💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

| Field | Value |
| --- | --- |
| Owner | [huggingface](https://repo.masters-masters.com/en/owners/huggingface) |
| Stars | 11,217 |
| Forks | 1,243 |
| Language | [Rust](https://repo.masters-masters.com/en/languages/rust) |
| License | Apache-2.0 |
| Created | 2019 |
| Topics | nlp, natural-language-processing, natural-language-understanding, language-model, transformers, bert, gpt |
| Tier | [Stable (5,000 to 19,999 ★)](https://repo.masters-masters.com/en/ranks/cult) |
| View on GitHub | [huggingface/tokenizers](https://github.com/huggingface/tokenizers) |

## Repository

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

The tier for repos with 5,000 to 19,999 ★. Published odds per card pulled: 0.9%. Rarity: Very rare.

## More repos by huggingface

- [huggingface/transformers · ★ 168,463 · Python · LTS](https://repo.masters-masters.com/en/repos/huggingface/transformers)
- [huggingface/pytorch-image-models · ★ 37,261 · Python · LTS](https://repo.masters-masters.com/en/repos/huggingface/pytorch-image-models)
- [huggingface/diffusers · ★ 34,854 · Python · LTS](https://repo.masters-masters.com/en/repos/huggingface/diffusers)
- [huggingface/agents-course · ★ 34,387 · MDX · LTS](https://repo.masters-masters.com/en/repos/huggingface/agents-course)
- [huggingface/smolagents · ★ 30,195 · Python · LTS](https://repo.masters-masters.com/en/repos/huggingface/smolagents)
- [huggingface/lerobot · ★ 28,656 · Python · LTS](https://repo.masters-masters.com/en/repos/huggingface/lerobot)

## Nearby in Rust

- [Nasiko-Labs/nasiko · ★ 11,555 · Rust · Stable](https://repo.masters-masters.com/en/repos/Nasiko-Labs/nasiko)
- [perspective-dev/perspective · ★ 11,387 · Rust · Stable](https://repo.masters-masters.com/en/repos/perspective-dev/perspective)
- [storytold/vectorcraft · ★ 11,267 · Rust · Stable](https://repo.masters-masters.com/en/repos/storytold/vectorcraft)
- [dmtrKovalenko/fff · ★ 11,213 · Rust · Stable](https://repo.masters-masters.com/en/repos/dmtrKovalenko/fff)
- [QuipNetwork/hashsigs-rs · ★ 11,206 · Rust · Stable](https://repo.masters-masters.com/en/repos/QuipNetwork/hashsigs-rs)
- [pingcap/talent-plan · ★ 11,034 · Rust · Stable](https://repo.masters-masters.com/en/repos/pingcap/talent-plan)

---
Data: GitHub · Illustration: the project’s public image, pixelated by RepoMasters.