# yanqiangmiffy/how-to-train-tokenizer

<https://repo.masters-masters.com/en/repos/yanqiangmiffy/how-to-train-tokenizer>

[Home](https://repo.masters-masters.com/en) › [yanqiangmiffy](https://repo.masters-masters.com/en/owners/yanqiangmiffy) › [how-to-train-tokenizer](https://repo.masters-masters.com/en/repos/yanqiangmiffy/how-to-train-tokenizer)

yanqiangmiffy/how-to-train-tokenizer has 152 stars on GitHub. Its RepoMasters card: Prototype tier. 怎么训练一个LLM分词器 Language: Python.

| Field | Value |
| --- | --- |
| Owner | [yanqiangmiffy](https://repo.masters-masters.com/en/owners/yanqiangmiffy) |
| Stars | 152 |
| Forks | 28 |
| Language | [Python](https://repo.masters-masters.com/en/languages/python) |
| License | None |
| Created | 2023 |
| Tier | [Prototype (100 to 249 ★)](https://repo.masters-masters.com/en/ranks/rush) |
| View on GitHub | [yanqiangmiffy/how-to-train-tokenizer](https://github.com/yanqiangmiffy/how-to-train-tokenizer) |

## Repository

怎么训练一个LLM分词器

The tier for repos with 100 to 249 ★. Published odds per card pulled: 70%. Rarity: Common.

## More repos by yanqiangmiffy

- [yanqiangmiffy/Chinese-LangChain · ★ 2,821 · Python · RC](https://repo.masters-masters.com/en/repos/yanqiangmiffy/Chinese-LangChain)
- [yanqiangmiffy/InstructGLM · ★ 650 · Python · Alpha](https://repo.masters-masters.com/en/repos/yanqiangmiffy/InstructGLM)
- [yanqiangmiffy/sentence-similarity · ★ 371 · Python · Alpha](https://repo.masters-masters.com/en/repos/yanqiangmiffy/sentence-similarity)
- [yanqiangmiffy/NLP-Interview-Notes · ★ 352 · Jupyter Notebook · Alpha](https://repo.masters-masters.com/en/repos/yanqiangmiffy/NLP-Interview-Notes)

## Nearby in Python

- [canitcode/video2rig · ★ 152 · Python · Prototype](https://repo.masters-masters.com/en/repos/canitcode/video2rig)
- [holdcloud/mcd-calendar-buddy · ★ 152 · Python · Prototype](https://repo.masters-masters.com/en/repos/holdcloud/mcd-calendar-buddy)
- [Supra-3D/PhysAg3D · ★ 152 · Python · Prototype](https://repo.masters-masters.com/en/repos/Supra-3D/PhysAg3D)
- [johnboxall/django-ab · ★ 152 · Python · Prototype](https://repo.masters-masters.com/en/repos/johnboxall/django-ab)
- [bartTC/django-markup · ★ 152 · Python · Prototype](https://repo.masters-masters.com/en/repos/bartTC/django-markup)
- [tweepy/examples · ★ 152 · Python · Prototype](https://repo.masters-masters.com/en/repos/tweepy/examples)

---
Data: GitHub · Illustration drawn by RepoMasters from the repo’s ID and its language color.