AlphaUncommon
guillaume-be/rust-tokenizers
Rust-tokenizer offers high-performance tokenizers for modern language models, including WordPiece, Byte-Pair Encoding (BPE) and Unigram (SentencePiece) models
- Owner
- guillaume-be (personal account)
- Stars
- Stars: 345
- Forks
- Forks: 33
- Language
- Rust
- License
- Apache-2.0
- Created
- 2019
- Topics
- tokenizerrust-langtransformerdeep-learning
- Tier
- Alpha (250 to 699 ★) What this tier means
More repos by guillaume-be
All 2 repos by guillaume-be- guillaume-be/rust-bertStars: 3,091RustRC
Nearby in Rust
All Rust repos- nsaintot/cdj3k-emuStars: 345RustAlpha
- Dicklesworthstone/franken_ocrStars: 345RustAlpha
- SheepSeb/tinyworldformerStars: 345RustAlpha
- zonyitoo/rust-iniStars: 345RustAlpha
- reem/rust-ordered-floatStars: 345RustAlpha
- mattgathu/cuteStars: 345RustAlpha