AlphaUncommon
OpenNMT/Tokenizer
Fast and customizable text tokenization library with BPE and SentencePiece support
- Owner
- OpenNMT (organization)
- Stars
- Stars: 343
- Forks
- Forks: 83
- Language
- C++
- License
- MIT
- Created
- 2017
- Topics
- tokenizersentencepiecenatural-language-processingmachine-translationbpeunicodetokenizationicupythoncpp
- Tier
- Alpha (250 to 699 ★) What this tier means
More repos by OpenNMT
All 6 repos by OpenNMT- OpenNMT/OpenNMT-pyStars: 7,020PythonStable
- OpenNMT/CTranslate2Stars: 4,729C++RC
- OpenNMT/OpenNMTStars: 2,387LuaRC
- OpenNMT/OpenNMT-tfStars: 1,485PythonBeta
- OpenNMT/Im2TextStars: 137LuaPrototype
Nearby in C++
All C++ repos- johnBuffer/StableIndexVectorStars: 343C++Alpha
- hyprnux/hyprglassStars: 343C++Alpha
- MarioGalindoQ/Modern-CPP-Design-PatternsStars: 343C++Alpha
- philogb/v8-glStars: 343C++Alpha
- amcewen/HttpClientStars: 343C++Alpha
- nektra/Deviare-InProcStars: 343C++Alpha