Illustration drawn by RepoMasters from the repo’s ID and its language color. Credits

PrototypeCommon

ash-01xor/bpe.c

Simple Byte pair Encoding mechanism used for tokenization process . written purely in C

Owner: ash-01xor (personal account)
Stars: Stars: 156
Forks: Forks: 4
Language: C
License: MIT
Created: 2024
Topics: llmtokenization
Tier: Prototype (100 to 249 ★) What this tier means

Nearby in C

jangafx/FMAGStars: 156CPrototype
iamjhe08/VidGrabStars: 156CPrototype
ravensclawx/HBLauncherStars: 156CPrototype
nicklan/pnmixerStars: 156CPrototype
kjn/lbzip2Stars: 156CPrototype
libguestfs/hivexStars: 156CPrototype