PrototypeCommon
lucidrains/mixture-of-attention
Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts
- Owner
- lucidrains (personal account)
- Stars
- Stars: 121
- Forks
- Forks: 5
- Language
- Python
- License
- MIT
- Created
- 2023
- Topics
- artificial-intelligenceattention-mechanismsdeep-learningmixture-of-expertsrouted-attention
- Tier
- Prototype (100 to 249 ★) What this tier means
More repos by lucidrains
All 175 repos by lucidrains- lucidrains/vit-pytorchStars: 25,555PythonLTS
- lucidrains/DALLE2-pytorchStars: 11,306PythonStable
- lucidrains/denoising-diffusion-pytorchStars: 10,723PythonStable
- lucidrains/imagen-pytorchStars: 8,430PythonStable
- lucidrains/PaLM-rlhf-pytorchStars: 7,862PythonStable
- lucidrains/x-transformersStars: 5,962PythonStable
Nearby in Python
All Python repos- bagusprts/OutlineWeldStars: 121PythonPrototype
- CRFXN5C/mcd-mcp-kitStars: 121PythonPrototype
- thomasunise/argusStars: 121PythonPrototype
- pydanny/pydanny-event-notesStars: 121PythonPrototype
- peterbe/django-mongokitStars: 121PythonPrototype
- andreisavu/zookeeper-monitoringStars: 121PythonPrototype