AlphaUncommon
feifeibear/long-context-attention
USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference
- Owner
- feifeibear (personal account)
- Stars
- Stars: 696
- Forks
- Forks: 87
- Language
- Python
- License
- Apache-2.0
- Created
- 2024
- Topics
- attention-is-all-you-needllm-inferencellm-trainingpytorchring-attentiondeepspeed-ulysses
- Tier
- Alpha (250 to 699 ★) What this tier means
More repos by feifeibear
All 3 repos by feifeibear- feifeibear/LLMSpeculativeSamplingStars: 927PythonBeta
- feifeibear/LLMRooflineStars: 126Jupyter NotebookPrototype
Nearby in Python
All Python repos- EurekaClaw/EurekaClawStars: 696PythonAlpha
- Ar9av/PaperOrchestraStars: 696PythonAlpha
- Moxin-Tech/moworld_baseStars: 696PythonAlpha
- stephenmcd/django-forms-builderStars: 696PythonAlpha
- saxix/django-adminactionsStars: 696PythonAlpha
- perrygeo/simannealStars: 696PythonAlpha