# feifeibear/long-context-attention

<https://repo.masters-masters.com/en/repos/feifeibear/long-context-attention>

[Home](https://repo.masters-masters.com/en) › [feifeibear](https://repo.masters-masters.com/en/owners/feifeibear) › [long-context-attention](https://repo.masters-masters.com/en/repos/feifeibear/long-context-attention)

feifeibear/long-context-attention has 696 stars on GitHub. Its RepoMasters card: Alpha tier. USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for…

| Field | Value |
| --- | --- |
| Owner | [feifeibear](https://repo.masters-masters.com/en/owners/feifeibear) |
| Stars | 696 |
| Forks | 87 |
| Language | [Python](https://repo.masters-masters.com/en/languages/python) |
| License | Apache-2.0 |
| Created | 2024 |
| Topics | attention-is-all-you-need, llm-inference, llm-training, pytorch, ring-attention, deepspeed-ulysses |
| Tier | [Alpha (250 to 699 ★)](https://repo.masters-masters.com/en/ranks/b_movie) |
| View on GitHub | [feifeibear/long-context-attention](https://github.com/feifeibear/long-context-attention) |

## Repository

USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference

The tier for repos with 250 to 699 ★. Published odds per card pulled: 20%. Rarity: Uncommon.

## More repos by feifeibear

- [feifeibear/LLMSpeculativeSampling · ★ 927 · Python · Beta](https://repo.masters-masters.com/en/repos/feifeibear/LLMSpeculativeSampling)
- [feifeibear/LLMRoofline · ★ 126 · Jupyter Notebook · Prototype](https://repo.masters-masters.com/en/repos/feifeibear/LLMRoofline)

## Nearby in Python

- [kangarooking/kangarooking-skills · ★ 696 · Python · Alpha](https://repo.masters-masters.com/en/repos/kangarooking/kangarooking-skills)
- [EurekaClaw/EurekaClaw · ★ 696 · Python · Alpha](https://repo.masters-masters.com/en/repos/EurekaClaw/EurekaClaw)
- [RennAkira/spicy-monopoly · ★ 696 · Python · Alpha](https://repo.masters-masters.com/en/repos/RennAkira/spicy-monopoly)
- [stephenmcd/django-forms-builder · ★ 696 · Python · Alpha](https://repo.masters-masters.com/en/repos/stephenmcd/django-forms-builder)
- [saxix/django-adminactions · ★ 696 · Python · Alpha](https://repo.masters-masters.com/en/repos/saxix/django-adminactions)
- [perrygeo/simanneal · ★ 696 · Python · Alpha](https://repo.masters-masters.com/en/repos/perrygeo/simanneal)

---
Data: GitHub · Illustration drawn by RepoMasters from the repo’s ID and its language color.