# tomaarsen/attention\_sinks

<https://repo.masters-masters.com/en/repos/tomaarsen/attention_sinks>

[Home](https://repo.masters-masters.com/en) › [tomaarsen](https://repo.masters-masters.com/en/owners/tomaarsen) › [attention\_sinks](https://repo.masters-masters.com/en/repos/tomaarsen/attention_sinks)

tomaarsen/attention\_sinks has 734 stars on GitHub. Its RepoMasters card: Beta tier. Extend existing LLMs way beyond the original training length with constant…

| Field | Value |
| --- | --- |
| Owner | [tomaarsen](https://repo.masters-masters.com/en/owners/tomaarsen) |
| Stars | 734 |
| Forks | 43 |
| Language | [Python](https://repo.masters-masters.com/en/languages/python) |
| License | Apache-2.0 |
| Created | 2023 |
| Topics | llm, llms, nlp, python, transformers |
| Tier | [Beta (700 to 1,999 ★)](https://repo.masters-masters.com/en/ranks/box_office) |
| View on GitHub | [tomaarsen/attention\_sinks](https://github.com/tomaarsen/attention_sinks) |

## Repository

Extend existing LLMs way beyond the original training length with constant memory usage, without retraining

The tier for repos with 700 to 1,999 ★. Published odds per card pulled: 6%. Rarity: Rare.

## More repos by tomaarsen

- [tomaarsen/SpanMarkerNER · ★ 475 · Jupyter Notebook · Alpha](https://repo.masters-masters.com/en/repos/tomaarsen/SpanMarkerNER)
- [tomaarsen/TwitchMarkovChain · ★ 139 · Python · Prototype](https://repo.masters-masters.com/en/repos/tomaarsen/TwitchMarkovChain)

## Nearby in Python

- [tracefinity/tracefinity · ★ 734 · Python · Beta](https://repo.masters-masters.com/en/repos/tracefinity/tracefinity)
- [yzhao062/agent-style · ★ 734 · Python · Beta](https://repo.masters-masters.com/en/repos/yzhao062/agent-style)
- [ARahim3/mlx-dspark · ★ 734 · Python · Beta](https://repo.masters-masters.com/en/repos/ARahim3/mlx-dspark)
- [itdxer/neupy · ★ 734 · Python · Beta](https://repo.masters-masters.com/en/repos/itdxer/neupy)
- [czheo/syntax\_sugar\_python · ★ 734 · Python · Beta](https://repo.masters-masters.com/en/repos/czheo/syntax_sugar_python)
- [Cadene/vqa.pytorch · ★ 734 · Python · Beta](https://repo.masters-masters.com/en/repos/Cadene/vqa.pytorch)

---
Data: GitHub · Illustration drawn by RepoMasters from the repo’s ID and its language color.