# ofirpress/attention\_with\_linear\_biases

<https://repo.masters-masters.com/en/repos/ofirpress/attention_with_linear_biases>

[Home](https://repo.masters-masters.com/en) › [ofirpress](https://repo.masters-masters.com/en/owners/ofirpress) › [attention\_with\_linear\_biases](https://repo.masters-masters.com/en/repos/ofirpress/attention_with_linear_biases)

ofirpress/attention\_with\_linear\_biases has 557 stars on GitHub. Its RepoMasters card: Alpha tier. Code for the ALiBi method for transformer language models…

| Field | Value |
| --- | --- |
| Owner | [ofirpress](https://repo.masters-masters.com/en/owners/ofirpress) |
| Stars | 557 |
| Forks | 42 |
| Language | [Python](https://repo.masters-masters.com/en/languages/python) |
| License | MIT |
| Created | 2021 |
| Tier | [Alpha (250 to 699 ★)](https://repo.masters-masters.com/en/ranks/b_movie) |
| View on GitHub | [ofirpress/attention\_with\_linear\_biases](https://github.com/ofirpress/attention_with_linear_biases) |

## Repository

Code for the ALiBi method for transformer language models (ICLR 2022)

The tier for repos with 250 to 699 ★. Published odds per card pulled: 20%. Rarity: Uncommon.

## More repos by ofirpress

- [ofirpress/self-ask · ★ 325 · Jupyter Notebook · Alpha](https://repo.masters-masters.com/en/repos/ofirpress/self-ask)
- [ofirpress/YouMayNotNeedAttention · ★ 294 · Python · Alpha](https://repo.masters-masters.com/en/repos/ofirpress/YouMayNotNeedAttention)
- [ofirpress/shortformer · ★ 145 · Python · Prototype](https://repo.masters-masters.com/en/repos/ofirpress/shortformer)

## Nearby in Python

- [FireRedTeam/FireRedVAD · ★ 557 · Python · Alpha](https://repo.masters-masters.com/en/repos/FireRedTeam/FireRedVAD)
- [Rangizingo/cc-cache-fix · ★ 557 · Python · Alpha](https://repo.masters-masters.com/en/repos/Rangizingo/cc-cache-fix)
- [linuxrebel/DocuBrowser · ★ 557 · Python · Alpha](https://repo.masters-masters.com/en/repos/linuxrebel/DocuBrowser)
- [mnot/redbot · ★ 557 · Python · Alpha](https://repo.masters-masters.com/en/repos/mnot/redbot)
- [wrye-bash/wrye-bash · ★ 557 · Python · Alpha](https://repo.masters-masters.com/en/repos/wrye-bash/wrye-bash)
- [UDST/urbansim · ★ 557 · Python · Alpha](https://repo.masters-masters.com/en/repos/UDST/urbansim)

---
Data: GitHub · Illustration drawn by RepoMasters from the repo’s ID and its language color.