# fkodom/grouped-query-attention-pytorch

<https://repo.masters-masters.com/en/repos/fkodom/grouped-query-attention-pytorch>

[Home](https://repo.masters-masters.com/en) › [fkodom](https://repo.masters-masters.com/en/owners/fkodom) › [grouped-query-attention-pytorch](https://repo.masters-masters.com/en/repos/fkodom/grouped-query-attention-pytorch)

fkodom/grouped-query-attention-pytorch has 196 stars on GitHub. Its RepoMasters card: Prototype tier. (Unofficial) PyTorch implementation of grouped-query…

| Field | Value |
| --- | --- |
| Owner | [fkodom](https://repo.masters-masters.com/en/owners/fkodom) |
| Stars | 196 |
| Forks | 14 |
| Language | [Python](https://repo.masters-masters.com/en/languages/python) |
| License | MIT |
| Created | 2023 |
| Tier | [Prototype (100 to 249 ★)](https://repo.masters-masters.com/en/ranks/rush) |
| View on GitHub | [fkodom/grouped-query-attention-pytorch](https://github.com/fkodom/grouped-query-attention-pytorch) |

## Repository

(Unofficial) PyTorch implementation of grouped-query attention (GQA) from "GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints" (https://arxiv.org/pdf/2305.13245.pdf)

The tier for repos with 100 to 249 ★. Published odds per card pulled: 70%. Rarity: Common.

## More repos by fkodom

- [fkodom/fft-conv-pytorch · ★ 524 · Python · Alpha](https://repo.masters-masters.com/en/repos/fkodom/fft-conv-pytorch)
- [fkodom/clip-text-decoder · ★ 119 · Python · Prototype](https://repo.masters-masters.com/en/repos/fkodom/clip-text-decoder)
- [fkodom/yet-another-retnet · ★ 105 · Python · Prototype](https://repo.masters-masters.com/en/repos/fkodom/yet-another-retnet)

## Nearby in Python

- [dacnay816y62-hub/fantasy-movie-poster-skill · ★ 196 · Python · Prototype](https://repo.masters-masters.com/en/repos/dacnay816y62-hub/fantasy-movie-poster-skill)
- [zhezh/kddcup2026\_champion · ★ 196 · Python · Prototype](https://repo.masters-masters.com/en/repos/zhezh/kddcup2026_champion)
- [Jakeschincariol/chatgpt-youtube-agent-skill · ★ 196 · Python · Prototype](https://repo.masters-masters.com/en/repos/Jakeschincariol/chatgpt-youtube-agent-skill)
- [patpadgett/corkscrew · ★ 196 · Python · Prototype](https://repo.masters-masters.com/en/repos/patpadgett/corkscrew)
- [wbond/sublime\_prefixr · ★ 196 · Python · Prototype](https://repo.masters-masters.com/en/repos/wbond/sublime_prefixr)
- [seomoz/reppy · ★ 196 · Python · Prototype](https://repo.masters-masters.com/en/repos/seomoz/reppy)

---
Data: GitHub · Illustration drawn by RepoMasters from the repo’s ID and its language color.