# bigscience-workshop/Megatron-DeepSpeed

<https://repo.masters-masters.com/en/repos/bigscience-workshop/Megatron-DeepSpeed>

[Home](https://repo.masters-masters.com/en) › [bigscience-workshop](https://repo.masters-masters.com/en/owners/bigscience-workshop) › [Megatron-DeepSpeed](https://repo.masters-masters.com/en/repos/bigscience-workshop/Megatron-DeepSpeed)

bigscience-workshop/Megatron-DeepSpeed has 1,449 stars on GitHub. Its RepoMasters card: Beta tier. Ongoing research training transformer language models at…

| Field | Value |
| --- | --- |
| Owner | [bigscience-workshop](https://repo.masters-masters.com/en/owners/bigscience-workshop) |
| Stars | 1,449 |
| Forks | 225 |
| Language | [Python](https://repo.masters-masters.com/en/languages/python) |
| License | None |
| Created | 2021 |
| Tier | [Beta (700 to 1,999 ★)](https://repo.masters-masters.com/en/ranks/box_office) |
| View on GitHub | [bigscience-workshop/Megatron-DeepSpeed](https://github.com/bigscience-workshop/Megatron-DeepSpeed) |

## Repository

Ongoing research training transformer language models at scale, including: BERT & GPT-2

The tier for repos with 700 to 1,999 ★. Published odds per card pulled: 6%. Rarity: Rare.

## More repos by bigscience-workshop

- [bigscience-workshop/petals · ★ 10,680 · Python · Stable](https://repo.masters-masters.com/en/repos/bigscience-workshop/petals)
- [bigscience-workshop/promptsource · ★ 3,042 · Python · RC](https://repo.masters-masters.com/en/repos/bigscience-workshop/promptsource)
- [bigscience-workshop/bigscience · ★ 1,020 · Shell · Beta](https://repo.masters-masters.com/en/repos/bigscience-workshop/bigscience)
- [bigscience-workshop/xmtf · ★ 534 · Jupyter Notebook · Alpha](https://repo.masters-masters.com/en/repos/bigscience-workshop/xmtf)
- [bigscience-workshop/biomedical · ★ 507 · Python · Alpha](https://repo.masters-masters.com/en/repos/bigscience-workshop/biomedical)
- [bigscience-workshop/t-zero · ★ 463 · Python · Alpha](https://repo.masters-masters.com/en/repos/bigscience-workshop/t-zero)

## Nearby in Python

- [dleemiller/WordLlama · ★ 1,450 · Python · Beta](https://repo.masters-masters.com/en/repos/dleemiller/WordLlama)
- [oddcod3/Phantom-Evasion · ★ 1,449 · Python · Beta](https://repo.masters-masters.com/en/repos/oddcod3/Phantom-Evasion)
- [wenet-e2e/wespeaker · ★ 1,449 · Python · Beta](https://repo.masters-masters.com/en/repos/wenet-e2e/wespeaker)
- [bojone/attention · ★ 1,448 · Python · Beta](https://repo.masters-masters.com/en/repos/bojone/attention)
- [keredson/gnomecast · ★ 1,448 · Python · Beta](https://repo.masters-masters.com/en/repos/keredson/gnomecast)

---
Data: GitHub · Illustration: the project’s public image, pixelated by RepoMasters.