# foundation-model-stack/fms-fsdp

<https://repo.masters-masters.com/en/repos/foundation-model-stack/fms-fsdp>

[Home](https://repo.masters-masters.com/en) › [foundation-model-stack](https://repo.masters-masters.com/en/owners/foundation-model-stack) › [fms-fsdp](https://repo.masters-masters.com/en/repos/foundation-model-stack/fms-fsdp)

foundation-model-stack/fms-fsdp has 289 stars on GitHub. Its RepoMasters card: Alpha tier. 🚀 Efficiently (pre)training foundation models with native PyTorch…

| Field | Value |
| --- | --- |
| Owner | [foundation-model-stack](https://repo.masters-masters.com/en/owners/foundation-model-stack) |
| Stars | 289 |
| Forks | 47 |
| Language | [Python](https://repo.masters-masters.com/en/languages/python) |
| License | Apache-2.0 |
| Created | 2024 |
| Topics | distributed-training, llm, pytorch |
| Tier | [Alpha (250 to 699 ★)](https://repo.masters-masters.com/en/ranks/b_movie) |
| View on GitHub | [foundation-model-stack/fms-fsdp](https://github.com/foundation-model-stack/fms-fsdp) |

## Repository

🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash attention v2.

The tier for repos with 250 to 699 ★. Published odds per card pulled: 20%. Rarity: Uncommon.

## More repos by foundation-model-stack

- [foundation-model-stack/foundation-model-stack · ★ 234 · Python · Prototype](https://repo.masters-masters.com/en/repos/foundation-model-stack/foundation-model-stack)
- [foundation-model-stack/fastsafetensors · ★ 181 · Python · Prototype](https://repo.masters-masters.com/en/repos/foundation-model-stack/fastsafetensors)
- [foundation-model-stack/bamba · ★ 140 · Python · Prototype](https://repo.masters-masters.com/en/repos/foundation-model-stack/bamba)

## Nearby in Python

- [rokbenko/quackd · ★ 289 · Python · Alpha](https://repo.masters-masters.com/en/repos/rokbenko/quackd)
- [techjanitor/botmaker · ★ 289 · Python · Alpha](https://repo.masters-masters.com/en/repos/techjanitor/botmaker)
- [ZhFkl/resumeworkflow · ★ 289 · Python · Alpha](https://repo.masters-masters.com/en/repos/ZhFkl/resumeworkflow)
- [seomoz/pyreBloom · ★ 289 · Python · Alpha](https://repo.masters-masters.com/en/repos/seomoz/pyreBloom)
- [chikatoike/IMESupport · ★ 289 · Python · Alpha](https://repo.masters-masters.com/en/repos/chikatoike/IMESupport)
- [mediacloud/backend · ★ 289 · Python · Alpha](https://repo.masters-masters.com/en/repos/mediacloud/backend)

---
Data: GitHub · Illustration drawn by RepoMasters from the repo’s ID and its language color.