PrototypeCommon
shjwudp/c4-dataset-script
Inspired by google c4, here is a series of colossal clean data cleaning scripts focused on CommonCrawl data processing. Including Chinese data processing and cleaning methods in MassiveText.
- Owner
- shjwudp (personal account)
- Stars
- Stars: 137
- Forks
- Forks: 18
- Language
- Python
- License
- MIT
- Created
- 2022
- Topics
- commoncrawldatasetnlpmassivetextpythonspark
- Tier
- Prototype (100 to 249 ★) What this tier means
More repos by shjwudp
All 2 repos by shjwudp- shjwudp/shuStars: 222PythonPrototype
Nearby in Python
All Python repos- a-shygun/dripfetchStars: 137PythonPrototype
- Worldbuilder013/HEXISStars: 137PythonPrototype
- evolvent-ai/RSIGymStars: 137PythonPrototype
- svetlyak40wt/django-tagging-ngStars: 137PythonPrototype
- dayorbyte/MongoAlchemyStars: 137PythonPrototype
- seantis/seantis-questionnaireStars: 137PythonPrototype