AlphaUncommon
wangy8961/python3-concurrency-pics-02
爬取 www.mzitu.com 全站图片，截至目前共5162个图集，16.5万多张美女图片，使用 asyncio 和 aiohttp 实现的异步版本只需要不到2小时就能爬取完成。按日期创建图集目录，保存更合理。控制台只显示下载的进度条，详细信息保存在日志文件中。支持异常处理，不会终止爬虫程序。失败的请求，下次再执行爬虫程序时会自动下载
- Owner
- wangy8961 (personal account)
- Stars
- Stars: 332
- Forks
- Forks: 82
- Language
- Python
- License
- None
- Created
- 2018
- Topics
- python3concurrencyasyncioaiohttprequestsprogressbarfuturesthreadpoolcoroutine
- Tier
- Alpha (250 to 699 ★) What this tier means
More repos by wangy8961
All 2 repos by wangy8961- wangy8961/flask-vuejs-madblogStars: 880VueBeta
Nearby in Python
All Python repos- BeamusWayne/simp-skillStars: 332PythonAlpha
- yixiangshijie/ai-agents-in-action-2nd-edition-cnStars: 332PythonAlpha
- NVlabs/ENPIREStars: 332PythonAlpha
- rdegges/django-twilioStars: 332PythonAlpha
- lincolnloop/django-layoutStars: 332PythonAlpha
- dancaron/Django-ORMStars: 332PythonAlpha