PrototypeCommon
out0fmemory/GuozhongCrawler
GuozhongCrawler的是一个无须配置、便于二次开发的爬虫开源框架，它提供简单灵活的API，只需少量代码即可实现一个爬虫。其设计灵感来源于多个爬虫国内外爬虫框架的总结。采用完全模块化的设计，功能覆盖整个爬虫的生命周期(链接提取、页面下载、内容抽取、持久化)，支持多线程抓取，分布式抓取，并支持自动重试，定制执行js、自定义cookie等功能。在处理网站抓取多次后被封IP的问题上，guozhongCrawler采用动态轮换IP机制有效防止IP被封。另外，源码中的注释及Log输出全部采用通俗易懂的中文。让初学者能有更加深刻的理解
- Owner
- out0fmemory (personal account)
- Stars
- Stars: 103
- Forks
- Forks: 65
- Language
- Java
- License
- None
- Created
- 2015
- Tier
- Prototype (100 to 249 ★) What this tier means
More repos by out0fmemory
All 4 repos by out0fmemory- out0fmemory/qqwry.datStars: 1,217PythonBeta
- out0fmemory/GoAgent-Always-AvailableStars: 1,124PythonBeta
- out0fmemory/Goproxy-Always-AvailableStars: 165GoPrototype
Nearby in Java
All Java repos- max-messenger-client/max-bot-sdk-javaStars: 103JavaPrototype
- mastercook777/Heimdall-AYN-Thor-AssistantStars: 103JavaPrototype
- zl1nq/JacoStars: 103JavaPrototype
- apache/maven-scmStars: 103JavaPrototype
- awsmithson/tcx2nikeplusStars: 103JavaPrototype
- intervigilium/MicDroidStars: 103JavaPrototype