BetaRare
PuerkitoBio/fetchbot
A simple and flexible web crawler that follows the robots.txt policies and crawl delays.
- Owner
- PuerkitoBio (organization)
- Stars
- Stars: 791
- Forks
- Forks: 92
- Language
- Go
- License
- BSD-3-Clause
- Created
- 2014
- Topics
- crawlerrobots-txt
- Tier
- Beta (700 to 1,999 ★) What this tier means
More repos by PuerkitoBio
All 5 repos by PuerkitoBio- PuerkitoBio/goqueryStars: 14,989GoStable
- PuerkitoBio/gocrawlStars: 2,052GoRC
- PuerkitoBio/purellStars: 485GoAlpha
- PuerkitoBio/rehttpStars: 223GoPrototype
Nearby in Go
All Go repos- insaaniManav/prompt-forgeStars: 791GoBeta
- ShahabSL/SkirkStars: 791GoBeta
- MichaelKinsy/PiGStars: 791GoBeta