Illustration drawn by RepoMasters from the repo’s ID and its language color. Credits
PrototypeCommon

samclarke/robots-parser

NodeJS robots.txt parser with support for wildcard (*) matching.

Owner
samclarke (personal account)
Stars
Stars: 168
Forks
Forks: 19
Language
JavaScript
License
MIT
Created
2014
Topics
user-agentjavascriptnodejsrobots-txtrobots-exclusion-standardrobots-parserrfc-9309
Tier
Prototype (100 to 249 ★) What this tier means
View on GitHub

More repos by samclarke

All 2 repos by samclarke

Nearby in JavaScript

All JavaScript repos