PrototypeCommon
Aavache/LLMWebCrawler
A Web Crawler based on LLMs implemented with Ray and Huggingface. The embeddings are saved into a vector database for fast clustering and retrieval. Use it for your RAG.
- Owner
- Aavache (personal account)
- Stars
- Stars: 107
- Forks
- Forks: 13
- Language
- Python
- License
- None
- Created
- 2023
- Topics
- pythonrayraylibdistributed-computinghuggingfacellmmilvustransformervector-databasewebcrawler
- Tier
- Prototype (100 to 249 ★) What this tier means
Nearby in Python
All Python repos- tuwang2020/mcd-fit-buddyStars: 107PythonPrototype
- Ayi914/papercheckerStars: 107PythonPrototype
- token-gate-llms/TokenGateStars: 107PythonPrototype
- chrisdrackett/django-typogrifyStars: 107PythonPrototype
- laanwj/decudaStars: 107PythonPrototype
- akaihola/pgtricksStars: 107PythonPrototype