noncrawlA web crawler creating link soups | |
Download |
noncrawl Ranking & Summary
Advertisement
- License:
- GPL
- Price:
- FREE
- Publisher Name:
- Niels Serup
- Publisher web site:
- http://metanohi.org
noncrawl Tags
noncrawl Description
A web crawler creating link soups noncrawl is a web crawler that saves only links. It crawls the web but does not attempt to do everything. Instead, its only purpose is to recursively check sites for links to other sites, which are then also checked for links to other sites, etc.So, if site Y links to site X, that piece of information is saved, and if site X has not been checked yet, it will be crawled just like site Y was. Requirements: · Python
noncrawl Related Software