WWW::CheckSite::Spider

WWW::CheckSite::Spider is a base class for spidering the web.
Download

WWW::CheckSite::Spider Ranking & Summary

Advertisement

  • Rating:
  • License:
  • Perl Artistic License
  • Price:
  • FREE
  • Publisher Name:
  • Abe Timmerman
  • Publisher web site:
  • http://search.cpan.org/~abeltje/WWW-CheckSite-0.018/lib/WWW/CheckSite/Spider.pm

WWW::CheckSite::Spider Tags


WWW::CheckSite::Spider Description

WWW::CheckSite::Spider is a base class for spidering the web. WWW::CheckSite::Spider is a base Perl class for spidering the web.SYNOPSIS use WWW::CheckSite::Spider; my $sp = WWW::CheckSite::Spider->new( uri => 'http://www.test-smoke.org', ); while ( my $page = $sp->get_page ) { # $page is a hashref with basic information }or to spider a site behind HTTP basic authentication: package BA_Mech; use base 'WWW::Mechanize'; sub get_basic_credentials { ( 'abeltje', '********' ) } package main; use WWW::CheckSite::Spider; my $sp = WWW::CheckSite::Spider->new( ua_class => 'BA_Mech', uri => 'http://your.site.with.ba/', ); while ( my $page = $sp->get_page ) { # $page is a hashref with basic information }This module implements a basic web-spider, based on WWW::Mechanize. It takes care of putting pages on the "still-to-fetch" stack. Only uri's with the same origin will be stacked, taking the robots-rules on the server into account. Requirements: · Perl


WWW::CheckSite::Spider Related Software

About SoftwareSea