urlgrabber Python utility that will help you easily fetch files from the internet
Download

urlgrabber Ranking & Summary

urlgrabber Tags

grab fetch grabber grab url file url grabber fetch url

urlgrabber Description

Python utility that will help you easily fetch files from the internet urlgrabber is a free and open source Python package that drastically simplifies the fetching of files. It is designed to be used in programs that need common (but not necessarily simple) url-fetching features. It is extremely simple to drop into an existing program and provides a clean interface to protocol-independant file-access. Best of all, urlgrabber takes care of all those pesky file-fetching details, and lets you focus on whatever it is that your program is written to do! Using urlgrabber, data can be fetched in three basic ways:· urlgrab(url) - copy the file to the local filesystem· urlopen(url) - open the remote file and return a file object· urlread(url) - return the contents of the file as a string Here are some key features of "urlgrabber": · identical behavior for http://, ftp://, and file:// urls · http keepalive - faster downloads of many files by using only a single connection · byte ranges - fetch only a portion of the file · reget - for a urlgrab, resume a partial download · progress meters - the ability to report download progress automatically, even when using urlopen! · throttling - restrict bandwidth usage · batched downloads using threads - download multiple files simultaneously (feature still in progress) · retries - automatically retry a download if it fails. The number of retries and failure types are configurable · authenticated server access for http and ftp · proxy support - support for authenticated http and ftp proxies · mirror groups - treat a list of mirrors as a single source, automatically switching mirrors if there is a failure · broad support - unix and windows, python 2.3 - 2.5 Requirements: · Python

urlgrabber Related Software