fetcher Simple Python fetcher
Download

fetcher Ranking & Summary

fetcher Tags

Python fetcher Spider Python fetcher spider charset

fetcher Description

Simple Python fetcher fetcher is a simple Python fetcher module.Specific use, see test case:#!/usr/bin/env python#coding: utf-8from fetcher import *from unittest import TestCaseimport loggingdef test_fetcher_fetch(): fetcher = Fetcher() d = fetcher.fetch("http://www.163.com") assert d.code == 200def test_fetcher_head(): fetcher = Fetcher() d = fetcher.head("http://www.163.com") logging.info(d) assert "Content-Type" in d def test_charset(): fetcher = Fetcher() d = fetcher.fetch("http://www.taobao.com") assert "gb2312" == FetcherUtil.charset(d.content)def test_decode(): fetcher = Fetcher() d = fetcher.fetch("http://www.taobao.com") assert "·宝" not in d.content c = FetcherUtil.decode(d.content, "utf-8") assert "·宝" in cdef test_image_fetcher(): fetcher = ImageFetcher() images = fetcher.fetch("http://www.taobao.com") assert images Requirements: · Python

fetcher Related Software