libarc Library for processing internet archive ARC, CDX, and DAT files
Download

libarc Ranking & Summary

libarc Tags

process archive processor processing library archive processor process archive

libarc Description

Library for processing internet archive ARC, CDX, and DAT files Libarc is a C++ library for accessing the contents of GZIP compressed ARC files generated by the Internet Archive's Heritrix web crawler.NOTE: libarc is licensed and distributed under the terms of the BSD License. Here are some key features of "libarc": · Opening and scanning the contents of GZIP compressed ARC file. The library does not currently read CDX index files, though this feature will be added in a future release. · You can get an iterator to walk over the contents of the ARC file member by member. You can specify a media type to limit the types members you see. · You can access the information in the member's URL record and the response headers from the HTTP server. · You can access the member's data in a single API call. What's New in This Release: · Added several utility template functions to libarc.h - for_each, for_each_if, find_if, and count_if · Added version constants to libarc.h. · Significant enhancements to arcdump · You can now dump the contents of an ARC member · You can list the offsets and sizes of members in the ARC file · Formal documentation.

libarc Related Software