CCWget
Common Crawl wGET turns large-scale web archives into targeted, reproducible lookups. Find the record you need, then retrieve only the selected WARC object.
Benefits
- Search historical URLs — Enumerate exact URLs, hosts, domains and time ranges across indexed Common Crawl snapshots.
- Follow content by digest — Use hexadecimal or Base32 SHA-1 digests to locate matching archived content without guessing its URL.
- Retrieve the HTTP header and web object — Inspect metadata first, then retrieve the WARC header and the matching archived web object without transferring the full archive.
Access to CCWget must be requested from CIRCL. To request access, please contact CIRCL.