Web Crawls

The Web Archive of the Internet Archive started in late 1996, is made available through the Wayback Machine, and some collections are available in bulk to researchers. Many pages are archived by the Internet Archive for other contributors including partners of Archive-IT, and Save Page Now users. Other captures are donated to the Internet Archive by other partners such as Alexa Internet.

RSS

Sort by:

Internet Archive Web Crawls

1,923,156 items 17.3 petabytes

Save Page Now

417,518 items 4.1 petabytes

Archive Team

2,221,316 items 28.2 petabytes

Alexa Crawls

226,901 items 3.1 petabytes

Worldwide Web Crawls

634,931 items 5.7 petabytes

Archive-It Digital Collection

10,483,441 items 3.2 petabytes

Archive-It Partners

10,478,859 items 3.2 petabytes

Live Web Proxy Crawls

109,032 items 1 petabyte

Archive Team: URLs

565,046 items 6 petabytes

Focused Crawls

736,736 items 5.1 petabytes

Common Crawl

94,009 items 2.3 petabytes

Survey Crawls

100,902 items 922.3 terabytes

Top Domains

359,780 items 2.9 petabytes

Wiki Collections

3,403,652 items 1.7 petabytes

Wikipedia Outlinks

185,410 items 1.2 petabytes

Fix Broken Links Web Crawls

288,662 items 1.4 petabytes

ArchiveBot: The Archive Team Crowdsourced Crawler

39,782 items 4 petabytes

Custom Crawl Services

203,532 items 2.9 petabytes