Skip to main content

Internet Archive Web Crawls

The Internet Archive discovers and captures web pages through many different web crawls.

At any given time several distinct crawls are running, some for months, and some every day or longer.

View the web archive through the Wayback Machine.

More right-solid
SHOW DETAILS
up-solid down-solid
eye
Title
Date Archived
Creator
Live Web Proxy Crawls
web
eye 7.2M
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 7.2M
favorite 0
comment 0
Government Web & Data Archive
collection
6,269
ITEMS
6.7M
VIEWS
collection
eye 6.7M
This collaborative project is an extension of the 2016  End of Term  project, intended to document the federal government's web presence by archiving government websites and data. As part of this preservation effort, URLs supplied from partner institutions, as well as nominated by the public, will be crawled regularly to provide an on-going view of federal agencies' web and social media presence. Key partners on this effort are the Environmental Data & Governance...
Topics: government, data, federal, congress
Live Web Proxy Crawls
web
eye 6.2M
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 3.9M
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl344.us.archive.org:survey from Thu Oct 12 08:48:34 PDT 2017 to Thu Oct 12 01:56:31 PDT 2017.
Topic: crawldata
Internet Archive crawldata from Webwide Crawl, captured by crawl422.us.archive.org:wide from Wed Jan 4 01:00:14 PST 2017 to Tue Jan 3 19:50:56 PST 2017.
Topic: crawldata
Live Web Proxy Crawls
web
eye 2.4M
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 2.1M
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 1.8M
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 1.5M
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 1.5M
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 1.5M
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 1.3M
favorite 0
comment 0
Internet Archive crawldata from Webwide Crawl, captured by crawl428.us.archive.org:wide from Tue Jun 13 00:55:34 PDT 2017 to Mon Jun 12 19:36:27 PDT 2017.
Topic: crawldata
Internet Archive crawldata from Webwide Crawl, captured by crawl809.us.archive.org:wide from Tue Jun 6 00:51:50 PDT 2017 to Mon Jun 5 19:34:08 PDT 2017.
Topic: crawldata
Internet Archive crawldata from Webwide Crawl, captured by crawl801.us.archive.org:wide from Tue Jun 6 07:52:14 PDT 2017 to Tue Jun 6 03:02:49 PDT 2017.
Topic: crawldata
Live Web Proxy Crawls
web
eye 1.2M
favorite 0
comment 0
Internet Archive crawldata from Webwide Crawl, captured by crawl423.us.archive.org:wide from Tue Jun 6 06:46:26 PDT 2017 to Tue Jun 6 01:11:01 PDT 2017.
Topic: crawldata
Live Web Proxy Crawls
web
eye 1.2M
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 1.1M
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 1.1M
favorite 0
comment 0
Internet Archive crawldata from Webwide Crawl, captured by crawl813.us.archive.org:wide from Tue Jun 6 23:57:11 PDT 2017 to Tue Jun 6 17:57:05 PDT 2017.
Topic: crawldata
Internet Archive crawldata from Webwide Crawl, captured by crawl422.us.archive.org:wide from Wed Jun 7 00:20:30 PDT 2017 to Tue Jun 6 18:18:45 PDT 2017.
Topic: crawldata
Internet Archive crawldata from Webwide Crawl, captured by crawl808.us.archive.org:wide from Wed Jun 7 01:07:21 PDT 2017 to Tue Jun 6 19:06:13 PDT 2017.
Topic: crawldata
Live Web Proxy Crawls
web
eye 1M
favorite 0
comment 0
Internet Archive crawldata from Webwide Crawl, captured by crawl808.us.archive.org:wide from Tue Jun 6 23:48:51 PDT 2017 to Tue Jun 6 17:43:39 PDT 2017.
Topic: crawldata
Live Web Proxy Crawls
web
eye 965,492
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 962,607
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 933,077
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 923,269
favorite 0
comment 0
Survey Crawl Number 6: Sep 11th, 2017 - running now
web
eye 895,741
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl344.us.archive.org:survey from Sun Oct 29 08:31:23 PDT 2017 to Sun Oct 29 02:17:27 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web
eye 893,043
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl339.us.archive.org:survey from Sat Sep 30 04:27:26 PDT 2017 to Fri Sep 29 21:56:42 PDT 2017.
Topic: crawldata
Live Web Proxy Crawls
web
eye 881,839
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 867,076
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 856,569
favorite 0
comment 0
Wide Crawl Number 16: Started June 3rd, 2017 - Still running
web
eye 842,118
favorite 0
comment 0
Internet Archive crawldata from Webwide Crawl, captured by crawl814.us.archive.org:wide from Wed Jun 7 01:39:13 PDT 2017 to Tue Jun 6 19:25:06 PDT 2017.
Topic: crawldata
Live Web Proxy Crawls
web
eye 804,308
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 795,064
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 776,984
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 772,713
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 722,272
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 711,049
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 695,099
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 664,717
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 662,340
favorite 0
comment 0
Survey Crawl Number 6: Sep 11th, 2017 - running now
web
eye 650,412
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl344.us.archive.org:survey from Sat Oct 7 15:34:23 PDT 2017 to Sat Oct 7 08:50:07 PDT 2017.
Topic: crawldata
Live Web Proxy Crawls
web
eye 619,567
favorite 0
comment 0
Wide Crawl Number 16: Started June 3rd, 2017 - Still running
web
eye 596,295
favorite 0
comment 0
Internet Archive crawldata from Webwide Crawl, captured by crawl429.us.archive.org:wide from Sat Jun 3 20:55:35 PDT 2017 to Sat Jun 3 14:24:26 PDT 2017.
Topic: crawldata
Live Web Proxy Crawls
web
eye 578,093
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 568,828
favorite 0
comment 0
Wide Crawl Number 16: Started June 3rd, 2017 - Still running
web
eye 554,094
favorite 0
comment 0
Internet Archive crawldata from Webwide Crawl, captured by crawl424.us.archive.org:wide from Wed Jul 5 22:52:40 PDT 2017 to Wed Jul 5 17:21:36 PDT 2017.
Topic: crawldata
Live Web Proxy Crawls
web
eye 551,839
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 546,481
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 544,648
favorite 0
comment 0
Wide Crawl Number 16: Started June 3rd, 2017 - Still running
web
eye 540,928
favorite 0
comment 0
Internet Archive crawldata from Webwide Crawl, captured by crawl421.us.archive.org:wide from Sat Jun 3 01:20:30 PDT 2017 to Sat Jun 3 14:17:04 PDT 2017.
Topic: crawldata
Live Web Proxy Crawls
web
eye 519,000
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 516,093
favorite 0
comment 0
Live Web Proxy Crawls
web
eye 513,248
favorite 0
comment 0
Survey Crawl Number 6: Sep 11th, 2017 - running now
web
eye 511,629
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl825.us.archive.org:survey from Tue Sep 12 05:26:47 PDT 2017 to Wed Sep 13 06:35:45 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web
eye 504,980
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl824.us.archive.org:survey from Sat Sep 16 02:29:17 PDT 2017 to Tue Sep 19 12:30:46 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web
eye 504,305
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl818.us.archive.org:survey from Mon Sep 11 13:36:04 PDT 2017 to Mon Sep 11 22:51:52 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web
eye 502,529
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl825.us.archive.org:survey from Mon Sep 11 13:31:57 PDT 2017 to Mon Sep 11 22:26:47 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web
eye 502,457
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl835.us.archive.org:survey from Mon Sep 11 13:32:08 PDT 2017 to Mon Sep 11 22:37:40 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web
eye 500,956
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl339.us.archive.org:survey from Mon Sep 11 13:31:15 PDT 2017 to Mon Sep 11 22:22:53 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web
eye 500,333
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl824.us.archive.org:survey from Tue Sep 12 05:26:17 PDT 2017 to Wed Sep 13 06:44:40 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web
eye 500,057
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl817.us.archive.org:survey from Mon Sep 11 04:34:29 PDT 2017 to Mon Sep 11 22:23:11 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web
eye 497,138
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl818.us.archive.org:survey from Tue Sep 12 05:51:52 PDT 2017 to Wed Sep 13 02:15:41 PDT 2017.
Topic: crawldata
Live Web Proxy Crawls
web
eye 497,037
favorite 0
comment 0
Survey Crawl Number 6: Sep 11th, 2017 - running now
web
eye 495,748
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl836.us.archive.org:survey from Wed Oct 25 09:28:50 PDT 2017 to Wed Oct 25 03:36:14 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web
eye 495,211
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl824.us.archive.org:survey from Mon Sep 11 13:31:49 PDT 2017 to Mon Sep 11 22:26:17 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web
eye 494,582
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl825.us.archive.org:survey from Wed Sep 13 13:35:46 PDT 2017 to Fri Sep 15 23:02:56 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web
eye 493,683
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl835.us.archive.org:survey from Tue Sep 12 05:37:40 PDT 2017 to Wed Sep 13 02:01:07 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web
eye 493,301
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl838.us.archive.org:survey from Mon Sep 11 13:32:37 PDT 2017 to Mon Sep 11 21:52:51 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web
eye 493,130
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl339.us.archive.org:survey from Sat Sep 16 03:22:07 PDT 2017 to Tue Sep 19 12:55:17 PDT 2017.
Topic: crawldata