Skip to main content

ArchiveBot: The Archive Team Crowdsourced Crawler

ArchiveBot is an IRC bot designed to automate the archival of smaller websites (e.g. up to a few hundred thousand URLs). You give it a URL to start at, and it grabs all content under that URL, records it in a WARC, and then uploads that WARC to ArchiveTeam servers for eventual injection into the Internet Archive (or other archive sites).

To use ArchiveBot, drop by #archivebot on EFNet. To interact with ArchiveBot, you issue commands by typing it into the channel. Note you will need channel operator permissions in order to issue archiving jobs. The dashboard shows the sites being downloaded currently.

There is a dashboard running for the archivebot process at http://www.archivebot.com.

ArchiveBot's source code can be found at https://github.com/ArchiveTeam/ArchiveBot.

14,489
RESULTS
rss


PART OF
Archive Team
Media Type
14,489
web
Year
9,197
2017
3,909
2016
942
2015
424
2014
16
2013
Topics & Subjects
12,377
archivebot
1
184.180.244.41
1
3dblogger.typepad.com
1
ahkscript.org
1
approachingaro.org
1
arstechnica.com
More right-solid
Collection
More right-solid
Creator
2,108
archive team
SHOW DETAILS
up-solid down-solid
eye
Title
Date Archived
Creator
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 1.5M
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 1.3M
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 922,623
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 815,001
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 560,248
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 513,612
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 455,946
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 445,216
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 404,322
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 401,630
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 399,958
favorite 1
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 387,781
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 386,700
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 376,076
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 375,219
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 350,956
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 334,281
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 332,906
favorite 1
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 328,393
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 315,997
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 312,398
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 311,796
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 309,879
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 308,430
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
Topic: archivebot
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 303,534
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 303,023
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 299,115
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 298,494
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 297,450
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 293,132
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 291,809
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 290,241
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 286,210
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 285,992
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 280,343
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 279,838
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 279,748
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 273,759
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 273,132
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 267,281
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 266,653
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 265,470
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 264,880
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 263,630
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 262,590
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 259,015
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 258,591
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 257,835
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 257,088
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 255,894
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 254,776
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 254,413
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 253,655
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 253,524
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 252,490
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 252,116
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 251,812
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 251,811
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 249,432
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 248,422
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 245,073
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
Topic: archivebot
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 243,016
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 241,749
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 241,396
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 239,730
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
Topic: archivebot
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 238,809
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 237,253
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 234,519
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 233,001
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 232,513
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 232,315
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 231,473
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.