Join GitHub today
GitHub is home to over 20 million developers working together to host and review code, manage projects, and build software together.
Python
Ruby
HTML
Haxe
JavaScript
Shell
Other
Cannot retrieve the latest commit at this time.
| Failed to load latest commit information. | |||
|
|
bot | ||
|
|
cogs | ||
|
|
dashboard | ||
|
|
db | ||
|
|
lib | ||
|
|
pipeline | ||
|
|
spec | ||
|
|
.gitignore | ||
|
|
.gitmodules | ||
|
|
COMMANDS | ||
|
|
Gemfile | ||
|
|
Gemfile.lock | ||
|
|
INSTALL | ||
|
|
LICENSE | ||
|
|
README | ||
README
1. ArchiveBot
<SketchCow> Coders, I have a question.
<SketchCow> Or, a request, etc.
<SketchCow> I spent some time with xmc discussing something we could
do to make things easier around here.
<SketchCow> What we came up with is a trigger for a bot, which can
be triggered by people with ops.
<SketchCow> You tell it a website. It crawls it. WARC. Uploads it to
archive.org. Boom.
<SketchCow> I can supply machine as needed.
<SketchCow> Obviously there's some sanitation issues, and it is root
all the way down or nothing.
<SketchCow> I think that would help a lot for smaller sites
<SketchCow> Sites where it's 100 pages or 1000 pages even, pretty
simple.
<SketchCow> And just being able to go "bot, get a sanity dump"
2. More info
For the user's guide, read the COMMANDS file.
For a half-assed installation and operation guide, read INSTALL.
For a polished installation guide, submit a pull request.
3. License
Copyright 2013 David Yip; made available under the MIT license. See
LICENSE for details.
4. Special thanks
Dragonette, Barnaby Bright, Vienna Teng, NONONO.
The memory hole of the Web has gone too far.
Don't look down, never look away; ArchiveBot's like the wind.
vim:ts=2:sw=2:tw=72:et