ArchiveTeam/ArchiveBot

moar minor text edits

moar minor text edits

added new control node documentation, linked redis documentation

minor text edits as per yipdw's feedback

edits to new admin documentation

added new control node documentation, linked redis documentation

updated documentation

updated documentation

…alias

Add !ug as an alias for !unignore

Master

Added QQ share to global ignores

Motivation: IRC crops long lines at a certain number of characters. If you add a too long pattern with !ig, it is impossible to remove it again because the line with !unig would be two characters too long. This alias works around that.

Cf. #218

More forum ignores for Vanilla forum URL patterns

…per-socializer-ignore

Ignore links by the Wordpress plugin "Super Socializer"

Add some RFC5735 networks that should not be crawled [RFR]

Update AddToAny

Add the following networks to the global ignore list as listed by RFC5735:

0.0.0.0/8           "This" Network             RFC 1122, Section 3.2.1.3
169.254.0.0/16      Link Local                 RFC 3927
192.0.2.0/24        TEST-NET-1                 RFC 5737
198.18.0.0/15       Network Interconnect
                    Device Benchmark Testing   RFC 2544
198.51.100.0/24     TEST-NET-2                 RFC 5737
203.0.113.0/24      TEST-NET-3                 RFC 5737

Motivated by failures for TEST-NET-1 starting to show up in actual crawls.

Please review before applying as this is untested regexps in the filter that is always is applied.

AddToAny has switched to https, so need to update global ignore list

…k-ignore

Ignore VKontakte share link also on vk.com domain

…tioncode

Ignore verification code/captcha on SMF which causes an infinite loop

Ignore OAuth URLs by Google and Facebook

…k-ignores

Added some ignore patterns for tracking links and pixels on Reddit