Python, 5333 4991

I believe this is the first contender to score significantly better than a random oracle.

def H(s):n=int(s.encode('hex'),16);return n%(8**8-ord('+%:5O![/5;QwrXsIf]\'k#!__u5O}nQ~{;/~{CutM;ItulA{uOk_7"ud-o?y<Cn~-`bl_Yb'[n%70]))

Python 2, 140 bytes, 4662 4471 4362 colliding words

def f(s):n=int(s.encode('hex'),16);return n%(8**8+3-60*ord('4BZp%(jTvy"WTf.[Lbjk6,-[LVbSvF[Vtw2e,NsR?:VxC0h5%m}F5,%d7Kt5@SxSYX-=$N>'[n%71]))

Inspired by the form of kasperd’s solution, obviously—but with the important addition of an affine transformation on the modulus space, and entirely different parameters.

Python, 6446 6372

This solution achieves lower collision count than all previous entries, and it needs only 44 of the 140 bytes allowed for code:

H=lambda s:int(s.encode('hex'),16)%16727401

JavaScript (ES6), 6389

The hash function (105 bytes):

s=>[...s.replace(/[A-Z]/g,a=>(b=a.toLowerCase())+b+b)].reduce((a,b)=>(a<<3)*28-a^b.charCodeAt(),0)<<8>>>8

The scoring function (NodeJS) (170 bytes):

h={},c=0,l=require('fs').readFileSync(process.argv[2],'utf8').split('\n').map(a=>h[b=F(a)]=-~h[b])
for(w of Object.getOwnPropertyNames(h)){c+=h[w]>1&&h[w]}
console.log(c)

Call as node hash.js dictionary.txt, where hash.js is the script, dictionary.txt is the dictionary text file (without the final newline), and F is defined as the hashing function.

Thanks Neil for shaving 9 bytes off the hashing function!

CJam, 6273

{49f^245b16777213%}

XOR each character with 49, reduce the resulting string via x, y ↦ 245x + y, and take the residue modulo 16,777,213 (the largest 24-bit prime).

Scoring

$ cat hash.cjam
qN% {49f^245b16777213%} %N*N
$ all=$(wc -l < british-english-huge.txt)
$ unique=$(cjam hash.cjam < british-english-huge.txt | sort | uniq -u | wc -l)
$ echo $[all - unique]
6273

CJam, 4125

0000000: 7b 5f 37 62 32 30 30 25 5f 22 51 f0 5b 0a  {_7b200%_"Q.[.
000000e: 25 c6 ca 91 94 3c 16 85 db 59 b7 d0 7c f3  %....<...Y..|.
000001c: 83 3c 8c 8a 07 94 20 10 10 f5 82 24 ab 14  .<.... ....$..
000002a: 33 28 ce 33 db ce 1b ac 99 5c 22 d4 a0 be  3(.3.....\"...
0000038: 2e 74 84 14 15 f7 14 10 d4 ca 60 2d 5a 59  .t........`-ZY
0000046: dc 49 b1 a6 21 1a 60 e2 0b a5 a7 52 36 35  .I..!.`....R65
0000054: 1d 75 03 49 b2 7c 57 14 a9 8f 25 4a c2 53  .u.I.|W...%J.S
0000062: 17 bd 59 78 59 f1 1b c1 53 f1 12 7b 69 22  ..YxY...S..{i"
0000070: 32 35 36 62 47 62 3d 32 2a 31 34 39 2b 40  256bGb=2*149+@
000007e: 62 38 33 38 37 33 25 32 30 30 2a 2b 7d     b83873%200*+}

This approach divides domain and codomain into 200 disjoint sets, and defines a different hash function for each pair.

The involved constants have been chosen more or less arbitrarily, and tweaking them will hopefully improve the score. I'll try to find better ones when I have time.

Scoring

$ echo $LANG
en_US
$ cat gen.cjam
"qN%{_7b200%_"
[5 1 15 0 5 11 0 10 2 5 12 6 12 10 9 1 9 4 3 12 1 6 8 5 13 11 5 9 11 7 13 0 7 12 15 3 8 3 3 12 8 12 8 10 0 7 9 4 2 0 1 0 1 0 15 5 8 2 2 4 10 11 1 4 3 3 2 8 12 14 3 3 13 11 12 14 1 11 10 12 9 9 2 2 13 4 10 0 11 14 2 14 7 4 8 4 1 4 1 5 15 7 1 4 1 0 13 4 12 10 6 0 2 13 5 10 5 9 13 12 4 9 11 1 10 6 2 1 1 10 6 0 14 2 0 11 10 5 10 7 5 2 3 6 3 5 1 13 7 5 0 3 4 9 11 2 7 12 5 7 1 4 10 9 8 15 2 5 4 10 12 2 5 3 1 7 11 13 5 9 7 8 5 9 15 1 1 11 12 1 5 3 15 1 1 2 7 11 6 9]
Gb256b:c`
"256bGb=2*149+@b83873%200*+}%$e`0f={1>},1bN"
$ cjam gen.cjam > test.cjam
$ cjam test.cjam < british-english-huge.txt
4125

Mathematica, 6473

The next step up... instead of summing the character codes we treat them as the digits of a base-151 number, before taking them modulo 2²⁴.

hash[word_] := Mod[FromDigits[ToCharacterCode @ word, 151], 2^24]

Here is a short script to determine the number of collisions:

Total[Last /@ DeleteCases[Tally[hash /@ words], {_, 1}]]

I've just tried all bases systematically from 1 onwards, and so far base 151 yielded the fewest collisions. I'll try a few more to bring down the score a bit further, but the testing is a bit slow.

Javascript (ES5), 6765

This is CRC24 shaved down to 140 Bytes. Could golf more but wanted to get my answer in :)

function(s){c=0xb704ce;i=0;while(s[i]){c^=(s.charCodeAt(i++)&255)<<16;for(j=0;j++<8;){c<<=1;if(c&0x1000000)c^=0x1864cfb}}return c&0xffffff}

Validator in node.js:

var col = new Array(16777215);
var n = 0;

var crc24_140 = 
function(s){c=0xb704ce;i=0;while(s[i]){c^=(s.charCodeAt(i++)&255)<<16;for(j=0;j++<8;){c<<=1;if(c&0x1000000)c^=0x1864cfb}}return c&0xffffff}

require('fs').readFileSync('./dict.txt','utf8').split('\n').map(function(s){ 
    var h = crc24_140(s);
    if (col[h]===1) {
        col[h]=2;
        n+=2;
    } else if (col[h]===2) {
        n++;
    } else {
        col[h]=1;
    }
});

console.log(n);

Python, 9310

Yeah, not the best, but at least it is something. As we say in crypto, never write your own hash function.

This is exactly 140 bytes long, as well.

F=lambda x,o=ord,m=map:int((int(''.join(m(lambda z:str(o(z)^o(x[-x.find(z)])^o(x[o(z)%len(x)])),x)))^(sum(m(int,m(o,x))))^o(x[-1]))%(2**24))

Matlab, 30828 8620 6848

It builds the hash by assigning a prime number to each ascii character/position combo and calculating their product for each word modulo the largest prime smaller than 2^24. Note that for testing I moved the call to primes outside into the tester directly before the while loop and passed it into the hash function, because it sped it up by about a factor of 1000, but this version works, and is self-contained. It may crash with words longer than about 40 characters.

function h = H(s)
p = primes(1e6);
h = 1;
for i=1:length(s)
    h = mod(h*p(double(s(i))*i),16777213);
end
end

Tester:

clc
clear variables
close all

file = fopen('british-english-huge.txt');
hashes = containers.Map('KeyType','uint64','ValueType','uint64');

words = 0;
p = primes(1e6);
while ~feof(file)
    words = words + 1;
    word = fgetl(file);
    hash = H(word,p);
    if hashes.isKey(hash)
        hashes(hash) = hashes(hash) + 1;
    else
        hashes(hash) = 1;
    end
end

collisions = 0;
for key=keys(hashes)

    if hashes(key{1})>1
        collisions = collisions + hashes(key{1});
    end
end

Python, 340053

A terrible score from a terrible algorithm, this answer exists more to give a small Python script that displays scoring.

H=lambda s:sum(map(ord, s))%(2**24)

To score:

hashes = []
with open("british-english-huge.txt") as f:
    for line in f:
        word = line.rstrip("\n")
        hashes.append(H(word))

from collections import Counter
print(sum(v for k, v in Counter(hashes).items() if v > 1))

Python, 6390 6376 6359

H=lambda s:reduce(lambda a,x:a*178+ord(x),s,0)%(2**24-48)

May be considered a trivial modification to Martin Büttner's answer.

C#, 6251 6335

int H(String s){int h = 733;foreach (char c in s){h = (h * 533 + c);}return h & 0xFFFFFF;}

The constants 533 and 733 889 and 155 give the best score out of all of the ones I've searched so far.

C++: 7112 6694 6483 6479 6412 6339 collisions, 90 bytes

I implemented a naïve genetic algorithm for my coefficient array. I'll update this code as it finds better ones. :)

int h(const char*s){uint32_t t=0,p=0;while(*s)t="cJ~Z]q"[p++%6]*t+*s++;return t%16777213;}

Test function:

int main(void)
{
    std::map<int, int> shared;

    std::string s;
    while (std::cin >> s) {
        shared[h(s.c_str())]++;
    }

    int count = 0;
    for (auto c : shared) {
        if ((c.first & 0xFFFFFF) != c.first) { std::cerr << "invalid hash: " << c.first << std::endl; }
        if (c.second > 1) { count += c.second; }
    }

    std::cout << count << std::endl;
    return 0;
}

Ruby, 9309 collisions, 107 bytes

def hash(s);require'prime';p=Prime.first(70);(0...s.size).reduce(0){|a,i|a+=p[i]**(s[i].ord)}%(2**24-1);end 

Not a good contender, but I wanted to explore a different idea from other entries.

Assign the first n primes to the first n positions of the string, then sum all prime[i] ** (ascii code of string[i]), then mod 2**24-1.

Java 8, 7054 6467

This is inspired by (but not copied from) the builtin java.lang.String.hashCode function, so feel free to disallow according to rule #2.

w -> { return w.chars().reduce(53, (acc, c) -> Math.abs(acc * 79 + c)) % 16777216; };

To score:

import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.function.Function;

public class TweetableHash {
    public static void main(String[] args) throws Exception {
        List<String> words = Files.readAllLines(Paths.get("british-english-huge.txt"));

        Function<String, Integer> hashFunc = w -> { return w.chars().reduce(53, (acc, c) -> Math.abs(acc * 79 + c)) % 16777216; };

        Map<Integer, Integer> hashes = new HashMap<>();
        for (String word : words) {
            int hash = hashFunc.apply(word);
            if (hash < 0 || hash >= 16777216) {
                throw new Exception("hash too long for word: " + word + " hash: " + hash);
            }

            Integer numOccurences = hashes.get(hash);
            if (numOccurences == null) {
                numOccurences = 0;
            }
            numOccurences++;

            hashes.put(hash, numOccurences);
        }

        int numCollisions = hashes.values().stream().filter(i -> i > 1).reduce(Integer::sum).get();
        System.out.println("num collisions: " + numCollisions);
    }
}

Ruby, 6473 collisions, 129 bytes

h=->(w){@p=@p||(2..999).select{|i|(2..i**0.5).select{|j|i%j==0}==[]};c=w.chars.reduce(1){|a,s|(a*@p[s.ord%92]+179)%((1<<24)-3)}}

The @p variable is filled with all the primes below 999.

This converts ascii values into primes numbers and takes their product modulo a large prime. The fudge factor of 179 deals with the fact that the original algorithm was for use in finding anagrams, where all words that are rearrangements of the same letters get the same hash. By adding the factor in the loop, it makes anagrams have distinct codes.

I could remove the **0.5 (sqrt test for prime) at the expense of poorer performance to shorten the code. I could even make the prime number finder execute in the loop to remove nine more characters, leaving 115 bytes.

To test, the following tries to find the best value for the fudge factor in the range 1 to 300. It assume that the word file in in the /tmp directory:

h=->(w,y){
  @p=@p||(2..999).
    select{|i|(2..i**0.5). 
    select{|j|i%j==0}==[]};
  c=w.chars.reduce(1){|a,s|(a*@p[s.ord%92]+y)%((1<<24)-3)}
}

american_dictionary = "/usr/share/dict/words"
british_dictionary = "/tmp/british-english-huge.txt"
words = (IO.readlines british_dictionary).map{|word| word.chomp}.uniq
wordcount = words.size

fewest_collisions = 9999
(1..300).each do |y|
  whash = Hash.new(0)
  words.each do |w|
    code=h.call(w,y)
    whash[code] += 1
  end
  hashcount = whash.size
  collisions = whash.values.select{|count| count > 1}.inject(:+)
  if (collisions < fewest_collisions)
    puts "y = #{y}. #{collisions} Collisions. #{wordcount} Unique words. #{hashcount} Unique hash values"
    fewest_collisions = collisions
  end
end

Python, 6995 6862 6732

Just a simple RSA function. Pretty lame, but beats some answers.

M=0x5437b3a3b1
P=0x65204c34d
def H(s):
    n=0
    for i in range(len(s)):
        n+=pow(ord(s[i]),P,M)<<i
    return n%(8**8)