LOWER DENS

In the two weeks since the election, I’ve seen a lot of talk about “standing up” for science. Science Magazine put an ad for new AAAS memberships under a photograph of the white house, with a call to “stand strong” and “speak up for science” emphasizing funding and research investment, science communication, and climate change.

In the same two weeks, I also saw:

• a pre-print of a computer science article about using machine learning to identify “criminality” in photographs of faces
• an article about the science of “educational genomics,” where students can be tracked into different classes according to their genetic potential
• and an article speculating that China would be the first to use CRISPR to create “superhumans”

When the EPA transition is being led by someone who said “I am not a great believer in man-made climate change,” it is vital that we all stand up for the future of our world. But when overt racism is now daily coming from the most powerful people in the country and hate crimes are rampant, we must also stand up and recognize when science further reinforces oppression and speak up for marginalized people.

The examples above show how “objective” scientific treatment of social phenomena can backfire and enable racist policies. An algorithm to detect “criminality” from facial features is going to be biased the same way our criminal justice system is biased along racial and socioeconomic lines. A program to match students to courswork based on genetics will be biased the same way our educational system is along racial and socioeconomic lines. Speculation about “superhumans” traffics in racist stereotypes about Chinese ethics and the problematic belief that ability is defined more by our genes than by our environments.

Scientists aren’t immune to the blind spots and biases of our culture and science has long been an instrument of normalizing and naturalizing prejudice. Today, those scientists are more likely to complain about “political correctness” than appeal to our sense of “natural” hierarchies. At the same time, white supremacists tout James Watson’s Nobel Prize alongside his abhorrent beliefs about intelligence and race (let’s save discussion of the “anti-science” label for another time).

To stand up for science in our uncertain political time, we have to think beyond budgets—I’m going to stand up for a better science. I’m going to stand up for scientists who are being harrassed, whose identities leave them vulnerable, and stand with all who are standing up for a science that includes diverse voices. I’m going to stand for science that works for all of us.

So it turns out you can train a neural network to generate paint colors if you give it a list of 7,700 Sherwin-Williams paint colors as input. How a neural network basically works is it looks at a set of data - in this case, a long list of Sherwin-Williams paint color names and RGB (red, green, blue) numbers that represent the color - and it tries to form its own rules about how to generate more data like it. 

Last time I reported results that were, well… mixed. The neural network produced colors, all right, but it hadn’t gotten the hang of producing appealing names to go with them - instead producing names like Rose Hork, Stanky Bean, and Turdly. It also had trouble matching names to colors, and would often produce an “Ice Gray” that was a mustard yellow, for example, or a “Ferry Purple” that was decidedly brown.  

These were not great names.

There are lots of things that affect how well the algorithm does, however.

One simple change turns out to be the “temperature” (think: creativity) variable, which adjusts whether the neural network always picks the most likely next character as it’s generating text, or whether it will go with something farther down the list. I had the temperature originally set pretty high, but it turns out that when I turn it down ever so slightly, the algorithm does a lot better. Not only do the names better match the colors, but it begins to reproduce color gradients that must have been in the original dataset all along. Colors tend to be grouped together in these gradients, so it shifts gradually from greens to browns to blues to yellows, etc. and does eventually cover the rainbow, not just beige.

Apparently it was trying to give me better results, but I kept screwing it up.

Raw output from RGB neural net, now less-annoyed by my temperature setting

People also sent in suggestions on how to improve the algorithm. One of the most-frequent was to try a different way of representing color - it turns out that RGB (with a single color represented by the amount of Red, Green, and Blue in it) isn’t very well matched to the way human eyes perceive color.

These are some results from a different color representation, known as HSV. In HSV representation, a single color is represented by three numbers like in RGB, but this time they stand for Hue, Saturation, and Value. You can think of the Hue number as representing the color, Saturation as representing how intense (vs gray) the color is, and Value as representing the brightness. Other than the way of representing the color, everything else about the dataset and the neural network are the same. (char-rnn, 512 neurons and 2 layers, dropout 0.8, 50 epochs)

Raw output from HSV neural net:

And here are some results from a third color representation, known as LAB. In this color space, the first number stands for lightness, the second number stands for the amount of green vs red, and the third number stands for the the amount of blue vs yellow.

Raw output from LAB neural net:

It turns out that the color representation doesn’t make a very big difference in how good the results are (at least as far as I can tell with my very simple experiment). RGB seems to be surprisingly the best able to reproduce the gradients from the original dataset - maybe it’s more resistant to disruption when the temperature setting introduces randomness.

And the color names are pretty bad, no matter how the colors themselves are represented.

However, a blog reader compiled this dataset, which has paint colors from other companies such as Behr and Benjamin Moore, as well as a bunch of user-submitted colors from a big XKCD survey. He also changed all the names to lowercase, so the neural network wouldn’t have to learn two versions of each letter.

And the results were… surprisingly good. Pretty much every name was a plausible match to its color (even if it wasn’t a plausible color you’d find in the paint store). The answer seems to be, as it often is for neural networks: more data.

Raw output using The Big RGB Dataset:

I leave you with the Hall of Fame:

RGB:

HSV:

LAB:

Big RGB dataset: