Congrats to the graduates of our first Google Brain Residency class, and welcome to our new class

It's hard to believe the first year of our Google Brain Residency program has come to an end already for our initial class of residents, and that this week we're welcoming our second class of residents. This blog post highlights some of the research that residents in our inaugural class did in their one year residency (they've been a very productive bunch, as you can see by reading the blog post).

Blog post:
https://research.googleblog.com/2017/07/the-google-brain-residency-program-one.html

More about the residency program: http://g.co/brainresidency

TensorFlow Research Cloud: 1000 free Cloud TPUs for open machine learning research

Complementing our announcement of Cloud TPUs today, I'm also extremely excited that Google is offering 1,000 Cloud TPU devices, each capable of 180 TFLOPS of computation, for free to researchers doing open machine learning research. That's a total of 180 petaflops of computation (for comparison, the #1 supercomputer in the world now has an Rpeak of ~125 petaflops, per top500.org). As we've talked to machine learning researchers around the world, many of them tell us that they are computation limited: they believe that they could advance their research more quickly with access to more computation. TFRC is our way of helping the open machine learning research community move even more quickly and to tackle problems that are currently computation limited (as long as they are willing to openly publish their research and share feedback with the Cloud TPU team). See the blog post for details.

Nice work in getting TFRC to this point, +Zak Stone!

Relevant links:
TFRC blog post: https://research.googleblog.com/2017/05/introducing-tensorflow-research-cloud.html
Cloud TPU blog post: https://blog.google/topics/google-cloud/google-cloud-offer-tpus-machine-learning/
Cloud TPUs page: https://cloud.google.com/tpu/
Signup form to express interest in TFRC, Cloud TPUs, and TPU Pods: https://g.co/tpusignup

I'm very excited about the work many people at Google have been doing to develop our own custom systems for accelerating training machine learning models. Version 2 of our Tensor Processing Unit is a device that offers 180 teraflops of compute, and is also designed to be connected together with an ultra-high-speed network into a TPU Pod of 64 of these devices, offering 11.5 petaflops of compute. These systems are programmed using TensorFlow.

These TPU devices will be coming to Google Cloud as Cloud VMs with TPUs.

A nice write-up of how the Google Streetview team uses deep learning to extract useful data about the world from Streetview images, to improve the quality of our maps product with more accurate data. Nice work, +Julian Ibarz​!

Something sketchy going on with machine learning

David Ha, one of this year's Brain Residents (g.co/brainresidency), wrote up a very nice Google Research blog post about the work that he and +Douglas Eck have been doing on generative models for sketch-style drawing.

David and Doug have a new Arxiv paper that was just posted about the details of the work at titled "A Neural Representation of Sketch Drawings" for the details, but the blog post gives a really nice overview of what these sorts of generative models can do. The blog post goes through many variations of pretty cool things you can do with this model:

o "Draw me a cat", and it can generate different many different cats
o "Draw me a cat that looks roughly like toothbrush", and it can draw a cat that looks like a toothbrush
o Smoothly transition between cat pictures that are just faces versus ones that show the whole body by manipulating a latent variable that is learned by the model
o Do "vector arithmetic in drawing space": "cat head + (pig with body - pig just head)" gives "cat with body"
o Complete drawings given a category and just a line or two: "Finish this fire truck"

Just a taste of the kinds of creative tools that machine learning will be able to provide in the coming years.

Blog post:
https://research.googleblog.com/2017/04/teaching-machines-to-draw.html
Arxiv paper with details: https://arxiv.org/abs/1704.03477

Google, Google Brain and Canada

Researchers in Canada have been the source of some of the best AI research of the past several decades, with great work coming from University of Toronto, Université de Montréal, McGill University, and elsewhere. More recently, the Canada and Ontario governments have given large grants to support basic AI research in Montreal (https://mila.umontreal.ca/en/) and Toronto (http://vectorinstitute.ai/), and Google has contributed support for both of these institutes as well:

https://canada.googleblog.com/2017/03/canadas-ai-moment.html
https://canada.googleblog.com/2016/11/google-invests-45-million-in-montreal.html

The Google Brain team (g.co/brain) has had a presence in Canada since 2013. We have Google Brain locations in Toronto (headed by +Geoffrey Hinton) and in Montreal (headed by +Hugo Larochelle). These
labs do basic research and collaborate with the local universities and
institutes. We're hiring top machine learning researchers in these
labs (as well as a other locations including Mountain View,
San Francisco, Cambridge (MA), New York City, and Zürich), so if
you're interested in joining the Google Brain team, learn more at
http://g.co/brain and consider applying at:

https://careers.google.com/jobs#t=sq&j=brain+research


Neural nets can do pretty well at some quantum chemistry problems

Members of the Google Brain team, in collaboration with members of the Google Accelerated Sciences team, DeepMind, and the University of Basel, have posted some new exciting work on using neural nets to solve important problems in quantum chemistry. Traditionally, these problems are tackled with very slow and expensive numerical simulations based on Dense Functional Theory (DFT: https://en.wikipedia.org/wiki/Density_functional_theory). Quoting from the blog post, the neural-net based approaches "_can predict 11 of these properties accurately enough to potentially be useful to chemists, but up to 300,000 times faster than it would take to simulate them using DFT_" (this is a trend I'm starting to see more often across lots of sciences: use a neural net to cheaply approximate a much more computationally expensive simulator). This is hugely useful, because one could use the neural net to rapidly screen hundreds of thousands of times as many possible compounds in the same amount of time, more rapidly identifying important compounds for drug development or exotic material design.

Blog: https://research.googleblog.com/2017/04/predicting-properties-of-molecules-with.html

The blog post references two new papers that have just been posted to Arxiv. The first is focused more for a chemistry audience, and the second describes the underlying machine learning model that was developed to solve these chemistry problems:

"Fast machine learning models of electronic and energetic properties consistently reach approximation errors better than DFT accuracy", https://arxiv.org/abs/1702.05532

"Neural Message Passing for Quantum Chemistry", https://arxiv.org/abs/1704.01212

It's also nice to see that these papers both have Google Brain Residents as co-authors (see g.co/brainresidency).

Nice work, +George Dahl, +Oriol Vinyals, Felix Faber, Luke Hutchison, Bing Huang, Justin Gilmer, Samuel Schoenholz, Steven Kearnes, Patrick Riley, and Anatole von Lilienfeld!

Edit: Adding missing quotation mark

ISCA paper preprint about Google's Tensor Processing Unit

Paper: https://drive.google.com/file/d/0Bx4hafXDDq2EMzRNcy1vSUxtcEk/view
Blog post by +Norm Jouppi: https://cloudplatform.googleblog.com/2017/04/quantifying-the-performance-of-the-TPU-our-first-machine-learning-chip.html

Last June at Google I/O, +Sundar Pichai showed an example of a new type of custom ASIC that Google had developed to accelerate machine learning workloads, called a Tensor Processing Unit (TPU), but didn't give very many details. The TPU is used to run large neural networks very efficiently and with low latency throughout many Google products, including Search, Photos, Translate, and also powered the AlphaGo system used during the match against Lee Sedol in Korea last March, and offers 92 trillion operations per second (TOPs) per chip with a modest power budget. I'm happy to announce that we now have a detailed paper In-Datacenter Performance Analysis of a Tensor Processing Unit​ that will appear in this year's International Symposium on Computer Architecture (ISCA) conference in Toronto in June. Today we've published a pre-print of the paper and a companion blog post, and +David Patterson will be giving a talk about the TPU at the Computer History Museum in Mountain View this afternoon (https://sites.google.com/corp/view/naeregionalsymposium: sadly no more space is available).

Various news articles:
https://www.nextplatform.com/2017/04/05/first-depth-look-googles-tpu-architecture/
https://www.wired.com/2017/04/building-ai-chip-saved-google-building-dozen-new-data-centers/
Hacker News discussion: https://news.ycombinator.com/item?id=14043059