https://distill.pub Homepage articles from Distill Tue, 02 Aug 2022 21:49:04 -0400 https://distill.pub/favicon.png https://distill.pub en-us 60 https://distill.pub/2021/understanding-gnns Understanding the building blocks and design choices of graph neural networks. https://distill.pub/2021/understanding-gnns Thu, 02 Sep 2021 20:0:0 Z https://distill.pub/2021/gnn-intro What components are needed for building learning algorithms that leverage the structure and properties of graphs? https://distill.pub/2021/gnn-intro Thu, 02 Sep 2021 20:0:0 Z https://distill.pub/2021/distill-hiatus After five years, Distill will be taking a break. https://distill.pub/2021/distill-hiatus Fri, 02 Jul 2021 20:0:0 Z https://distill.pub/selforg/2021/adversarial Reprogramming Neural CA to exhibit novel behaviour, using adversarial attacks. https://distill.pub/selforg/2021/adversarial Thu, 06 May 2021 20:0:0 Z https://distill.pub/2020/circuits/weight-banding Weights in the final layer of common visual models appear as horizontal bands. We investigate how and why. https://distill.pub/2020/circuits/weight-banding Thu, 08 Apr 2021 20:0:0 Z https://distill.pub/2020/circuits/branch-specialization When a neural network layer is divided into multiple branches, neurons self-organize into coherent groupings. https://distill.pub/2020/circuits/branch-specialization Mon, 05 Apr 2021 20:0:0 Z https://distill.pub/2021/multimodal-neurons We report the existence of multimodal neurons in artificial neural networks, similar to those found in the human brain. https://distill.pub/2021/multimodal-neurons Thu, 04 Mar 2021 20:0:0 Z https://distill.pub/selforg/2021/textures Neural Cellular Automata learn to generate textures, exhibiting surprising properties. https://distill.pub/selforg/2021/textures Thu, 11 Feb 2021 20:0:0 Z https://distill.pub/2020/circuits/visualizing-weights We present techniques for visualizing, contextualizing, and understanding neural network weights. https://distill.pub/2020/circuits/visualizing-weights Thu, 04 Feb 2021 20:0:0 Z https://distill.pub/2020/circuits/curve-circuits Reverse engineering the curve detection algorithm from InceptionV1 and reimplementing it from scratch. https://distill.pub/2020/circuits/curve-circuits Sat, 30 Jan 2021 20:0:0 Z https://distill.pub/2020/circuits/frequency-edges A family of early-vision neurons reacting to directional transitions from high to low spatial frequency. https://distill.pub/2020/circuits/frequency-edges Wed, 27 Jan 2021 20:0:0 Z https://distill.pub/2020/circuits/equivariance Neural networks naturally learn many transformed copies of the same feature, connected by symmetric weights. https://distill.pub/2020/circuits/equivariance Tue, 08 Dec 2020 20:0:0 Z https://distill.pub/2020/understanding-rl-vision With diverse environments, we can analyze, diagnose and edit deep reinforcement learning models using attribution. https://distill.pub/2020/understanding-rl-vision Tue, 17 Nov 2020 20:0:0 Z https://distill.pub/2020/communicating-with-interactive-articles Examining the design of interactive articles by synthesizing theory from disciplines such as education, journalism, and visualization. https://distill.pub/2020/communicating-with-interactive-articles Fri, 11 Sep 2020 20:0:0 Z https://distill.pub/2020/selforg A collection of articles and comments with the goal of understanding how to design robust and general purpose self-organizing systems. https://distill.pub/2020/selforg Thu, 27 Aug 2020 20:0:0 Z https://distill.pub/2020/selforg/mnist Training an end-to-end differentiable, self-organising cellular automata for classifying MNIST digits. https://distill.pub/2020/selforg/mnist Thu, 27 Aug 2020 20:0:0 Z https://distill.pub/2020/circuits/curve-detectors Part one of a three part deep dive into the curve neuron family. https://distill.pub/2020/circuits/curve-detectors Wed, 17 Jun 2020 20:0:0 Z https://distill.pub/2020/bayesian-optimization How to tune hyperparameters for your machine learning model using Bayesian optimization. https://distill.pub/2020/bayesian-optimization Tue, 05 May 2020 20:0:0 Z https://distill.pub/2020/circuits/early-vision An overview of all the neurons in the first five layers of InceptionV1, organized into a taxonomy of 'neuron groups.' https://distill.pub/2020/circuits/early-vision Wed, 01 Apr 2020 20:0:0 Z https://distill.pub/2020/grand-tour By focusing on linear dimensionality reduction, we show how to visualize many dynamic phenomena in neural networks. https://distill.pub/2020/grand-tour Mon, 16 Mar 2020 20:0:0 Z https://distill.pub/2020/circuits What can we learn if we invest heavily in reverse engineering a single neural network? https://distill.pub/2020/circuits Tue, 10 Mar 2020 20:0:0 Z https://distill.pub/2020/circuits/zoom-in By studying the connections between neurons, we can find meaningful algorithms in the weights of neural networks. https://distill.pub/2020/circuits/zoom-in Tue, 10 Mar 2020 20:0:0 Z https://distill.pub/2020/growing-ca Training an end-to-end differentiable, self-organising cellular automata model of morphogenesis, able to both grow and regenerate specific patterns. https://distill.pub/2020/growing-ca Tue, 11 Feb 2020 20:0:0 Z https://distill.pub/2020/attribution-baselines Exploring the baseline input hyperparameter, and how it impacts interpretations of neural network behavior. https://distill.pub/2020/attribution-baselines Fri, 10 Jan 2020 20:0:0 Z https://distill.pub/2019/computing-receptive-fields Detailed derivations and open-source code to analyze the receptive fields of convnets. https://distill.pub/2019/computing-receptive-fields Mon, 04 Nov 2019 20:0:0 Z https://distill.pub/2019/paths-perspective-on-value-learning A closer look at how Temporal Difference Learning merges paths of experience for greater statistical efficiency https://distill.pub/2019/paths-perspective-on-value-learning Mon, 30 Sep 2019 20:0:0 Z https://distill.pub/2019/advex-bugs-discussion Six comments from the community and responses from the original authors https://distill.pub/2019/advex-bugs-discussion Tue, 06 Aug 2019 20:0:0 Z https://distill.pub/2019/advex-bugs-discussion/response-1 The main hypothesis in Ilyas et al. (2019) happens to be a special case of a more general principle that is commonly accepted in the robustness to distributional shift literature https://distill.pub/2019/advex-bugs-discussion/response-1 Tue, 06 Aug 2019 20:0:0 Z https://distill.pub/2019/advex-bugs-discussion/response-2 An example project using webpack and svelte-loader and ejs to inline SVGs https://distill.pub/2019/advex-bugs-discussion/response-2 Tue, 06 Aug 2019 20:0:0 Z https://distill.pub/2019/advex-bugs-discussion/response-3 An example project using webpack and svelte-loader and ejs to inline SVGs https://distill.pub/2019/advex-bugs-discussion/response-3 Tue, 06 Aug 2019 20:0:0 Z https://distill.pub/2019/advex-bugs-discussion/response-4 An experiment showing adversarial robustness makes neural style transfer work on a non-VGG architecture https://distill.pub/2019/advex-bugs-discussion/response-4 Tue, 06 Aug 2019 20:0:0 Z https://distill.pub/2019/advex-bugs-discussion/response-5 Refining the source of adversarial examples https://distill.pub/2019/advex-bugs-discussion/response-5 Tue, 06 Aug 2019 20:0:0 Z https://distill.pub/2019/advex-bugs-discussion/response-6 Section 3.2 of Ilyas et al. (2019) shows that training a model on only adversarial errors leads to non-trivial generalization on the original test set. We show that these experiments are a specific case of learning from errors. https://distill.pub/2019/advex-bugs-discussion/response-6 Tue, 06 Aug 2019 20:0:0 Z https://distill.pub/2019/advex-bugs-discussion/original-authors https://distill.pub/2019/advex-bugs-discussion/original-authors Tue, 06 Aug 2019 20:0:0 Z https://distill.pub/2019/gan-open-problems What we'd like to find out about GANs that we don't know yet. https://distill.pub/2019/gan-open-problems Tue, 09 Apr 2019 20:0:0 Z https://distill.pub/2019/visual-exploration-gaussian-processes How to turn a collection of small building blocks into a versatile tool for solving regression problems. https://distill.pub/2019/visual-exploration-gaussian-processes Tue, 02 Apr 2019 20:0:0 Z https://distill.pub/2019/memorization-in-rnns Inspecting gradient magnitudes in context can be a powerful tool to see when recurrent units use short-term or long-term contextual understanding. https://distill.pub/2019/memorization-in-rnns Mon, 25 Mar 2019 20:0:0 Z https://distill.pub/2019/activation-atlas By using feature inversion to visualize millions of activations from an image classification network, we create an explorable activation atlas of features the network has learned and what concepts it typically represents. https://distill.pub/2019/activation-atlas Wed, 06 Mar 2019 20:0:0 Z https://distill.pub/2019/safety-needs-social-scientists If we want to train AI to do what humans want, we need to study humans. https://distill.pub/2019/safety-needs-social-scientists Tue, 19 Feb 2019 20:0:0 Z https://distill.pub/2018/editorial-update An Update from the Editorial Team https://distill.pub/2018/editorial-update Tue, 14 Aug 2018 20:0:0 Z https://distill.pub/2018/differentiable-parameterizations A powerful, under-explored tool for neural network visualizations and art. https://distill.pub/2018/differentiable-parameterizations Wed, 25 Jul 2018 20:0:0 Z https://distill.pub/2018/feature-wise-transformations A simple and surprisingly effective family of conditioning mechanisms. https://distill.pub/2018/feature-wise-transformations Mon, 09 Jul 2018 20:0:0 Z https://distill.pub/2018/building-blocks Interpretability techniques are normally studied in isolation. We explore the powerful interfaces that arise when you combine them -- and the rich structure of this combinatorial space. https://distill.pub/2018/building-blocks Tue, 06 Mar 2018 20:0:0 Z https://distill.pub/2017/aia By creating user interfaces which let us work with the representations inside machine learning models, we can give people new tools for reasoning. https://distill.pub/2017/aia Mon, 04 Dec 2017 20:0:0 Z https://distill.pub/2017/ctc A visual guide to Connectionist Temporal Classification, an algorithm used to train deep neural networks in speech recognition, handwriting recognition and other sequence problems. https://distill.pub/2017/ctc Mon, 27 Nov 2017 20:0:0 Z https://distill.pub/2017/feature-visualization How neural networks build up their understanding of images https://distill.pub/2017/feature-visualization Tue, 07 Nov 2017 20:0:0 Z http://distill.pub/2017/momentum We often think of optimization with momentum as a ball rolling down a hill. This isn't wrong, but there is much more to the story. http://distill.pub/2017/momentum Tue, 04 Apr 2017 16:00:00 -0400 https://distill.pub/2017/research-debt Science is a human activity. When we fail to distill and explain research, we accumulate a kind of debt... https://distill.pub/2017/research-debt Wed, 22 Mar 2017 20:0:0 Z http://distill.pub/2016/handwriting Several interactive visualizations of a generative model of handwriting. Some are fun, some are serious. http://distill.pub/2016/handwriting Tue, 06 Dec 2016 15:00:00 -0500 http://distill.pub/2016/deconv-checkerboard When we look very closely at images generated by neural networks, we often see a strange checkerboard pattern of artifacts. http://distill.pub/2016/deconv-checkerboard Mon, 17 Oct 2016 16:00:00 -0400 http://distill.pub/2016/misread-tsne Although extremely useful for visualizing high-dimensional data, t-SNE plots can sometimes be mysterious or misleading. http://distill.pub/2016/misread-tsne Thu, 13 Oct 2016 16:00:00 -0400 http://distill.pub/2016/augmented-rnns A visual overview of neural attention, and the powerful extensions of neural networks being built on top of it. http://distill.pub/2016/augmented-rnns Thu, 08 Sep 2016 16:00:00 -0400