neuroscience · slides · talks & lectures

Conference talk | Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex @ SfN 2018

Conference talk based on the research paper by
Ilya Kuzovkin, Raul Vicente, Mathilde Petton, Jean-Philippe Lachaux, Monica Baciu, Philippe Kahane, Sylvain Rheims, Juan R. Vidal, Jaan Aru
Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex
Society for Neuroscience (SfN) 2018 annual meeting


We have analyzed human brain activity during the task of visual categorization. The activity was recorded using intracranial electrodes from over 100 human subjects. We compared the activity in different frequency bands and temporal windows to the activation of an artificial neural network trained to perform the same task of visual categorization on the same set of test images.

You can download the slides of the talk here: Ilya-Kuzovkin-Activations-of-deep-convolutional-neural-networks-are-aligned-with-gamma-band-activity-of-human-visual-cortex-SfN-2018.pdf

We consistently found that signals in low-gamma (31–70 Hz) frequencies across all time windows and high-gamma (71–150 Hz) frequencies in the 150–350 ms window are aligned with the DCNN in a specific way: an increase in the complexity of features along the layers of the DCNN was roughly matched by the transformation in the representational geometry of responses to the stimuli along the ventral stream. In other words, the lower and higher layers of the DCNN explained gamma band signals from earlier and later visual areas, respectively.

Our second important observation was that the volume of responses (sum of significant correlations) had a clear pattern when compared across layers of a DCNN: convolutional layers elicited correlation with gamma and high gamma band activity, while the fully connected layers correlated with early responses in the lower (under 30 Hz) frequencies.


Slide 1 of 27: Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex Ilya Kuzovkin, Raul Vicente, Mathilde Petton, Jean-Philippe Lachaux, Monica Baciu, Philippe Kahane, Sylvain Rheims, Juan R. Vidal and Jaan AruSlide 2 of 27: Biological system of visionSlide 3 of 27: Biological system of visionSlide 4 of 27: Biological system of visionSlide 5 of 27: Artificial system of visionSlide 6 of 27: Artificial system of visionSlide 7 of 27: Artificial system of visionSlide 8 of 27: Artificial system of visionSlide 9 of 27: Artificial system of visionSlide 10 of 27: Artificial system of visionSlide 11 of 27: Is there a correspondence between the hierarchies of features of artificial and biological systems of vision?Slide 12 of 27: Güçlü, U. & van Gerven, M. A. Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream. J. Neurosci. 35, 10005–10014 (2015). Yamins, D. L. & DiCarlo, J. J. Using goal-driven deep learning models to understand sensory cortex .…Slide 13 of 27: x100Slide 14 of 27: x100Slide 15 of 27: x100 ~100 probes per subject 9073 probes totalSlide 16 of 27Slide 17 of 27Slide 18 of 27Slide 19 of 27Slide 20 of 27Slide 21 of 27Slide 22 of 27Slide 23 of 27Slide 24 of 27: Quantify the diagonal trend ρ align = Spearman( Assignment to DNN layers, Assignment to Brodmann areas)Slide 25 of 27: Quantify the diagonal trend ρ align = Spearman( Assignment to DNN layers, Assignment to Brodmann areas)Slide 26 of 27: - simple features (mapped to lower layers of DCNN) - complex features (higher layers)Slide 27 of 27: Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex https://github.com/kuz/Human-Intracranial-Recordings-and-DCNN-to-Compare- Biological-and-Artificial-Mechanisms-of-Vision https://www.nature.com/articles/s42003-018-0110-y
1 / 27

Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex