neuroscience · slides

Paper overview | Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Slide 1 of 63: Article overview by Ilya Kuzovkin Umut Güclü and Marcel A. J. van Gerven Computational Neuroscience Seminar University of Tartu 2015 Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral StreamSlide 2 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral StreamSlide 3 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream … pixels classes Linear “spider” “cat”Slide 4 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream … pixels classes … hidden layer Non-linear “cat” “spider”Slide 5 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream … pixels classes … hidden layer … hidden layer Deep “cat” “spider”Slide 6 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream “spider” “cat” important featureSlide 7 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream “spider” important feature RUN! “cat”Slide 8 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream “spider” important feature RUN! Convolutional filter “cat”Slide 9 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream Convolutional (and pooling) layerSlide 10 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream … pixels classes … hidden layer … hidden layer … convolutional layer Deep Convolutional Neural Network “cat” “spider”Slide 11 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral StreamSlide 12 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral StreamSlide 13 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral StreamSlide 14 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream Matthew D. Zeiler, Rob Fergus Visualizing and Understanding Convolutional Networks 2013Slide 15 of 63: Matthew D. Zeiler, Rob Fergus Visualizing and Understanding Convolutional Networks 2013 Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral StreamSlide 16 of 63: Matthew D. Zeiler, Rob Fergus Visualizing and Understanding Convolutional Networks 2013 Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral StreamSlide 17 of 63: Matthew D. Zeiler, Rob Fergus Visualizing and Understanding Convolutional Networks 2013 Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral StreamSlide 18 of 63: Matthew D. Zeiler, Rob Fergus Visualizing and Understanding Convolutional Networks 2013 Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral StreamSlide 19 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral StreamSlide 20 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream Two-stream hypothesisSlide 21 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral StreamSlide 22 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral StreamSlide 23 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral StreamSlide 24 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral StreamSlide 25 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral StreamSlide 26 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral StreamSlide 27 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream ?Slide 28 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral StreamSlide 29 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral StreamSlide 30 of 63Slide 31 of 63Slide 32 of 63: 96 x 37x 37 = 131,424Slide 33 of 63: 96 x 37x 37 = 131,424Slide 34 of 63: 96 x 37x 37 = 131,424 256 x 17x 17 = 73,984Slide 35 of 63: 96 x 37x 37 = 131,424 256 x 17x 17 = 73,984Slide 36 of 63: 96 x 37x 37 = 131,424 256 x 17x 17 = 73,984Slide 37 of 63: 96 x 37x 37 = 131,424 256 x 17x 17 = 73,984Slide 38 of 63: 96 x 37x 37 = 131,424 256 x 17x 17 = 73,984 Train linear regression modelSlide 39 of 63: 96 x 37x 37 = 131,424 256 x 17x 17 = 73,984 Train linear regression model Test itSlide 40 of 63: 96 x 37x 37 = 131,424 256 x 17x 17 = 73,984 Train linear regression model Test it r = 0.22Slide 41 of 63: 96 x 37x 37 = 131,424 256 x 17x 17 = 73,984 Train linear regression model Test it r = 0.22 Train linear regression model Test itSlide 42 of 63: 96 x 37x 37 = 131,424 256 x 17x 17 = 73,984 Train linear regression model Test it r = 0.22 Train linear regression model Test it r = 0.67Slide 43 of 63: 96 x 37x 37 = 131,424 256 x 17x 17 = 73,984 Train linear regression model Test it r = 0.22 Train linear regression model Test it r = 0.67Slide 44 of 63: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral StreamSlide 45 of 63: NEXT COOL THING: CATEGORIES OF FEATURES … ImageNet validation setSlide 46 of 63: NEXT COOL THING: CATEGORIES OF FEATURES … ImageNet validation set . . . . 1888Slide 47 of 63: NEXT COOL THING: CATEGORIES OF FEATURES … ImageNet validation set . . . . 1888Slide 48 of 63: NEXT COOL THING: CATEGORIES OF FEATURES … ImageNet validation set . . . . . 1888Slide 49 of 63: NEXT COOL THING: CATEGORIES OF FEATURES … ImageNet validation set . . . . . 1888 deconvolution .Slide 50 of 63: NEXT COOL THING: CATEGORIES OF FEATURES … ImageNet validation set . . . . . 1888 deconvolution . Low Mid High • blob • contrast • edge • contour • shape • texture • pattern • object • object part human-assigned to 9 categoriesSlide 51 of 63: NEXT COOL THING: CATEGORIES OF FEATURES … ImageNet validation set . . . . . 1888 deconvolution . Low Mid High • blob • contrast • edge • contour • shape • texture • pattern • object • object part human-assigned to 9 categories 1. Divide 1888 neurons into 9 categoriesSlide 52 of 63: NEXT COOL THING: CATEGORIES OF FEATURES … ImageNet validation set . . . . . 1888 deconvolution . Low Mid High • blob • contrast • edge • contour • shape • texture • pattern • object • object part human-assigned to 9 categories 1. Divide 1888 neurons into 9 categories 2. Predict…Slide 53 of 63: NEXT COOL THING: CATEGORIES OF FEATURES … ImageNet validation set . . . . . 1888 deconvolution . Low Mid High • blob • contrast • edge • contour • shape • texture • pattern • object • object part human-assigned to 9 categories 1. Divide 1888 neurons into 9 categories 2. Predict…Slide 54 of 63: NEXT COOL THING: CATEGORIES OF FEATURES … ImageNet validation set . . . . . 1888 deconvolution . Low Mid High • blob • contrast • edge • contour • shape • texture • pattern • object • object part human-assigned to 9 categories 1. Divide 1888 neurons into 9 categories 2. Predict…Slide 55 of 63: NEXT COOL THING: CATEGORIES OF FEATURES … ImageNet validation set . . . . . 1888 deconvolution . Low Mid High • blob • contrast • edge • contour • shape • texture • pattern • object • object part human-assigned to 9 categories 1. Divide 1888 neurons into 9 categories 2. Predict…Slide 56 of 63: NEXT COOL THING: CATEGORIES OF FEATURESSlide 57 of 63: OTHER RESULTS Correlation between predicted responses between pairs of voxel groupsSlide 58 of 63: OTHER RESULTS Selectivity of visual areas to feature maps of varying complexitySlide 59 of 63: OTHER RESULTS Distribution of the receptive field centersSlide 60 of 63: OTHER RESULTS Biclustering of voxels and feature mapsSlide 61 of 63: SUMMARYSlide 62 of 63: An intracranial dataset we have. How to repeat the result?Slide 63 of 63: An intracranial dataset we have. How to repeat the result? vs.
1 / 63

Article overview: Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

The article presents the comparison of the complexity of the representation of visual features in the deep convolutional neural network and in our brain. DNN activity layer-by-layer is used to predict voxel activations and it is shown that lower layers of the DNN are better at predicting V1, V2 and that higher layers of the DNN are better at predicting activity in LO and higher areas of the ventral stream. The result effectively demonstrates that the layer-by-layer complexity of visual features we see in DNNs is also present in the visual cortex.

Download PDF