Movie frame selectivity in hippocampal neurons

(a) Raster plots of two different dentate gyrus (DG) neurons as a function of the movie frame (top). The corresponding mean firing rate response over 60 trials is also shown (bottom). These two cells had significantly increased firing activity in specific parts of the movie. 33.1% of dentate neurons were significantly modulated by the movie (right, green bar), far greater than chance (gray bar). Total active, broad spiking neurons for each brain region indicated at top (Ntuned /Ncells=506/1531). (b) Same as (a), for CA3 (168/969, 17.3%), (c) CA1 (2326/6914, 33.6%) and (d) subiculum (379/849, 44.6%) neurons.

Multi-peaked, mega-scale movie-fields across all brain areas

(a) Distribution of the number of movie-fields per tuned cell (See Methods) in different brain regions (shown by different colors, top line inset, arranged in their hierarchical order). Hippocampal regions (blue-green shades) were significantly different from each other (KS-test p<0.04), except DG-CA3. All visual regions were significantly different from each other (KS-test p<7.0×10−11). All visual-hippocampal region pair-wise comparisons were also significantly different (KS-test p<1.8×10−44). CA1 had the lowest number of movie-fields per cell (2.0±0.02, mean±s.e.m.) while V1 had the highest (10.4±0.1). (b) Distribution of the durations of movie-fields identified in (a), across all tuned neurons from a given brain region. These were significantly different for all brain region pairs (KS-test p<7.3×10−3). The longest movie-fields were in subiculum (3169.9±169.8 ms), and the shortest in V1 (156.6±9.2ms). (c) Snippets of movie-fields from an example cell from V1, with 2 of the fields zoomed in, showing 60x difference in duration. Black bar at top indicates 50ms, and gray bar indicates 1s. Each frame corresponds to 33.3ms. Average response (solid trace, y-axis on the right) is superimposed on the trial wise spiking response (dots, y-axis on the left). Color of dots corresponds to frame numbers as in Fig. 1. (d) Same as (c), for a CA1 neuron with 54x difference in duration. (e) The ratio of longest to shortest field duration within a single cell, i.e., mega-scale index, was largest in V1 (56.7±2.2) and least in subiculum (8.0±9.7). All visual-visual and visual-hippocampal brain region pairs were significantly different on this metric (KS-test p<0.02). Among the hippocampal-hippocampal pairs, only CA3-SUB were significantly different (p=0.03). (f) For each cell, the total duration of all movie-fields, i.e., cumulative duration of significantly elevated activity, was comparable across brain regions. The largest cumulative duration (10.2±0.46s, CA3) was only 1.66x of the smallest (6.2±0.09 sec, V1). Visual-hippocampal and visual-visual brain region pairs’ cumulative duration distributions were significantly different (KS-test p<0.001), but not hippocampal pairs (p>0.07). Distribution of the firing within fields, normalized by that in the shuffle response. All fields from all tuned neurons in a brain region were used. Firing in movie-fields was significantly different across all brain region pairs (KS-test, p<1.0×10−7), except DG-CA3. Movie-field firing was largest in V1 (2.9±0.03) and smallest in subiculum (1.14±0.03). (h) Snippets of movie-fields from representative tuned cells, from LGN showing a long movie-field (233 frames, or 7.8s, panel 1), and from AM-PM and from hippocampus showing short fields (2 frames or 66.6ms wide or less).

Population averaged movie-tuning varies across brain areas.

(a) Stack plot of all the movie-fields detected from all tuned neurons of a brain region. Color indicates relative firing rate, normalized by the maximum firing rate in that movie-field. The movie-fields are sorted according to the frame with the maximal response. Note accumulation of fields in certain parts of the movie, especially in subiculum and AM-PM. (b) Similar to (a), but using only a single, tallest movie-field peak from each neuron showing a similar pattern, with pronounced overrepresentation of some portions of the movie in most brain areas. Each neuron’s response is normalized by its maximum firing rate. (c) Multiple single unit activity (MSUA) in a given brain region, obtained as the average response across all tuned cells, by using maxima-normalized response for each cell from (b). Gray lines indicate mean±4*std response from the shuffle data corresponding to p=0.025 after Bonferroni correction for multiple comparisons (see Methods). AM-PM had the largest MSUA modulation (sparsity=0.01) and CA1 had the smallest (sparsity=1.8×10−4). The MSUA modulation across several brain region pairs –AM&PM-DG, V1-CA3, DG-CA3, CA3-CA1 and CA1-SUB were not significantly correlated (Pearson correlation coefficient p>0.05). Some brain region pairs, DG-LGN, DG-V1, AM&PM-CA3, LGN-CA1, V1-CA1, DG-SUB and CA3-SUB, were significantly negatively correlated (r<-0.18, p<4.0×10−7). All other brain region pairs were significantly positively correlated (r>0.07, p<0.03). (d) Number of frames for which the observed MSUA deviates from the z=±4 range from (c), termed significant deviation. V1 and AM-PM had the largest positive deviant frames (289), and CA3 had the least (zero). (e) Firing in deviant frames above (or below) chance level, as a percentage of the average response. Above chance level deviation was greater or equal to that below, for all brain regions, with the largest positive deviation in AM-PM (9.3%), largest negative deviation in V1 (6.0%), and least in CA3 (zero each). (f) Total firing rate response of visual regions across tuned neurons. All regions had significant negative correlation (r<-0.39, p<3.4×10−34) between the ensemble response and the frame-to-frame (F2F) correlation (gray line, y-axis on the left) across movie frames. (g) Similar to (f), for hippocampal regions. CA3 response were not significantly correlated with the frame-to-frame correlation, dentate gyrus (r=0.26, p=4.0×10−15) and CA1 (r=0.21, p=1.5×10−10) responses were positively correlated, and subiculum response was negatively correlated (r=-0.44, p=2.2×10−43). Note the substantially higher mean firing rates of LGN in (f) and subiculum neurons in (g) (colored lines closer to the top) compared to other brain areas.

Larger reduction of selectivity in hippocampal than visual regions due to scrambled presentation.

(a) Similarity between the visual content of one frame with the subsequent one, quantified as the Pearson correlation coefficient between pixel-pixel across adjacent frames for the continuous movie (pink) and the scrambled sequence (lavender), termed F2F correlation. Similar to Fig. 3g. For the scrambled movie, the frame number here corresponds to the chronological frame sequence, as presented. (b) Fraction of broad spiking neurons significantly modulated by either the continuous movie or the scrambled sequence using z-scored sparsity measures (similar to Fig. 1, see Methods). For all brain regions, continuous movie generated greater selectivity than scrambled sequence (KS-test p<7.4×10−4). (c) Percentage change in the magnitude of tuning between the continuous and scrambled movies for cells significantly modulated by either continuous or scrambled movie, termed visual continuity index. Largest drop in selectivity due to scrambled movie occurred in CA1 (90.3±2.0%), and least in V1 (−1.5±0.6%). Continuous to scrambled tuning change was significantly different between all brain region pairs (KS-test p<0.03) and significantly greater for hippocampal areas than visual (8.2-fold, p<10−100). (d) Raster plots (top) and mean rate responses (color, bottom) showing increased spiking responses to only one or two scrambled movie frames, lasting about 50ms. Tuned responses to scrambled movie were found in all brain regions, but these were the least frequent in DG and CA1. (e) One representative cell each from V1 (left) and CA1 (right), where the frame rearrangement of scrambled responses resulted in a response with high correlation to the continuous movie response for V1, but not CA1. Pearson correlation coefficient values of continuous movie and rearranged scrambled responses are indicated on top. (f) Average decoding error for observed data (see Methods), over 60 trials for continuous movie (maroon), was significantly lower than shuffled data (gray) (KS-test p<8.2×10− 22). Solid line – mean error across 60 trials, shaded box – s.e.m. (g) Similar to (f), decoding of scrambled trials was significantly worse than that for the continuous movie (KS-test p<3.6×10−8), except V1 (p=0.13), where the errors were not significantly different (2.6 vs. 2.7 frames). Scrambled responses, in their “as is”, chronological order were used herein. LGN decoding error for scrambled presentation was 6.5x greater than that for continuous movie, whereas the difference in errors was least for V1 (1.04x). Scrambled movie decoding error for all visual areas and for CA1 and subiculum was significantly smaller than chance level (KS-test p<2.6×10−3), but not DG and CA3 (p>0.13). Only the middle 20 trials of the continuous movie were used for comparison with the scrambled movie since the scrambled movie was only presented 20 times. Middle trials of the continuous movie were chosen as the appropriate subset since they were chronologically closest to the scrambled movie presentation.

The movie.

The 30 second long, isoluminant movie with frame numbers denoting key episodes in this continuous segment.

Movie selectivity across brain areas.

(a) Similar to Fig. 1, representative single cells from LGN showing selective movie responses. Fraction selective are shown by the bar chart on the right. (b) Same as (a), for V1. (c) Same as (a), for higher visual areas AM-PM. (d) Cumulative distribution of movie selectivity across all broad spiking cells, including tuned cells (z>2 vertical black line, see Methods). Largest prevalence of selectivity in broad spiking neurons was seen in primary visual cortex (V1, 97.3%, 2606 out of 2679) and least in CA3 hippocampus (17.3%, 168 out of 969). (e) All brain regions analyzed showed far greater selectivity than the chance level (dashed gray line).

Movie tuning is unaffected by locomotive state of the mouse

(a) Similar to Fig. 1, a representative cell from each brain region showing significant modulation movie tuning using only the data when the mouse was immobile, while excluding the data when the mouse was running (stationary data, see Methods). All cells except for DG are from Fig. 1 or Extended Data Fig. 2. (b) Fraction of selective neurons was significantly above chance in all brain regions, ranging from 94.7% in V1 up to 7.1% in CA3 in the stationary data. (c) To explicitly test the effect of running on movie selectivity, we compared the results in (b) with a random subsample of data that included running and stationary, to control for the loss of data (see Methods). Prevalence of movie selectivity was not significantly different (KS-test p>0.18) in these 2 subsamples, except in CA1 (p=0.007, 13.1% in stationary data, 15.0% in the equivalent subsample). Only sessions with at least 300 seconds of stationary data were used in this analysis to ensure sufficient statistical power.

Simultaneously recorded hippocampal cells have different movie tuning.

Four simultaneously recorded and significantly movie-tuned cells from each from (a) Dentate gyrus, (b) CA3, (c) CA1 and (d) Subiculum. Each cell shows different movie selectivity. Average responses are overlaid (on raster plots), and their color corresponds to the different brain regions, described in Fig. 1 legend.

Few hippocampal neurons had greater than 5 movie-fields.

Handful of movie-tuned neurons from dentate gyrus (row 1 and 2), CA3 (row 3 and 4), CA1 and subiculum (bottom-right), had multiple movie-fields. Average responses are overlaid (on raster plots), similar to Fig. 1 and Extended Data Fig. 4.

Movie-field duration ratios are shorter than expected by chance in visual, but not hippocampal areas.

(a) Distribution of median duration of movie-fields, computed across all fields of a neuron. All visual-hippocampal region pairs were significantly different (KS-test p<7.1×10−31). DG-CA3 and DG-CA1 were not significantly different, but other visual-visual and hippocampal-hippocampal region pairs were significantly different. (KS-test p<0.04). CA3 had the largest median field duration (6.3±0.48s), and V1 had the smallest (0.27±0.03sec). Surprisingly, LGN movie-field durations (0.57±0.13s) were significantly longer than V1 (p=2.5×10−21), and comparable to those in the higher order brain areas (0.71±0.05s). (b) Median firing in movie-fields, normalized by that in the shuffle response, obtained as the median from all fields of a neuron. This metric is significantly different across all brain region pairs (KS-test p<3.4×10−5), except DG-CA3, CA3-CA1 and DG-CA1 pairs. The largest median firing was seen for V1 (2.5±0.05), and the smallest in subiculum (1.13±0.03). (c) Cumulative firing in movie-fields, normalized by that in the shuffle response, obtained by adding the firing within all fields of a neuron was significantly different across all brain region pairs (KS-test p<3.0×10−7), except DG-CA3, CA3-CA1 and DG-CA1. V1 response was largest (1.93±0.04), and subiculum was the smallest (1.11±0.02). (d) For each brain region, the movie-field duration ratio was recalculated by randomly reassigning the cell ids to all the movie peaks from that brain region. Using this new assignment of movie peaks to a cell, we obtain the chance level of mega-scale index (largest/smallest peak duration) within a cell. The observed mega-scale index was lesser than chance in all the visual areas (KS-test p<3.2×10−3, median was 77.5%, 56.2% and 41.7% of chance for LGN, V1 and AM-PM respectively). This was not the case in hippocampal regions (p>0.23). (e) Histogram of movie-fields, binned for their durations and their prominence, on a log-log scale. The most prominent fields tended to be wider in most brain areas, and this effect was stronger in hippocampal regions, than visual. Note that the histogram color is also logarithmically scaled.

Population vector overlap is broader in hippocampus and more stable for tuned than untuned cells.

(a) Population vector overlap between even and odd trials for the population of tuned neurons show highest overlap along the diagonal, i.e. the same movie frame, for all brain regions. Each neuron’s response was normalized by its mean rate and the average response in even as well as odd trials was smoothed by a Gaussian window of 2 frames (66.6ms, see Methods). Dashed black lines indicate the -300 and +300 frames away from the diagonal. Notice large correlations (close to unity, horizontal color bar) indicating stable responses. The correlations decay quickly to smaller values for the visual areas but slowly for hippocampal areas, due to their broader movie-fields. (b) Same as (a), but for untuned neurons, resulting in a salt and pepper overlap pattern and low values of correlation, indicating lesser stability than the tuned neurons. Since the majority of cells in the visual areas were tuned, the untuned population was smaller, leading to more variable population vector overlap. (c) The average overlap as a function of the number of movie frames away from the diagonal. It had a large value in visual regions for the 0th diagonal (colored lines) indicating stable responses, whereas the untuned neuron population (gray lines) were unstable, with values near zero, or chance level. The highest population vector overlap in hippocampal regions was smaller than visual areas but persisted for more frames, due to their broader movie-fields (Full width at half maximum of the peak – 17.3 frames for LGN, 22.7-V1, 39.0-AM&PM, 49.8-DG, 57.4-CA3, 64.7-CA1 and 59.2-subiculum).

Distribution of movie-fields reflects frame to frame correlation structure of the movie.

(a) Histogram of movie-fields across all tuned neurons in a brain region, as a function of the movie frame, showing non-uniform distribution. Framen to framen+1 correlation coefficient (F2F correlation), indicating the similarity of 2 consecutive frames, is shown for reference in gray. All distributions were deemed significantly different than a uniform distribution based on a Chi-square goodness-of-fit test (p<3.8×10−6). All distributions were significantly negatively correlated with F2F correlation (r<-0.18, p<10−7). (b) Same as (a), but for the median duration of movie-fields. F2F correlation shown in gray, with large correlation between consecutive frames between frames 400-800 reflected in larger movie-field durations in visual areas. All distributions were significantly different than a uniform distribution based on a Chi-square goodness-of-fit test (p<10−100) and all distributions were significantly positively correlated with F2F correlation (r>0.24, p<2×10−13). Note-y-axes for the histogram are log-scaled and show larger median durations for hippocampal regions than visual. (c) Total firing rate across all broad spiking neurons in different brain regions, showing similar non-uniformity as Fig. 3. All brain regions had significantly negative correlation with the F2F correlation (r<-0.08, p<0.03), except DG, which was significantly positively correlated (r=0.21, p=2.4×10−10). Largest number of above chance deviations were seen for AM-PM (340 frames), and least for CA3 (57 frames). Below chance level deviations were least common in LGN (25 frames), and most common in AM-PM (441 frames). Similar to Fig. 3c-e.

Scrambled movie elicits narrower but more movie-fields per cell than the continuous movie in the visual regions.

(a) Total number of fields per cell for the scrambled sequence were hierarchically arranged, with largest number of fields in LGN (mean±s.e.m., 31.8±2.0), followed by V1 (24.0±0.38) and last AM-PM (11.1±2.1). All three brain regions were significantly different from each other (KS-test p<2.0×10−5). (b) Median scrambled movie-field duration was shortest in LGN (43.9±131.2ms), intermediate in V1 (46.2±24.8) and widest in AM-PM (77.6±40.1ms), and differences were significant (p<7.0×10−4). This was much smaller than for the continuous movie (Fig. 2). (c) Durations of fields for scrambled sequence across all fields of all neurons from a brain region. These were narrowest in LGN (31.3±6.5ms), followed by V1 (38.6±0.2) and last AM-PM (64.3±6.7). All differences were significant (KS-test p<7.2×10−136). (d) Despite these differences, the cumulative duration of movie-fields was comparable across the three brain regions (1.69±0.05sec for V1, 2.03±0.07 for AM-PM and 2.4±0.2 for LGN), but significantly different (p<1.7×10−5). Note the linear scale on the x-axis in this panel compared to the log-scale in other panels. (e) Ratio of field durations, i.e., mega-scale index, was smallest in V1 (15.5±1.6), intermediate in LGN (16.3±5.4) and largest in AM-PM (23.4±2.1), and not significantly different between V1 and LGN (p=0.28). V1-AM&PM and LGN-AM&PM were significantly different (p<5.7×10−5). (f) Cumulative firing activity, summed across all movie-fields of a given neuron was largest in V1 (3.8±0.1), intermediate in LGN (2.3±0.1) and smallest in AM-PM (2.0±0.07), and significantly different between all brain region pairs (p<0.02).

Scrambled sequence evoked movie-fields were narrower than the continuous movie-fields in all visual regions, cell by cell comparison.

Data for only those visual area neurons that were significantly modulated by both the continuous and scrambled movie were used. (a) The number of movie-fields per cell for the continuous movie was significantly smaller than that for scrambled sequence in all brain areas (LGN – continuous mean±s.e.m.=10.7±0.42, scrambled=31.8±2.0, KS-test p=2.0×10−23, V1-10.8±0.11 vs. 24.0±0.38, KS-test p=3.7×10−210, AM&PM-6.9±0.07, vs. 11.1±0.21, KS-test p=1.3×10−57). Data are additionally scattered by a small random number for ease of visualization. (b) Median duration of movie-fields for a cell was significantly larger for continuous movie, compared to scrambled sequence in all visual regions. (LGN continuous=0.46±0.08sec, scrambled=0.04±0.13, KS-test p=7.1×10−65, V1-0.25±0.03 vs. 0.04±0.02, KS-test p<10−150, AM&PM-0.65±0.04, vs. 0.08±0.04, KS-test p<10−150). (c) Cumulative duration of all movie-fields for a cell was significantly larger for continuous movie, compared to scrambled sequence in all visual regions. (continuous=8.9±0.23sec, scrambled=2.4±0.19, KS-test p=3.2×10−69, V1-6.1±0.09 vs. 1.69±0.05, KS-test p=3.3x10−296, AM&PM-7.8±0.1, vs. 2.0±0.07, KS-test p=9.0×10−318). (d) Histogram of number of fields per cell, for continuous and scrambled movies. (e) Logarithmically spaced histogram of median field durations was significantly different between continuous and scrambled sequence. (f) Similar to (e), histogram of cumulative duration of movie-fields for each cell. (g) The ratio of number of fields per cell between continuous and scrambled movies was biased to smaller than unity values for all brain regions, with the largest bias for LGN (0.46±0.08), intermediate for V1 (0.5±0.04), and least for AM-PM (0.77±0.05). (h) The median field duration ratio was biased to values greater than unity, with the largest bias for LGN (7.4±1.4), least for V1 (4.5±0.68), and intermediate for AM-PM (5.5±0.82). (i) The cumulative field duration ratio was also biased to values greater than unity, with similar biases for LGN (3.37±0.36), V1 (3.1±0.3), and AM-PM (3.3±0.67).

Multiple-single unit activity (MSUA) across all movie-tuned neurons in a brain region shows greater modulation than chance levels for scrambled sequence in all visual areas.

(a) Stack plot of tuned responses to the scrambled movie presentation from each brain region, arranged in their increasing order of the frame corresponding to the peak firing response. Each response is normalized by the peak response. (b) Colored trace-average response, across all tuned responses from (b). gray trace - chance level, z=±4, corresponding to the p=0.025 level after Bonferroni correction. (c) Number of frames for which the observed response exceeds (or falls below) z=±4 cutoff from (b), called significantly deviant frames. V1 had the largest number of positive (279 frames) and negative (297) deviant frames, AM-PM had intermediate (225 & 235), and LGN had the least (31 & 29). (d) Firing rate deviation above chance levels, corresponding to the significant frames, as identified in (c), normalized by the mean rate of the MSUA. Largest deviation was observed in V1 (above-3.1 and below-2.7%), and least in LGN (1.1% and 0.45%) Compare with Fig. 3. (e) Frame to frame correlation, from Fig. 4a for comparison. This was not significantly correlated with the MSUA responses in (b), for any of the brain regions (Pearson correlation coefficient LGN p=0.06, V1 p=0.07, AM-PM p=0.26).

Latency of responses to scrambled-sequence correspond to the anatomical hierarchy of visual areas.

(a) Average response for one representative cell from each visual region, that had high similarity between continuous movie and rearranged scrambled sequence responses (see Methods). Gray response in background corresponds to the chronological scrambled sequence. (b) Cumulative histogram of z-scored correlation between continuous and scrambled-rearranged tuning responses (see Methods). Dotted black line indicates significance threshold of z>2. (c) The latency at which continuous and scrambled-rearranged responses were maximally correlated showed high values (heuristically above 0.25) in a short range of positive latencies for LGN, V1 and AM-PM neurons. This analysis was restricted to neurons tuned in continuous as well as scrambled movies. Similar analysis for hippocampal regions resulted in almost no correlations above 0.25. (d) Cumulative histogram of latencies when the continuous and scrambled-rearranged responses were maximally correlated was smallest for LGN (59.5±4.6ms), and largest for higher visual areas, AM-PM (91.6±1.6ms). Hippocampal regions were excluded, owing to lack of data with correlation about 0.25.

Population vector overlap was narrow at the diagonal with scrambled movie

(a) Population vector overlap between even and odd trials for tuned neurons showing higher overlap along the diagonal for all brain regions. Black lines indicate the -300 and +300 diagonal, whereas the main diagonal is the 0th diagonal. (b) Same as (a) but for untuned neurons, resulting in a salt and pepper overlap without higher correlation around the diagonal. (c) The average overlap along diagonals had a large value in visual regions for the 0th diagonal, which was not true for the untuned neuron population. Average correlation in hippocampal regions was broader and lesser in magnitude compared to visual regions. Similar to Extended Data Fig. 7. Full width at half maximum of the peak – 4.4 frames for LGN, 4.8-V1, 5.2-AM&PM, 7.6-DG, 5.7-CA3, 10.8-CA1 and 15.1-subiculum.

Movie presentation did not alter hippocampal firing rates and the mega-scale coding was unrelated to cluster quality.

(a) More than 50% of hippocampal place cells shut down during maze exploration. In contrast, there was no consistent pattern of neural activation or shutdown during the movie presentation in all brain areas. To make a more conservative estimate, this comparison was restricted to units whose firing rates did not differ by more than 20% across the two movie blocks. Further only the data when the animals were immobile was used to avoid confounding effects of running. (b) The mega-scale index was only weakly correlated with the mean firing rate of a neuron in V1 (Pearson’s correlation coefficient r=0.08, p=7.3×10−5), CA1 (r=-0.14, p=3.5×10−8) and subiculum (r=-0.14, p=0.02), and was uncorrelated for other brain regions (p>0.05) (c) The refractory violations index was uncorrelated with the mega-scale index (lower index means better cluster quality31,79) for all brain regions (p>0.05). To remove potential confounding effect of mean firing rates, we computed the partial correlation coefficient, by factoring out the mean firing rate). (d) Similar to (c), the isolation index (greater isolation index means better cluster quality31,80) was uncorrelated with the mega-scale index for all brain regions (partial correlation coefficient, by factoring out the mean firing rate, p>0.12). Factoring out mean firing rate was deemed necessary since the isolation index was typically positively correlated, and the refractory violations index was typically negatively correlated with mean rate. The mega-scale index comparisons were restricted to movie active, tuned neurons with at least two movie peaks. Note-log spaced axes for (a)-(d)