Model discovery to link neural activity to behavioral tasks
- Department of Neuroscience, The Ohio State University College of Medicine, United States
- Interdisciplinary Biophysics Graduate Program, United States
Figures
Figure 1 with 1 supplement
Model identification of neural encoding (MINE) accommodates a large set of predictor–activity relationships.
(A) Schematic of the convolutional neural network (CNN) used. (B) The predictors that make up the ground-truth dataset. S1 and S2 are continuously variable predictors akin to sensory variables while …
Figure 1—figure supplement 1
Model identification of neural encoding (MINE) accommodates a large set of predictor–activity relationships.
(A) Heatmap of all generated responses in the ground-truth dataset 220 total across 11 groups including pure noise responses. (B) Schematic of the expanded linear model that includes all first-order …
Figure 2 with 1 supplement
Truncations of the Taylor expansion assign computational complexity.
(A) Schematic of the approach. At the data mean, the output of the network is differentiated with respect to the inputs. The first-order derivative (gradient J) and the second-order derivatives …
Figure 2—figure supplement 1
Truncations of the Taylor expansion assign computational complexity.
(A–C) Linear correlations between input predictors and ground-truth data ‘neural activity’ for different mixtures. Colors of correlation plots correspond to colored examples in Figure 2. (A) Mixing …
Figure 3 with 1 supplement
Model identification of neural encoding (MINE) characterizes linear and nonlinear receptive fields.
(A) Schematic of the test response generation. Inputs S (either white-noise or slow fluctuating) are convolved in parallel with two receptive fields acting as filters. The result of one convolution …
Figure 3—figure supplement 1
Model identification of neural encoding (MINE) characterizes linear and nonlinear receptive fields.
(A) Comparison of example Gaussian white noise (gray) and random wave stimulus (red). (B) When presenting a Gaussian white noise stimulus, the quality of receptive fields extracted by MINE …
Figure 4 with 1 supplement
Taylor decomposition reveals contributing single factors and interactions.
(A) The neural model translates predictors into neural activity. By Taylor decomposition, the function implemented by the convolutional neural network (CNN) can be linearized locally. Relating …
Figure 4—figure supplement 1
Taylor decomposition reveals contributing single factors and interactions.
(A) Schematic highlighting key parts of the decomposition that isolates individual factors. (B–H) Remaining responses and associated Taylor metrics. Red bars indicate predictors that are expected to …
Figure 5 with 1 supplement
Model identification of neural encoding (MINE) identifies cortical features of sensorimotor processing during a learned task.
(A) Simplified schematic of the widefield imaging experiment conducted in Musall et al., 2019a. (B) MINE test data vs. training data correlations on 200 temporal components from 13 sessions across …
Figure 5—figure supplement 1
Model identification of neural encoding (MINE) identifies cortical features of sensorimotor processing during a learned task.
(A) Per-pixel Taylor metric scores for the right visual stimulus (‘rVisStim’) subtracted from those of the left visual stimulus (‘lVisStim’) for the 12 mice not shown in Figure 5F. (B) As (A) but …
Figure 6 with 1 supplement
Using model identification of neural encoding (MINE) to probe larval zebrafish thermoregulatory circuits.
(A) Experimental design. Larval zebrafish expressing nuclear GCaMP6s in all neurons and mCherry in glutamatergic neurons are imaged under a two-photon microscope while random heat stimuli are …
Figure 6—figure supplement 1
Using model identification of neural encoding (MINE) to probe larval zebrafish thermoregulatory circuits.
(A) Across all imaged planes of all fish (orange lines), the autocorrelation of the provided stimulus. Black line is the average across all planes (). (B) Boxenplot of the inter-trial correlations …
Figure 7 with 1 supplement
Functional subdivisions of thermoregulatory circuits.
(A) Anatomical clustering of neurons encoding swim starts (left), swim vigor (middle), and swim direction (right). Neurons were clustered based on spatial proximity, and clusters with fewer than 10 …
Figure 7—figure supplement 1
Functional subdivisions of thermoregulatory circuits.
(A–C) For each analyzed Z-Brain region, the units encoding swim starts (A), vigor (B), or directionality (C) as a fraction of the sum across these three groups. Red bars highlight the regions with …
Tables
Table 1
The programmatic interface to model identification of neural encoding (MINE).
Details and example usage of the programmatic interface to MINE.
| class MineData | class Mine |
|---|---|
| correlations_trained (n_neurons x 1) | train_fraction (float) |
| Correlation of CNN prediction and activity on training portion of data | Which fraction of the data is used for training with remainder used for testing |
| correlations_test (n_neurons x 1) | model_history (integer) |
| Correlation of CNN prediction and activity on test portion of data | Number of timepoints the model receives as input |
| taylor_scores (n_neurons x n_components x 2) | corr_cut (float) |
| Taylor metric for each component (predictor and first -order interaction terms). The first entry along the last dimension is the mean score, the second entry is the bootstrap standard error. | If test correlation is less than this value for a neuron, it is considered ‘not fit.’ |
| model_lin_approx_scores (n_neurons x 1) | compute_taylor (bool) |
| Goodness of fit of linear Taylor model around the data mean to determine nonlinearity. | If true, compute Taylor metrics and complexity analysis (linear and second-order approximations). |
| mean_exp_scores (n_neurons x 1) | return_jacobians (bool) |
| Goodness of fit of second-order Taylor model around the data mean to derive complexity. | If true, return linear receptive fields. |
| jacobians (n_fit_neurons x (n_timepoints x n_predictors)) | taylor_look_ahead (integer) |
| For each fit neuron, the receptive field of each predictor across time. | The number of timepoints to predict ahead when calculating Taylor metrics. |
| hessians (n_fit_neurons x (n_timepoints x n_predictors) x (n_timepoints x n_predictors)) | taylor_pred_every (integer) |
| For each fit neuron, the matrix of second-order partial derivatives. Useful to extract second-order receptive fields. | Every how many frames a Taylor expansion should be performed to calculate Taylor metrics. |
| Additional settable properties: | |
| return_hessians (bool, default False) If true, return matrices of second-order derivatives model_weight_store (hdf5 file or group, default None) If set, trained model weights for all models will be organized and stored in the file/group n_epochs (integer, default 100) The number of training epochs. | |
| save_to_hdf5(file_object, overwrite = False) | analyze_data(pred_data: List, response_data: Matrix) ->MineData |
| Saves the result data to an hdf5 file or group | Takes a list of n_timepoints long predictors and a matrix of n_neurons x n_timepoints size and applies MINE iteratively to fit and characterize CNN relating all predictors to each individual neuron. |
| Example usage: | |
| predictors = [Stimulus, Behavior, State] | |
| responses = ca_data | |
| # NOTE: If predictors and response are not z-scored, | |
| # (mean = 0; standard deviation = 1) Mine will print | |
| # a warning | |
| miner = Mine (2/3, 50, 0.71, True, True, 25, 5) | |
| miner.model_weight_store = h5py.File(“m_weights.h5”, ‘a’) | |
| result_data = miner.analyze_data (predictors, responses) | |
| all_fit = result_data.correlations_test >= 0.71 | |
| is_nonlinear = result_data.model_lin_approx_scores <0.8 | |
| is_stim_driven = (result_data.taylor_scores[:, 0, 0] – 3 x result_data.taylor_scores[:, 0, 1]) >0 |
Additional files
-
Supplementary file 1
Z-Brain region abbreviations.
- https://cdn.elifesciences.org/articles/83289/elife-83289-supp1-v2.docx
-
MDAR checklist
- https://cdn.elifesciences.org/articles/83289/elife-83289-mdarchecklist1-v2.docx
