Predictive processing circuit model variants postulating a testable motif of interactions between neurons of deep and superficial cortical layers.

(A) Two classes of neurons postulated by predictive processing. Top: A comparator circuit. Prediction error neurons compute the difference between the teaching signal and prediction. Positive prediction errors (PE+) are computed as Teaching signal minus Prediction (T – P), negative prediction errors (PE-) are computed as Prediction minus Teaching signal (P – T). Here and in the subsequent panels, arrows represent excitatory connections and blunt arrows inhibitory connections (inhibitory interneurons are omitted for simplicity – see (Attinger et al., 2017a; Widmer et al., 2022) for a full circuit description). Bottom: Internal representation circuit. Internal representation neurons (R) integrate prediction errors and shape the prediction (or teaching) signals sent to other areas. (B) Schematic of the hierarchical implementation of predictive processing. (C) Schematic of the non-hierarchical implementation of predictive processing. (D) The distinct predictions made by hierarchical and non-hierarchical implementations on the functional influence of deep layer neurons on superficial layer neurons.

Functional identification of positive and negative prediction error neurons in layer 2/3 of V1.

(A) AAV vector injections were used to express ChrimsonR-tdTomato in different Cre-positive deep layer neurons (Figures 3-5), and GCaMP6f in layer 2/3 neurons of V1. The ChrimsonR virus was omitted in no-ChrimsonR control mice. (B) Schematic of the two-photon microscope and a virtual reality system. (C) Schematic of the visuomotor stimuli used to probe for prediction error responses. Grating onsets were used to identify positive prediction error neurons and visuomotor mismatch stimuli were used to identify negative prediction error neurons. (D) Top: Mean layer 2/3 population response to grating onset. Here and in the subsequent panels, the solid line represents the hierarchical bootstrap estimates of the mean trace. Shading around the mean is the bootstrap error, defined as one standard deviation of the bootstrap distribution at each time bin. Bottom: Heatmap of mean responses of all layer 2/3 neurons. Neurons are sorted by the amplitude of their grating onset response, dashed gray lines mark the sorting window. (E) As in D, but for visuomotor mismatch. Purple shading marks the duration of the mismatch stimulus. (F) Mean response of positive (light green) and negative (dark green) prediction error neurons to grating onset. Here and in the subsequent panels, response curves are compared for each time bin: in the horizontal bars above the plot, black marks time bins where p<0.05 and gray mark time bins where p>0.05. Colored bars to the left indicate which lines are compared. (G) Mean response of positive (light green) and negative (dark green) prediction error neurons to visuomotor mismatch. (H) Scatter plot of the correlation between neuronal activity and optical flow speed during grating sessions against the correlation between neuronal activity and locomotion speed for all layer 2/3 neurons. Dot color indicates the amplitude of the response to visuomotor mismatch.

Opposing functional influence of Tlx3 layer 5 neurons on positive and negative prediction error neurons of layer 2/3.

(A) We optogenetically activated Tlx3 layer 5 neurons while recording calcium activity in layer 2/3 neurons. (B) Mean layer 2/3 population responses to optogenetic stimulation of Tlx3 layer 5 neurons (dark gray), control light stimulation in the same mice (yellow), and stimulation in no-ChrimsonR control mice (light gray). Red shading marks the stimulation interval. (C) Heatmap of mean layer 2/3 neuronal responses to optogenetic stimulation of Tlx3 layer 5 neurons (left) and control light stimulation (right). Neurons are sorted by the amplitude of their response to the optogenetic stimulation. (D) Mean responses of positive (light green) and negative (dark green) prediction error neurons in layer 2/3 to optogenetic stimulation of Tlx3 layer 5 neurons. Red shading marks the optogenetic stimulation interval. (E) Box plots showing average responses of positive prediction error neurons (light green), negative prediction error neurons (dark green), and all layer 2/3 neurons (black) to optogenetic stimulation of Tlx3 layer 5 neurons. Boxes indicate the interquartile range, the central line marks the median, and whiskers the 10th and 90th percentiles. Orange circles mark the mean response. Here and elsewhere, n.s.: not significant, *: p<0.05, **: p<0.01, ***: p<0.001. (F) Scatter plot of the correlation between neuronal activity and optical flow speed against the correlation between neuronal activity and locomotion speed during grating sessions for all layer 2/3 neurons. Dot color indicates the amplitude of the response to optogenetic stimulation of Tlx3 layer 5 neurons.

Opposing functional influence of Fezf2 layer 5 neurons on positive and negative prediction error neurons of layer 2/3.

(A) We optogenetically activated Fezf2 layer 5 neurons while recording calcium activity in layer 2/3 neurons. (B) Mean layer 2/3 population responses to optogenetic stimulation of Fezf2 layer 5 neurons (dark gray), control light stimulation in the same mice (yellow), and stimulation in no-ChrimsonR control mice (light gray). Red shading marks the stimulation interval. (C) Heatmap of mean layer 2/3 neuronal responses to optogenetic stimulation of Fezf2 layer 5 neurons (left) and control light stimulation (right). Neurons are sorted by the amplitude of their response to the optogenetic stimulation. (D) Mean responses of positive (light green) and negative (dark green) prediction error neurons in layer 2/3 to optogenetic stimulation of Fezf2 layer 5 neurons. Red shading marks the optogenetic stimulation interval. (E) Box plots showing average responses of positive prediction error neurons (light green), negative prediction error neurons (dark green), and all layer 2/3 neurons (black) to optogenetic stimulation of Fezf2 layer 5 neurons. Boxes indicate the interquartile range, the central line marks the median, and whiskers the 10th and 90th percentiles. Orange circles mark the mean response. (F) Scatter plot of the correlation between neuronal activity and optical flow speed against the correlation between neuronal activity and locomotion speed during grating sessions for all layer 2/3 neurons. Dot color indicates the amplitude of the response to optogenetic stimulation of Fezf2 layer 5 neurons.

Functional influence of Ntsr1 layer 6 neurons on positive and negative prediction error neurons of layer 2/3.

(A) We optogenetically activated Ntsr1 layer 6 neurons while recording calcium activity in layer 2/3 neurons. (B) Mean layer 2/3 population responses to optogenetic stimulation of Ntsr1 layer 6 neurons (dark gray), control light stimulation in the same mice (yellow), and stimulation in no-ChrimsonR control mice (light gray). Red shading marks the stimulation interval. (C) Heatmap of mean layer 2/3 neuronal responses to optogenetic stimulation of Ntsr1 layer 6 neurons (left) and control light stimulation (right). Neurons are sorted by the amplitude of their response to the optogenetic stimulation. (D) Mean responses of positive (light green) and negative (dark green) prediction error neurons in layer 2/3 to optogenetic stimulation of Ntsr1 layer 6 neurons. Red shading marks the optogenetic stimulation interval. (E) Box plots showing average responses of positive prediction error neurons (light green), negative prediction error neurons (dark green), and all layer 2/3 neurons (black) to optogenetic stimulation of Ntsr1 layer 6 neurons. Boxes indicate the interquartile range, the central line marks the median, and whiskers the 10th and 90th percentiles. Orange circles mark the mean response. (F) Scatter plot of the correlation between neuronal activity and optical flow speed against the correlation between neuronal activity and locomotion speed during grating sessions for all layer 2/3 neurons. Dot color indicates the amplitude of the response to optogenetic stimulation of Ntsr1 layer 6 neurons.

Opposing functional influence of positive and negative prediction error neurons on Tlx3 layer 5 IT neurons.

(A) Schematic of the distinct experimental predictions made by hierarchical and non-hierarchical predictive processing implementations on the functional influence of superficial layer neurons on deep layer neurons. (B) Two artificial regulatory elements (either AP.Baz1a.1 or AP.Adamts2.1) were used in AAV vectors to bias expression of soma-targeted (ST) ChrimsonR to either positive (Rrad) or negative (Adamts2) prediction error neurons respectively. AAV vectors were injected in V1 of Tlx3-Cre x Ai148 mice. (C) We optogenetically activated Rrad neurons while recording calcium activity in Tlx3 layer 5 IT neurons. (D) Left: Mean layer 5 IT populational response to optogenetic stimulation of Rrad neurons (red) and control light stimulation in the same mice (yellow). Red shading marks the stimulation interval. Right: Heatmap of mean layer 5 IT neuronal responses to optogenetic stimulation of Rrad neurons. Neurons are sorted by the amplitude of their response to the optogenetic stimulation. (E) We optogenetically activated Adamts2 layer 2/3 neurons while recording calcium activity in Tlx3 layer 5 IT neurons. (F) As in D, but for stimulation of Adamts2 neurons. (G) Comparison of the responses of Tlx3 layer 5 IT neurons to optogenetic stimulation of Rrad or Adamts2 neurons. Boxes indicate the interquartile range, the central line the median, and whiskers the 10th and 90th percentiles. Orange circles mark the mean response.

Layer 2/3 responds to manipulations of layer 5 activity as if layer 5 were a teaching signal for layer 2/3.

(A) Mice first experienced a visuomotor closed loop condition, that was then followed by an optogenetic closed loop. In optogenetic closed loop, mice were exposed to constant visual illumination on the screen while the strength of optogenetic stimulation of Tlx3 layer 5 neurons was coupled to locomotion speed of the mouse instead. Following this, mice were exposed to an optogenetic open loop condition, in which the pattern of optogenetic stimulation the mouse had self-generated in the preceding optogenetic closed loop session was replayed. (B) Mean layer 2/3 population response to optomotor mismatch and optogenetic open loop stimulation halt. Pink shading indicates the duration of the stimulus. (C) Mean layer 2/3 population response to visuomotor mismatch. Purple shading indicates the duration of the visuomotor mismatch stimulus. (D) We binned layer 2/3 neurons by their visuomotor mismatch responses into 5 bins and plotted the mean optomotor mismatch responses of the neurons in these bins. Error bars indicate SEM.

Functional influence of Scnn1a layer 4 neurons on positive and negative prediction error neurons of layer 2/3 is unspecific.

(A) We optogenetically activated Scnn1a layer 4 neurons while recording calcium activity in layer 2/3 neurons. (B) Mean layer 2/3 population responses to optogenetic stimulation of Scnn1a layer 4 neurons (dark gray), control light stimulation in the same mice (yellow), and stimulation in no-ChrimsonR control mice (light gray). Red shading marks the stimulation interval. (C) Heatmap of mean layer 2/3 neuronal responses to optogenetic stimulation of Scnn1a layer 4 neurons (left) and control light stimulation (right). Neurons are sorted by the amplitude of their response to the optogenetic stimulation. (D) Mean responses of positive (light green) and negative (dark green) prediction error neurons in layer 2/3 to optogenetic stimulation of Scnn1a layer 4 neurons. Red shading marks the optogenetic stimulation interval. (E) Box plots showing average responses of positive prediction error neurons (light green), negative prediction error neurons (dark green), and all layer 2/3 neurons (black) to optogenetic stimulation of Scnn1a layer 4 neurons. Boxes indicate the interquartile range, the central line marks the median, and whiskers the 10th and 90th percentiles. Orange circles mark the mean response. (F) Scatter plot of the correlation between neuronal activity and optical flow speed against the correlation between neuronal activity and locomotion speed during grating sessions for all layer 2/3 neurons. Dot color indicates the amplitude of the response to optogenetic stimulation of Scnn1a layer 4 neurons.

How a JEPA architecture could be mapped onto the cortical circuit.

(A) Schematic of a JEPA architecture for self-supervised learning (adapted from Assran et al., 2023). The output of one encoder A is trained to predict the output of the other encoder B. The prediction is compared to the target output in a comparator network D. (B) A proposal for a JEPA implementation in cortical circuits. Superficial and deep cortical layers function as the two separate encoder networks postulated by JEPA. Prediction error neurons in layer 2/3 compare the local layer 5 activity to a prediction of that activity.

Locomotion onset responses of layer 2/3 neurons differ between closed loop and open loop conditions.

(A) Mean layer 2/3 population response to locomotion onset in closed loop (solid line) and open loop (dashed line) conditions. Here and in the subsequent panels, response curves represent the hierarchical bootstrap estimates of the mean trace. Shading around the line is the bootstrap error, defined as one standard deviation of the bootstrap distribution at each time bin. Response curves are compared for each time bin: in the horizontal bars above the plot, black marks time bins where p<0.05 and gray mark time bins where p>0.05. (B) Mean response of positive (left, light green) and negative (right, dark green) prediction error neurons to locomotion onset in closed loop (solid lines) and open loop (dashed lines) conditions.

Responses of layer 2/3 neurons to control light stimulation do not correlate with optogenetic stimulation responses and resemble responses in no-ChrimsonR control mice.

(A) Scatterplot of the responses of layer 2/3 neurons to optogenetic stimulation of Tlx3 layer 5 neurons against their responses to control light stimulation. Each dot represents a neuron. (B) Comparison of average responses of layer 2/3 neurons to control light stimulation (yellow) and stimulation light in no-ChrimsonR control mice (light gray). Boxes indicate the interquartile range, the central line marks the median, and whiskers the 10th and 90th percentiles. Orange circles mark the mean response.

Functionally specific modulation by Tlx3 layer 5 neurons is robust to functional group size and is absent for control light stimulation.

(A) We optogenetically activated Tlx3 layer 5 neurons while recording calcium activity in layer 2/3 neurons. (B) As in Figure 3D, but using the 10% (instead of 15%) most strongly responding neurons to grating onset and visuomotor mismatch. (C) As in Figure 3D, but using the 20% (instead of 15%) most strongly responding neurons to grating onset and visuomotor mismatch. (D) Mean responses of positive prediction error neurons (light green), negative prediction error neurons (dark green), and the layer 2/3 population (black) to control light stimulation. Yellow shading marks the control stimulation interval.

No-ChrimsonR control stimulation does not induce functionally specific modulation of layer 2/3 neurons.

(A) Mean responses of positive prediction error neurons (light green), negative prediction error neurons (dark green), and the layer 2/3 population (black) to stimulation light in no-ChrimsonR control mice. Red shading marks the stimulation interval.

Stimulation of layer 5 likely separates positive and negative error neurons irrespective of feature-selectivity of those neurons.

(A) Mean layer 2/3 population response to preferred grating (pink) and orthogonal grating (black) onsets. (B) Mean responses of positive prediction error neurons (light green) and negative prediction error neurons (dark green) to optogenetic stimulation of Tlx3 layer 5 neurons. Positive prediction error neurons were selected using a single grating orientation, either 135° (left) or 45° (right). Red shading marks the stimulation interval. (C) Schematic of the inverted visuomotor coupling in the virtual reality system. Locomotion of the mouse on the treadmill is coupled to forward moving visual flow. We define inverted visuomotor mismatch stimulus as a halt in the inverted visual flow during locomotion of the mouse. (D) Inverted visuomotor mismatch drives a population of neurons different from those activated by normal visuomotor mismatch. Left: quantification of measurement noise in normal visuomotor mismatch responses. Scatter plot of visuomotor mismatch response on even trials versus the same response on odd trials. Each dot is a neuron. The black line marks the diagonal. Right: Responses of layer 2/3 neurons to normal visuomotor mismatch versus their responses to inverted visuomotor mismatch. The correlation between the two is significantly lower than would be expected from measurement noise. Inset: Bootstrap distribution of odd versus even trial correlations of normal visuomotor mismatch responses. Vertical red line marks the observed correlation between normal and inverted visuomotor mismatch responses. (E) Mean response of inverted visuomotor mismatch neurons to grating onset. Like normal visuomotor mismatch neurons (Figure 2F), inverted visuomotor mismatch neurons decrease their activity on visual stimulation. (F) Mean responses of positive prediction error neurons (light green) and negative prediction error neurons (dark green) to optogenetic stimulation of Tlx3 layer 5 neurons. Negative prediction error neurons are selected using inverted visuomotor mismatch stimulus. Red shading marks the stimulation interval.

Functionally specific modulation by Fezf2 layer 5 neurons is robust to functional group size and is absent for control light stimulation.

(A) We optogenetically activated Fezf2 layer 5 neurons while recording calcium activity in layer 2/3 neurons. (B) As in Figure 4D, but using the 10% (instead of 15%) most strongly responding neurons to grating onset and visuomotor mismatch. (C) As in Figure 4D, but using the 20% (instead of 15%) most strongly responding neurons to grating onset and visuomotor mismatch. (D) Mean responses of positive prediction error neurons (light green), negative prediction error neurons (dark green), and the layer 2/3 population (black) to control light stimulation. Yellow shading marks the control stimulation interval.

Functionally specific modulation by Ntsr1 layer 6 neurons is robust to functional group size and is absent for control light stimulation.

(A) We optogenetically activated Ntsr1 layer 6 neurons while recording calcium activity in layer 2/3 neurons. (B) As in Figure 5D, but using the 10% (instead of 15%) most strongly responding neurons to grating onset and visuomotor mismatch. (C) As in Figure 5D, but using the 20% (instead of 15%) most strongly responding neurons to grating onset and visuomotor mismatch. (D) Mean responses of positive prediction error neurons (light green), negative prediction error neurons (dark green), and the layer 2/3 population (black) to control light stimulation. Yellow shading marks the control stimulation interval.

Locomotion unmasks inhibitory influence of layer 5 IT neurons, but not layer 4 neurons, on layer 2/3 neurons.

(A) Mean layer 2/3 population response to optogenetic stimulation of Tlx3 layer 5 neurons split by whether the mouse is stationary (pink) or locomoting (blue). (B) Scatterplot of layer 2/3 neuronal responses to stimulation of Tlx3 layer 5 neurons while mice were locomoting versus their responses while mice were stationary. Each dot is a neuron. For the analysis shown in C, we define neurons below and to the left of the negative diagonal as inhibited, and those to the right and above as excited. (C) Left: Mean response of inhibited layer 2/3 neurons (as defined in B). Right: Mean response of excited layer 2/3 neurons. (D-F) As in A-C, but for responses of Tlx3 layer 5 neurons to optogenetic stimulation of the same population. (G-I) As in A-C, but for responses of layer 2/3 neurons to optogenetic stimulation of Scnn1a layer 4 neurons.

Neither optogenetic stimulation of Scnn1a layer 4 neurons nor control light stimulation results in functionally specific modulation in layer 2/3.

(A) We optogenetically activated Scnn1a layer 4 neurons while recording calcium activity in layer 2/3 neurons. (B) As in Figure 8D, but using the 10% (instead of 15%) most strongly responding neurons to grating onset and visuomotor mismatch. (C) As in Figure 8D, but using the 20% (instead of 15%) most strongly responding neurons to grating onset and visuomotor mismatch. (D) Mean responses of positive prediction error neurons (light green), negative prediction error neurons (dark green), and the layer 2/3 population (black) to control light stimulation. Yellow shading marks the control stimulation interval.

Sub-circuits for reciprocal predictive processing in a single cortical area.

(A) Schematic of the first sub-circuit. The action (A) input functions as a prediction and the sensory input (S) as a teaching signal. The internal representation neuron represents an estimate of the sensory input. (B) Schematic of the second sub-circuit. The action (A) input functions as a teaching signal, and the sensory input (S) as a prediction. The internal representation neuron represents an estimate of the action.

Statistics

All information on statistical tests used in the manuscript are shown in Table S1. We used hierarchical bootstrap or a correlation coefficient for all comparisons.

Number of mice per experiment