Mapping the Wake-Sleep algorithm onto cortical architecture.

Left: Network architecture. We model early sensory processing in the cortex with a multilayer network, r, receiving stimuli s. Center: individual pyramidal neurons receive top-down inputs (red) at the apical dendritic compartment, and bottom-up inputs at the basal dendritic compartment (blue). 5-HT2a receptors are expressed on the apical dendritic shaft (red bar), and on PV interneurons (red triangle); both sites may play a role in gating basal input. Right: Over the course of Wake-Sleep training, basal inputs dominate activity during the Wake phase (α = 0) and are used to train apical synapses, whereas apical inputs dominate activity during the Sleep phase (α = 1) and are used to train basal synapses.

Visualizing the effects of psychedelics in the model.

We model the effects of classical psychedelics by progressively increasing α from 0 to 1 in our model, where α = 1 is equivalent to the Sleep phase. We visualize the effects of psychedelics on the network representation by inspecting the stimulus layer s. a) Example stimulus-layer activity (rows) in response to an MNIST digit presentation as psychedelic dose increases (columns, left to right). b) Same as (a) but for ‘eyes-closed’ conditions where an entirely black image is presented. c-d) Same as (a-b), but for the CIFAR10 dataset.

Effects of psychedelics on single model neurons.

a) Correlations between the apical and basal dendritic compartments of either the same network neuron or between randomly selected neurons. b) Total plasticity for apical (left) and basal (right) synapses as α increases in the model when plasticity is either gated or not gated by α. Error bars indicate +/-1 s.e.m. c) Cosine similarity between plasticity induced under psychedelic conditions compared to baseline for apical (left) and basal (right) synapses.

Effects of psychedelics on neural variability.

a) Stimulus-conditioned variability for neurons in the network as α increases, as compared to variability in neural activity across stimuli (rightmost bar). Error bars indicate +/-1 s.e.m. b) Proportion correct for a classifier trained to detect the label of presented MNIST digits as α increases. c) Variability in the logit outputs of the trained classifier as α increases.

Network-level effects of psychedelics.

a) Pairwise correlation matrices computed for neurons in layer 2 across stimuli for α = 0 (left), α = 0.5 (center), and α = 1.0 (right). b) Correlation similarity metric between the pairwise correlation matrices of the network in the absence of hallucination (α = 0) as compared to hallucinating network states (α > 0). c) Proportion of explained variability as a function of principal component (PC) number for α ∈ { 0, 0.5, 1}. d) Ratio of across-stimulus variance in individual stimulus layer neurons when the apical dendrites have been inactivated, versus baseline conditions across different α values. e) Ratio of across-stimulus variance in individual neurons in the stimulus layer when neurons at the deepest network layer have been inactivated, versus baseline conditions across different α values. Error bars indicate +/-1 s.e.m.

Visualizing the effects of psychedelics in pretrained VDVAE models.

Decoded outputs of a pretrained VDVAE model trained on Tiny ImageNet (Top) and FFHQ-256 (Bottom) based on hallucinations generated in the top 35 layers of the model. Image samples vary along rows, and hallucination intensity, parameterized by α, increases along columns.

Summarizing testable predictions of the ‘oneirogen hypothesis’.

Models: OH - oneirogen hypothesis; EC - Ermentrout & Cowan [94]; REBUS - Relaxed Beliefs Under Psychedelics [69]; DD - DeepDream [95]. Key: ✓- model is consistent with the prediction; ✓-model is inconsistent with the prediction; n/a - model is neither inconsistent nor consistent with the prediction.

Visualizing the effects of psychedelics for alternative model architectures.

We model the effects of classical psychedelics by progressively increasing α from 0 to 1 in alternative model architectures. We visualize the effects of psychedelics on the network representation by inspecting the stimulus layer s. a) Example stimulus-layer activity (rows) in response to an MNIST digit presentation as psychedelic dose increases (columns, left to right) in the recurrent network model. b) Same as (a) but for our single compartment neuron model. c) Same as (a) using the multicompartment neuron model used for our main results, but for our noise-based hallucination hallucination protocol. d) Same as (c), but in a network in which neither the generative nor inference pathways have been trained beyond initialization.

Example generated images for different model architectures and datasets.

Generated images sampled from Eq. (1) with α = 1 for: a) Our primary multicompartment neuron model trained on MNIST, b) A multicompartment neuron model trained on CIFAR10, c) The recurrent network model, d) The single compartment neuron model.

Alignment between apical and basal dendritic compartments for different model architectures and datasets.

Apical-basal alignment for: a) An untrained multicompartment neuron model trained on MNIST, b) A single compartment neuron model, c) A recurrent network model, d) A multicompartment neuron model trained on CIFAR10.

Hallucination-induced synaptic plasticity for different neuron models.

a) Basal (top) and apical (bottom) plasticity as a function of α for a multicompartment neuron model trained on MNIST, using our noise-based hallucination protocol as a control. b) Same as (a) for a single compartment neuron model, using our primary hallucination protocol. c) Same as (b) for a recurrent network model, d) Same as (b) for a multicompartment neuron model trained on CIFAR10. Error bars indicate +/-1 s.e.m.

Neural variability changes for different neuron models.

a) Stimulus-conditioned variability (top), classifier accuracy (middle), and classifier output variability (bottom) as a function of α for a multicompartment neuron model trained on MNIST, using our noise-based hallucination protocol as a control. b) Same as (b) for a single compartment neuron model, using our primary hallucination protocol. c) Same as (b) for a recurrent network model, d) Same as (b) for a multicompartment neuron model trained on CIFAR10. Error bars indicate +/-1 s.e.m.

Network-level effects of psychedelics for different network architectures and training datasets.

For each network architecture, we examine: correlation similarity as a function of α (top row), the proportion explained variance across stimuli as a function of principal component number (second row), the ratio of across-stimulus variance in stimulus layer neurons when apical dendrites have been inactivated compared to baseline conditions across different α values (third row), and the ratio of across-stimulus variance in stimulus layer neurons when the deepest network layer has been inactivated across different α values (fourth row). a) Results for an untrained multicompartment neuron. b) Results for a multicompartment neuron model trained on MNIST, using our noise-based hallucination protocol. c) Results for a single compartment neuron model. d) Results for a recurrent network model. e) Results for a multicompartment neuron model trained on CIFAR10. Error bars indicate +/-1 s.e.m.

Visualizing the eyes-closed effects of psychedelics in pretrained VDVAE models.

Decoded outputs of a pretrained VDVAE model trained on Tiny ImageNet (Top) and FFHQ-256 (Bottom) based on hallucinations generated in the top 35 layers of the model. Black input images were used for samples in all rows. Hallucination intensity, parameterized by α, increases along columns.

Analyzing the image- and network-level effects of psychedelics in a Tiny ImageNet-pretrained VDVAE model.

a) Laplacian pyramid features for a grayscale example input image from the Tiny ImageNet dataset (top left). Pyramid levels increase along columns, corresponding to decreasing resolution, and hallucination intensity increases along rows. b) Correlation similarity across pyramid levels between the base image and a hallucinated image across different α values, averaged over 100 image samples. c) Stimulus-conditioned variance of units in layer 30 (descending from the top of the network) across different α values, averaged over 100 sample images and 32 distinct trials. d) Correlation similarity calculated between correlation matrices for units in layer 30 across different α values, averaged over spatial positions and 100 sample images. e) Ratio of across-stimulus variance in individual units of layer 30 when the highest 20 layers of the network have been inactivated, versus baseline conditions across different α values. Error bars indicate +/-1 s.e.m.

MNIST multicompartment network hyperparameters.

CIFAR10 multicompartment network hyperparameters.

Recurrent network hyperparameters.