Abstract
Animals, including humans, rely on contextual information to interpret ambiguous stimuli. Impaired context processing is a hallmark of several neuropsychiatric disorders, including schizophrenia, autism, PTSD, and addiction. While sex differences in the prevalence and manifestations of these disorders are well established, potential sex differences in context processing remain uncertain. Here we examined sex differences in the contextual control over cue-evoked reward seeking and its neural correlates, in rats. Male and female rats were trained in a bidirectional occasion-setting preparation in which the validity of two reward-predictive cues was informed by the presence, or absence, of a background contextual feature (A:X+ / X-/ A:Y-/ Y+). Females were significantly slower to acquire contextual control over cue-evoked reward seeking. However, once established, the contextual control over behavior was more robust in female rats; it showed less within-session variability (less influence of prior reward) and greater resistance to acute stress. This superior contextual control achieved by females was accompanied by an increased activation of the orbitofrontal cortex compared to males. Critically, these behavioral and neural sex differences were specific to the contextual modulation process and not observed in simple, context-independent, reward prediction tasks. These results indicate a sex-biased trade-off between the speed of acquisition and the robustness of performance in the contextual modulation of cued reward seeking. The different distribution of sexes along the Fast learning ↔ Steady performance continuum might reflect different levels of engagement of the orbitofrontal cortex, and might have implications for our understanding of sex differences in psychiatric disorders.
eLife assessment
This valuable manuscript reveals sex differences in bi-conditioning Pavlovian learning and conditional behavior. Males learn hierarchical context-cue-outcome associations more quickly, but females show more stable and robust task performance. These sex differences are related to cellular activation in the orbitofrontal cortex. Although the evidence for the claims is solid, the claim of sex differences in context-dependent discrimination behaviour is not fully supported by the data. Nevertheless, the results will be of interest to many behavioural neuroscientists, particularly those who investigate sex-specific behaviours.
Introduction
Pavlovian learning is often conceived as the formation of a binary association between two events, where event #1 (the stimulus) evokes the anticipation of event #2 (the outcome). Under this strict definition, the background circumstances (the context) in which those events occur play little role. However, this simple form of learning is often augmented by higher level cognitive systems that allow animals (including humans) to learn and retrieve context-specific associations (e.g., the word “shot” might evoke different representations in a bar vs. a vaccine clinic) [1–3].
Deficits in contextual modulation are observed in several neuropsychiatric disorders including schizophrenia, autism, post-traumatic stress disorders (PTSD), and substance use disorders (SUD) [4–11], and may contribute to the intrusive thoughts and maladaptive (situation-inappropriate) behaviors that characterize these disorders. Importantly, the prevalence and symptomatology of these disorders vary between sexes. For example, the incidence of schizophrenia and autism is higher in men [12] while women are more at risk of developing SUD or PTSD following initial drug use or trauma [13, 14]. This disparity might be due, in part, to subtle sex differences in executive strategies [15–17]. Stress, a contributing factor in all aforementioned disorders, can further exacerbate these sex differences, resulting in sex-specific cognitive vulnerabilities [17–19]. Sex differences in executive functions and sensitivity to stress have been reproduced in animal models using behavioral assays of attention, impulsivity, and working memory [16, 20, 21]. Whether these sex differences extend to the contextual modulation of associative predictions remains unknown.
In animal models, the contextual modulation of associative predictions can be studied in occasion-setting preparations. In positive occasion-setting, animals learn that a target cue (X) results in a reward outcome (+) only when that cue is accompanied by a contextual feature (A); the same cue presented in absence of this contextual feature remains without consequence (A:X+ / X-). Conversely, in negative occasion-setting, a target cue results in reward, except when accompanied by a specific contextual feature (A:X-/ X+). In both preparations, the contextual feature is thought to hierarchically modulate (positively or negatively) the association between the target cue and the outcome [3, 22]. Strong evidence for this hierarchical modulation is the fact that a contextual feature can simultaneously function as a positive modulator for one target cue and as a negative modulator for another target cue (A:X+ / X-/ A:Y-/ Y+). In such bidirectional occasion-setting procedures all cues and contexts have equal probabilities of reward, therefore, accurate reward predictions cannot be achieved via simple associative summation but instead require hierarchical contextual modulation.
Compared to simple forms of associative learning, the neural bases of context-gated associative predictions (occasion-setting) remain severely understudied. Although recent studies have implicated the orbitofrontal cortex (OFC) in occasion-setting [23–25], most of the behavioral and neurobiological occasion-setting studies have exclusively used male subjects and/or did not explicitly address sex as a biological variable.
Therefore, the purpose of this study was two-fold: 1) investigate sex differences in the (bidirectional) contextual gating of reward prediction and its sensitivity to stress, and 2) investigate sex differences in OFC engagement during the contextual gating of reward predictions. We found that female rats were slower than males to acquire discriminated (context-gated) reward-seeking. However, once established, context-gated performance was more stable in females; it showed less variability within a session and was more resistant to acute stress. Moreover, we showed that this superior contextual control achieved by females was accompanied by an increased activation of the orbitofrontal cortex.
Results
Male and female rats were randomly assigned to one of four Pavlovian discrimination tasks outlined in Figure 1A. In these tasks [adapted from 26], all rats were exposed to two auditory cues (X and Y, 10s) presented within two distinct, alternating, visual contexts (A and noA, 2min). All rats received similar amount of reward, and produced a similar behavioral response (magazine approach). However, the different reward contingencies featured in these tasks presumably promoted the emergence of different associative architectures for reward prediction (Fig. 1B).
![](https://prod--epp.elifesciences.org/iiif/2/93509%2Fv1%2Fcontent%2F555217v2_fig1.tif/full/max/0/default.jpg)
Sex differences in the acquisition of context-gated but not context-independent reward predictions.
A Apparatus and general task design. During each session, the local context alternates every 2min between a Ctx.A (houselight on) and Ctx.noA (houselight off). Embedded in these context cycles, brief auditory stimuli (X or Y; 10s) are presented and potentially followed by reward (chocolate milk or banana pellets). B Reward contingencies and corresponding associative structures for each training condition. Ctx-dep. O1: the local context informs the predictive status of the ambiguous reward cues X and Y. Simple discrimination: reward cues are not ambiguous (only X is rewarded) and the local context is irrelevant. No Discrimination: reward cues X and Y are potentially rewarded and the local context is irrelevant. Ctx-dep. O1/O2: Context-dependent discrimination with differential outcomes; in this case, the local context is relevant but may contribute to context-sensitive performance via simple (non-hierarchical) associations (summation of cue- and context-evoked predictions). C Time in port (% cue time) during rewarded and non-rewarded trials across acquisition for male and female rats trained in each task. D Discrimination ratio across acquisition for male and female rats trained in each task. Error bars and error bands indicate ± s.e.m. *: P < 0.05, t-tests
In the Ctx-dep. O1 task, cue X was rewarded only in context A and cue Y was rewarded only in context noA; all rewarded trials resulted in the same outcome (O1). Accurate reward prediction in this task critically relies on the context-informed hierarchical modulation associative predictions (i.e occasion-setting). The Ctx-dep. O1/O2 task featured similar contingencies with the exception that the two rewarded trials (A:X and Y) resulted in two different outcomes (O1 and O2, respectively). This allowed for distinct context-outcome associations (Ctx.A→O1 and Ctx.noA→O2), as well as distinct cue-outcome associations (X→O1 and Y→O2). As a result, context-sensitive performance could potentially be achieved via simple (nonhierarchical) summation process. In the Simple discrimination task, cue X was always rewarded and cue Y was never rewarded, regardless of context. Finally, in the No discrimination task, both cues X and Y were probabilistically rewarded in both contexts. Importantly, the only task that unambiguously engages context-gated reward predictions (or contextual occasion setting) is the first task: context-dependent discrimination with a single outcome (ctx-dep. O1).
Male rats acquire context-regulation over reward-seeking faster than females
Fig.1 C-D displays the course of acquisition of discriminated Pavlovian approach in all four tasks. In the context-dependent discrimination with a single outcome (Ctx dep. O1), Linear Mixed Models (LMM) analysis revealed a significant Trial x Session interaction, (F(17, 181.488) = 73.343, P < 0.001), as rats gradually learned to respond more during rewarded than non-rewarded trials. Critically, the analysis also revealed a significant Sex x Trial x Session interaction (F(17, 181.488) = 4.683, P < 0.001). Post-hoc comparisons revealed that males and females displayed similar levels of responding to rewarded trials (Ps > 0.260), but females maintained higher responding during non-rewarded trials, causing both sexes to differ in this measure from session 40 onwards (Ps < 0.047).
To facilitate comparisons between groups, we calculated for each animal a discrimination ratio, defined as the time in port during rewarded trials divided by the total time in port during all trials (Fig 1D). With the exception of animals trained in the No Discrimination task (for which discrimination between rewarded and non-rewarded trials was impossible by design), all training groups demonstrated a gradual increase in discrimination ratio (main Session effect: No Discr. P = 0.477; all other groups Ps < 0.001). Importantly, only in Ctx-dep.O1 did we detect a main effect of Sex (F(1, 69.876) = 15.185, P < 0.001) and a Sex x Session interaction (F(17, 365.569) = 3.054, P < 0.001). Indeed, acquisition of discriminated performance was faster in males, causing significant sex differences from session 28 onwards (Ps < 0.014). No main effect or interaction with Sex was detected for any other training group (Ps > 0.305).
We next calculated the proportion of male and female rats that successfully acquired discriminated responding during the training period (Fig. 2). The criterion for successful discrimination was defined as a time in port during rewarded trials that exceeded the time in port during non-rewarded trials by at least 50% (Time rew ≥ 1.5* Time nonrew, which corresponds to a discrimination ratio ≥ 0.6) for a minimum of 4 out of 5 consecutive sessions. Rats in the Ctx-dep. discrimination groups had to reach this criterion for both cues to be considered “discriminators” (A:X ≥ 1.5* X and Y ≥ 1.5 A:Y). A significant sex difference in the proportion of discriminators was observed in the Ctx-dep. O1 group (χ2 = 5.35; P = 0.021), as a greater proportion of males reached criterion (Fig. 2 B-C). No sex differences were observed in the proportion of discriminators in any other tasks.
![](https://prod--epp.elifesciences.org/iiif/2/93509%2Fv1%2Fcontent%2F555217v2_fig2.tif/full/max/0/default.jpg)
Individual differences in the acquisition of discriminated Pavlovian approach.
A Time in port (% cue time) during the four different trial types (A:X / X / A:Y / Y) at the end of the training period (last 3 days average) for male and female rats trained in each discrimination task. B Discrimination ratio at the end of the training period (last 3 days average) for male and female rats trained in each discrimination task. Dark symbol: discriminator; Grey symbol: non-discriminator. Horizontal bar = group median; Dashed horizontal line represents the indifference level (no discrimination). C Proportion of male and female rats classified as discriminator or non-discriminator in each task. D Speed of acquisition (sessions to criterion) for discriminator rats, in each task. Horizontal bar = group median *: P < 0.05 Males vs. Females, t-tests; §: P < 0.05 chi-square test.
We then analyzed the number of sessions required for rats to reach the discrimination criterion (Fig. 2D). Rats that failed to reach the discrimination criterion were excluded from this analysis. A Kruskal-Wallis one-way ANOVA revealed a significant effect of the behavioral task (H (2) = 87.361, P < 0.001) with different sessions-to-criterion for all the discrimination tasks (Ctx-dep. O1 > Ctx-dep. O1/O2 > Simple; Ps < 0.001, Dunn’s tests). Sex differences in the speed of acquisition (i.e. sessions-to-criterion) were analyzed separately within each task. In Ctx-dep. O1, males were significantly faster to reach the discrimination criterion (T(38) = 2.502, P = 0.017). No sex differences in the speed of acquisition were observed in the other tasks (Ps ≥ 0.175).
These results indicate that female rats were generally slower to acquire contextual control over cue-evoked reward-seeking. This sex difference in the development of discriminated performance was observed only in Ctx-dep. O1, the only task that unambiguously engages contextual modulation of reward predictions. Note that estrous cycle did not affect performance in female discriminators (Fig. S1).
Trial history influences context-gated reward-seeking in male but not female rats
The delivery of reward, while critical for learning, can also interfere with the expression of context-dependent discrimination [27]. Therefore, we investigated to which extent within-session reward history (i.e. prior rewarded or non-rewarded trials) influenced performance in the Ctx-dep. O1 task, in male and female discriminators (Fig. 3). Performance was analyzed over 10 consecutive sessions (sessions 71-80; 800 trials total). The first trial of a session was excluded as it was not preceded by any trial. For the remaining 790 trials (79 trials x 10 sessions), we determined if the trial was preceded by a rewarded or non-rewarded trial. Responding during non-rewarded and rewarded trials was analyzed separately.
![](https://prod--epp.elifesciences.org/iiif/2/93509%2Fv1%2Fcontent%2F555217v2_fig3.tif/full/max/0/default.jpg)
Recent rewards disrupt contextual control of reward seeking in male, but not female, rats.
A Time in port (% cue time) during non-rewarded trials as a function of prior trial (rewarded: prior R, or non-rewarded: prior NR) for male and female rats trained in Ctx-dep. O1 task. Inset: increase in responding caused by prior reward for male and female rats (horizontal bar = median difference score). B Time in port during non-rewarded trials as a function of prior consecutive non-rewarded or rewarded trials. C Response probability on non-rewarded trials as a function of prior trial (rewarded: prior R, or non-rewarded: prior NR). Inset: increase in the probability of responding caused by prior reward for male and female rats (horizontal bar = median difference score). D Response probability on non-rewarded trials as a function of prior consecutive non-rewarded or rewarded trials. E Time in port (% cue time) during rewarded trials as a function of prior trial (rewarded: prior R, or non-rewarded: prior NR). Inset: difference in responding caused by prior reward for male and female rats. F Time in port during rewarded trials as a function of prior consecutive non-rewarded or rewarded trials. G Response probability on rewarded trials as a function of prior trial (rewarded: prior R, or non-rewarded: prior NR). Inset: difference in the probability of responding caused by prior reward for male and female rats. H Response probability on rewarded trials as a function of prior consecutive non-rewarded or rewarded trials. Error bars indicate ± s.e.m. * P < 0.05 Males vs. Females; # P < 0.05; ## < 0.01 Prior R vs. prior NR; t-tests
Fig. 3A shows the time in port during non-rewarded trials, as a function of the outcome of the previous trial (prior reward, or non-reward). A RM-ANOVA revealed a main effect of Prior Rewards (F(1, 38) = 27.144; P < 0.001) as prior rewards increased responding during subsequent non-rewarded trials. The analysis showed no main effect of Sex (F(1, 38) = 1.783; P = 0.190), but a marginal Sex x Prior Reward interaction (F(1, 38) = 4.015; P = 0.052), as males were slightly more influenced by prior rewards. The effect of trial history appears to be carried exclusively by the outcome of the previous trial (t-1) (no effect of consecutive prior rewards or non-rewards; Ps ≥ 0.147, Bonferroni t-tests; Fig. 3B).
We then analyzed another metric of conditioned responding: the response probability (i.e. probability subject enters the reward port during cue presentation). We reasoned that this metric might be more prone to detect brief lapses in discrimination performance (false alarms). Fig. 3C shows the probability of responding to a non-rewarded cue as a function of the outcome of the prior trial (prior reward or non-reward). A RM-ANOVA revealed a main effect of Prior Reward (F(1, 38) = 25.607; P < 0.001), and a significant Sex x Prior Reward interaction (F(1, 38) = 9.247; P = 0.004). In males, prior reward increased the probability of responding on subsequent non-rewarded trials (T(38) = 43.752; P < 0.001) while in females, trial history had no effect (T(38) = 1.631; P = 0.209). As a result, males and females differed in their probability of responding to non-rewarded trials when those trials were preceded by reward (T(38) = 4.508; P = 0.040). Here again we found no influence of the number of consecutive past reward or non-rewards (Ps ≥ 0.710, Fig. 3D).
Responding to rewarded cues remained high throughout the sessions and was not influenced by trial history (Ps ≥ 0.113); moreover no main effect or interaction with Sex was observed on these rewarded trials (Ps ≥ 0.127) (Fig. 3E-H). These results indicate that male rats have difficulty inhibiting responding to non-rewarded (contextually irrelevant) cues following a recent reward. In contrast, female rats maintained high discrimination performance throughout the session and recent rewards had little to no effect on their discrimination accuracy.
Acute stress disrupts context-gated reward-seeking in male but not female rats
We then tested the effect of an acute stress (90-min restrain) on discrimination performance (expressed as discrimination ratio), for all training groups (Fig. 4). With the exception of the No Discrimination group, acute stress reduced discrimination accuracy in all tasks (main Stress effect: Ps ≤ 0.004). However, only in the Ctx-dep. O1 task did we observe a significant Stress x Sex interaction F(1, 24) = 5.665; P = 0.026) as only males showed reduced discrimination performance following acute stress (Stress vs. Baseline; Males: T(14) = 5.165; P < 0.001; Females: T(10) = 1.290; P = 0.209). No main effect or interaction with Sex was observed in any other task (Ps ≥ 0.178). Therefore, although acute stress disrupted discrimination performance in all tasks, females trained in Ctx-dep. O1 appear to be protected from this disrupting effect of stress.
![](https://prod--epp.elifesciences.org/iiif/2/93509%2Fv1%2Fcontent%2F555217v2_fig4.tif/full/max/0/default.jpg)
Acute stress disrupts context-gated reward seeking in male but not female rats
A-B Time in port (% cue time) during the four different trial types (A:X / X / A:Y / Y) at baseline and after acute stress for male (A) and female (B) rats trained in each discrimination task. C Discrimination ratio at baseline and after acute stress, for male and female rats trained in each discrimination task. D Stress-induced disruption in discrimination performance (stress – baseline) for male and female rats trained in each discrimination task (horizontal bar = median difference score). Error bars indicate ± s.e.m; * P < 0.05; *** P < 0.001, t-tests
Female rats show higher orbitofrontal engagement during context-gated reward predictions
For a subset of animals, we quantified c-Fos-immunoreactive (cfos+) cells in the OFC (medial, ventral and lateral subregions) following regular behavior testing (no-stress condition). We also quantified cfos+ in two control regions, M1J and S1J, where we did not expect task-specific activation. Compared to baseline (homecage controls), behavioral testing increased cfos+ expression in all regions of interest, regardless of task (Fig. 5; homecage c-Fos+ counts are shown but not included in the following statistical analyses). A RM-ANOVA revealed a main effect of Region F(4, 192) = 85.037; P ≤ 0.001) and a significant Task x Sex x Region interaction (F(12, 192) = 1.9; P = 0.036). Follow-up planned contrast analysis compared males and females within each task and brain region. In the Ctx-dep. O1 task, females showed increased c-fos+ expression compared to males, in all OFC subregions (T(14) ≥ 2.387; P ≤ 0.032). No sex differences were found in any other task (Ps ≥ 0.255). Moreover, no sex differences were found in the control regions M1J and S1J, regardless of task (Ps ≥ 0.139).
![](https://prod--epp.elifesciences.org/iiif/2/93509%2Fv1%2Fcontent%2F555217v2_fig5.tif/full/max/0/default.jpg)
Female rats show higher orbitofrontal engagement during context-gated reward seeking.
A Experimental design. Brains were collected 2h after the start of the behavioral session. Brain sections were stained for c-Fos (activity marker) and NeuN (general neuronal marker). B Behavioral performance (discrimination ratio) ratio during the final session, before tissue collection for male and female rats trained in each task. C-G c-Fos+ density in the primary jaw motor cortex (C), primary jaw somatosensory cortex (D), medial orbitofrontal cortex (E), ventral orbitofrontal cortex (F), and lateral orbitofrontal cortex (G) for males and females trained in each discrimination task. Error bars indicate ± s.e.m. Gray bands indicate average c-fos+ density for animals who remained in their home cage prior to tissue collection (homecage controls; n=5). *: P < 0.05 Males vs. Females, t-tests.
Discussion
We investigated sex differences in contextual modulation of cue-evoked reward predictions and its neural correlates. Male and female rats were trained on a context-dependent discrimination task (Ctx-dep. O1) in which a contextual feature informs the validity of two reward cues (i.e. bidirectional occasion-setting). Overall, males acquired discriminated, context-gated behavior faster than females. However, once established, contextual control over behavior was more robust in females; it was more stable within a session (less influence of prior reward) and more resistant to acute stress. This superior contextual control achieved by females was accompanied by elevated c-Fos expression in all OFC subregions (medial, ventral and lateral). Critically, no sex differences (behavioral or neurobiological) were observed among animals trained on simpler discrimination tasks that do not engage contextual gating (Simple Discrimination, No discrimination, Ctx-dep. O1/O2).
For Ctx-dep. O1, context is an occasion-setter, i.e. a stimulus that hierarchically modulates the associative strength between a target cue and its outcome. However, our task differs from classic occasion-setting preparations in several ways. Historically, Pavlovian modulation has been studied in preparations in which the modulating stimuli (the occasion-setter) exerts unidirectional modulation, either strengthening (A:X+ / X-) or weakening (A:X-/ X+) a cue-outcome association. In such preparations, the occasion-setter might also establish a direct association (excitatory or inhibitory) with the outcome [1, 28, 29], thereby entangling the contributions of hierarchical modulation and direct stimuli-evoked predictions. To circumvent this issue, occasion-setting preparations commonly introduce a temporal gap (5-30s) between the termination of a discrete occasion-setting stimulus and its target cue. This approach has generally been successful in limiting direct associations between the occasion-setter and the outcome, instead promoting hierarchical modulation processes [30, 31, but see 32]. This temporal gap however introduces a working-memory requirement (animals must remember if the target cue was preceded by the occasion-setter) which complicates results interpretations. Here, we used a bidirectional occasion-setting preparation with a single reward outcome (Ctx-dep. O1) as an alternative approach to investigate contextual modulation [26]. Here, contextual feature (A) positively modulates one target association (A:X+ / X-) and negatively modulates another association (A:Y-/ Y+). Critically, there are no direct informative associations between contextual feature A and the single reward outcome (rewards are equally frequent in the presence or absence of A), so accurate discrimination necessitates hierarchical modulation [22, 33]. Moreover, the contextual feature precedes and overlaps with the target cue, thereby eliminating working-memory requirements. Thus, sex differences observed in Ctx-dep. O1, strongly indicate sex differences in hierarchical (context-informed) associative modulations. Alternatively, males and females might differ in terms of attentional resources. Indeed, unlike Simple Discrimination or No discrimination tasks, Ctx-dep. O1 requires high multimodal attention (animals must simultaneously attend to the visual context and auditory cues). However, no sex differences were observed in context-dependent discrimination with two outcomes (Ctx-dep. O1/O2), a task that shares similar attentional requirements with Ctx-dep. O1, but does not necessarily engage hierarchical contextual modulation (context potentially informing predictions via direct context-outcome associations). Together, these results indicate that the sex differences observed here are not attributable to simple associative, motivational, working-memory, or attentional processes, but are specific to the neurocomputational operations required for the hierarchical, contextual control of behavior.
Sex differences in context-dependent behaviors have previously been reported in fear conditioning preparations where females often display generalized fear responses that evade precise contextual control [34–37]. Elevated baseline anxiety and fear reactivity in females might contribute to this relative failure to regulate fear responses [38, 39, but see 40]. However, we show here that reduced contextual control in females (at least in the early stages of acquisition) is not limited to aversive behaviors but extends to appetitive, reward-seeking behaviors. This suggests a broader, domain-general, delay in the engagement of contextual, top-down regulation processes in female rodents. This is consistent with higher sign-tracking tendencies in female rats [41–45] —another form of appetitive behavior that reflects reduced top-down cognitive control (opposed to goal-tracking behavior) [46–48]. Note that the reduced top-down contextual control displayed by females in our study was limited to delayed acquisition. Provided sufficient training, females actually established more robust contextual control over cue-evoked reward seeking than males.
The biological basis for the delayed acquisition of contextual control in females is unknown. Although estrous cycle had no effect on the expression of context-dependent behavior in female discriminators, fluctuating sex hormones (and resulting neurobiological changes) could have contributed to the slower acquisition of contextual control in females. Indeed, learning the hierarchical contextual rules for reward prediction is a complex and lengthy process that probably requires distributed plasticity across several brain regions including the striatum, amygdala, hippocampus and neocortex [3, 49]. Estrous cycle affects synaptic physiology and plasticity at all these sites via changes in dopamine reuptake [50, 51], neuronal excitability [52–56], spine density [57, 58], gene expression [59, 60], or neurogenesis [61]. This dynamic neurobiological environment might constitute a challenge for females attempting to solve the Ctx-dep. O1 task, and might contribute to their delayed acquisition of contextual control. Interestingly, this dynamic neurobiological environment might also contribute to the more stable performance ultimately achieved by females. Indeed, in deep learning models, the injection of random “noise” in (artificial) neural networks during training is commonly used to build redundancy and increase network performance and stability [62, 63]. This ‘robustness through perturbations’ approach often comes at the expense of learning speed, similar to what we observed in females. This hypothesis of a role of cyclical sex hormones in the trade-off between learning speed and performance stability remains to be confirmed experimentally.
Regardless of the precise mechanism, our results indicate that, compared to male rats, females ultimately achieved more stable contextual control over cued reward-seeking; their behavior remained context-regulated under stress or after recent rewards. This behavioral sex difference was accompanied by neural sex differences. Compared to males, females displayed increased OFC activation during Ctx-dep. O1. Importantly, no behavioral or neural sex differences were observed for any other task. Together, these results strongly implicate the OFC in the sex-biased expression of contextual modulation of reward-seeking. This is consistent with recent studies demonstrating a critical role of the OFC in hierarchical control over cue-evoked behaviors [23–25]. More generally, the OFC is proposed to encode task structure including context-dependent contingencies [64– 66]. Accordingly, the OFC appears critical for decision-making when optimal choice requires knowledge of the broader task structure (i.e. when stimuli cannot be taken at face value) [67, 68]. Although causality remains to be established, our data suggest that the increased orbitofrontal engagement displayed by females might contribute to their resilient contextual regulation of behavior.
The contextual regulation of behavior is at the heart of cognitive control [69]. Deficits in this process are integral to the pathophysiology of numerous neuropsychiatric disorders [4]. Although extrapolation of animal data to humans requires caution, this study might offer insight into the biological sex differences observed in certain disorders. For instance, the initial delay in establishing contextual control observed here in females might contribute to the rapid progression of SUD or PTSD observed in women following initial drug use or trauma [13, 14, 70, 71]. Conversely, the more stable contextual control ultimately achieved by females after extended training observed here might contribute to the better prognosis for women following behavioral and cognitive treatments for these disorders [71–73], and might be due, in part, to increased OFC engagement [74].
Methods
Subjects
Subjects were male and female Long Evans rats (Charles Rivers), 8-10 weeks at arrival. Rats were housed in same-sex pairs in a temperature-controlled vivarium (22 °C) with a 12-hr light/dark cycle; behavioral experiments were conducted during the light cycle. Rats were given free access to water in their home cage for the duration of the experiments. After a week of acclimation to the colony room, rats were mildly food-restricted to maintain ∼95% of age-matched free-feeding weights. All experimental procedures were conducted in accordance with UCSB Institutional Animal Care and Use Committees and the US National Institute of Health guidelines.
Apparatus
Behavioral training was conducted in 12 identical conditioning chambers enclosed in individual sound- and light-attenuating cubicles (Med Associates, St. Albans, VT). Six chambers were dedicated to male subjects and six chambers were dedicated to female subjects; animals of both sexes were trained simultaneously. A recessed reward delivery port was located at the center of the front panel and was connected to a pellet dispenser and a syringe pump for liquid reward delivery (syringe pump located outside of the sound-attenuating cubicle). Subjects’ presence in the reward port was detected by the interruption of an infrared beam. Two ceiling-facing chamber lights located on the front and back internal walls of the cubicle provided diffuse chamber illumination and were used to manipulate the background visual context. Auditory stimuli were delivered via a white noise speaker and a clicker (76dB each), located in the back and front panel, respectively. A fan mounted on the cubicle provided ventilation and low background noise. A computer equipped with Med-PC software (Med Associates) controlled all experimental programs and data collection.
Rewards
Food rewards consisted of a 45mg banana-flavored food pellet (BioServe Dustless Precision Pellets, F0059 formula) or a 0.17mL bolus of sweetened chocolate milk (2 parts chocolate Nesquik® + 1 part water + 5% sucrose w/v) delivered over 3s. These rewards are highly discriminable but equicaloric and were equally preferred by rats in preliminary studies. The identity of the rewarding outcomes was counterbalanced within behavioral tasks and within sexes.
Discrimination Training
Rats were randomly assigned to one of four Pavlovian discrimination conditions: Context dependent discrimination with single outcome (Ctx-dep. O1, n = 30F + 32M), Context dependent discrimination with dual outcome (Ctx-dep. O1/O2, n = 14F + 14M), Simple discrimination (Simple, n = 12F + 22M), or No Discrimination (NoDiscr., n = 8F + 8M). Animals were individually placed in the conditioning chambers for 80-min training sessions 5-6 days/week. During each session, the visual background –the local context– alternated every two minutes between flashing houselights (0.25s on, 0.25s off; Context A) or darkness (Context noA). Within each context phase, two 10-s auditory cues, X or Y (white noise or clicker, counterbalanced), were presented individually, in two separate trials, and were potentially followed by reward. The presentation of each cue within a context phase was selected semi-randomly, so that the identity of the first cue had no incidence on the identity of the second cue (i.e. the same cue could repeat twice within a context phase, so the outcome of the first trial did not inform the outcome of the second trial). Each session consisted of 40 context cycles and 80 cue presentations (i.e. 80 trials) with an average intertrial of 50s ± 20s (rectangular distribution). The reward contingencies in each task are outlined in Figure 1.
Context dependent discrimination with single outcome (Ctx-dep. O1)
In this task, the reward-predictive validity of each cue was informed by the context, so that cue X was rewarded only in Ctx.A and cue Y was rewarded only in Ctx.noA. All rewarded trials resulted in the same reward outcome (O1). Considered individually, both cues and both contexts have equal probabilities of resulting in the same outcome, therefore accurate performance in this task cannot be understood in terms of summed associative strength between cues and contexts. Instead, accurate performance in this task critically relies on contextual modulation (i.e. occasion-setting). During the first 35 training sessions, rats experienced an equal number of rewarded and unrewarded trials (20 trials for each context/cue combination per session). On the 36th session and for the remainder of the experiment, the relative proportion of rewarded trials was reduced (5 A:X+; 15 X-; 15 A:Y-; 5 Y+) in an effort to promote discrimination.
Context dependent discrimination with dual outcome (Ctx-dep. O1/O2)
This task followed similar contingencies as previously described, with the exception that the two rewarded trial types (A:X and noA:Y) resulted in two different food outcomes (O1 and O2, respectively). This has profound implications for the learning processes engaged in the task. The delivery of different outcomes in different contexts allows for distinct context-outcome associations (Ctx.A→O1 and Ctx.noA→O2), as well as distinct cue-outcome associations (X→O1 and Y→O2). These different associative structures could potentially facilitate discrimination performance via simple summation process.
Simple Discrimination
Cue X was always rewarded and cue Y was never rewarded, regardless of the context. Presumably, this training protocol promotes accurate reward prediction via simple associative learning (context-independent predictions).
No Discrimination
Both cues X and Y were probabilistically rewarded in both context (the background context provides no disambiguating information). Presumably, this training protocol promotes reward prediction via simple associative learning (context-independent predictions) although the reward cues retain some ambiguity.
Assessment of Estrous Cycle
Estrous cycle phases were monitored via vaginal cytology for a subset of females (n = 10) over 15 consecutive days (sessions 66-80). Immediately after the behavioral session, the tip of a saline-moistened cotton swab was inserted into the vagina and rotated to dislodge cells from the vaginal wall. Swabs were immediately rolled onto a glass slide, and the samples were preserved with a spray fixative (M-FixTM, MilliporeSigma) without allowing the cells to dry. Slides were then stained with a modified Papanicolau staining procedure as follows: 50% ethyl alcohol, 3 min; tap water, 10 dips (×2); Gill’s hematoxylin 1, 6 min; tap water, 10 dips (×2); Scott’s water, 4 min; tap water, 10 dips (×2); 95% ethyl alcohol, 10 dips (×2); modified orange-greenish 6 (OG-6), 1 min; 95% ethyl alcohol, 10 dips; 95% ethyl alcohol 8 dips; 95% ethyl alcohol 6 dips; modified eosin azure 36 (EA-36), 20 min; 95% ethyl alcohol, 40 dips; 95% ethyl alcohol, 30 dips; 95% ethyl alcohol, 20 dips; 100% ethyl alcohol, 10 dips (×2); xylene, 10 dips (×2); coverslip immediately. Samples were observed with light microscopy (4x). Four criteria were used to determine the specific estrus cycle: 1. cell density, 2. percentage of nucleated epithelial cells, 3. cornified epithelial cells, and 4. Percentage of leukocytes [75–77]. Each sample was independently rated by two trained observers (S.P. and A.M.), blind to the behavioral data. Inter-rater agreement was high (88.2%). Only behavioral sessions for which both raters agreed on the estrous phase were selected for analysis. As a result, 2-4 behavioral sessions were considered for each subject at each phase of the estrous cycle. Behavioral data on these sessions were averaged to express a single data point for each rat at each stage of the estrous cycle. Females that did not show a regular cycle were excluded from the analysis (n = 2).
Acute Restraint Stress
Before testing the effect of acute stress on discrimination performance, the relative proportion of rewarded trials was adjusted in all groups to match the proportion of rewarded trials in the context-dependent discrimination with a single outcome (Ctx-dep. O1). As a result, rewarded cues became less frequent and represented only 1/4 of the trials. For all groups, these rewarded trials were equally distributed across both contexts. After 5-6 sessions of adjustment to these new conditions, we tested the effect of an acute stress on discrimination performance in all training groups. Acute stress was induced by restraining rats in standard Plexiglas rat restraint tubes for 90 min, in a novel brightly lit room. To account for the size differences between male and females, two different sized restraint tubes were used. After placing the rats in the tube, the length of the restrainer was adjusted to immobilize the rat without causing pain or interfering with animals’ breathing. After 90 min of restraint, the rats were returned to their homecage and left undisturbed for 5 min before being placed in the conditioning chambers for a normal behavioral session.
cFos Immunofluorescence
Following stress test, animal were retrained for 5-6 sessions, under standard conditions (no stress; equal number of rewards for all training groups). After their last behavioral session, animals remained in the conditioning chambers for 40 min (120 min total); they were then deeply anesthetized with a lethal dose of Euthasol and perfused transcardially with ice-cold 1x PBS followed by ice-cold 4% PFA in PB. Brains were extracted, post-fixed (4% PFA overnight) and cryoprotected (30% sucrose + 0.1% sodium azide, 1x PBS) until processed for immunostaining. Brains were frozen and sectioned into 40μm with a sliding cryostat (Leica CM1860). Free floating coronal sections were washed for 10 min in PBS, followed by a 2-h incubation in a blocking solution (5% NGS, 0.2% Triton X-100 in 1x PBS) at room temperature. Sections were then incubated with rabbit anti-cFos (1:1000, Cell Signaling, CAT#: 2250) and mouse anti-NeuN (neuron-specific protein; 1:2000 or 1:3000, Novus, CAT#: NBP1-92693) primary antibodies, in the blocking solution, for 20 h a 4°C. Following primary antibody incubation, sections were washed in PBS 3 times for 10 min and then incubated with anti-rabbit Alexa Fluor® 647-conjugated secondary antibody (far-red) (1:250, Jackson ImmunoResearch, CAT#: 111-605-144) and anti-mouse Alexa Fluor® 488-conjugated secondary antibody (green) (1:500, Jackson ImmunoResearch, CAT#: 115-545-003) for 90 min at room temperature. Sections were then washed in PBS three times for 10 min, mounted on slides and immediately coverslipped with a hardset mounting medium (Vector laboratories, CAT#: H-1700). Images were acquired at a 10x magnification, using a Keyence BZ-X fluorescence microscope.
Brain regions of interest for c-Fos positive neurons quantification included the medial, ventral, and lateral orbitofrontal cortex (mOFC, vOFC, and lOFC). Additionally, two control regions were included, for which we did not expect task-specific activation: the jaw primary motor (M1J) and somatosensory cortex (S1J). These regions of interests were manually isolated from the original images, using only the green / NeuN channel (the far-red / c-fos channel was temporarily disabled for blinding purposes). A custom ImageJ script (National Institute of Health) was then used to count c-Fos positive neurons in the selected regions and normalize this count to the surface area of the selected region (c-Fos positive neurons / mm2). For each brain region, c-Fos positive neurons were counted bilaterally from 2-4 sections per animal and a mean count was computed for each animal.
Statistical analysis
Data were collected over three replication cohorts, each composed of approximately equal number of male and female rats. Cue-evoked reward seeking was quantified primarily as the percentage of time a rat spent in the food cup during the last 5 seconds of cue presentation. Discrimination accuracy was expressed as a discrimination ratio, defined as the time in port during rewarded trials divided by the total time in port during all trials. A ratio of 0.5 indicates equal responding during rewarded and non-rewarded trials (i.e. no discrimination) whereas a ratio of 1 indicates responding exclusively during rewarded trials (i.e. perfect discrimination). Time in port and discrimination ratio during acquisition were aggregated in 4-session bins. These longitudinal repeated measures were analyzed with Linear Mixed-Models (LMM) with a Restricted Maximum Likelihood (REML) estimation method. All covariance structures were explored and the best fitting model was determined by the lower Akaike information criterion score [78, 79]. The selection of a covariance structure influenced the degrees of freedom of the model, possibly resulting in non-integer values. The remaining of the analyses consisted generally of mixed-models repeated measures (RM) ANOVAs with Sex and/or Task as a between-subject factor, and Trial history, Stress, or Brain region as within-subject factors. The Geisser-Greenhouse correction was applied when data lacked sphericity, possibly resulting in non-integer degrees of freedom. Post-hoc or planned contrast pairwise comparisons were carried with Bonferroni-corrected t-tests. When appropriate, nonparametric tests were conducted and consisted of the Kruskal-Wallis H test followed by post-hoc Bonferroni-corrected Dunn’s pairwise comparisons. All tests were two-tailed, significance was assessed against a type I error rate of 0.05. Statistical analyses were conducted with SPSS Statistics package (version 28.0.0.0; IBM SPSS). Graphs were generated using GraphPad Prism 9.
Funding
This work was supported by UC Santa Barbara Academic Senate Research Grant.
Competing Interests
The authors have nothing to disclose.
![](https://prod--epp.elifesciences.org/iiif/2/93509%2Fv1%2Fcontent%2F555217v2_figS1.tif/full/max/0/default.jpg)
Estrus cycle does not influence the expression of context-gated reward seeking
A. Vaginal cytology was used to track the stage of estrous cycle in a subset of female discriminators trained in the Ctx-dep. O1 task (n=8). B. Time in port during the four trial types (A:X / X / A:Y / Y) was not influenced by the estrous cycle (Estrous Stage F(3, 21) = 0.450; P = 0.720; Estrous Stage x Trial: F(2.258, 15.808) = 1.293; P = 0.305). C. Discrimination ratio was not influenced by the estrous cycle (F(3, 21) = 1.601; P = 0.219). Error bars indicate ± s.e.m.
References
- 1.Context-dependent learning and causal structurePsychon. Bull. Rev 24:557–565
- 2.Modulatory mechanisms in Pavlovian conditioningAnim Learn Behav 23:123–143
- 3.Occasion settingBehav. Neurosci 133:145–175
- 4.The contextual brain: implications for fear conditioning, extinction and psychopathologyNat. Rev. Neurosci 14:417–428
- 5.Context blindness in autism spectrum disorderFocus Autism Other Dev. Disabl 30:182–192
- 6.Contextual processing and its alterations in patients with addictive disordersAddiction Neuroscience 7
- 7.The schizophrenic experience: taken out of context? SchizophrBull 31:43–53
- 8.Schizophrenic deficits in the processing of contextA test of a theoretical model. Arch. Gen. Psychiatry 53:1105–1112
- 9.Context Processing and the Neurobiology of Post-Traumatic Stress DisorderNeuron 92:14–30
- 10.Autism: beyond “theory of mind”Cognition 50:115–132
- 11.Context-processing abilities in chronic cocaine usersPsychol Addict Behav 27:687–695
- 12.Male sex bias in early and late onset neurodevelopmental disorders: Shared aspects and differences in Autism Spectrum Disorder, Attention Deficit/hyperactivity Disorder, and SchizophreniaNeurosci. Biobehav. Rev 135
- 13.Sex/ Gender Differences in the Time-Course for the Development of Substance Use Disorder: A Focus on the Telescoping EffectPharmacol. Rev 75:217–249
- 14.Sex differences in trauma and posttraumatic stress disorder: A quantitative review of 25 years of researchPsychol. Bull 132:959–992
- 15.The extreme male brain theory of autismTrends Cogn. Sci. (Regul. Ed 6:248–254
- 16.Let’s call the whole thing off: evaluating gender and sex differences in executive functionNeuropsychopharmacology 44:86–96
- 17.Evolutionary perspective on sex differences in the expression of neurological diseasesProg. Neurobiol 176:33–53
- 18.Sex differences in neural stress responses and correlation with subjective stress and stress regulationNeurobiol. Stress 11
- 19.Sex-specific mechanisms for responding to stressJ. Neurosci. Res 95:75–82
- 20.Sex differences in animal models of decision makingJ. Neurosci. Res 95:260–269
- 21.Estrogen mediates sex differences in stress-induced prefrontal cortex dysfunctionMol. Psychiatry 9:531–538
- 22.US specificity of occasion setting: hierarchical or configural learning? BehavProcesses 90:311–322
- 23.Basolateral amygdala and orbitofrontal cortex, but not dorsal hippocampus, are necessary for the control of reward-seeking by occasion settersPsychopharmacology 240:623–635
- 24.Imbalanced activity in the orbitofrontal cortex and nucleus accumbens impairs behavioral inhibitionCurr. Biol 26:2834–2839
- 25.Selective Modulation of Orbitofrontal Network Activity during Negative Occasion SettingJ. Neurosci 37:9415–9423
- 26.Differential outcome effects in pavlovian biconditional and ambiguous occasion setting tasksJ Exp Psychol Anim Behav Process 36:471–481
- 27.Dissociating task acquisition from expression during learning reveals latent knowledgeNat. Commun 10
- 28.The functions of contexts in associative learningBehav. Processes 104:2–12
- 29.Modulation of a discrete Pavlovian conditioned reflex by a putative emotive Pavlovian conditioned stimulusJ Exp Psychol Anim Behav Process 17:299–311
- 30.Conditioning of simultaneous and serial feature-positive discriminationsAnim Learn Behav 9:293–303
- 31.Temporal control in Pavlovian occasion settingBehav. Processes 44:225–236
- 32.Do rats learn conditional independence? RSoc. Open Sci 4
- 33.Can existing associative principles explain occasion setting? Some old ideas and some new dataBehav. Processes 137:5–18
- 34.Sex differences in discriminating between cues predicting threat and safetyNeurobiol. Learn. Mem 133:196–203
- 35.Sex differences in fear regulation and reward-seeking behaviors in a fear-safety-reward discrimination taskBehav. Brain Res 368
- 36.Sex Differences in Context Fear Generalization and Recruitment of Hippocampus and Amygdala during RetrievalNeuropsychopharmacology 42:397–407
- 37.Decreased cued fear discrimination learning in female rats as a function of estrous phaseLearn. Mem 27:254–257
- 38.Sex effects in defensive behavior: baseline differences and drug interactionsNeurosci. Biobehav. Rev 15:461–468
- 39.Sex differences in learning processes of classical and operant conditioningPhysiol. Behav 97:229–238
- 40.Sex differences in anxiety-like behaviors in ratsPhysiol. Behav 211
- 41.Sex differences in nicotine-enhanced Pavlovian conditioned approach in ratsBiol. Sex Differ 10
- 42.Sex-dependent associations between addiction-related behaviors and the microbiome in outbred ratsEBioMedicine 55
- 43.Sex-dependent impact of early-life stress and adult immobilization in the attribution of incentive salience in ratsPLoS One 13
- 44.Individual variation in the propensity to attribute incentive salience to a food cue: influence of sexBehav. Brain Res 278:462–469
- 45.Incentive salience attribution, “sensation-seeking” and “novelty-seeking” are independent traits in a large sample of male and female heterogeneous stock ratsSci. Rep 9
- 46.The neuroscience of cognitive-motivational styles: Sign-and goal-trackers as animal modelsBehav. Neurosci 132:1–12
- 47.Neurobiological Basis of Individual Variation in Stimulus-Reward LearningCurr. Opin. Behav. Sci 13:178–185
- 48.Mechanisms of Shared Vulnerability to Post-traumatic Stress Disorder and Substance Use DisordersFront. Behav. Neurosci 14
- 49.Contextual behavior and neural circuitsFront. Neural Circuits 7
- 50.Sex and estrous cycle variations of rat striatal dopamine uptake sitesNeuroendocrinology 58:16–22
- 51.Dopaminergic dynamics underlying sex-specific cocaine rewardNat. Commun 8
- 52.Effects of the estrous cycle and ovarian hormones on cue-triggered motivation and intrinsic excitability of medium spiny neurons in the Nucleus Accumbens core of female ratsHorm. Behav 116
- 53.Estrus-Cycle Regulation of Cortical InhibitionCurr. Biol 29:605–615
- 54.Estrous Cycle Mediates Midbrain Neuron Excitability Altering Social Behavior upon StressJ. Neurosci 43:736–74
- 55.Hippocampal excitability increases during the estrous cycle in the rat: a potential role for brain-derived neurotrophic factorJ. Neurosci 23:11641–11652
- 56.Sex- and Estrus-Dependent Differences in Rat Basolateral AmygdalaJ. Neurosci 37:10567–10586
- 57.Naturally occurring fluctuation in dendritic spine density on adult hippocampal pyramidal neuronsJ. Neurosci 10:4035–4039
- 58.Estrous cycle impacts on dendritic spine plasticity in rat nucleus accumbens core and shell and caudate-putamenJ. Comp. Neurol 531:759–774
- 59.A functional cellular framework for sex and estrous cycle-dependent gene expression and behaviorCell 185:654–671
- 60.The estrous cycle surpasses sex differences in regulating the transcriptome in the rat medial prefrontal cortex and reveals an underlying role of early growth response 1Genome Biol 16
- 61.Sex hormones and adult hippocampal neurogenesis: Regulation, implications, and potential mechanismsFront Neuroendocrinol 41:129–152
- 62.Dropout: A Simple Way to Prevent Neural Networks from OverfittingJournal of Machine Learning Research
- 63.Dive into Deep LearningarXiv
- 64.The hippocampus and orbitofrontal cortex jointly represent task structure during memory-guided decision makingCell Rep 37
- 65.Orbitofrontal cortex as a cognitive map of task spaceNeuron 81:267–279
- 66.Orbitofrontal cortex encodes memories within value-based schemas and represents contexts that guide memory retrievalJ. Neurosci 35:8333–8344
- 67.Medial orbitofrontal cortex mediates outcome retrieval in partially observable task situationsNeuron 88:1268–1280
- 68.Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortexNat. Neurosci 14:1590–1597
- 69.An integrative theory of prefrontal cortex functionAnnu. Rev. Neurosci 24:167–202
- 70.Sex differences in neural mechanisms mediating reward and addictionNeuropsychopharmacology 44:166–183
- 71.Sex and gender differences in post-traumatic stress disorder: an updateEur. J. Psychotraumatol 8
- 72.Gender differences in alcohol and substance use relapseClin. Psychol. Rev 26:128–148
- 73.Sex differences in recovery from PTSD in male and female interpersonal assault survivorsBehav. Res. Ther 51:247–255
- 74.Integrative structural, functional, and transcriptomic analyses of sex-biased brain organization in humansProc. Natl. Acad. Sci. USA 117:18788–18798
- 75.Vaginal cytology of the laboratory rat and mouse: review and criteria for the staging of the estrous cycle using stained vaginal smearsToxicol. Pathol 43:776–793
- 76.The rodent estrous cycle: characterization of vaginal cytology and its utility in toxicological studiesBirth Defects Res B Dev Reprod Toxicol 80:84–97
- 77.Estrous cycle and ovarian changes in a rat mammary carcinogenesis model after irradiation, tamoxifen chemoprevention, and agingComp Med 53:532–538
- 78.Analysis of longitudinal data from animals with missing values using SPSSNat. Protoc 11:1112–1129
- 79.Analyzing longitudinal data with the linear mixed models procedure in SPSSEval. Health Prof 32:207–228
Article and author information
Version history
- Sent for peer review:
- Preprint posted:
- Reviewed Preprint version 1:
- Reviewed Preprint version 2:
- Reviewed Preprint version 3:
- Version of Record published:
Copyright
© 2023, Peterson et al.
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.
Metrics
- views
- 725
- downloads
- 30
- citations
- 0
Views, downloads and citations are aggregated across all versions of this paper published by eLife.