1 Introduction

Decision-making is often framed as the selection among discrete options. However, many real-world decisions are continuous in time and instead require determining how long to persist in an ongoing activity before disengaging. This class of decisions, when to stop, is central to adaptive behavior, shaping how organisms allocate time across competing pursuits. Failures of this process are implicated in neuropsychiatric conditions such as addiction, depression, and OCD (Addicott et al., 2017; Huys et al., 2015; Marton et al., 2019). Despite its importance, the neural mechanisms that govern the timing of disengagement from ongoing behavior remain poorly understood.

1.1 Ecological Models of Time Allocation

These types of continuous, temporal decisions are common within ecological foraging paradigms. Specifically, patch foraging presents animals with an environment where resources are clustered in patches. The utility of searching a patch declines over time as its resources are harvested, requiring the animal to decide when to abandon it in search of a new one. Many ethological studies have characterized various factors that affect patch occupancy time, such as patch richness (Keller and Sullivan, 2023; Marshall et al., 2013; Utsumi et al., 2009), inter-patch distance (Searle et al., 2006; Utsumi et al., 2009), predation risk (Brown, 1988; Eccard et al., 2020; Ward et al., 2000), and competition (Brown, 1988; Laguë et al., 2012). To understand how long an animal should continue searching a given patch, we use the normative principle of reward-rate maximization, which posits that the best strategy is the one that results in the highest overall rate of reward. For typical patch-foraging tasks, where the patch’s reward probability decays monotonically over time, the optimal policy is to leave a patch when the rate of reward inside the patch falls below the overall rate of reward achieved in the environment, as delineated by the Marginal Value Theorem (MVT) (Charnov, 1976). The MVT provides the mathematically optimal time to exit a patch, yet patch occupancy across organisms is consistently reported to be over-patient compared to optimal (Baum, 1983; Bettinger and Grote, 2016; Güldener et al., 2024; Hall-McMaster et al., 2021; Harhen and Bornstein, 2023; Kilpatrick et al., 2021; Lloyd et al., 2023; Marshall et al., 2013; Nonacs, 2001; Wajnberg et al., 2000).

1.2 Neural Substrates of Decision-Making and Timing

The basal ganglia play a key role in action selection and option evaluation during decision-making (Cisek, 2007; Mink, 1996; Redgrave et al., 1999). The primary input nucleus of the basal ganglia, the striatum, receives cortical projections from a wide variety of regions, integrating information about current goals and shifting environmental variables to guide behavior (Balleine et al., 2007; Ferenczi et al., 2016; Frank et al., 2004). Subregions of the striatum engage with different aspects of optimizing behavior, with the ventral striatum processing reward-related information (Berridge and Robinson, 2003; Salamone and Correa, 2012), the dorsolateral striatum mediating habit formation (Packard and Knowlton, 2002; Smith and Graybiel, 2013; Yin et al., 2004), and DMS engaged in goal-directed decision-making (Balleine et al., 2007; Yin et al., 2005). Lesions of DMS severely impair behavioral flexibility in animals, causing them to lose their ability to adapt to shifting action-outcome contingencies (Castañé et al., 2010; Pisa and Cyr, 1990; Ragozzino, 2007; Torres et al., 2016; Yin et al., 2005), while electrophysiological recordings show that DMS neurons dynamically encode decision variables, including action values, predicted outcomes, and choice signals (Burton et al., 2015; Ito and Doya, 2015; Kim et al., 2013; Lau and Glimcher, 2008; Parker et al., 2016; Thorn et al., 2010).

However, much of the decision-making research in DMS uses tasks in which subjects choose between discrete options, such as two-alternative forced choice and multi-armed bandit tasks (Balleine et al., 2007; Bogacz et al., 2006; Samejima et al., 2005; Stott and Redish, 2014; Tang et al., 2022). Patch foraging presents a different type of decision where the choice measured is a continuous variable. Rather than deciding what action to take, subjects must determine when to take action. The striatum has also been strongly implicated as a region central to timing, with lesions causing impairments in interval timing and temporal discrimination (Matell and Meck, 2004; Meck, 2006; Merchant et al., 2013). DMS neurons have been shown to encode temporal information through activity patterns such as ramping (Emmons et al., 2017; Ponzi and Wickens, 2022; Vandaele et al., 2021), and temporal receptive fields (Gouvea et al., 2015; Jin et al., 2009; Mello et al., 2015). The convergent role of the DMS in both reward-related decision-making and interval timing makes it well-suited for the temporal decisions required during patch foraging.

1.3 Study Overview and Objectives

Here, we investigate how DMS supports decisions about when to abandon a depleting pursuit using a patch-foraging task in freely moving mice. In this task, the optimal policy depends only on total time invested in the patch, independent of individual reward times. Contrary to this prediction, we find that mice adopt a reward-reset timing strategy, in which the decision to leave is governed primarily by the time elapsed since the most recent reward, with systematic modulation by patch residence time and environmental reward rate. This strategy reflects a temporally structured process in which each reward event resets an internal timing process that determines how long the animal is willing to continue waiting.

At the neural level, we identify a corresponding population code in DMS. Individual neurons undergo discrete, reward-triggered transitions between firing-rate states at characteristic delays, collectively tiling the post-reward interval. Across the population, the number of transitioned units increases over time following each reward, forming an accumulation signal that resets with each new reward. The rate of accumulation scales with environmental reward rate and invested patch residence time, and predicts the animal’s exit timing, reaching a threshold at the moment of patch departure.

These findings reveal a mechanism by which DMS encodes a dynamically rescaled signal encoding the time to leave, integrating invested patch residence time and environmental reward rate into a unified signal that governs when to stop. More broadly, they suggest that temporal decisions to disengage may be implemented through a time-cost-variable reward-triggered accumulation process, providing a bridge between interval timing and decision-making in naturalistic settings.

2 Results

2.1 Behavioral Characterization of Patch-Foraging Strategy

We began by characterizing time investment behavior in a patch-foraging task that required subjects to decide how to invest time in rewarding pursuits in their effort to maximize the overall reward rate reaped from an environment. Mice were placed in custom behavioral chambers with access to two nose-poke ports, the ‘time-investment port’ and the ‘context port’ (Figure la). The time-investment port served as the ‘patch’, where water rewards were randomly delivered from a lick spout with an exponentially decreasing probability over time. Mice invested time in the patch by maintaining their heads in the time-investment port, licking continuously as they sought to harvest rewards. As the likelihood of reward decreased with time, mice could choose to abandon the time-investment port at any moment in favor of the context port. The context port provided a fixed number of regularly spaced rewards over a fixed interval, after which a light cue turned on to indicate that the context port was depleted and the time-investment port had been replenished. As soon as the mouse returned to the time-investment port, the light cue turned off, signaling that the context port was replenished and available, posing the challenge of deciding how much time to invest in the time-investment port. Mice learned to alternate back and forth between the two ports throughout each behavioral session, investing a self-determined amount of time in the time-investment port.

Figure 1.

Sessions were broken into blocks where the context port switched between being ‘high’ value (four rewards in five seconds) and ‘low’ value (four rewards in ten seconds), while the reward probability distribution in the time-investment port remained constant (Figure la). By changing the reward rate and time spent in the context port, we altered the cost of time by manipulating the overall reward rate of the environment (Charnov, 1976; McNamara and Houston, 1985; Namboodiri et al., 2014; Sutlief et al., 2025). Changing the cost of time incentivizes reward-rate maximizing agents to leave the time-investment port earlier during high value blocks and later during low value blocks. Critically, this experimental design shifted the reward-rate optimizing patch-exit strategy solely by manipulating factors external to the patch (Figure lh). The central behavioral measure of interest was the amount of time invested in the time-investment port during each trial. We found that, on average, mice invested significantly more time in the time-investment port during ‘low’ context port blocks compared to ‘high’ context port blocks (Figure lb), progressively increasing their likelihood of departure as time passed in the time-investment port (Figure lc). While the direction of this contextdependent shift aligns with theoretical expectations (Charnov, 1976; McNamara and Houston, 1985; Sutlief et al., 2025) and prior observations (Constantino and Daw, 2015; Kilpatrick et al., 2021; Marshall et al., 2013), mice typically stayed in the patch longer than is reward-rate optimal, consistent with the over-patience commonly reported in patch-foraging tasks (Charnov, 1976; McNamara and Houston, 1985; Constantino and Daw, 2015; Hayden et al., 2011; Hutchinson et al., 2008; Lenow et al., 2017; Nonacs, 2001; Wilke et al., 2009). Across sessions, mice achieved an overall reward rate of 28.7 ± 0.2 rewards/min (mean ± SEM, n l34 sessions), with higher rates during high-context blocks (34.2 ± 0.3) than low-context blocks (23.3 ± 0.2; paired t-test, p l.l ×10⁻⁹¹). This corresponded to 88.2 and 93.l of the reward-rate-optimal policy for high and low blocks, respectively.

2.2 Time-cost-variable, Reward-Triggered Exit Time Policy

Since the probability of reward depends only on the time since entering the port, the optimal departure time is not affected by the occurrence and timing of rewards on any given trial. Nonetheless, the departure strategy exhibited by mice was very sensitive to the timing of rewards received in the time-investment port. While mice were extensively trained on the underlying reward probability distribution, the time at which the most recent reward was received relative to port entry was the strongest factor determining port exit (Figure lg), followed by patch residence time, and finally by environmental reward-rate context. To better understand how reward events shaped exit policy, we examined the hazard rate of leaving the patch during the interval following rewards, split by context (Figure le) and the time of the reward relative to port entry (Figure lf). We found that all three of these factors contributed to exit behavior, where each reward received would “reset” the amount of time the mouse would be willing to wait for a subsequent reward prior to departure, being determined by the patch occupancy time thus far and the environmental reward-rate context. Mice thus appeared to predominantly time from the last reward, though the time they were willing to wait following each reward decreased the later the reward was received and was shorter under the highthan under the low-reward-rate context. This strategy of timing from the last reward rather than patch entry, though suboptimal, is improved by the willingness to wait progressively shorter amounts of time the longer they stay in the patch.

2.3 DMS Neurons Undergo Firing Rate State Transitions Following Rewards

Given that behavior was organized around reward-triggered timing, and that this timing was sensitive to both patch dwell time and time’s cost, we asked whether DMS activity expresses a corresponding representation of the post-reward interval that similarly exhibits these sensitivities, and whether that representation predicts time investment decisions.

Many DMS units showed reward-locked dynamics that resembled ramping when averaged across rewards (Figure lc, bottom). However, inspection of single-trial responses revealed that these apparent ramps arose from discrete transitions between low and high firing-rate states following reward. This structure became evident when responses were aligned to each neuron’s inferred transition time rather than to the reward itself (Figure 2d). Figure 2a illustrates an example neuron exhibiting a rewardtriggered suppression followed by activation, a pattern that recurred across the population.

Figure 2.

We developed a classification procedure to identify step-like responses and estimate transition times on individual reward intervals (see Methods). Of l,849 total units, 898 met criteria for rewardtriggered state transitions (Figure 2e); validation with simulated flat-rate Poisson neurons confirmed a false positive rate of l.35% (Figure 2-figure supplement 3). Across these step-like units, mean transition times were distributed across the post-reward interval, effectively tiling the time axis (Figure 2f). Within units, transition times across rewards were approximately normally distributed, with variability increasing for later mean transition times (Figure 2-figure supplement l). Apart from this reward-triggered state-switching activity pattern, several other identifiable task-related activity patterns were observed across the population, with units exhibiting, in varying combinations, responses to a variety of different task aspects, primarily to movement between ports and to licking (Figure 2-figure supplement 2). Additionally, 320 units were characterized as non-responsive and did not show any identifiable task-related activity patterns.

2.4 Neural State Switches Correlate with Behavioral Exit Policy

We next examined whether variability in neural transition times tracked the animal’s departure policy. Transition times covaried with several task parameters, most prominently the timing of reward relative to patch entry and context, with larger parameter correlations among units with later mean transition times (Figure 3-figure supplement l).

Figure 3.

To relate neural dynamics to the mouse’s intended leave time following each reward, we derived a “projected exit time” for each reward event: an estimate of how long the mouse would have waited after that reward if a subsequent reward had not occurred (see Methods section 4.9). Although exit timing is variable when pooling across sessions (Figure lb), within-session policy is comparatively stable (Figure l-figure supplement la). We examined several related possible predictors from the reward dynamics in the time-investment port and found that the time of the last reward relative to port entry was the most correlated with leave time from the final reward for individual sessions (Figure l-figure supplement lg). We therefore fitted a session-specific regression model predicting leave time from the last reward as a function of context and the time of the last reward relative to port entry (Figure l-figure supplement 2), and used the fitted coefficients to project exit times on reward-reward intervals.

Neural transition times tracked these projected exit times. For the four example neurons in Figure 3, the probability of being in the high-firing state shifted later when the projected exit time was later and earlier when the projected exit time was earlier (Figure 3a). For each example neuron, transition time increased approximately linearly with projected exit time (Figure 3b). At the population level (n 898 step-like units), cumulative transition dynamics diverged most strongly when grouped by projected exit time: accumulation was steeper when projected exit time was earlier and shallower when projected exit time was later (all pairwise Kolmogorov-Smirnov tests significant, p ≤ 0.008; Figure 3c). This grouping produced greater separation than grouping by time since entry (Figure 3d) or by context alone (Figure 3e), consistent with projected exit time being the most behaviorally aligned regressor (Figure 3-figure supplement l).

2.5 Accumulation of Neural State Transitions Predicts Exit Time

Finally, we asked whether reward-reset accumulation of state transitions provides a quantitative, trial-by-trial predictor of patch exit. We analyzed 53 sessions with a minimum of seven simultaneously recorded step-like units (see Methods). Figure 4a shows representative trials from an example session with l9 step-like units. Following each reward, units transitioned progressively into their post-step states, producing an accumulation of transitioned units that reset at each reward.

Figure 4.

Across reward-to-exit intervals, accumulation dynamics scaled with exit timing: accumulation ramped faster when the mouse exited sooner and slower when the mouse exited later (Figure 4b). Notably, leading up to each exit, the accumulated count approached a similar level (l5 transitioned units), consistent with a threshold mechanism. For each reward-to-exit interval, we fit a line to the accumulation trajectory and found that the slope was significantly negatively correlated with exit time (Figure 4c), whereas the intercept was not (Figure 4d).

This pattern generalized across sessions. Accumulation slopes were significantly negatively correlated with exit time (mean r −0.55l, one-sample t-test vs. 0: p l.4 ×10⁻²⁶, 48 of 53 sessions individually significant; Figure 4e), while intercepts showed no significant correlation (mean r 0.0l3, p 0.73; Figure 4f), consistent with a variable accumulation rate from a stable starting point. Using a threshold-crossing rule applied to accumulation trajectories, we generated neurally predicted exit times that explained 39% of variance in observed exit times (R² 0.386, p 3.l ×10⁻²⁵⁸; Figure 4g). Applying the same analysis framework to inter-reward intervals (excluding reward-to-exit intervals) and comparing accumulation dynamics to behaviorally projected exit times yielded similar results (Figure 4h-j). Together, these results identify a reward-reset accumulation-to-threshold computation in DMS that integrates elapsed time, reward history, and environmental context to predict when the animal will abandon a depleting pursuit.

3 Discussion

Decisions about when to disengage from an ongoing activity are fundamental to adaptive behavior, yet the neural mechanisms that govern such temporal decisions remain poorly understood. Here, we show that during patch foraging, mice adopt a time-cost-variable, reward-reset timing policy, and that this policy is reflected in a corresponding neural computation in dorsomedial striatum (DMS). Specifically, we identify a population-level signal in which discrete, reward-triggered neural state transitions accumulate over time and reach a threshold at the moment of patch exit. The rate of this accumulation varies systematically with the cost of time and predicts exit timing on a trial-by-trial basis. Together, these findings show how DMS implements an accumulation-to-threshold computation that integrates elapsed time from reward and elapsed time from pursuit initiation with environmental reward rate to determine when to depart.

3.1 Neural Mechanisms of Temporal Decision-Making

At the individual neuron level, we observed step-like responses in DMS neurons, differing from the ramping or sparse temporal response dynamics typically associated with timing processes. Previous studies on interval timing in the striatum have predominantly reported firing rates that increase steadily over the course of the interval (Gouvea et al., 2015) or show sparse encoding of time fields (Mello et al., 2015). At the population level, our findings indicate that the striatum exhibits activity patterns that accord with the features of the behavioral strategy employed, where the intended interval to depart from a patch appears as timed from the last reward, though modified by the context in which the patch is embedded, and by the amount of time already spent in the patch. Specifically, the number of transitioned units resets and accumulates linearly after each reward, with the slope of this accumulation predicting the actual exit time from the patch. As with behavior, the time already spent in the patch as well as the patch context impact timing, affecting the rate of accumulation of transitioned units. When rewards occur earlier in the patch, the slope of accumulation is shallower, predicting greater time investment, whereas if rewards occur later in the patch, the slope of accumulation is steeper, predicting early departure times. When the time spent outside the patch in the context port is greater, the reward rate of the context port is lower, so that the cost of time is lower, resulting in a greater willingness to invest time in the patch. This is reflected in the correspondingly lower rates of accumulation of transitioned units in comparison to when the patch’s context is ‘high’. Collectively, these observations indicate a neural mechanism in which the DMS conveys the intended waiting interval after a reward, integrating elapsed time since last reward, elapsed time since patch entry, and the context of the patch.

3.2 Relationship to Accumulation-to-Bound Frameworks

The accumulation-to-threshold dynamics we observe in DMS share formal features with drift-diffusion and related accumulation-to-bound models, which have been widely applied to perceptual and valuebased decisions (Shadlen and Newsome, 1996; Gold and Shadlen, 2007; Brody and Hanks, 2016). In those frameworks, noisy evidence accumulates until a bound is reached, triggering a commitment to action. Here, the accumulated quantity is not sensory evidence but the cumulative count of discrete, reward-triggered neural state transitions representing the passage of time, with the bound corresponding to the threshold at which the animal departs the patch. This parallel suggests that accumulation-to-threshold computations may serve as a general motif across decision domains, extending from perceptual discrimination to self-timed decisions about behavioral persistence. Theoretical work has explored this connection deeply (Church and Deluty, 1977; Gibbon, 1977; Gibbon et al., 1984; Killeen and Fetterman, 1988; Matell and Meck, 2000; Namboodiri et al., 2014; Simen et al., 2011; Treisman, 1963), with varying models framing interval timing as an accumulation-to-threshold process that differ mainly in the form and generation of time’s representation and how it is read-out. Additionally, models such as TOPDDM (Simen et al., 2013) and TIMERR-DDM (Namboodiri et al., 2016; Namboodiri and Hussain Shuler, 2016), inspired by the Behavioral Theory of Timing (BeT) (Killeen and Fetterman, 1988), adapt their timing process to the temporal structure of reward within an environment. The linear accumulation to a singular threshold we observe here accords with TOPDDM, for whereas TOPDDM encodes interval timing by adjusting the rate of linear accumulation to a threshold, TIMERR-DDM encodes interval timing by adjusting the threshold of a non-linear accumulation process. Finally, the rate-scaling we observe, in which accumulation is faster under higher environmental reward rates and longer patch residence times, is also consistent with the proposal that the average reward rate impacts the cost of time, modulating the vigor of behavioral and neural processes (Niv et al., 2007).

Several prior findings in the dorsal striatum converge with our observations. Gouvea et al. (2015) showed that striatal population dynamics predict duration judgments, and Mello et al. (2015) identified a scalable population code for time in striatum. Emmons et al. (2017) reported ramping activity in DMS during interval timing and argued that frontostriatal temporal signals are consistent with drift-diffusion models. Regarding decision-making, Yartsev et al. (2018) demonstrated a causal role for anterior dorsal striatum in evidence accumulation, and Bolkan et al. (2022) showed that DMS pathway contributions to behavior depend on cognitive demand, with strong effects when tasks require evidence accumulation. Together, these studies establish that the dorsal striatum supports both temporal processing and accumulation-dependent decision-making. Our findings extend this work by identifying, in a naturalistic foraging context, a reward-reset accumulation signal in DMS for self-timed patch departure, implemented not as continuous rate changes but as the cumulative recruitment of neurons undergoing discrete post-reward state transitions.

3.3 Reward-Reset Strategy and Behavioral Optimality

In contrast to the behavioral predictions of classical optimal foraging theory, we observed the use of a reward-reset strategy, where the time spent in the port before a reward predicted how long the mouse would wait following that reward. According to the Marginal Value Theorem, the optimal patch departure time occurs when the instantaneous reward rate falls below the environmental average (Charnov, 1976; Kolling et al., 2012), where the timing of individual rewards in a fully stochastic environment should have no effect. However, mice consistently used the time since the most recent reward as their main decision variable, similar to the strategies previously observed in parasitic wasps (Haccou et al., 1991; van Alphen et al., 2003; Wajnberg et al., 2000). Other variables, such as the reward-rate context and patch occupancy time, had modest but significant effects on the time to exit following each reset, demonstrating sensitivity to the time-evolving value of the patch, and to the environmental reward-rate context.

The suboptimality we observed (mice staying in patches longer than predicted by MVT) is consistent with previous reports of “over-patience” in patch-foraging tasks (Constantino and Daw, 2015). This pattern may reflect some inherent cognitive constraints in temporal decision-making (Cuthill et al., 1990; Kacelnik and Krebs, 1985; Krebs et al., 1974) or could represent an adaptive response to environmental uncertainty (Kacelnik and Todd, 1992; Kacelnik and Bateson, 1996; Olsson and Brown, 2006), where a longer occupancy time allows for more information collection before a certainty threshold is crossed. The reward-reset strategy may be particularly advantageous in environments where patch quality varies unpredictably, since recent rewards would provide more reliable information about the patch’s current reward rate than a previously accumulated estimate of environmental statistics. While this is notably not the case for the task described in this study, the observation that mice use a reward-reset strategy nonetheless may suggest a generalized approach to similar foraging environments.

3.4 Conclusion

In summary, we identify a neural mechanism in which discrete, reward-triggered state transitions in DMS accumulate to a threshold that determines when to abandon a pursuit. This time-cost-variable, reward-reset accumulation provides a parsimonious account of how temporal information, reward history, and environmental context are integrated to guide decisions about when to stop. We suggest that DMS plays a central role in generating the temporal signals that underlie time-investment strategies during foraging, with downstream circuits likely converting this timing representation into the motor action to depart. Our findings are correlational, and causal manipulations will be needed to determine whether the accumulation dynamics we observe are necessary for patch departure decisions or instead reflect timing signals inherited from cortical or other inputs to DMS. Additionally, the generality of the reward-reset accumulation mechanism across different foraging environments and task structures remains to be tested. Nonetheless, by linking behavioral strategy to a specific population-level neural computation, these findings bridge interval timing and decision-making, and offer a framework for understanding how the brain controls persistence and disengagement in dynamic environments.

4 Methods

Key resources table

4.1 Compliance Statement

All animal procedures were conducted in accordance with the National Institutes of Health guidelines and were approved by the Johns Hopkins School of Medicine Institutional Animal Care and Use Committee (IACUC).

4.2 Subjects

Ten C57BL/6J mice (4 male, 6 female) were used in this study. Mice were purchased from The Jackson Laboratory and housed in standard vivarium cages. All animals had ad libitum access to food and water, except during periods of behavioral training and electrophysiological recording, when water was restricted. All animals maintained 85-90% of their original body weight throughout water restriction.

4.3 Task Design

Mice were trained on a patch-foraging task designed to specifically observe and measure the moment when a mouse decides to give up on a depleting patch (Figure la). The timing of that decision optimally depends on the relative expected value of remaining in the patch compared with the overall expected value of the environment. In our task, mice are required to harvest from a patch under low and high reward-rate environments while the underlying reward probability function within the patch remains consistent. Broadly, the task requires the mouse to switch back and forth between two ports as it harvests water rewards. The two ports are the time-investment port, which functions as a patch with reward delivered stochastically according to an exponentially decaying distribution, and the context port, which delivers four evenly spaced rewards over either a short or a long interval, depending on block.

The task is performed in a custom-built 7” x 5” box where the mouse has access to two ports into which it may poke its head. The ports are equipped with lick spouts that deliver l µL water rewards, infrared sensors that detect head entry and licks, and LED light indicators that signal whether or not the port is active (LED off = port is active/rewards are available, LED on = port inactive/rewards are not available). The mouse begins in the first of the two ports, the context port, where it receives 4 water rewards regularly spaced over either 5 or l0 seconds, depending on the current block. After the final reward is delivered, the context port light turns on and the time-investment port light turns off, indicating the mouse may then switch ports. Travel between the ports takes approximately 300 ms. Upon entering the time-investment port, the context port light turns off, signaling that the mouse may abandon the time-investment port in favor of the predictable context port rewards at any time. For as long as the mouse chooses to stay, the time-investment port delivers rewards probabilistically, with the likelihood of reward decreasing exponentially as time passes. The cumulative probability of the exponential decay curve sums to eight, meaning that on average, staying indefinitely in the timeinvestment port would yield eight rewards, while the actual number of rewards that can be delivered is unbounded. Eventually, the moment of experimental interest arrives, being when the mouse decides to exit the time-investment port and return to the context port. As soon as it reenters the context port, the time-investment port light turns on and a single cycle (trial) is complete. Each time the mouse consumes a set of four context port rewards, the time-investment port reward probability is refreshed to its starting value, allowing the mouse to continuously harvest by alternating between the ports.

Mice learned to operate the task without the need for any shaping steps. To consider a mouse fully trained, we required that they did not leave the context port prematurely on more than 20% of the trials and that they move to the time-investment port within 3 seconds of the light cue turning on in the context port. This ensured that they experienced the intended reward rate when deciding when to leave the time-investment port. Mice typically reached criterion within one to two months of daily training (two sessions per day). No requirements are placed on their strategy in the time-investment port.

Each behavior session lasted l8 minutes and was divided into six three-minute blocks. Across blocks, the dynamics of the context port alternated between providing a high reward-rate context (4 rewards in 5 seconds) and a low reward-rate context (4 rewards in l0 seconds). The value of the starting block (‘low’ or ‘high’) was randomly determined for each session. The reward probability distribution of the time-investment port remained consistent throughout all trials across all mice. All of the logic of the task, as well as control of the sensors, LEDs, and solenoids, was custom-written in Python and deployed on a Raspberry Pi. All time-stamped events were saved to a text file for later analysis.

4.4 Analysis - Support Vector Machine (SVM) Model

To understand the factors contributing to the decision of when to leave the time-investment port, a Support Vector Machine (SVM) classifier was used to identify the conditions under which a mouse would either stay or leave. Three factors were supplied to the SVM: The time since entry into the time-investment port, the time since the most recent reward in the time-investment port, and the reward-rate context (either low or high). Each trial from a given session was divided into l0 ms time bins and assigned to the ‘Stay’ group. The five seconds following each exit was similarly divided and assigned to the ‘Leave’ group. The SVM then used a linear kernel to calculate the plane that best separated the two groups. The coefficients of each normalized feature were then compared across sessions to determine the relative importance of the time from reward, the time from entry, and the reward-rate context to the patch-leave decision strategies used by mice across sessions.

4.5 Surgery for Neuropixel Implantation

Mice underwent stereotaxic surgery to implant Neuropixel 2.0 (multi-shank) probes using a custom 3D printed headcap and probe insertion assembly. All surgeries were performed in aseptic conditions and followed an IACUC approved protocol. Animals were anesthetized with isoflurane and their body temperature was maintained at 37^°C. Ophthalmic ointment was applied to the eyes and Lidocaine (0.002l mL/g) was injected subcutaneously under the scalp. Fur was removed from the scalp before it was disinfected with alternating Betadine and 70% isopropyl alcohol. A midline incision exposed the skull, with excess skin removed to expose a large surface area of skull to ensure the headcap could be securely attached. The periosteum was cleared with alternating 3% hydrogen peroxide and 70% isopropyl alcohol. The skull was then leveled and the craniotomy target (dorsomedial striatum, AP: 0.7 mm, ML: l.2 mm) was measured from bregma. The skull surface was thoroughly dried and a thin layer of Metabond dental cement was applied to the entire exposed surface, which was given l0 minutes to fully set. The craniotomy target was then marked on the Metabond, and an oval section (approximately l.2 mm × l mm) was drilled through both metabond and skull. A separate small hole was drilled for a ground pin posteriorly on the contralateral side. A small section of dura was carefully removed from the craniotomy. The custom headcap was then placed over the craniotomy, with an oval lip fitted inside to prevent skull regrowth where the Neuropixel probe would be inserted. A gold grounding pin was placed in the posterior hole. Metabond was used to secure both headcap and pin to the skull, ensuring the craniotomy and ground pin hole were fully enclosed. Buprenorphine (0.0083 mL/g) was then administered subcutaneously on the lower back, approximately 5 minutes prior to the end of the surgery. The Neuropixel 2.0 probe was then slowly inserted into DMS, using a built-in screw and shuttle system custom designed to interface with the headcap. The probe was lowered to a target depth of 5 mm at a rate of l mm/min. The ground pin was connected to the grounding pad of the Neuropixel probe via a soldered silver wire and the Neuropixel housing assembly was secured to the headcap with two external screws. The weight of the entire piece reached approximately 3 grams. Mice were supplied with hydration gel and monitored as they recovered in a heated cage until they were fully awake.

4.6 Electrophysiology

Neural activity was recorded from DMS of fully trained mice using Neuropixels 2.0 multi-shank probes. Each probe has four l0 mm shanks, with a total of 5l20 recording sites across them. The acquisition system used included a National Instruments PXIe-l07l chassis, with a PXIe-838l module. Neural signals were recorded with a sampling rate of 30 kHz across 384 channels using SpikeGLX software. Before recording, activity across the probe shanks was observed. The surface of the brain was located and used to estimate the band of recording sites most likely to cover the DMS, which the 384 channels were then assigned from which to record. Surface row information was saved to later validate histologically determined unit locations.

Probes were implanted using custom 3D printed fixtures for chronic, freely moving recordings, and the locations of all identified units were verified post hoc (Figure 2b). After recovery from surgical probe implantation, mice were retrained and DMS activity recorded for up to a month, depending on stability. Recordings were preprocessed using built-in methods from SpikeInterface (Buccino et al., 2020) and sorted using a consensus-based algorithm combining Kilosort 2.5 and Kilosort 3 (Pachitariu et al., 2016), all of which were orchestrated and managed with the DataJoint infrastructure (Yatsenko et al., 2021).

4.7 Histology

At the end of the experiment, mice were anesthetized with isoflurane and injected with sodium pentobarbital (200 mg/kg). Mice were then transcardially perfused with phosphate-buffered saline (PBS), then 4% paraformaldehyde (PFA). Brains were removed and placed in 4% PFA overnight, then transferred to PBS. Brains were sliced coronally at 50 µ m using a vibratome (Leica VTl000 S), then mounted on glass slides. Prior to surgery, the probes were coated in Vybrant DiI (V22885) and the tracks were then imaged using a fluorescence microscope (Keyence BZ-X800). Probe track locations were reconstructed within the Allen Mouse Brain Atlas using HERBS (https://github.com/Whitlock-Group/HERBS).

4.8 Neural Data Preprocessing and Spike Sorting

Recording sessions were processed using a cloud-based pipeline implemented with DataJoint (Yatsenko et al., 2021) and running on AWS cloud services. Code for the preprocessing and sorting steps is publicly available at https://github.com/dj-sciops/jhu_shulerab_element-array-ephys.

Preprocessing was conducted using the SpikeInterface toolbox. First, a bandpass filter was applied to the raw signals to remove frequencies below 300 Hz and above 6000 Hz. Broken channels were identified and interpolated over so they would not disrupt the spike sorting algorithm. Phase shift correction was applied to account for the phase delays between channels during the system’s acquisition cycles. Lastly, common average referencing (CAR) was applied across all channels to reduce common noise artifacts.

Spike sorting was performed independently with Kilosort 2.5 and Kilosort 3. Because the two algorithms complement each other in detecting different signal types, we used the SpikeInterface consensus method to identify units that were reliably detected by both sorters, ensuring that only high-confidence units were carried forward. Quality metrics were calculated for all consensus units, then assessed during manual curation performed using Phy, where remaining noise units were removed. The final set of curated units for each session was stored in the cloud pipeline and made available for analysis. Software versions for all preprocessing, spike sorting, and analysis pipelines are specified in the accompanying code repositories (see Data Availability Statement). Statistical analyses were performed in Python 3.l2 using SciPy l.l7.0, NumPy l.26.4, scikit-learn l.8.0, and statsmodels 0.l4.6.

4.9 Analysis - Behaviorally Projected Exit Time Calculation

To predict how long a given mouse on a given session would be willing to wait following each reward, we fit the leave time following each final reward (the dependent variable) as a function of when that reward occurred relative to port entry (the predictor), using simple linear regression.

Separate models were fit to trials from each context block, to allow for shifts in both the slope and intercept of the correlation. We then used the regression models to predict how long following every reward the mouse would have been willing to wait, if no further rewards had been delivered on that trial.

4.10 Analysis - Neural Step-like Classification

To characterize neurons exhibiting step-like responses at varying temporal delays from reward, an algorithm was developed to identify high and low firing rate states across trials and precisely detect the timing of transitions between states. To do so, first the instantaneous firing rate of individual neurons was calculated as one over the interspike interval for 10 ms time bins. For each unit, all of the post-reward responses in the time-investment port were pooled together and a sigmoid curve was fitted to the unsmoothed firing rate data. The parameters (y-offset, height, slope, and center) of this session sigmoid were then used as initial parameters for sigmoid fits to every interval individually. Parameter bounds were set to force the individual interval (reward-to-reward and reward-to-exit) sigmoid fits to match the high and low firing-rate states identified in the session sigmoid fit. The absolute value of the sigmoid slope was required to be greater than 11 (in units of delta(spikes/second)/second), preventing the sigmoid from fitting ramping activity with shallow curves. The center parameter (inflection point of the sigmoid) was bounded to the duration of the interval plus or minus 10%. The primary criterion for a successful fit was for the sigmoid to fit the center within the interval (at least l00 ms after the start and l00 ms before the end). On intervals where there was no clear step, the sigmoid’s best fit pushed the center to one of the far edges of the interval, essentially fitting the interval as entirely low firing or entirely high firing, due to the strict bounds on the other parameters. Because intervals could be shorter than a given unit’s typical transition latency, classification was restricted to intervals long enough for a transition to plausibly occur. Specifically, units were classified as step-like if the algorithm successfully identified a step on at least 50% of intervals whose duration exceeded the unit’s mean transition time plus one standard deviation of its transition time distribution. To validate this pipeline, matched homogeneous Poisson process neurons (flat-rate, no step structure) were generated for each recorded unit and processed through the identical classification procedure, yielding a false positive rate of l.35% (95% CI [0.85%, 2.04%]; Figure 2-figure supplement 3).

4.11 Analysis - Neural Accumulation

Sessions with at least seven simultaneously recorded step-like units were included in the accumulation analysis. This minimum was chosen to ensure sufficient population sampling for reliable linear regression of cumulative transition counts; results were qualitatively consistent across a range of inclusion thresholds. The cumulative number of units in their post-transition state was measured over the course of each trial. The slope and intercept of the accumulation across each reward-reward and reward-exit interval was calculated using simple linear regression. To predict the leave time from the neural activity, a single cumulative unit threshold per session was calculated as the median number of transitioned units at the time of each true exit within that session. This approach was selected after evaluating several threshold-setting methods; a single session-wide median threshold sufficiently captured the accumulation-to-threshold pattern observed in the data. The predicted leave time for each interval was then calculated as the intersection of its regression line with the session threshold.

Figure Supplements

Figure 1–figure supplement 1:

Figure 1–figure supplement 2:

Figure 1–figure supplement 3:

Figure 2–figure supplement 1:

Figure 2–figure supplement 2:

Figure 2–figure supplement 3:

Figure 3–figure supplement 1:

Data availability

All original data (including raw behavioral and neural recordings) and analysis code will be deposited and made publicly available on a DataJoint repository. The raw data, upon which all subsequent analyses were performed, will be available alongside the code required to reproduce the analyses and figures.

Acknowledgements

We thank Dr. Patricia Janak, Dr. Dan O’Connor, and Dr. Jeremiah Cohen for project guidance and valuable comments on the manuscript. We are also grateful to the DataJoint team for technical support. We also thank Rebekah Zhang, Dr. Simon Allard, and Dr. Tanya Martin, and the rest of the Hussain Shuler lab for their support.

Additional information

Author Contributions (CRediT)

Elissa Sutlief (First Author): Conceptualization, Investigation, Data Curation, Formal Analysis, Methodology, Project Administration, Software, Visualization, Writing - Original Draft, Writing - Review & Editing

Shichen Zhang (Author 2): Investigation, Methodology, Formal Analysis, Writing - Review & Editing

Kate Forsberg (Author 3): Investigation, Methodology

Marshall G Hussain Shuler (Senior Author / PI): Conceptualization, Supervision, Funding Acquisition, Project Administration, Writing - Review & Editing

Funding Sources

NIMH (5R0lMHl23446), NIA (RFlAG063783, R0lAG063783) to MGHS. NEI (T32EY007l43) to ES.

Funding

HHS | NIH | National Institute of Mental Health (NIMH) (R01MH123446)

• Marshall G Hussain Shuler

HHS | NIH | National Institute on Aging (NIA) (RF1AG063783)

• Marshall G Hussain Shuler

HHS | NIH | National Institute on Aging (NIA) (R01AG063783)

• Marshall G Hussain Shuler

HHS | NIH | National Eye Institute (NEI) (T32EY007143)

• Elissa Sutlief

Kavli Neuroscience Discovery Institute

• Marshall G Hussain Shuler