Stochasticity in the miR9/Hes1 oscillatory network can account for clonal heterogeneity in the timing of differentiation
Abstract
Recent studies suggest that cells make stochastic choices with respect to differentiation or division. However, the molecular mechanism underlying such stochasticity is unknown. We previously proposed that the timing of vertebrate neuronal differentiation is regulated by molecular oscillations of a transcriptional repressor, HES1, tuned by a posttranscriptional repressor, miR9. Here, we computationally model the effects of intrinsic noise on the Hes1/miR9 oscillator as a consequence of low molecular numbers of interacting species, determined experimentally. We report that increased stochasticity spreads the timing of differentiation in a population, such that initially equivalent cells differentiate over a period of time. Surprisingly, inherent stochasticity also increases the robustness of the progenitor state and lessens the impact of unequal, random distribution of molecules at cell division on the temporal spread of differentiation at the population level. This advantageous use of biological noise contrasts with the view that noise needs to be counteracted.
https://doi.org/10.7554/eLife.16118.001Introduction
The nervous system is a complex organ system with many different cell types. These are generated by actively dividing progenitor cells in a sequential order during embryogenesis, while progenitor cells themselves change their character along the way (Götz and Huttner, 2005). In cases where histogenesis of neural tissues has been examined, such as the mammalian cortex and the vertebrate retina, a temporal order has been found such that certain cell types are generated early while others are generated late (Franco et al., 2012; Bertrand et al., 2002; Boije et al., 2014; Greig et al., 2013; Franco and Müller, 2013; Goetz et al., 2014; Shen et al., 2006; Gaspard et al., 2008). In order to understand neural development it is essential to understand how the timing of differentiation is regulated.
The predominant view is that the timing of differentiation is precise and tightly controlled. Indeed, given that the final developed nervous system is consistently structurally organised, this is an intuitive assumption. Therefore, it was surprising that recent experiments based on singlecell lineage mapping in the zebrafish retina have shown that the timing of differentiation can be a fundamentally stochastic process, although the collective behaviour of many progenitors forms a reproducible retina (He et al., 2012). A similar conclusion was reached in a study where rat retinal progenitor cells were cultured at singlecell density and constant extracellular environment (Gomes et al., 2011). In addition to the retina, lineage tracing has also demonstrated that progenitors produce a temporally ordered but variable number of neurons within the mouse neocortex (Shen et al., 2006; Gaspard et al., 2008) although other studies reported a high degree of determinism in this system (Gao et al., 2014). Nevertheless, inherent stochasticity may be a widespread phenomenon in differentiation.
Previous models aiming to explain this phenomenon, such as the branching models of population dynamics (Rué and Martinez Arias, 2015; Klein and Simons, 2011), take into account 3 types of division: symmetric proliferative, asymmetric and symmetric differentiative. The probabilities of division type are either fixed (e.g. in adult tissue homeostasis; reviewed in Simons and Clevers (Simons and Clevers, 2011)) or change with time (e.g. in developing tissues (Slater et al., 2009)), but are usually fitted to the data (He et al., 2012; Gomes et al., 2011). Such models are useful in understanding population dynamics but lack a molecular mechanistic explanation.
At a molecular level, the balance between differentiation and the maintenance of progenitors in the developing nervous system is regulated by transcriptional repressors such as the Hes gene family, including Hes1 and Hes5 (Kageyama et al., 2007, 2008). When Hes genes are absent, neural progenitors prematurely differentiate and cause a wide range of defects in brain formation (Hatakeyama et al., 2004; Nakamura et al., 2000). Conversely, overexpression of Hes genes leads to inhibition of neurogenesis and overmaintenance of neural progenitors (Ishibashi et al., 1995).
The development of livecell imaging with unstable Luciferase (LUC) reporters has shown that the dynamics of Hes1 gene expression changes during neural development (Imayoshi et al., 2013). Hes1 is expressed in an oscillatory manner with a period of around 2 hr in neural progenitors (Imayoshi et al., 2013; Shimojo et al., 2008) but is expressed at a low steady state in differentiated neurons (Imayoshi et al., 2013; Sasai et al., 1992). Based on expression dynamics and the functional studies mentioned above, it has been proposed that a Hes1 oscillatory state is necessary for the maintenance of progenitors, while low, nonoscillatory levels are associated with a transition to neuronal differentiation (Kageyama et al., 2008). The most direct evidence for the functional importance of oscillatory dynamics in general, comes from optogenetics studies of the Hes1 targets Ascl1 (Imayoshi et al., 2013) and Dll1 (Shimojo et al., 2016). It was shown that lightinduced oscillatory expression of Ascl1 increased the proportion of dividing cells in Ascl1null neural progenitors, whereas sustained Ascl1 expression increased the efficiency of neuronal differentiation (Imayoshi et al., 2013). It has similarly been shown that lightinduced sustained expression of the Delta ligand Dll1 leads to higher levels of the cell cycle inhibitor p21 than oscillatory expression (Shimojo et al., 2016). Together, these suggested that the expression dynamics of Ascl1 and Dll1 encode information for a choice between proliferation and differentiation within neural progenitors. The hypothesis that gene expression dynamics change as cells make cellstate transitions in development is consistent with previous studies in theoretical biology (Furusawa and Kaneko, 2012; Huang, 2011; GarciaOjalvo and Martinez Arias, 2012; Rué and Martinez Arias, 2015). In these studies, the interactions of multiple genes in regulatory networks can lead to the emergence of transient stem cell dynamics, which evolve to an attracting stable configuration of gene expression corresponding to distinct cell types.
Experimental and theoretical work has shown that Hes1 oscillates in neural progenitors possibly due to a combination of delayed negative selfrepression and relatively fast degradation of Hes1 mRNA and HES1 protein, previously measured in fibroblasts (Jensen et al., 2003; Monk, 2003; Hirata et al., 2002; Momiji and Monk, 2008). Until recently it was not understood how oscillations of Hes1 could be terminated and the timing of differentiation controlled. Recent experimental results have shown that Hes1 is a primary target of the microRNA miR9, and HES1 also periodically represses the transcription of miR9, thus forming a double negative feedback loop (Bonev et al., 2012). However, mature miR9 is very stable and accumulates over time in a gradual manner. It has been proposed that accumulating levels of miR9 beyond a certain level can cause oscillations of Hes1 to cease, leading to differentiation (Bonev et al., 2012; Tan et al., 2012). Experimentally it has been shown that Hes1 is a target of miR9, that depleting miR9 prevents or delays differentiation and that Hes1 changes dynamics of expression as cells differentiate (Bonev et al., 2012; Bonev et al., 2011; Imayoshi et al., 2013; Tan et al., 2012; Coolen et al., 2012). However, a theoretical approach unifying these phenomena was lacking.
The Hes1/miR9 interaction has been mathematically modelled using deterministic delay differential equations (Goodfellow et al., 2014). This showed that interactions within the Hes1/miR9 network can lead to the emergence of bistability (alternative cell fates) and Hopf bifurcations (separating stable states from an oscillatory regime). Computationally it was shown that indeed, accumulation of miR9 can lead to a transition from oscillatory to stable expression of HES1 protein and that the timing of this transition is controlled by the parameters and the initial conditions of the model. Although this mathematical model provided a useful starting point, a deterministic system is not able to explain the stochastic nature of the differentiation process that has been reported in vivo.
Here, we show that high celltocell variability exists in the abundance of Hes1 mRNA and protein in undifferentiated neural cells (progenitor and stem cells) as evidenced by quantitative singlemolecule Fluorescent in Situ Hybridisation (smFISH) and Fluorescence Correlation Spectroscopy (FCS), respectively. Furthermore, the copy number of Hes1 mRNA, protein and miR9 per cell is low, (determined by smFISH, FCS and stem loop qRTPCR, respectively) supporting the inclusion of 'finite number' intrinsic stochasticity in the computational model of the Hes1/miR9 interaction. Oscillations produced by a stochastic model match singlecell imaging data well and lead to systematic changes in the dynamics and output of this network. In the deterministic model, equivalent cells (i.e clonal or isogenic) that start from the same initial conditions terminate oscillations i.e. 'differentiate' at exactly the same time. However, in the stochastic framework the timetodifferentiation of such cells becomes distributed, with the spread of the distribution inversely related to the absolute number of interacting molecules. Due to the ability of intrinsic noise to induce oscillatory dynamics, Hes1 oscillations are maintained for a greater range of parameters than in the deterministic system, which can be interpreted as increased robustness of the progenitor state in the stochastic model. Computationally, the average time of differentiation can be shifted by varying the molecular copy number or by varying the initial miR9 levels. This supports the idea that the number of interacting molecules within a cell is an important parameter that can be tuned to control the onset of differentiation. We validate predictions of our stochastic model by measuring the time distribution of differentiation when miR9 is added to the system. Finally, we show that the stochastic network is able to better 'absorb' noise attributable to unequal distribution of some molecular components at division, maintaining its output as if the divisions were symmetric and thus giving a robust outcome at a population level.
Together, our work demonstrates that randomness originating in the finite copy number within transcriptional programmes provides a theoretical framework to understand the heterogeneity observed in singlecell time series of neural stem (NS) cells and its consequences for neural development. Developing systems may exploit stochasticity in gene regulatory networks to stagger differentiation time, cope with noise introduced by unequal inheritance of components and safeguard the progenitor state.
Results
Developing a stochastic model for the Hes1/miR9 oscillator
Previous analysis focussed on a model of the Hes1/miR9 network using delay differential equations (Goodfellow et al., 2014), which are themselves an extension of earlier models by Jensen et al. (2003) and Monk (2003); Momiji and Monk (2008), as outlined in the Materials and methods.
A deterministic system relies on differential equations, making an assumption that variables such as mRNA and protein can take a continuous range. This is very familiar to the biologist and such equations are commonly used for modelling biological processes (Alon, 2007). However, in reality, biochemical reactions are the result of random encounters between discrete numbers of molecules, and some of these molecules may be present at low numbers. The finite number of molecules interacting within the system leads to inherent randomness known as intrinsic stochasticity (Swain et al., 2002).
Thus, while the units of the deterministic differential equations are concentrations, a stochastic model requires a description in terms of discrete numbers of molecules. To convert from concentration into the number of molecules we use the system size parameter, $\mathrm{\Omega}$, which can be seen as a measure of the volume of the biological system (Kaern et al., 2005; Elf and Ehrenberg, 2003). Concentrations and copy numbers (i.e. number of molecules) are then related via
The copy number can increase when the system size (volume) increases and the concentration is constant, or when the system size (volume) is constant and the concentration increases.
In order to investigate the effects of intrinsic noise on the dynamics of differentiation we reformulated the Hes1/miR9 model in a stochastic framework, which describes the system in terms of its underlying microscopic reactions, namely the production and degradation of Hes1 mRNA, HES1 protein and microRNA miR9. We use three complementary stochastic modelling approaches: simulation of the Chemical Master Equation (CME) with the Stochastic Simulation Algorithm with delay (dSSA), simulation of the Chemical Langevin Equation (CLE) and the Linear Noise Approximation (LNA) as described fully in the Materials and methods.
Briefly, the CME uses the network of interactions and their associated rates to formulate how each reaction acts to increase or decrease the likelihood of transitioning to a new state. For example, if the number of protein molecules is very high, the next most likely reaction may be a protein degradation event. Other reactions can still be triggered, however, and this leads to the heterogeneity over time.
Single realisations of the CME were simulated computationally using Gillespie’s Stochastic Simulation Algorithm with delay (dSSA) (Gillespie, 1977; Anderson, 2007; Cai, 2007) which is a statistically exact solution of the master equation but can be computationally expensive, particularly at high system size ($\mathrm{\Omega}$) because every reaction is simulated. Therefore, to investigate different parameters and to understand the behaviour of a population of cells, both of which require a large number of repeats, we used methods that are more computationally efficient such as Chemical Langevin Equations (CLE) and the Linear Noise Approximation (LNA), which add a random fluctuating term by bundling reactions together (Gillespie, 2000; Grima et al., 2011). The Linear Noise Approximation (LNA) is a (semi)analytical approach that does not require multiple simulations and is hence the fastest of all the methods for scanning parameter space. However, it can be less accurate at low system size (Grima, 2012).
A 'finite number' stochastically simulated network is justified by experimental data
In our conceptual framework, the timing of differentiation corresponds to the time required to reach a low stable HES1 state in the model, brought about by an increase in miR9 over time (Goodfellow et al., 2014). The increase of miR9 over time in cultured neural progenitor cells has been shown before (Bonev et al., 2012). Here, using a miR9 sensor we show that the activity of miR9 increases specifically in neuronally differentiated cells (Figure 1—figure supplement 1).
In order to justify the finitenumber approach in our modelling we used smFISH to determine the absolute copy number of Hes1 mRNA in C17.2 neural progenitor cells and neural stem cells (NS) expressing GFP from the Mapt (tau) locus (TauGFP NS cells). Multiple fluorescent probes were designed to target the Hes1 transcript (Figure 1A) and the accuracy of smFISH for absolute quantitation was validated in Figure 1—figure supplement 2. The Hes1 mRNA copy number was highly variable between individual cells grown under identical conditions and also between cell lines (Figure 1B and D), but generally quite low. In C17.2 neural progenitor cells, we detected an average of 72 copies Hes1 mRNA/cell by smFISH (Figure 1C), and an average of 88 copies by qRTPCR (data not shown), confirming the result. In TauGFP NS cells grown under proliferative conditions (GFP negative cells; Figure 1D and E), the average was only 11 copies of Hes1 mRNA/cell by smFISH. The longtail distribution of mRNA molecules in NS cells may be indicative of slow switching between transcriptionally active and inactive promoter states (Munsky et al., 2012). Other studies using smFISH have reported numbers of mRNA molecules ranging from a few copies to ${10}^{3}$ (Neuert et al., 2013; Bahar Halpern et al., 2015; Battich et al., 2015). A few spontaneously differentiated cells, identified by highintensity GFP expression (Figure 1—figure supplement 3), were mostly negative for Hes1 mRNA transcript, indicating that the gene is switched off upon differentiation (average 0.8 copies per cell) (Figure 1D and F). This was confirmed by using an intronic probe for Hes1 which showed transcription in GFP negative cells (Figure 1H, corresponding exonic shown in G, I) but no transcription in GFP positive neuronally differentiated cells (Figure 1J).

Figure 1—source data 1
 https://doi.org/10.7554/eLife.16118.003

Figure 1—source data 2
 https://doi.org/10.7554/eLife.16118.004

Figure 1—source data 3
 https://doi.org/10.7554/eLife.16118.005

Figure 1—source data 4
 https://doi.org/10.7554/eLife.16118.006

Figure 1—source data 5
 https://doi.org/10.7554/eLife.16118.007
For mature miR9, we used a stemloop Taqman qRTPCR and synthetic miR9 (Figure 1—figure supplement 4) to estimate the absolute copy number of endogenous miR9, expressed as a population average copy number per cell in C17.2 cells. Mature miR9 was present in an average copy number of 1100 ± 20 (S.E.M) molecules per cell, significantly lower than other reported microRNAs and surprisingly low considering the hundreds of miR9 targets (Bonev et al., 2011) (Figure 1—figure supplement 5).
Finally, to quantify the abundance of endogenous HES1 protein per nucleus in NS cells we generated two stable HES1 reporter celllines: VENUS:HES1 driven either by the constitutive UbC promoter or the 2.7 kb Hes1 promoter (Figure 2A,B). FCS was used to count the number of VENUS:HES1 fluorophores (see Experimental methods). This gave an estimate of 3660 ± 120 (S.E.M) molecules of VENUS:HES1 per nucleus when driven by the UbC promoter (Figure 2D and Figure 2—figure supplement 1). We next used western blotting to find the ratio of expression between VENUS:HES1 and endogenous HES1 (Figure 2C) and subsequently normalised to the percentage of VENUS:HES1 positive cells in the population. This gave variable ratios between experiments and showed an approximate ratio of 1:0.8 ectopic to endogenous HES1. This suggests an average abundance of endogenous HES1 of 2930 molecules per nucleus which is significantly lower than the median protein abundance of 50,000 copies per cell previously reported from global quantification studies (Schwanhausser et al., 2011). Furthermore, FCS quantification of VENUS:HES1 driven by the Hes1 promoter gave a similar value, that is 2450 ± 190 (S.E.M) molecules per nucleus (Figure 2D). These complementary approaches highlight the low average abundance and high variability of HES1 molecules in the cell.

Figure 2—source data 1
 https://doi.org/10.7554/eLife.16118.014
We then investigated whether a stochastic or deterministic simulation of the Hes1/miR9 interaction model is a better fit to the experimental data of HES1 protein and mRNA dynamics at the singlecell and population level. We simulated the model of the Hes1/miR9 network using the default parameters of Goodfellow et al. (Goodfellow et al., 2014) (parameter set 1 – Appendix 1—table 1) in both a deterministic and stochastic framework (using the dSSA algorithm). While oscillations in the deterministic model remain coherent (Figure 3B), the stochastic model shows variability in its dynamics and period (Figure 3C). Singlecell luciferase imaging of primary NS cells containing a Luciferase2HES1 (LUC2:HES1) fusion BAC reporter (Imayoshi et al., 2013) shows a high degree of noise evident as peaktopeak variability in amplitude and period. This was also evident in our experiments (Figure 3A and Figure 3—figure supplement 1 and 2 for images and traces of multiple cells). At the population level, analysed by qRTPCR of serum synchronised C17.2 cells, oscillations in Hes1 mRNA quickly dampen, presumably because cells drift out of synchrony (Figure 3D). The average expression of a population of 2000 stochastic simulations using the dSSA shows faster dampening (Figure 3F) than deterministic simulations (Figure 3E). Thus, the stochastic simulation matches the experimental data better, both at the singlecell and the population level.

Figure 3—source data 1
 https://doi.org/10.7554/eLife.16118.017

Figure 3—source data 2
 https://doi.org/10.7554/eLife.16118.018
Stochasticity spreads the timing of differentiation
By measuring the time taken to reach a low HES1 state in multiple simulations we can describe the distribution of timing events expected in a population of cells. The contribution of stochasticity was investigated by varying the system size parameter $\mathrm{\Omega}$, which changes the number of molecules in the system. All analysis was performed using parameters for which the amplitude of oscillations is large and the transition to the low HES1 state is very clear, (parameter set 2 – Appendix 1—table 1). Once the HES1 protein expression reaches a low state we observed that it never again exceeds a low threshold of 20x$\mathrm{\Omega}$. We therefore defined 'differentiation' as the time taken for HES1 protein to decrease below this threshold.
In deterministic simulations, all cells that start with the same parameters and conditions differentiate at the same time. At a high system size ($\mathrm{\Omega}$=50), corresponding to high molecule number (mRNA∼500; protein and miR9∼30,000), there is low variability in the stochastic dynamics and oscillations appear regular (Figure 4A). The time taken to reach the low HES1 protein threshold was calculated for 2000 simulations using the dSSA, and the distribution of times to reach the low HES1 state is narrow for $\mathrm{\Omega}$=50 (Figure 4B), approaching the deterministic solution.

Figure 4—source data 1
 https://doi.org/10.7554/eLife.16118.022
As the system size is decreased to $\mathrm{\Omega}$=5 (mRNA∼50; protein and miR9∼3000) and then further to $\mathrm{\Omega}$=1 (mRNA∼10; protein and miR9∼600) molecular fluctuations cause increasingly varying amplitude and period of HES1 oscillations within each simulation (Figures 4D and G, respectively). At the same time, the distribution in the timing of differentiation of 2000 independent simulations becomes increasingly broad as the system size decreases (Figures 4E and H). Similar results were obtained with the CLE (Figures 4CI). We conclude that the greater random molecular fluctuations caused by decreased molecular copy number acts to increase the uncertainty in the timing to reach a low HES1 protein state in each cell, such that differentiation events become distributed in the population over a longer period of time.
Stochasticity shifts the average onset of differentiation
When we reduced the system size we unexpectedly found that the average of the distribution also became gradually shifted towards earlier times (e.g. compare Figure 4B with Figure 4E and H, where scaling of mean with system size shown in Figure 4—figure supplement 1). This shift indicates that stochasticity also causes a systematic change in timing, whereby the low HES1 state is reached on average earlier at low system sizes, even though the parameters of the model remain exactly the same. The origin of the systematic change in timing can be understood by considering how the production of miR9 changes in the presence of stochasticity.
Transcription of miR9 is repressed by HES1 protein, and consequently the production of miR9 ($r$) increases when HES1 protein levels are low, assuming activation in the absence of repression. The production of miR9 is therefore intrinsically linked to HES1 protein, and stochastic variability in HES1 protein causes stochasticity in the production of miR9 (Figure 5A, miR9 production is blue line). As the repression of miR9 transcription by HES1 protein is modelled as a nonlinear Hilltype function, the higher incidence of low HES1 troughs at lower system sizes gives rise to large spikes of miR9 production (compare blue line in Figure 5A with Figure 5B). In turn, this leads to a faster increase of the stable mature miR9 (red line in Figure 5A and B) and a reduction in the average time to differentiation.

Figure 5—source data 1
 https://doi.org/10.7554/eLife.16118.025
However, the systematic change in timing caused by stochasticity is not guaranteed to speed up the system. Instead of low troughs in HES1 causing spikes in miR9 production, in an alternative parameter set the peaks in protein could cause a disproportionate repression in miR9 production. This occurs, for example, when the transcriptional repression threshold of miR9 production is roughly twice that of the Hes1 promoter (${p}_{1}$ = 750), as shown in Figure 5C. In this scenario the systematic effect of stochasticity on timing is reversed and the low system size simulations reach the low HES1 state at a later time. Note that the relative effect on timing is smaller, but this may be expected as HES1 is only weakly repressing miR9 transcription for this parameter setting.
Increased miR9 concentration shifts timing and reduces the spread of differentiation
The concentration of miR9 is a key driver of the timing to differentiation of a single cell (single simulation) in the deterministic model (Goodfellow et al., 2014). We therefore tested the effect of altering the concentration of this key component in the stochastic model and in a populationlevel simulation.
In the stochastic model increasing the amount of miR9 brought forward and reduced the spread of time to differentiation events in a simulated population (Figure 6A–C). To experimentally validate this theoretical finding we measured the timing and spread of differentiation in a population of C17.2 and murine TauGFP NS cells transfected with miR9 mimic. Increasing miR9 in C17.2 neural progenitor cells increased the number of Tuj1positive neurons earlier than in control cultures after several days, without affecting the total cell number (Figure 6—figure supplement 1). The shorter time to differentiate was confirmed by FACS analysis of TauGFP NS cells treated again with miR9 mimic (Figure 6—figure supplement 2). Transfecting the C17.2 cells with a control mimic and measuring the number of Tuj1positive neuronal cells showed a broad spread in the timing of differentiation (standard deviation = 48 hr, Figure 6D), as expected from a stochastic process. When cells were transfected with 50 nM or 100 nM of miR9 mimic the average time to differentiation decreased and the spread became narrower in a miR9 initial concentration dependent way (standard deviation = 24 hr and 17 hr respectively, Figure 6E and F). Thus, both computational prediction and experimental validation suggest that the initial concentration of a single species, miR9, is sufficient to change the variance in the time to differentiation.

Figure 6—source data 1
 https://doi.org/10.7554/eLife.16118.027

Figure 6—source data 2
 https://doi.org/10.7554/eLife.16118.028

Figure 6—source data 3
 https://doi.org/10.7554/eLife.16118.029
Stochasticity increases the robustness of the progenitor state
The progenitor state needs to be robust to avoid premature depletion of progenitors during the lengthy course of development. In order to investigate the robustness of the Hes1/miR9 regulatory network we compared the termination of HES1 oscillations in stochastic and deterministic frameworks. This comparison was carried out systematically in the space of the parameters ${r}_{1}$ and ${m}_{1}$ using the linear noise approximation (LNA) (see Appendix for details). These control how much miR9 is required to repress Hes1 mRNA translation by half, and how nonlinear (sigmoidal) the translational repression is as a function of miR9. We have previously shown that translational repression of Hes1 mRNA by miR9 is particularly important in terminating the Hes1 oscillator with a low HES1 protein level (Goodfellow et al., 2014).
Different behaviours between the deterministic and the stochastic model are shown in this parameter space (Figures 7A and B). In the first region, where translational repression is highly nonlinear (high ${m}_{1}$) and translational repression strength is low (high amount of miR9 required; high ${r}_{1}$), miR9 never accumulates to the levels required to terminate oscillations (Figure 7A area 1). As a result, deterministic simulations in this parameter region show selfsustained limit cycle oscillations (${m}_{1}$ = 3.5, ${r}_{1}$ = 900) (Figure 7C). The equivalent stochastic simulations using the dSSA also display sustained periodic behaviour (Figure 7D). This shows that when the translational repressive effect of miR9 is weak, oscillations are not terminated in either deterministic or stochastic settings.

Figure 7—source data 1
 https://doi.org/10.7554/eLife.16118.033

Figure 7—source data 2
 https://doi.org/10.7554/eLife.16118.034
In the second region, (Area 2 in the deterministic model; Figure 7A), defined by highly nonlinear translational repression (high ${m}_{1}$) and high translational repression strength (low amount of miR9 required to repress translation; low ${r}_{1}$), both deterministic and stochastic models produce HES1 protein levels that decrease and reach a steady state (Figure 7E and F, respectively). The power spectrum of the stochastic simulations at ${m}_{1}$ = 3.5, ${r}_{1}$ = 600 shows no peak, and hence the coherence (defined in Materials and methods, Figure 7—figure supplement 1) is zero (Figure 7—figure supplement 2A) The resulting behaviour of such a process is aperiodic (Figure 7—figure supplement 2B); therefore even though HES1 protein still shows some fluctuations (Figure 7F), the lack of periodicity allows us to conclude that oscillations are terminated in the stochastic setting. Thus, in this area oscillations are terminated in both models.
Of particular interest is the third region (Area 3 in the stochastic model; Figure 7B), which is characterised by less nonlinear repression (low ${m}_{1}$) and low translational repression (i.e. high amount of miR9 to repress translation by half, high ${r}_{1}$). In this area of parameter space the periodic behaviour is distinct between the deterministic and stochastic framework. The deterministic solution in this regime exhibits damped protein oscillations that eventually reach a steady state (Figure 7G). In contrast, the time series of a stochastic simulation using the dSSA at this parameter configuration shows that even though miR9 accumulates over time, it is unable to terminate protein oscillations (Figure 7H). When the system oscillates in the stochastic regime (${m}_{1}$ = 1.5, ${r}_{1}$ = 600), there is a welldefined peak in the power spectrum (Figure 7—figure supplement 2C). The peak at 0.04 min${}^{1}$ corresponds to a period of 160 min in the time series (Figure 7—figure supplement 2D). Thus, this region represents an expansion of the range of parameters in which HES1 protein oscillates.
We propose that stochasticity has made the oscillatory state more robust to an increase in miR9 in the sense that the stochastic HES1 oscillator continues to operate under conditions (i.e. parameter values) where the deterministic equivalent will have stopped. It is still possible for the accumulation of miR9 to terminate HES1 oscillations in the stochastic model but the parameter space in which this happens is reduced.
Robustness of timing to perturbations from asymmetric inheritance at cell division
Having seen that stochastic oscillations are inherently more robust in that they are observed for an increased parameter space, we also tested whether they would show increased robustness to external perturbations. Such perturbations can arise by the unequal distribution of molecules during cell division and can be modelled by changing the initial conditions of the simulations. We consider two models of cell division: one where molecular constituents are distributed equally between daughter cells (and hence initial conditions are the same at 50% of the parent cell) and another where molecules are randomly inherited.
First, we investigated how Hes1 mRNA is actually partitioned in NS cells that are undergoing fate symmetric divisions (i.e. grown under proliferative divisions). Very few of these divisions partitioned Hes1 mRNA equally and some showed a highly asymmetric distribution (Figure 8A–D). The majority of cell divisions (36/51) were mostly consistent with a binomial mode of division (pvalue>0.05, binomial distribution with Bonferroni multiple testing adjusted pvalue), with a few outliers. A binomial distribution would be expected if each mRNA molecule is partitioned independently into the two daughter cells and where the probability of inheriting an mRNA is equal to the ratio of the cell areas of the daughter cells. Previous studies in bacteria have also shown that partitioning is approximately binomial (Golding et al., 2005).

Figure 8—source data 1
 https://doi.org/10.7554/eLife.16118.038
Using the binomial distribution we then calculated the impact of partitioning noise into the performance of the HES1 oscillator under deterministic and stochastic conditions. To generate initial conditions we simulated a parent cell deterministically for 12 hr, starting with parameter set 2 and initial conditions mRNA = 20x$\mathrm{\Omega}$, protein = 400x$\mathrm{\Omega}$, miR9 = 0. Using these parent cell conditions, we then generated daughter cells assuming either equal 50/50 inheritance (no heterogeneity) or a binomial distribution of inheritance for each species (schematic shown Figure 9A). Figure 9 shows the difference in the time distribution of differentiation assuming equal or binomial partitioning, with deterministic (Figure 9B,C) or stochastic (Figure 9D,E) dynamics.

Figure 9—source data 1
 https://doi.org/10.7554/eLife.16118.040
With 50/50 inheritance at cell division (i.e. no heterogeneity) and deterministic dynamics there is no randomness in the system and hence all cells 'differentiate' at the same time (Figure 9B). The time is the same for all system sizes, as the number of molecules has no effect on the dynamics. Next, we introduce heterogeneity in the starting conditions of the cell, occurring only through random partitioning and using a binomial distribution. At a high system size, the distribution of timetodifferentiation is very narrow (standard deviation = 8.58) (Figure 9C). This makes sense, because the relative variance of a binomial distribution is lower at high copy number and hence at large molecule number the daughter cells have similar inheritance and therefore similar timing to differentiation. As the system size is lowered ($\mathrm{\Omega}$=1 and 0.2), the variance increases and the variability in the daughter cell initial conditions causes the timetodifferentiation to become more spread out, compared either to the high system size (either mode of partitioning) or the low system size with equal partitioning (standard deviation = 165, pvalue<0.05 for KolmogorovSmirnov test between $\mathrm{\Omega}$ = 0.2 and 50) (Figure 9C). This finding denotes a lack of robustness to partitioning errors when one looks at the outcome of the HES1 oscillator at the population level.
We now consider the effect of unequal inheritance in the case of stochastic dynamics. At both high and low system size (low and high intrinsic noise, respectively) there is little difference between the distribution created with 50/50 (constant) or unequal (binomial) initial conditions as indicated by a low value of the KolmogorovSmirnov distance between the two distributions and a pvalue indicating a nonsignificant difference (Figure 9D and E).
We conclude that the stochastic system is more robust than the deterministic system to noise introduced at cell division because it produces the same distribution of differentiation at the population level irrespective of whether the cells distribute molecules randomly or equally at cell division, at all system sizes. In other words, the stochastic system shows tolerance to the unequal distribution of species at division and maintains its performance. By contrast, in the deterministic framework the distribution of differentiation events is changed significantly by random partitioning of molecules, and this change increases as the system size is lowered.
Discussion
At the molecular level, gene expression involves discrete reaction steps that result in the birth and death of individual molecules and it is therefore subject to the intrinsic noise generated by the random encounters of a finite number of interacting molecules (Lestas et al., 2010). It is widely assumed that such noise is detrimental to the robust performance of biological systems and indeed, a number of mechanisms to filter noise have been described (Arias and Hayward, 2006; Bahar Halpern et al., 2015). It remains an open question whether biological systems exploit noise, as is the question of whether noise optimization itself is an evolvable trait in development (reviewed in Rao et al., 2002; Kaern et al., 2005; Balázsi et al., 2011). Here, we have considered the influence of intrinsic noise generated by a low molecular number of interacting species, on the output of HES1 protein oscillations that are generated by delayed negative feedback and are tuned by miR9. Our findings led us to propose that stochasticity generated by intrinsic noise offers a number of advantages to the differentiation process.
Modelling the differentiation process as an exit from the oscillatory state with low HES1 protein (Goodfellow et al., 2014), we found that in the stochastic framework the time taken to exit HES1 oscillations varied from cell to cell, even though they started from identical conditions. Hence, there was a distribution in the timing of differentiation. This is an important feature of the stochastic model, as it predicts that it is possible for an initially equivalent population of cells (e.g. clonally related, with the same transcriptional profile and under identical conditions) to change cellular identity at different times as a result of internal noise alone and not as a consequence of any other external influences; at the population level this would create a distribution in the timing of differentiation (Gomes et al., 2011; Boije et al., 2014). We suggest that it may be advantageous for differentiation events to have a broad distribution, as this could be used to generate celltype diversity, if coupled with mechanisms of celltype specification (Qian et al., 2000; Temple, 2001). However, a distribution in timing of differentiation is not only linked to the generation of different cell types, because when one considers the generation of a single particular cell type, there is again a distribution in the timing of birth of this cell type (Gaspard et al., 2008; Gomes et al., 2011). This suggests that a distribution in the timing of differentiation is a widespread property, which is perhaps advantageous in allowing a time window for feedback control such that the right cell numbers are produced.
Our modelling also showed that there is a continuum of behaviours since the spread of the timing distribution is a function of the system size, where high system size gives rise to a narrow distribution and low system size to a broad distribution. We speculate that this may partly explain the existence of differentiation models that are highly deterministic, as for example in Drosophila neuroblasts which generate cells with a rigidly programmed sequence of fates (Pearson and Doe, 2004; Kohwi and Doe, 2013; Rué and Martinez Arias, 2015) and others that are highly stochastic, as the above mentioned example of vertebrate retinal development.
Our findings suggest that the incorporation of stochasticity into the HES1 oscillator model is advantageous because it increases the robustness of the progenitor state to intrinsic and extrinsic noise. The robustness to increased intrinsic noise is demonstrated by the increase in the region of the parameter space within which HES1 oscillates. This happens because in the parameter space near the oscillatory regime (Hopf bifurcation), the deterministic solution would exhibit damped oscillations but noise keeps pushing the system away from the steady state. This is consistent with previous reports that described noise induced oscillations (McKane and Newman, 2005; Barrio et al., 2006; Galla, 2009). However, while previous studies have looked at whether noise can induce oscillations in systems at steady state (statistically stationary), we have used LNA to look at whether initial transient oscillations are eventually terminated. In this way, we showed that miR9 accumulation can terminate stochastic oscillations if the threshold of miR9 required for translational repression is sufficiently low, and the Hill coefficient of repression is sufficiently high. This is important, as normal neural development relies on the balance of prolonged progenitor maintenance (i.e. robust HES1 oscillations) and differentiation (i.e. the ability to terminate these).
The outcome of deterministic dynamics is very different in symmetric versus randomly asymmetric divisions, meaning that deterministic dynamics are not robust to heterogeneities introduced at cell division. In particular, the narrow spread of differentiation that is produced by a deterministic molecular network would be impossible to maintain if the distribution of molecules at division is random and the system size low, both of which are experimentally observed. By contrast, the stochastic system has a more robust performance to extrinsic noise introduced by celldivision asymmetries, as the outcome does not change between equal or random division. Interestingly, at low system size the random partitioning of molecules at division produces a distributed time of differentiation, even when oscillations are modelled in a deterministic framework. We interpret this to mean that noise due to celldivision asymmetries has a similar effect to noise due to low system size on the outcome of the HES1 oscillator. This fits well with previous data on stochasticity introduced by partitioning errors (Huh and Paulsson, 2011). Our findings agree that noise introduced in this way is very difficult to distinguish from other sources of noise (Huh and Paulsson, 2011) and both could contribute in the heterogeneity of timing of clonal cells, although the effect does not appear to be additive.
In addition to the spread of differentiation events, the mean timetodifferentiation was also systematically affected in the stochastic framework, which can be explained by the dynamics of HES1 and miR9 production in conjunction with the parameterisation of the system. This shows that in addition to parameters and initial conditions (Goodfellow et al., 2014), the average timetodifferentiation can also be tuned by the absolute number of molecules reacting in the system through the system size. The mean timetodifferentiation is important as it controls the total cellular number of a developed organ, along with other quantities such as the proliferation rate.
Here, we have assumed that the intrinsic noise is a result of low interacting numbers of molecules in the Hes1/miR9 network. Indeed, this was experimentally supported by the absolute quantitation of Hes1 mRNA, protein and miR9. The average but absolute Hes1 mRNA, protein and miR9 numbers are consistent with a medium to low system size between $\mathrm{\Omega}$=5 and $\mathrm{\Omega}$=1, with variability observed both between cells and between different cell lines. In particular, while the Hes1 mRNA copy number is close to the reported global median of 17 copies per cell, the HES1 protein abundance is well below the median value of 50,000 copies per cell (Schwanhäusser et al., 2011). The average measured miR9 copy number (1100 copies) in C17.2 cells is also on the low side of the previously reported range of 10 to more than 30,000 copies per cell (Chen et al., 2005). We consider the abundance of HES1 protein and miR9 to be low not only compared to average global abundancies, but also considering the number of their predicted molecular targets (3,074 for HES1, (Margolin et al., 2009); and more than 500 for miR9, (Bonev et al., 2011)). However, the low molecular copy number is not an exclusive source of celltocell heterogeneity. The Hes1 gene is also involved in the Notch signalling pathway of cellcell communication, and other studies have considered the effect of noise in the Notch pathway (Jenkins et al., 2015). Additionally, spatial models of Notch signalling have considered the effect of stochastic motions of the nucleus within a gradient of Notch signalling molecules (Aggarwal et al., 2016). Such sources of heterogeneity are not mutually exclusive, and future studies may extend the model to include these effects.
In our model there is inherent stochasticity in every reaction controlling the birth and death of each species but the transcription rate can take a continuous range of values. Other studies of the Hes1/Her1 network have considered binary promoter noise, where the transcription rate switches discontinuously between an ON and OFF state (Lewis, 2003; Sturrock et al., 2013; Jenkins et al., 2015). In the promoter ON/OFF model, periods of gene activity are interspersed by transcriptional silence, leading to bursty behaviour (Suter et al., 2011). Bursting models of transcription have been shown to fit the statistics of mRNA distributions measured by smFISH in multiple different systems (Munsky et al., 2012; Neuert et al., 2013; Bahar Halpern et al., 2015), and have also been fitted to live time series of transcription (Harper et al., 2011; Corrigan and Chubb, 2014; Zechner et al., 2014). Other recent studies have explicitly modelled the coupling of the Hes1 system to the cell cycle (Pfeuty, 2015a, 2015b), but carry the disadvantage that they contain many parameters that are not reliably known.
The challenge is to integrate these different phenomena, each potentially contributing a different source of noise, with the ultimate goal of connecting mechanistic explanations of developmental gene expression programmes with models of population dynamics. Our model provided an experimentally grounded and molecularly explicit theoretical framework to understand the underlying heterogeneity in the differentiation of clonal isogenic stem or progenitor cells, which have been uncovered by lineage tracing experiments in vivo, or in vitro cultures under the same conditions (Rapaport et al., 2004). Our work has also highlighted the need for integrated computational modelling and the value of an absolute quantitation in the experimental approach.
Materials and methods
Computational methods
Request a detailed protocolPrevious analysis focussed on a model of the Hes1/miR9 network using delay differential equations (Goodfellow et al., 2014), which are themselves an extension of earlier models by Monk et al (2003), as follows
where $m$, $p$ and $r$ are the concentrations of Hes1 mRNA, HES1 protein and miR9 respectively. The model parameters $\mu}_{p$ and $\mu}_{r$ are the halflives of HES1 protein and miR9 degradation, respectively. The quantities $G(p(t\tau ))$, $S(r(t))$, $F(r(t))$ and ${G}_{r}(p(t))$ are Hilltype functions that control the rates of synthesis and degradation. The variable τ indicates the time delay in the repression of Hes1 transcription by HES1 protein, accounting for the total time taken from mRNA transcriptional initiation to HES1 protein returning to the nucleus to repress transcription. For a given set of initial conditions and parameters the solution of these differential equations is always the same, and they are therefore described as deterministic.
HES1 protein represses Hes1 mRNA production through binding to the promoter, and this is modelled with a monotonically decreasing Hill function. In the model it takes the form
where $\alpha}_{m$ is the basal transcription rate of Hes1 mRNA in the absence of HES1 protein (previously set to 1; [Goodfellow et al., 2014]).
The Hill function is characterized by two parameters: the repression threshold ${p}_{0}$, which represents the amount of protein required to repress transcription to half of basal levels, and the Hill coefficient ${n}_{0}$, which controls how steep (sigmoidal) the repression is as a function of protein concentration.
The degradation of Hes1 mRNA is dependent on the levels of miR9 through the Hill function $S(r)$
The function $S(r)$ represents an effective degradation rate of Hes1 mRNA, and ${b}_{l}$ and ${b}_{u}$ impose lower and upper bounds for Hes1 mRNA halflife, respectively, which are set using halflives measured in Bonev et al. (Bonev et al., 2012).
The function $F(r)$ represents translation repression of HES1 protein by miR9, and the functional form is also a Hilltype function, as previously assumed in Osella et al. (Osella et al., 2011).
Finally, the HES1 protein represses miR9 production through binding to the promoter, and this is again modelled with a Hill function
All analyses and simulations were performed using MATLAB 7.12 (RRID:SCR_001622)(MathWorks, US), and code is provided as associated figure source files. For deterministic simulations of the model, we used the MATLAB function ‘dde23’ with options ‘RelTol’ = 10${}^{5}$ and ‘AbsTol’ = 10${}^{8}$. For history vectors we used a single value repeated over prior time points. Numerical bifurcation analysis was performed using DDEBIFTOOL (Engelborghs et al., 2002).
Developing a stochastic model for the Hes1/miR9 oscillator
Request a detailed protocolIn order to investigate the effects of intrinsic noise on the dynamics of differentiation we reformulated the Hes1/miR9 model in a stochastic framework, which describes the system in terms of its underlying microscopic reactions. The numbers of molecules are increased or decreased through six reactions, namely the production and degradation of mRNA, protein and microRNA as follows:
The transcription reaction involves a delay, indicated by a double arrow, while all other reactions are assumed to occur instantaneously with the rates of reaction indicated above the arrows. The stochastic dynamics of the Hes1/miR9 reaction system were captured by the chemical master equation (CME) for the number ${n}_{m}$ of mRNA molecules, ${n}_{p}$ of protein molecules and ${n}_{r}$ of miR9 molecules. The CME describes how the probability of the copy number $\mathbf{n}=({n}_{m},{n}_{p},{n}_{r})$ of the different molecular species evolves over time.
The six reactions in the system result in a stochastic process described by the CME
where $P(\mathbf{n};t)$ is the probability of finding the system in state $\mathbf{n}$ at time $t$, and $P(\mathbf{n};t;{\mathbf{n}}^{\mathrm{\prime}},{t}^{\mathrm{\prime}})$ is the probability for the system to be in state $\mathbf{n}$ at $t$ and in state $\mathbf{n}}^{\mathrm{\prime}$ at time $t}^{\mathrm{\prime}$. $\mathbb{E}}_{m$, $\mathbb{E}}_{p$ and $\mathbb{E}}_{r$ are raising operators acting on functions of $n}_{m$, ${n}_{p}$ and ${n}_{r}$ via ${\mathbb{E}}_{m}g({n}_{m},{n}_{p},{n}_{r})=g({n}_{m}+1,{n}_{p},{n}_{r})$, ${\mathbb{E}}_{p}g({n}_{m},{n}_{p},{n}_{r})=g({n}_{m},{n}_{p}+1,{n}_{r})$ and ${\mathbb{E}}_{r}g({n}_{m},{n}_{p},{n}_{r})=g({n}_{m},{n}_{p},{n}_{r}+1)$. $\mathbb{E}}^{1$ stands for the inverse operation. In the case of twotime quantities, e.g. $P(\mathbf{n};t;{\mathbf{n}}^{\mathrm{\prime}},{t}^{\mathrm{\prime}})$ the raising applies with respect to the second argument $n}^{\mathrm{\prime}$.
We have also developed a set of Chemical Langevin Equations (CLE) (Brett and Galla, 2014), where $\zeta (t)$ is a rapidly fluctuating random term. The CLE is more computationally efficient at large system sizes, because it does not keep track of every reaction, as they are effectively bundled into time steps.
This random term is assumed to have a Gaussian distribution with zero mean, and the fluctuations added at two consecutive time points are uncorrelated (white noise). The variance of the white noise terms are given by
The CLE was simulated numerically using an EulerMaruyama scheme with time steps of 0.1 mins. For history vectors we used a single value repeated over prior time points.
To perform parameter scans, which requires a large number of repeats, we used the Linear Noise Approximation (LNA) because it does not require multiple simulations and is therefore much faster. The LNA is obtained through performing a system size expansion on the CME (Galla, 2009). The system size expansion decomposes the time evolution of each species into a deterministic and stochastic contribution, where the relative amplitude of stochastic fluctuations decreases with the inverse of the square root of the system size (i.e. relative fluctuations decrease with higher molecule number)
Where ${n}_{i}$ is the number of molecules of species $i$, $\varphi}_{i$ is the concentration of species $i$ as determined by the deterministic rate equations, and $\u03f5}_{i$ is the random fluctuation with continuous degrees of freedom around the deterministic solution.
After performing the system size expansion, the deterministic equations are recovered when the system size is infinite (no stochasticity). The linear noise approximation (LNA) refers to the next level of approximation from the system size expansion, valid in the limit of large (but not infinite) system size. The LNA describes stochastic fluctuations around the deterministic trajectory. However, there is a tradeoff with accuracy, and the system size expansion can be less accurate than either the dSSA or CLE at low system sizes (Grima et al., 2011; Thomas et al., 2012). The full detailed description of the system size expansion for the Hes1/miR9 model is shown in Appendix. Even though the CLE and LNA are less accurate than the CME at low system sizes, good agreement between them was observed.
Finally, in addition to the assumptions of the deterministic model (Goodfellow et al., 2014), which mostly refer to unknown kinetic parameter values, the stochastic model makes several conceptual assumptions. We assume the chemical system is in wellmixed conditions and the system size is a fixed value. We also assume that the cells are of the same size (volume), and hence the system size parameter is the same within a population of cells and after cell division. When cell division is included in the model, the time of cell division is not constrained and can occur at any point of the HES1 oscillation. Transcription of Hes1 and miR9 is controlled by negative feedback and we assume that it is constitutively active when not inhibited. Note that both Hes1 mRNA and miR9 are transcribed from DNA, which is assumed to be at constant concentration, and hence both Hes1 mRNA and miR9 are produced from the void. In order to make direct comparisons with the previous deterministic description, we do not explicitly model the noise associated with bursting transcription (Suter et al., 2011), which is therefore not explicitly distinguished from other sources of inherent noise.
Comparison of population level simulations with different modes of division
Request a detailed protocolWe simulated the time to differentiation in 3000 cells using the CLE, with different system sizes ranging from 0.2 to 50 and looked at the outcome of 50/50 or binomial partitioning at division. The timing distributions using constant or random initial condition models were compared using the KolmogorovSmirnov distance, which quantifies the distance between two empirical distributions and can take values between 0 and 1, with lower values indicating higher similarity,
where ${F}_{b}(t)$ is the cumulative distribution function of the timing with binomial initial conditions, and ${F}_{e}(t)$ is the cumulative distribution function of the timing with equal 50/50 initial conditions.
Evaluation of oscillatory dynamics
Request a detailed protocolCentral to oscillations is the notion of periodicity. In a stochastic setting, it becomes desirable to be able to distinguish oscillations from random fluctuations, which are dynamic in time but aperiodic. A common method for detecting periodicity is the Fourier transform, which decomposes a given signal into simple sinusoidal oscillating functions. The simple sinusoidal oscillating functions are described by their frequency, and so a Fourier transform takes data in the time domain and represents them in the frequency domain. The Fourier transform is then squared to obtain the power spectrum, and an oscillator is now defined as any process with a peak in its power spectrum at a particular (nonzero) frequency. If the peak is high and sharp, the time series will exhibit high quality oscillations at the dominant frequency with little variability in the peaktopeak in the time series. However, if the peak in the power spectrum is shallow and broad, there can be significant variation in the peaktopeak times. When there is no peak in the power spectrum at a nonzero frequency, the process will no longer oscillate and will instead consist of aperiodic fluctuations. A continuum of behaviours is possible, and a suitable metric needs to be defined to measure how sharp the peak in the power spectrum is.
One such measure is the coherence, which is defined as the area under the curve within 20% of the peak frequency as a fraction of the total area under the curve (Alonso et al., 2007), as shown in Figure 7—figure supplement 1. The coherence is easily calculated, since the power spectrum of a stochastic process can be approximated efficiently using the LNA. Unlike the CLE, the LNA is a linear stochastic differential equation, and it is therefore possible to directly calculate the power spectrum of the stochastic dynamics, even with delay (as described in Appendix).
Experimental methods
Cell culture
Request a detailed protocolC17.2 cells (RRID:CVCL_4511)(07062902, Sigma, UK) were grown in DMEM (ThermoFisher Scientific, UK ) with 10% FBS. Primary NS cells were isolated from dissected forebrains of E13.5–15.5 embryos from LUC2:HES1 BAC reporter mice (RRID:IMSR_RBRC06013) and cultured as previously described (Pollard, 2013). LUC2:HES1 BAC reporter mice were provided by the RIKEN BRC through the National BioResource Project of the MEXT, Japan (Imayoshi et al., 2013). TauGFP NS cells were generated from ES cells expressing GFP from the Mapt (tau) locus (Bibel et al., 2004) as previously described (Conti et al., 2005). TauGFP ES and NSE (NS derived from E14 ES cells) cells were a gift from Jennifer Nichols (Cambridge Stem Cell Institute, UK). All reported cell lines tested negative for mycoplasma.
Cell transfection, differentiation and generation of lentivirus reporter cell lines
Request a detailed protocolC17.2 cells were transfected with Lipofectamine 2000 (ThermoFisher Scientific, UK) and TauGFP NS cells with Lipofectamine 3000 (ThermoFisher Scientific, UK) as per manufacturers’ instructions. 24 hr following transfection, C17.2 cells were differentiated by culture in DMEM with 0.2% FBS (Lundqvist et al., 2013) whereas TauGFP NS were differentiated towards neurons by gradual removal of the EGF and FGF2 growth factors (Pollard, 2013). miRVana miR9 mimic and control mimic were obtained from ThermoFisher Scientific, UK. pDsRedmiR9 Sensor was a gift from Lynn Hudson (Addgene plasmid # 22742) (Lau et al., 2008) and control sensor was pDsRed (Clontech, France). pEGFP N1 (Clontech, France) was used as a transfection control plasmid. To generate the VENUS:HES1 reporter constructs the mouse Hes1 coding (Accession Number NM008235) and 3’UTR sequence was synthesised with flanking Gateway® att recombination sites and inserted into pUC57 vector, resulting in the pUC57Hes1 vector. The Hes1 gene was then transferred to the 3rd generation ‘pLNTVenus:#’ lentiviral transfer by LR clonase recombination reaction (Bagnall et al., 2015). The resultant vector is termed pLNT UbCVENUS:HES1 and allows for constitutive expression of Nterminally fused HES1 using Ubiquitinligase C promoter (UbC). Additionally, the UbC promoter was replaced with a 2.7 kb sequence of the Hes1 promoter (upstream of the start codon) using Pac1 and Nhe1 restriction enzymes; the derived vector is termed pLNT 2.7 kbHes1prVENUS:HES1. Lentivirus production and transduction protocols were performed as described before (Bagnall et al., 2015). NSE UbCVENUS:HES1 cells were FACS sorted on BD Influx Cell Sorter (BD Biosciences, UK) to maximize the VENUS:HES1 positive population.
qRTPCR
Request a detailed protocolFor synchronised population qRTPCR, C17.2 cells were serumstarved for 24 hr. Following activation with 10% FBS and at timepoints of 30 min, cells were lysed and RNA extracted with TRizol (ThermoFisher Scientific, UK). cDNA was prepared using Superscript III (Invitrogen, UK) as per manufacturers’ instructions and qRTPCR for Hes1 and Gapdh was performed with Taqman gene expression assays (Hes1  Mm01342805 m1, GapdhMm03302249 g1, ThermoFisher, Scientific, UK). In order to obtain an absolute measurement of miR9 copy numbers we compared miR9 content of total RNA extractions of C17.2 cells with a standard curve of known miR9 dilutions. The standard curve was generated using the following synthetic mature miR9 sequences (Eurogentec); 23nt UCUUUGGUUAUCUAGCUGUAUGA, 22 nt UCUUUGGUUAUCUAGCUGUAUG, 21 nt UCUUUGGUUAUCUAGCUGUAU, 20 nt UCUUUGGUUAUCUAGCUGUA, 19 nt UCUUUGGUUAUCUAGCUGU (in order of abundance according to miRBase). Synthetic microRNA was 10X serially diluted to enable a standard curve of 1x10${}^{5}$ – 10 copies (5 data points). C17.2 cells were grown in four T25 cm${}^{2}$ tissue culture flasks (Corning, UK) in DMEM with 10% FBS to approx. 80% confluency ( 6x10${}^{5}$ cells). One flask was trypsinised (Trypsin EDTA Sigma, UK) and total cell number counted. The remaining flasks were lysed and total RNA extracted using Ambion miRVana kit (AM1561, ThermoFisher, UK) as per manufacturers’ instructions. The efficiency of RNA extraction was calculated as 67% and this was incorporated into subsequent copy number calculations. The standard curve samples and 50ng of each total RNA sample were reverse transcribed with the Applied Biosystems TaqMan microRNA Reverse Transcription kit (4366596) as per the manufacturers’ protocol. The primers used for reverse transcription and Taqman qRTPCR were the stemloop primer set for mouse miR95p (product 4427975, assay ID 001089, ThermoFisher, UK). TaqMan qRTPCR was performed using TaqMan Fast Advanced master mix reagent (Applied Biosystems 4444557) on Applied Biosystems StepONE plus real time PCR system.
Bioluminescence imaging
Request a detailed protocol200000 primary LUC2:HES1 NS cells up to passage 15 were plated on laminin (Sigma, UK) coated 35 mm glassbottom dishes (GreinerBio One) and imaged in NS proliferation media containing 1 mM Dluciferin (Promega, UK) using a 40x oil objective on an Olympus UK LV200 inverted bioluminescence microscope. 10minute exposure and 2x2 binning were used. Dishes were maintained at 37°C in 5% CO2. Bioluminescent movies were analysed on Imaris (RRID:SCR_007370)(Version 7.2.2, Bitplane). Images were first subjected to a 3x3 median filter to remove bright spot artefacts from cosmic rays, then individual cells were tracked manually using the 'spots' function.
Singlemolecule Fluorescent In Situ Hybridization (smFISH) and immunofluorescence
Request a detailed protocolA set of 29 Quasar 570 5’ end labeled 20 nt smFISH probes were designed against Hes1 exonic sequence, and 38 Quasar 670 5’ end labeled 18 nt smFISH probes designed against Hes1 intronic sequence (Biosearch Technologies, US). TauGFP NS cells were grown on coverslips in NS cell media (Pollard, 2013) for 72 hr at 37°C with 5% CO2. Coverslips were washed with 1XPBS, fixed for 20 min with 4% formaldehyde in 1XPBS, washed with 1XPBS, and permeabilized in 70% EtOH at 4°C for 1 hr. Cells were then washed for 5 min in 2XSSC+10% deionized formamide. smFISH probes were diluted to 100 nM in 200 ul hybridization buffer per coverslip (2XSSC, 10% deionized formamide, 10% dextran sulphate). Probes were hybridized for 5 hr in a humidified chamber at 37°C, then cells washed in 2XSSC+10% deionized formamide at 37°C for 30 min. Immunofluorescence was performed post hybridization, incubating with primary antibodies at 4°C for 15 hr, and with secondary antibody at room temperature for 1 hr. Coverslips were mounted using ProLong diamond antifade mountant with DAPI (P36962, ThermoFisher Scientific, UK). Primary antibodies: antiTuj1 (RRID:AB_444319)(ab18207, Abcam, UK), rabbit antiGFP (RRID:AB_10073917)(A11122, ThermoFisher Scientific, UK) and rabbit antiAuroraB kinase (RRID:AB_302923)(ab2254 Abcam, UK). Secondary antibody: goat antirabbit Alexa Fluor 488 (RRID:AB_143165)(A11008, ThermoFisher Scientific, UK).
smFISH imaging and analysis
Request a detailed protocolZstack images were acquired at 0.2 μm zincrements by Delta Vision widefield microscopy (Olympus IX71 microscope body, Coolsnap HQ CCD camera (Photometrics, US), API XYZ stage), using a 60X (1.42NA) objective. Image stacks were viewed using Imaris (Version 7.2.2, Bitplane), and cell areas segmented manually using background autofluorescence in the Quasar 570 channel on a maximum projection of the image. Hes1 mRNA spots per cell were counted by a semiautomated method, using the inbuilt 'analyze spots' function in Imaris. The following settings were used: estimated XY diameter = 0.3 μM, estimated Z diameter = 0.6 μM, background subtraction = yes, filter type = maximum intensity. The threshold cutoff between signal and background was placed manually at the ‘trough’ in the spots profile for the maximum intensity filter. Active transcription sites were identified manually, defined as Hes1 intronic smFISH spots within the nucleus that colocalised with exonic Hes1 smFISH spots.
Flow cytometry
Request a detailed protocolTauGFP NS cells were harvested at 96 hr, 120 hr, 144 hr and 168 hr of neuronal differentiation following transfection with 40 nM of miR9 mimic or control mimic. Cells were dissociated using accutase (Sigma, UK) and washed in DMEMF12 supplemented with 0.2% BSA. Flow cytometry analysis for GFP detection was performed on LSR Fortessa (BD Biosciences, UK) and data were analysed with the FlowJo software (RRID:SCR_008520)(TreeStar, US).
Fluorescence Correlation Spectroscopy (FCS) protein quantitation
Request a detailed protocolFCS was used to count the number of fluorescent protein molecules expressed in live NSE cells that were transduced with lentiviral VENUS:HES1 constructs. FCS involves laser excitation and data collection in a tightly focused microscopic region known as the confocal volume. Fluorescent particles diffusing in the confocal volume give rise to fluorescent intensity fluctuations $F(t)$ (example shown in Figure 2—figure supplement 1A) which are analysed over lag time $\tau \text{}\text{}0$ to produce the normalised autocorrelation function (Figure 2—figure supplement 1B):
The amplitude of the autocorrelation function is proportional to 1/c where c represents the number of molecules detected in the confocal volume, i.e. molecule concentration. The shape of $R(\tau )$ can be explained by models taking into consideration multiple species of fluorescent particles characterised by different diffusion constants. Preliminary model selection indicated that two components were required to explain the variability in our data (results not shown) and a nonfluorescent component was added to account for transitions of fluorescent particles to a triplet state. Normalised autocorrelations were fitted with a two component model with triplet state previously used for live cells (Dross et al., 2009; Bagnall et al., 2015).
FCS experiments were carried out using a Zeiss LSM880 microscope with a PlanApochromat 40x, 1.4 NA oilimmersion objective on cells cultured and maintained at 37^{o}C and $5\%$ CO2. Venus (EYFP) fluorescence was excited with 514 nm laser light and emission collected between 516 and 570 nm. Data from individual cell nuclei was collected using 5 x 5 s runs at 0.2 to 0.3% laser power for each measurement using Zen 2.1 (RRID:SCR_013672)(Zeiss) software. We used custom Matlab (RRID:SCR_001622) routines to fit the autocorrelation data (Figure 2—figure supplement 1B) and obtain protein concentration in the confocal volume. Measurements collected from cells with count per molecule lower than 500 kHz were excluded from the final results. Singlecell data of number of molecules in the cell nucleus was obtained by adjusting concentration to the average volumetric ratio between nuclear volume and confocal volume. NSE cells have a nuclear volume of mean 721 fL ± 105 fL (S.D.) estimated using nuclear staining and 3D reconstruction from zstack images in Imaris. The confocal volume had been previously determined with mean 0.57fL ± 11 fL (S.D.) (Bagnall et al., 2015).
Western blotting
Request a detailed protocolNuclear extracts of UbCVENUS:HES1 NSE cells treated with DMSO or MG132 (MerckMillipore, UK) for 3 hr were generated as described in Schreiber et al. (1989). Western blots were performed using 10% NUPAGEBisTris gels (ThermoFisher, UK) and Whatman Protran nitrocellulose membrane (Sigma, UK) according to manufacturer’s instructions. Antibodies used were antiHES1 (RRID:AB_590682)(D1343, MBL, Japan), antiLAMIN B1 (RRID:AB_443298)(ab16048, Abcam, UK), HRPlinked antirat IgG (RRID:AB_10694715)(7077, CST, UK) and HRPlinked antirabbit IgG (RRID:AB_2099233)(7074, CST, UK).
Statistical analysis
Request a detailed protocolData were analysed using pairwise hypothesis testing methods and multiple hypothesis tests as indicated in the figure legends using Prism 7 (RRID:SCR_002798 )(GraphPad, U.S.A). Statistical significance is reported for pvalues <0.05 (*), <0.01(**) and <0.001 (***). Errors are reported as standard error of the mean (S.E.M) calculated from pooled data taking into account standard deviation (S.D) and sample size (n) as S.E.M=S.D./sqrt(n).
Appendix
Model parameters
Deterministic dynamics
Starting with the microscopic dynamics of the Hes1/miR9 described in the Materials and Methods, we apply the system size expansion as described in Brett and Galla (Brett and Galla, 2013) and Galla (Galla, 2009). To lowest order in $\mathrm{\Omega}}^{1/2$, one recovers the deterministic dynamics described in Goodfellow et al. (Goodfellow et al., 2014)
Linear noise approximation
The nexttoleading order in $\mathrm{\Omega}}^{1/2$ represents the LNA and recovers the probability distribution describing random deviations in molecule numbers around the deterministic trajectory. The fluctuations in mRNA, protein and miR9 molecule number are denoted $\u03f5}_{m$, $\u03f5}_{p$ and $\u03f5}_{r$, respectively. In order to account for noise induced dynamics below the Hopf bifurcation, we assume that the deterministic dynamics reach a fixedpoint. When the deterministic dynamics have reached a fixedpoint (i.e. after large time), the dynamics of the fluctuations can be approximated by the linear Langevin equations
where ${G}^{\mathrm{\prime}}({p}^{\ast})$, ${S}^{\mathrm{\prime}}({r}^{\ast})$, ${F}^{\mathrm{\prime}}({r}^{\ast})$ and ${G}_{r}^{\mathrm{\prime}}({p}^{\ast})$ represent are linearisations of the Hill functions $G,S$ and ${G}_{r}$ around the deterministic fixed point, and where $\zeta (t)$ represents white noise. Within the linearnoise approximation this noise is assumed to be additive. At the deterministic fixed point, the noise correlators are given by
The LNA therefore describes a process with linear delay relaxation dynamics supplemented by additive noise. This constitutes a multivariate OrnsteinUhlenbeck process (with delay). To analyse this further, we carry out a Fourier transform (with respect to time), and obtain
In matrix form this can be represented as
where
It is now useful to introduce the matrix of spectra, $\underset{\_}{\underset{\_}{S}}(\omega )$, with ${S}_{ij}(\omega )=\u27e8\stackrel{~}{{\u03f5}_{i}}(\omega )\stackrel{~}{{\u03f5}_{j}}(\omega {)}^{\u2020}\u27e9$, where $i,j$ represent the variables $m,p$ and $r$ above. One then has
where $\u27e8\stackrel{~}{\mathit{\zeta}}(\omega )\stackrel{~}{\mathit{\zeta}}(\omega {)}^{\u2020}\u27e9$ is
The further analysis is then based on a numerical evaluation of Equation 35.
References

Chapman and Hall/CRCAn introduction to systems biology  design principles of biological circuits, Chapman and Hall/CRC, London, Taylor and Francis group..

Stochastic amplification in epidemicsJournal of the Royal Society Interface 4:575–582.https://doi.org/10.1098/rsif.2006.0192

A modified next reaction method for simulating chemical systems with time dependent propensities and delaysThe Journal of Chemical Physics 127:214107.https://doi.org/10.1063/1.2799998

Filtering transcriptional noise during development: concepts and mechanismsNature Reviews. Genetics 7:34–44.https://doi.org/10.1038/nrg1750

Nuclear Retention of mRNA in Mammalian TissuesCell Reports 13:2653–2662.https://doi.org/10.1016/j.celrep.2015.11.036

Bursty gene expression in the intact mammalian liverMolecular Cell 58:147–156.https://doi.org/10.1016/j.molcel.2015.01.027

Oscillatory regulation of Hes1: Discrete stochastic delay modelling and simulationPLoS Computational Biology 2:e117.https://doi.org/10.1371/journal.pcbi.0020117

Proneural genes and the specification of neural cell typesNature Reviews. Neuroscience 3:517–530.https://doi.org/10.1038/nrn874

Differentiation of mouse embryonic stem cells into a defined neuronal lineageNature Neuroscience 7:1003–1009.https://doi.org/10.1038/nn1301

Reconciling competence and transcriptional hierarchies with stochasticity in retinal lineagesCurrent Opinion in Neurobiology 27:68–74.https://doi.org/10.1016/j.conb.2014.02.014

Gaussian approximations for stochastic systems with delay: chemical Langevin equation and application to a Brusselator systemThe Journal of Chemical Physics 140:124112.https://doi.org/10.1063/1.4867786

Exact stochastic simulation of coupled chemical reactions with delaysThe Journal of Chemical Physics 126:124108.https://doi.org/10.1063/1.2710253

Realtime quantification of microRNAs by stemloop RTPCRNucleic Acids Research 33:e179.https://doi.org/10.1093/nar/gni178

Numerical bifurcation analysis of delay differential equations using DDEBIFTOOLACM Transactions on Mathematical Software 28:1–21.https://doi.org/10.1145/513001.513002

Towards a statistical mechanics of cell fate decisionsCurrent Opinion in Genetics & Development 22:619–626.https://doi.org/10.1016/j.gde.2012.10.004

Exact stochastic simulation of coupled chemical reactionsThe Journal of Physical Chemistry 81:2340–2361.https://doi.org/10.1021/j100540a008

The chemical Langevin equationThe Journal of Chemical Physics 113:297.https://doi.org/10.1063/1.481811

International Review of Cell and Molecular Biology273–321, Making of a Retinal Cell, International Review of Cell and Molecular Biology, Elsevier Inc, 10.1016/b9780128000977.000075.

Molecular logic of neocortical projection neuron specification, development and diversityNature Reviews. Neuroscience 14:755–769.https://doi.org/10.1038/nrn3586

How accurate are the nonlinear chemical FokkerPlanck and chemical Langevin equations?The Journal of Chemical Physics 135:084103.https://doi.org/10.1063/1.3625958

A study of the accuracy of momentclosure approximations for stochastic chemical kineticsThe Journal of Chemical Physics 136:154105.https://doi.org/10.1063/1.3702848

The cell biology of neurogenesisNature Reviews. Molecular Cell Biology 6:777–788.https://doi.org/10.1038/nrm1739

Systems biology of stem cells: three useful perspectives to help overcome the paradigm of linear pathwaysPhilosophical Transactions of the Royal Society B: Biological Sciences 366:2247–2259.https://doi.org/10.1098/rstb.2011.0008

Nongenetic heterogeneity from stochastic partitioning at cell divisionNature Genetics 43:95–100.https://doi.org/10.1038/ng.729

Stochasticity in gene expression: from theories to phenotypesNature Reviews. Genetics 6:451–464.https://doi.org/10.1038/nrg1615

Dynamic Notch signaling in neural progenitor cells and a revised view of lateral inhibitionNature Neuroscience 11:1247–1251.https://doi.org/10.1038/nn.2208

Universal patterns of stem cell fate in cycling adult tissuesDevelopment 138:3103–3111.https://doi.org/10.1242/dev.060103

Temporal fate specification and neural progenitor competence during developmentNature Reviews Neuroscience 14:823–838.https://doi.org/10.1038/nrn3618

Identification of dynamically regulated microRNA and mRNA networks in developing oligodendrocytesThe Journal of Neuroscience : The Official Journal of the Society for Neuroscience 28:11720–11730.https://doi.org/10.1523/JNEUROSCI.193208.2008

Predatorprey cycles from resonant amplification of demographic stochasticityPhysical Review Letters 94:218102.https://doi.org/10.1103/PhysRevLett.94.218102

Dissecting the dynamics of the Hes1 genetic oscillatorJournal of Theoretical Biology 254:784–798.https://doi.org/10.1016/j.jtbi.2008.07.013

The bHLH gene hes1 as a repressor of the neuronal commitment of CNS stem cellsJournal of Neuroscience 20:283–293.

The role of incoherent microRNAmediated feedforward loops in noise bufferingPLoS Computational Biology 7:e1001101.https://doi.org/10.1371/journal.pcbi.1001101

Specification of temporal identity in the developing nervous systemAnnual Review of Cell and Developmental Biology 20:619–647.https://doi.org/10.1146/annurev.cellbio.19.111301.115142

In vitro expansion of fetal neural progenitors as adherent cell linesMethods in Molecular Biology 1059:13–24.https://doi.org/10.1007/9781627035743_2

Timing and topography of cell genesis in the rat retinaThe Journal of Comparative Neurology 474:304–324.https://doi.org/10.1002/cne.20134

Cell dynamics and gene expression control in tissue homeostasis and developmentMolecular Systems Biology 11:792.https://doi.org/10.15252/msb.20145549

Cell lineage tree models of neurogenesisJournal of Theoretical Biology 256:164–179.https://doi.org/10.1016/j.jtbi.2008.09.034

Spatial stochastic modelling of the Hes1 gene regulatory network: intrinsic noise can explain heterogeneity in embryonic stem cell differentiationJournal of the Royal Society Interface 10:20120988.https://doi.org/10.1098/rsif.2012.0988

Origins and consequences of transcriptional discontinuityCurrent Opinion in Cell Biology 23:657–662.https://doi.org/10.1016/j.ceb.2011.09.004

MicroRNA9 regulates neural stem cell differentiation by controlling Hes1 expression dynamics in the developing brainGenes to Cells : Devoted to Molecular & Cellular Mechanisms 17:952–961.https://doi.org/10.1111/gtc.12009
Decision letter

Alejandro Sánchez AlvaradoReviewing Editor; Stowers Institute for Medical Research, United States
In the interests of transparency, eLife includes the editorial decision letter and accompanying author responses. A lightly edited version of the letter sent to the authors after peer review is shown, indicating the most substantive concerns; minor comments are not usually included.
Thank you for submitting your article "Stochasticity in the miR/Hes1 oscillatory network can account for clonal heterogeneity in the timing of differentiation" for consideration by eLife. Your article has been reviewed by three peer reviewers, one of whom, Alejandro Sánchez Alvarado, is a member of our Board of Reviewing Editors and the evaluation has been overseen by Aviv Regev as the Senior Editor. The following individuals involved in review of your submission have agreed to reveal their identity: William A Harris (Reviewer #2).
The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.
Summary:
Timing in biological systems is a fundamental topic that has a long history, and is enjoying resurgence due to improvements in microscopy and theory. Phillips et al. computationally model the effects of intrinsic noise on the Hes1/miR9 oscillator, and propose that the stochasticity generated by such noise is primarily caused by low molecular numbers of interacting species. The model offers a number of advantages to understand cell differentiation over the more widely accepted deterministic model. In this proposed stochastic/probabilistic model, the authors show that increased stochasticity spreads the timing of differentiation in a population, which potentially can account for clonal heterogeneity in a developing population of cells. This is a feature of real development that is not inherent in deterministic models of this network in which when such equivalent cells are modelled using differential equations, as in such models all cells differentiate at the same time. The authors also show that such stochastic models can add robustness to the oscillations by causing cells that might have differentiated due to near threshold levels of Hes1 to remain oscillating. The paper thus gives a detailed mathematical potential explanation for recent observations of clonal variability. It suggest that intrinsic stochasticity based of small numbers of interacting molecules is a basis for the kind of variable output in the timing of differentiation in neural progenitors.
Essential revisions:
The following issues need to be addressed fully in a revised submission.
1) The manuscript is at times confusing when it uses the different models. In 3.1 we are given a very gentle introduction to noise in molecular systems, but we are told nothing about the CME, dSSA and CLE. The clarity of the manuscript would benefit if each of these methods were given a onetwo sentence introduction, whether in 3.1, or when they first appear so that the reader can understand the logic of the approach. For example, it is critical when deterministic and stochastic methods are compared that the reader understands what it means for them to have the same parameters. We realize the methods and supplements contain this information and there, it is wellwritten and solid, but others may not read these closely because of the mathematics and for them it may be opaque.
2) The proposed model is just one of several possible explanations that can account for the variability in clone sizes or differentiation statistics. The authors might want to discuss these other models in more detail, particularly their comparative strengths and weaknesses. For example, the authors show that if a progenitor divides such that the two daughters inherit different proportions of the parent cell's small number of Hes1 molecules, this too can drive stochastic differentiation timings even when a deterministic model based on differential equations is applied. Another recently published model (Agarwal et al. 2016 in PLoS One) shows that the variability of nuclear position in neural progenitors, which are thought to have a gradient of intracellular Notch ICD, is sufficient to explain the stochastic spread in differentiation seen in the zebrafish retina. There are other models that can inject stochasticity into these systems, either intrinsically (e.g., things that change the cell cycle) or extrinsically (based on interactions between cells).
3) The authors show that increasing miR9 promotes neuronal differentiation by counting percentage of cells expressing Tuj1 or TauGFP in population level (Figure 1A and B). However, the real distribution of timetodifferentiation at the single cell level via direct experimental measurement is missing. Since this is one of the key advancements of the study, showing the live image of dynamic gene expression (Hes1) and the distribution of timing of cell fate transition during normal differentiation, at least in the cultured cell lines used, seems necessary. Therefore, it would make the paper much stronger if the authors could somehow test the model (or some key aspect of it) experimentally. If live imaging is not feasible, another approach, for example, might be to focus on a key parameter of the model, called omega (based on the concentration of molecules within a given volume). Perhaps one could predict omega in a particular cell type because the distribution of oscillations and differentiation times works well if the value of omega is within a certain range and then test what exactly omega is in these cells. Is it close to the predicted value? Or perhaps they could experimentally change the starting omega conditions (quantitatively change the amount of miR9 or Hes1) in a population of cells and see if if gives the predicted outcomes.
4) How the results in Figure 1AC are directly relevant to this paper is not clear. Do they add something that was not clear from previous work and/or are required to motivate the examination of stochastic effects?
5) Single cell trace in Figure 2A: we are shown one cell, but cannot ascertain if this is typical. What was the culture density? Was this cell in contact with other cells in culture? What is the system being modeled? The cell is used to illustrate the point that if you use a deterministic model it resembles biological data more poorly that a stochastic model. As such, this is a trivial statement. For single cell data to meaningfully inform the discussion or contribute to our understanding there would have to be some kind of quantification and analysis of the data of a population of cells to say whether it fit a particular stochastic description that could be used to argue about the properties of the system. Ideally, one would compare two competing stochastic descriptions to determine which was more useful.
6) We also wonder how much fitting has had to be done to get these stochastic models to work well. There are a number of parameters and starting conditions that are used. Can any of these be fixed by experimental observations? The more unfixed parameters and starting conditions there are, the weaker the model is.
7) In the Results section, the authors claim that "spontaneously differentiated cells were mostly negative for Hes1 mRNA transcript." However, of the seven cells outlined in Figure 1F, almost all of them are showing GFP signal if one takes a closer look at them. To support their conclusion that Hes1 expression is switched off upon differentiation, the authors need to characterize the differentiation of these cells using more specific antibodies or functional assay. Also, quantitative and timeseries measurements of the abundance of hes1 and miR9 expression, along with the tauGFP appearance in single cell level will help to demonstrate a direct relationship.
8) The power spectrum technique is strong and appears to be a novel contribution, yet it is wellhidden. It is an ongoing problem a noisy time series to answer the question whether it comes from an oscillator or a fluctuating, but nonoscillating system. It would therefore be helpful to introduce this in a clearer manner, and show how it works on the deterministic case first.
9) Figure 7 and the text describing it are difficult to follow. Are we right in thinking this is essentially describing a negative result? It might be better to describe this the other way around – first to say what is the effect of 5050 distribution at division for a deterministic model, and how this is perturbed by a binomial distribution. Then, show how the stochastic model is "protected" against the effect of binomial distribution.
10) We are somewhat confused by the distinction the authors make about the "advantageous" use of biological noise. According to the narrative, stochasticity introduces a variation in the differentiation time due to the miR timing mechanism (which is good), but then stochasticity reduces the variation in timing due to cell division (which apparently is also good). Cannot one say that the total variation in timing is influenced by the effect of stochasticity on both the miR timing circuit and distribution at division?
11) Overall, the data set and data analysis are of high quality and should be of great interest to the readership of eLife. The one shortcoming of this article, which is acknowledged by the senior author, is a lack of experimental evidence to support key corollaries of the model. The use of prior data and some new experiments to hone and refine the model is laudable and make this article a rigorous one. However, putting forward experimental evidence to support the functional importance of the shift of the timing of differentiation on clonal heterogeneity, e.g., heterogeneous expression of cell markers, would have been welcomed. In its present form, I remain unconvinced about the spread distribution of timetodifferentiation is caused by decreased system size (molecular copy number) in the real developmental context.
https://doi.org/10.7554/eLife.16118.042Author response
1) The manuscript is at times confusing when it uses the different models. In 3.1 we are given a very gentle introduction to noise in molecular systems, but we are told nothing about the CME, dSSA and CLE. The clarity of the manuscript would benefit if each of these methods were given a onetwo sentence introduction, whether in 3.1, or when they first appear so that the reader can understand the logic of the approach. For example, it is critical when deterministic and stochastic methods are compared that the reader understands what it means for them to have the same parameters. We realize the methods and supplements contain this information and there, it is wellwritten and solid, but others may not read these closely because of the mathematics and for them it may be opaque.
We apologise for this opaqueness, which was due to space limitations. We have rewritten this section, adding general information about the mathematical methods into the main body of the paper (Results section). Some sentences were moved forward from the Materials and methods section (with the Journal’s permission) to increase the word count of the main body of the paper. We hope that this will help to making all parts of the manuscript accessible to the experimentalist as well as the theoretician.
2) The proposed model is just one of several possible explanations that can account for the variability in clone sizes or differentiation statistics. The authors might want to discuss these other models in more detail, particularly their comparative strengths and weaknesses. For example, the authors show that if a progenitor divides such that the two daughters inherit different proportions of the parent cell's small number of Hes1 molecules, this too can drive stochastic differentiation timings even when a deterministic model based on differential equations is applied. Another recently published model (Agarwal et al. 2016 in PLoS One) shows that the variability of nuclear position in neural progenitors, which are thought to have a gradient of intracellular Notch ICD, is sufficient to explain the stochastic spread in differentiation seen in the zebrafish retina. There are other models that can inject stochasticity into these systems, either intrinsically (e.g., things that change the cell cycle) or extrinsically (based on interactions between cells).
We have done experiments to strengthen the proposed model (described below) but we agree that there are other possible explanations, which may not be mutually exclusive. Thank you for the suggestion of the Aqgarwal et al. paper. We note that this paper was purely theoretical i.e. no experimental evidence was provided for the proposed steep NICD intracellular gradient. The Agqarwal et al. paper also assumes that Hes1 only exists in an ON or OFF state, so it ignores the evidence that Hes1 oscillates. Nevertheless, some variant of their model may apply; we have added this reference. We also discuss noise introduced by asymmetries at cell division (Huh and Paulson, 2011) and the effect of noise in the Notch pathway (Jenkins, 2015). We note that future work can extend our model to multicellular systems. Please let us know if we are missing other relevant references and we will gladly add them.
3) The authors show that increasing miR9 promotes neuronal differentiation by counting percentage of cells expressing Tuj1 or TauGFP in population level (Figure 1A and B). However, the real distribution of timetodifferentiation at the single cell level via direct experimental measurement is missing. Since this is one of the key advancements of the study, showing the live image of dynamic gene expression (Hes1) and the distribution of timing of cell fate transition during normal differentiation, at least in the cultured cell lines used, seems necessary. Therefore, it would make the paper much stronger if the authors could somehow test the model (or some key aspect of it) experimentally. If live imaging is not feasible, another approach, for example, might be to focus on a key parameter of the model, called omega (based on the concentration of molecules within a given volume). Perhaps one could predict omega in a particular cell type because the distribution of oscillations and differentiation times works well if the value of omega is within a certain range and then test what exactly omega is in these cells. Is it close to the predicted value? Or perhaps they could experimentally change the starting omega conditions (quantitatively change the amount of miR9 or Hes1) in a population of cells and see if if gives the predicted outcomes.
We have taken this suggestion on board and we have added new experimental data to support our model. As suggested, we have focused on a key parameter of the model, which is the starting concentration of miR9. We have done a computational experiment to increase the starting concentration of miR9 in a population of cells. Alongside this, we have experimentally overexpressed a miR9 mimic or control mimic in C17.2 neural progenitor cells and TauGFP NS cells. We show by several methods (i.e. Tuj1 expression in fixed populations, flow cytometry analysis for GFP expression in TauGFP NS cells), that as predicted by our model, overexpression of miR9 brings neuronal differentiation forward in time. Importantly, we also show that the addition of miR9 reduces the time distribution to differentiation at the population level, which agrees with the stochastic model and is a novel feature of it. This shows that the stochastic model is able to make predictions that are consistent with experimental measurements. The new results are shown in Figure 6 and Figure 6—figure supplements 1 and 2 and are presented in a new section entitled “Increased miR9 concentration shifts timing and reduces spread of differentiation”.
4) How the results in Figure 1AC are directly relevant to this paper is not clear. Do they add something that was not clear from previous work and/or are required to motivate the examination of stochastic effects?
We apologise that this figure caused confusion. Part of it was intended to motivate this study and part was to support the model. We have generated a lot more experimental data to motivate and support this study, such that figure 1 has now been redesigned and split into two. The old Figure 1C has been moved to new Figure 1—figure supplement 1. The old Figure 1A has been moved to Figure 6—figure supplement 1 and old figure 1B has been omitted as it has been superseded by the flow cytometry analysis shown in Figure 6—figure supplement 2 (New Figure 6 is a new figure with new data addressing major point 3 above). Figure 1 now starts with a description of the smFISH design (Figure 1A). See also new text in subsection “A “finite number” stochastically simulated network is justified by experimental data”)
5) Single cell trace in Figure 2A: we are shown one cell, but cannot ascertain if this is typical. What was the culture density? Was this cell in contact with other cells in culture? What is the system being modeled? The cell is used to illustrate the point that if you use a deterministic model it resembles biological data more poorly that a stochastic model. As such, this is a trivial statement. For single cell data to meaningfully inform the discussion or contribute to our understanding there would have to be some kind of quantification and analysis of the data of a population of cells to say whether it fit a particular stochastic description that could be used to argue about the properties of the system. Ideally, one would compare two competing stochastic descriptions to determine which was more useful.
Yes, the reviewer is correct in saying that Figure 2A was used to illustrate the point that a deterministic model resembles the data more poorly than a stochastic one. We don’t’ think that this is trivial, because the explicit comparison of the stochastic and deterministic models with experimental data has not been made before for the Hes1 system. It will of course be familiar to those working on the Hes1 oscillator from inspection of their own raw data but other oscillators (such as circadian) are often more regular. Nevertheless, we took this point on board and deepened our analysis. We are now including the traces of 15 representative cells in Figure 3—figure supplement 2 and we provide cell culture information in the Materials and methods. We also provide Luciferase imaging snapshots of a time series in Figure 3—figure supplement 1.
Quantification and analysis of the data of a population of cells was already in the figure. We used qRTPCR as a way to obtain population level data and to show that they fit synthetic population data in the stochastic framework better than in a deterministic one. This is shown in Figure 3D,E,F.
6) We also wonder how much fitting has had to be done to get these stochastic models to work well. There are a number of parameters and starting conditions that are used. Can any of these be fixed by experimental observations? The more unfixed parameters and starting conditions there are, the weaker the model is.
Some of the parameters of the model, such as mRNA and protein degradation rates, have been experimentally measured either from our work or the work of others. Other parameters have not been directly measured, but have been fitted in order to reproduce dynamical behavior. We apologise that this was not clear. We have now improved the table of parameters by including textual description of the terms and also references to previous literature (see Supplementary file Table 1)
7) In the Results section, the authors claim that "spontaneously differentiated cells were mostly negative for Hes1 mRNA transcript." However, of the seven cells outlined in Figure 1F, almost all of them are showing GFP signal if one takes a closer look at them. To support their conclusion that Hes1 expression is switched off upon differentiation, the authors need to characterize the differentiation of these cells using more specific antibodies or functional assay. Also, quantitative and timeseries measurements of the abundance of hes1 and miR9 expression, along with the tauGFP appearance in single cell level will help to demonstrate a direct relationship.
To strengthen the data in Figure 1, we have done what the reviewer requested. We quantitated GFP signal intensity following Hes1 smFISH in TauGFP NS cells, which allowed us to separate the GFP negative and GFP positive cells based on the intensity of staining. The new Figure 1—figure supplement 3, shows a clear separation of negative and positive populations. The quantitation of Hes1 smFISH (Figure 1D,E) was then done on the GFP negative population (undifferentiated cells). To further support the conclusion that Hes1 is switched off upon differentiation we performed smFISH with an intronic Hes1 probes, which in combination with the Hes1 exonic probes, reveals the sites of transcription. These data showed that TauGFP positive cells are negative for Hes1 transcription. This is shown in Figure 1G, H, I and J.
To address the issue of abundance we have added new data on the absolute quantification of miR9 (by stem loop PCR) and HES1 protein (by Fluorescence Correlation Spectroscopy (FCS) and Western Blotting), The absolute copy numbers, expressed as a population average, are well below the global reported median, justifying the finite number approach. These are in Figure 1—figure supplements 4 and 5 (miR9 stem loop qRTPCR) and in Figure 2 (protein FCS and WB), new section in the Results and discussed in the Discussion.
8) The power spectrum technique is strong and appears to be a novel contribution, yet it is wellhidden. It is an ongoing problem a noisy time series to answer the question whether it comes from an oscillator or a fluctuating, but nonoscillating system. It would therefore be helpful to introduce this in a clearer manner, and show how it works on the deterministic case first.
We are pleased to see that the reviewers find that this is an ongoing problem, which needs to be addressed. We have developed a novel method for classifying stochastic time series as oscillatory or not but the full method is beyond the scope of this paper. A manuscript describing this method is currently under review in PLOS Comp Biology (“Identifying stochastic oscillations in single cell live imaging time series using Gaussian processes”, by Nick E. Phillips, Cerys Manning, Nancy Papalopulu and Magnus Rattray).
9) Figure 7 and the text describing it are difficult to follow. Are we right in thinking this is essentially describing a negative result? It might be better to describe this the other way around – first to say what is the effect of 5050 distribution at division for a deterministic model, and how this is perturbed by a binomial distribution. Then, show how the stochastic model is "protected" against the effect of binomial distribution.
We agree that this was difficult to follow; we have extensively rewritten the text (subsection “Robustness of timing to perturbations from asymmetric inheritance at cell division”) describing these results and we have resigned the figure to start with the 50:50 distribution of the deterministic system, as suggested by the reviewers. This is now new Figure 9 and we think that it is much clearer; thank you for your comment.
10) We are somewhat confused by the distinction the authors make about the "advantageous" use of biological noise. According to the narrative, stochasticity introduces a variation in the differentiation time due to the miR timing mechanism (which is good), but then stochasticity reduces the variation in timing due to cell division (which apparently is also good). Cannot one say that the total variation in timing is influenced by the effect of stochasticity on both the miR timing circuit and distribution at division?
Apologies that we presented this in a way that was not clear. We do say that stochasticity introduces a variation in the differentiation time due to the miR9 timing mechanism and this is good, as the reviewer says, because it allows time for neuronal diversity and feedback control (as we speculate in the Discussion). However, the reviewer has misunderstood the second part. We say that stochasticity does not change the variation in timing due to cell division. In fact, our data show that the effect of “finite number” noise and “partitioning at cell division” noise are not additive. This can be seen by comparing columns D (50:50 division) and E (random division) of the stochastic network, in Figure 9. This why we say that the stochastic system is “robust” to partitioning noise. We have extensively rewritten this Result section (subsection Robustness of timing to perturbations from asymmetric inheritance at cell division) as well as the Discussion and we hope this point is now clearer.
11) Overall, the data set and data analysis are of high quality and should be of great interest to the readership of eLife. The one shortcoming of this article, which is acknowledged by the senior author, is a lack of experimental evidence to support key corollaries of the model. The use of prior data and some new experiments to hone and refine the model is laudable and make this article a rigorous one. However, putting forward experimental evidence to support the functional importance of the shift of the timing of differentiation on clonal heterogeneity, e.g., heterogeneous expression of cell markers, would have been welcomed. In its present form, I remain unconvinced about the spread distribution of timetodifferentiation is caused by decreased system size (molecular copy number) in the real developmental context.
Thank you for finding our paper of high quality and great interest. We have done several new experiments to strengthen the paper, as detailed in our response to the major and minor points above. In particular, to support the finite number stochastic simulations, in addition to the previous Hes1 mRNA copy number (Figure 1), we are now presenting absolute quantitative data for miR9 (Figure 1—figure supplements 4 and 5) and HES1 protein abundance (Figure 2); both of these support the idea of low molecular copy number. In addition, we have added theoretical and experimental data to show that increasing the initial concentration of miR9 speeds up differentiation and reduces its timing spread (Figure 6, Figure 6—figure supplements 1 and 2).
https://doi.org/10.7554/eLife.16118.043Article and author information
Author details
Funding
Biotechnology and Biological Sciences Research Council (Doctoral Training Centre in Systems Biology)
 Nick E Phillips
Wellcome Trust (Sir Henry Wellcome Postdoctoral Fellowship. 103986/Z/14/Z)
 Cerys S Manning
Wellcome Trust (WT ISSF, Biomedical research Consortia grant, 097820/Z/11/B)
 Tom Pettini
Medical Research Council (MR/K015885/1)
 James Boyd
 David G Spiller
 Michael RH White
Biotechnology and Biological Sciences Research Council (David Phillips Fellowship, BB/I017976/1)
 Pawel Paszek
Biotechnology and Biological Sciences Research Council (BB/K003097/1)
 David G Spiller
 Michael RH White
Engineering and Physical Sciences Research Council (EP/N014391/1)
 Marc Goodfellow
Wellcome Trust (WT105618MA)
 Marc Goodfellow
Wellcome Trust (090868/Z/09/Z)
 Nancy Papalopulu
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Acknowledgements
This work was supported by a Wellcome Trust Senior Research Fellowship to NP (090868/Z/09/Z), a Sir Henry Wellcome Fellowship to CM (103986/Z/14/Z), a Wellcome Trust Institutional Strategic Support Award (097820/Z/11/B) and a BBSRC Doctoral Training Centre in Systems Biology studentship to NEP. PP holds a BBSRC David Phillips Research Fellowship (BB/I017976/1). MG gratefully acknowledges the financial support of the EPSRC via grant EP/N014391/1. The contribution of MG was generously supported by a Wellcome Trust Institutional Strategic Support Award (WT105618MA). J Boyd was funded by MRC grant MR/K015885/1. DS and MW’s work is funded by an MRC grant MR/K015885/1 and a BBSRC grant BB/K003097/1. The authors would also like to thank the Biological Services Facility (BSF), the Bioimaging and Flow Cytometry Facilities of the University of Manchester for technical support, in particular to Dr Gareth Howell for expert advice on challenging FACS sorts. We thank Dr. Ximena Soto for advice and discussions and Dr Angelica SantiagoGomez from Robert B Clarke’s group at the MCRC, Manchester for technical support and advice with western blotting. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Ethics
Animal experimentation: All animal work was performed under regulations set out by the UK Home Office Legislation under the 1986 United Kingdom Animal Scientific Procedures Act.
Reviewing Editor
 Alejandro Sánchez Alvarado, Stowers Institute for Medical Research, United States
Publication history
 Received: March 17, 2016
 Accepted: August 24, 2016
 Version of Record published: October 4, 2016 (version 1)
Copyright
© 2016, Phillips et al.
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.
Metrics

 2,843
 Page views

 563
 Downloads

 28
 Citations
Article citation count generated by polling the highest count across the following sources: Crossref, Scopus, PubMed Central.
Download links
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Further reading

 Computational and Systems Biology
 Neuroscience
The locus coeruleus (LC) houses the vast majority of noradrenergic neurons in the brain and regulates many fundamental functions including fight and flight response, attention control, and sleep/wake cycles. While efferent projections of the LC have been extensively investigated, little is known about its local circuit organization. Here, we performed largescale multipatch recordings of noradrenergic neurons in adult mouse LC to profile their morphoelectric properties while simultaneously examining their interactions. LC noradrenergic neurons are diverse and could be classified into two major morphoelectric types. While fast excitatory synaptic transmission among LC noradrenergic neurons was not observed in our preparation, these mature LC neurons connected via gap junction at a rate similar to their early developmental stage and comparable to other brain regions. Most electrical connections form between dendrites and are restricted to narrowly spaced pairs or small clusters of neurons of the same type. In addition, more than two electrically coupled cell pairs were often identified across a cohort of neurons from individual multicell recording sets that followed a chainlike organizational pattern. The assembly of LC noradrenergic neurons thus follows a spatial and cell typespecific wiring principle that may be imposed by a unique chainlike rule.

 Computational and Systems Biology
Computational models starting from large ensembles of evolutionarily related protein sequences capture a representation of protein families and learn constraints associated to protein structure and function. They thus open the possibility for generating novel sequences belonging to protein families. Protein language models trained on multiple sequence alignments, such as MSA Transformer, are highly attractive candidates to this end. We propose and test an iterative method that directly employs the masked language modeling objective to generate sequences using MSA Transformer. We demonstrate that the resulting sequences score as well as natural sequences, for homology, coevolution and structurebased measures. For large protein families, our synthetic sequences have similar or better properties compared to sequences generated by Potts models, including experimentallyvalidated ones. Moreover, for small protein families, our generation method based on MSA Transformer outperforms Potts models. Our method also more accurately reproduces the higherorder statistics and the distribution of sequences in sequence space of natural data than Potts models. MSA Transformer is thus a strong candidate for protein sequence generation and protein design.