Replication Study: BET bromodomain inhibition as a therapeutic strategy to target c-Myc

Abstract
Introduction
Results and discussion
Materials and methods
References
Article and author information
Metrics

Abstract

In 2015, as part of the Reproducibility Project: Cancer Biology, we published a Registered Report (Kandela et al., 2015) that described how we intended to replicate selected experiments from the paper "BET bromodomain inhibition as a therapeutic strategy to target c-Myc" (Delmore et al., 2011). Here we report the results of those experiments. We found that treatment of human multiple myeloma (MM) cells with the small-molecular inhibitor of BET bromodomains, (+)-JQ1, selectively downregulated MYC transcription, which is similar to what was reported in the original study (Figure 3B; Delmore et al., 2011). Efficacy of (+)-JQ1 was evaluated in an orthotopically xenografted model of MM. Overall survival was increased in (+)-JQ1 treated mice compared to vehicle control, similar to the original study (Figure 7E; Delmore et al., 2011). Tumor burden, as determined by bioluminescence, was decreased in (+)-JQ1 treated mice compared to vehicle control; however, while the effect was in the same direction as the original study (Figure 7C-D; Delmore et al., 2011), it was not statistically significant. The opportunity to detect a statistically significant difference was limited though, due to the higher rate of early death in the control group, and increased overall survival in (+)-JQ1 treated mice before the pre-specified tumor burden analysis endpoint. Additionally, we evaluated the (−)-JQ1 enantiomer that is structurally incapable of inhibiting BET bromodomains, which resulted in a minimal impact on MYC transcription, but did not result in a statistically significant difference in tumor burden or survival distributions compared to treatment with (+)-JQ1. Finally, we report meta-analyses for each result.

https://doi.org/10.7554/eLife.21253.001

Introduction

The Reproducibility Project: Cancer Biology (RP:CB) is a collaboration between the Center for Open Science and Science Exchange that seeks to address concerns about reproducibility in scientific research by conducting replications of selected experiments from a number of high-profile papers in the field of cancer biology (Errington et al., 2014). For each of these papers a Registered Report detailing the proposed experimental designs and protocols for the replications was peer reviewed and published prior to data collection. The present paper is a Replication Study that reports the results of the replication experiments detailed in the Registered Report (Kandela et al., 2015), for a paper by Delmore et al., and uses a number of approaches to compare the outcomes of the original experiments and the replications.

In 2011, Delmore et al. demonstrated that inhibition of BET bromodomains with a selective small-molecule, (+)-JQ1, down-regulated the c-Myc transcriptional signaling network and reduced tumor burden and prolonged survival in vivo indicating that targeting of BET bromodomains is an effective strategy to modulate c-Myc function in multiple myeloma (MM). Time-dependent downregulation of MYC was observed in a human MM cell line (MM.1S) treated with (+)-JQ1, in agreement with other examined MM cell lines (Delmore et al., 2011). Using a bioluminescent MM xenograft model (MM.1S-luc) daily treatment with (+)-JQ1 resulted in a statistically significant decrease in tumor burden and, importantly, increased overall survival compared to vehicle control treated animals.

The Registered Report for the paper by Delmore et al. described the experiments to be replicated (Figures 3B and 7C–E), and summarized the current evidence for these findings (Kandela et al., 2015). Since that publication there have been additional studies examining the therapeutic strategy of targeting BET bromodomains in other types of cancer. This includes reports of antitumor effects using BET bromodomain inhibitors in MM(Chaidos et al., 2014 , Siu et al., 2016), ovarian cancer (Zhang et al., 2016), gastric cancer (Montenegro et al., 2014), childhood sarcoma (Bid et al., 2016), and triple negative breast cancer (da Motta et al., 2016; Shu et al., 2016). Acquired resistance to BET inhibitors have also been reported (Fong et al., 2015; Kumar et al., 2015; Rathert et al., 2015), with recent studies suggesting combinatorial drug treatment to overcome resistance mechanisms (Asangani et al., 2016; Kurimchak et al., 2016; Yao et al., 2015). In addition to efficacy, the nonclinical safety of BET inhibition has also been examined. In mesenchymal stem cells, (+)-JQ1 was reported to induce cell cycle arrest and downregulation of genes involved in self-renewal, mitosis, and DNA replication (Alghamdi et al., 2016), while mice treated with (+)-JQ1 at an efficacious dose resulted in lymphoid and hematopoietic toxicity (Lee et al., 2016). Currently, several BET bromodomain inhibitors, with slight variation in mechanism, are in clinical trials for patients with various hematologic and solid malignancies (Chaidos et al., 2015; French, 2016; Wadhwa and Nicolaides, 2016). Early results from a phase one study to establish the recommended dose of the OTX015/MK-8628 BET inhibitor in hematologic malignancies reported the drug was tolerated; however, thrombocytopenia was a common toxic effect observed (Amorim et al., 2016). In four patients with advanced stage NUT midline carcinoma, with confirmed BRD4-NUT fusions, early clinical benefit was reported for two, with a third achieving disease stabilization after treatment with OTX015/MK-8628 (Stathis et al., 2016).

The outcome measures reported in this Replication Study will be aggregated with those from the other Replication Studies to create a dataset that will be examined to provide evidence about reproducibility of cancer biology research, and to identify factors that influence reproducibility more generally.

Results and discussion

Evaluation of MYC expression in JQ1-treated MM.1S-luc Cells

We sought to independently replicate an experiment analyzing the expression of endogenous MYC during pharmacological inhibition of BET bromodomains with (+)-JQ1. This experiment is similar to what was reported in Figure 3B (Delmore et al., 2011) and assesses the levels of MYC by quantitative reverse transcription polymerase chain reaction (qRT-PCR) in a human MM cell line stabling expressing luciferase (MM.1S-luc) (Mitsiades et al., 2004). While the original study included a time course treatment with assessments at 0 hr, 0.5 hr, 1 hr, 4 hr, and 8 hr, the replication was restricted to the early- (0 hr and 1 hr) and late-treatment (8 hr) time points. Additionally, the replication experiment was extended to include additional control conditions to monitor MYC expression with the inactive (−)-JQ1 enantiomer (Filippakopoulos et al., 2010) and vehicle control at the same time points. We found that during the time course of cells treated with (+)-JQ1, MYC expression was inhibited by 0.146 times [n = 5, SD = 0.031] at 1 hr and 0.126 times [n = 5, SD = 0.036] at 8 hr following treatment compared to the amount of MYC expressed in cells at 0 hr [n = 5, M = 1.00, SD = 0.070] (Figure 1). This compares to the original study, which reported an estimated relative MYC expression of ~0.069 times and ~0.088 times for cells treated with (+)-JQ1 at 1 hr and 8 hr, respectively (Delmore et al., 2011). In MM.1S-luc cells treated with (−)-JQ1 or vehicle, the average relative MYC expression remained visually largely unchanged following treatment, indicating selectivity of MYC down-regulation through inhibition of BET bromodomains.

Figure 1

Download asset Open asset

*MYC* expression in JQ1-treated MM.1S-luc cells.

MM.1S-luc cells were treated with 500 nM (+)-JQ1, 500 nM (−)-JQ1, or an equivalent volume of DMSO. Total RNA was isolated at 0 hr, 1 hr, and 8 hr after treatment and qRT-PCR analysis was performed to detect *MYC* and *GAPDH* levels. Relative expression (*MYC/GAPDH*) is presented for each time point and condition normalized to (+)-JQ1 treated cells at 0 hr. Means reported and error bars represent s.d. from five independent biological repeats. Mixed-design analysis of variance (ANOVA) with time (0 hr, 1 hr, and 8 hr) as the within-subjects factor and treatment ((+)-JQ1, (−)-JQ1, or vehicle) as the between-subjects factor; interaction effect: F(4,24) = 268.9, p=1.49x10⁻¹⁹, treatment main effect: F(2,12) = 393.5, p=1.15x10⁻¹¹, time main effect: F(2, 24) = 368.0, p=9.84x10⁻¹⁹. Planned paired t-test of MM.1S-luc cells harvested 8 hr after (+)-JQ1 treatment compared to cells 0 hr after (+)-JQ1 treatment; t(4) = 38.92, uncorrected p=2.60x10⁻⁶, *a priori* Bonferroni adjusted significance threshold = 0.025; (Bonferroni corrected p=5.21x10⁻⁶). Planned paired t-test of MM.1S-luc cells harvested 1 hr after (+)-JQ1 treatment compared to cells 0 hr after (+)-JQ1 treatment; t(4) = 25.10, uncorrected p=1.50x10⁻⁵, *a priori* Bonferroni adjusted significance threshold = 0.025; (Bonferroni corrected p=2.99x10⁻⁵). Additional details for this experiment can be found at https://osf.io/9swnx/.

https://doi.org/10.7554/eLife.21253.002

There are multiple approaches that could be taken to explore this data; however, to provide a direct comparison to the original analysis, we are reporting the analysis specified a priori in the Registered Report (Kandela et al., 2015). The mixed-design analysis of variance (ANOVA) result was statistically significant for all effects. A further analysis of the simple main effects for each treatment type revealed a statistically significant effect with (+)-JQ1 treatment (F(2,8) = 777.5, p=6.86x10⁻¹⁰), indicating that the mean MYC expression was different among the three time points evaluated. As outlined a priori in the Registered Report (Kandela et al., 2015), two planned paired t-tests were conducted as follow-up tests using the Bonferroni correction to adjust for multiple comparisons, making the significance threshold 0.025. The comparison of MM.1S-luc cells harvested 8 hr after (+)-JQ1 treatment compared to cells 0 hr after (+)-JQ1 treatment was statistically significant (t(4) = 38.92, uncorrected p=2.60x10⁻⁶, corrected p=5.21x10⁻⁶), as well as the comparison of MM.1S-luc cells harvested 1 hr after (+)-JQ1 treatment compared to cells 0 hr after (+)-JQ1 treatment (t(4) = 25.10, uncorrected p=1.50x10⁻⁵, corrected p=2.99x10⁻⁵). As expected from the data presented in Figure 1, examination of the simple main effect for vehicle control treatment was not statistically significant (F(2,8) = 1.05, p=0.394). Interestingly, though, treatment with (−)-JQ1 yielded a statistically significant effect (F(2,8) = 19.40, p=0.00085). While this indicates that the null hypothesis that there is no difference among the different time points can be rejected, the magnitude of the effect size in (−)-JQ1 treated cells, $η$ _G² = 0.698, 90% CI [0.444, 0.882], is considerably smaller than (+)-JQ1 treated cells, $η$ _G² = 0.989, 90% CI [0.980, 0.996]. Furthermore, MYC expression at 0 hr [n = 5, M = 1.028, SD = 0.040], 1 hr [n = 5, M = 0.928, SD = 0.034], and 8 hr [n = 5, M = 1.005, SD = 0.016] after treatment with (−)-JQ1 is in stark contrast to what was observed when MM.1S-luc cells were treated with (+)-JQ1 (Figure 1).

In addition to our confirmatory analyses described above, the data were also explored as outlined in the Registered Report (Kandela et al., 2015). Although this was a planned analysis, it is exploratory because it differs from the methods used in the original study by analyzing the data as unpaired instead of paired. All of the effects of the two-way between-subjects ANOVA were statistically significant (interaction effect, F(4,36) = 175.9, p=4.28x10⁻²³; treatment main effect, F(2,36) = 665.9, p=3.68x10⁻²⁹; time main effect, F(2,36) = 240.6, p=1.47x10⁻²¹). The planned contrast of MM.1S-luc cells harvested 8 hr after (+)-JQ1 treatment and cells 0 hr after (+)-JQ1 treatment was statistically significant (t(36) = 29.27, p=1.06x10⁻²⁶, Cohen’s d = 15.82, 95% CI [8.14, 23.51]). Similarly, the planned contrast of MM.1S-luc cells harvested 1 hr after (+)-JQ1 treatment compared to cells 0 hr after (+)-JQ1 treatment also yielded statistically significant results (t(36) = 29.96, p=4.80x10⁻²⁶, Cohen's d = 15.71, 95% CI [8.08, 23.35]). These results are in agreement with the mixed-design ANOVA and planned paired t-tests.

Testing the efficacy of JQ1 treatment in mice harboring bioluminescent MM lesions

In order to test the effectiveness of (+)-JQ1 as a therapeutic strategy to target c-Myc, we sought to replicate experiments similar to what was reported in Figure 7C–E (Delmore et al., 2011). This included evaluation of tumor burden in mice orthotopically xenografted, as well as analysis of the overall rate of survival. Similar to the in vitro experiment, the inactive (−)-JQ1 enantiomer, which was not included in the original study, was added as an additional control to evaluate the specificity of the approach. Female SCID-beige mice were inoculated intravenously with a previously characterized bioluminescent MM model (MM.1S-luc) that recapitulates the clinical disease and allows for in vivo monitoring of tumor burden (Mitsiades et al., 2004). A pilot study was conducted to determine the appropriate timescale of tumor progression during this replication attempt after an initial attempt to engraft mice resulted in a longer delay from inoculation to detection than expected and reported in the original study. During the pilot study, mice were inoculated and monitored for tumor burden 8 days and 13 days after inoculation as described in the Registered Report (Kandela et al., 2015). The mice continued to be monitored three times a week until 57 days after inoculation when disease symptoms (e.g. significant weight loss and hind limb paralysis) associated with this xenograft model were detected in multiple mice. The baseline that was observed in the original study eight days after inoculation (~2×10⁶ bioluminescence) was not observed until day 27 post-inoculation in the pilot study. Interestingly, other studies that utilized almost the same model (strain, sex, and age of mice as well as mode of injection) have also reported variable times to detectable engraftment. Tai and colleagues did not report baseline signals until two weeks after inoculation, granted 5×10⁶ cells were injected opposed to the 2×10⁶ cells injected in the original study and this replication attempt (Tai et al., 2014). Similarly, Zhang and colleagues did not observe this level of baseline bioluminescence until 2–3 weeks when injecting 3×10⁶ cells (Zhang et al., 2014). Additionally, two other studies reported variable times to detectable engraftment, however these studies utilized different mouse strains and sex. When male SCID/NOD mice were injected with 5×10⁶ cells, a baseline signal was not reported until two weeks after inoculation (Mitsiades et al., 2004), while Azab and colleagues, which inoculated 2×10⁶ MM.1S-luc cells, reported that it took 3–4 weeks for sufficient tumor progression to occur and be detected (Azab et al., 2009).

Following the modified time frame, mice were inoculated with 2×10⁶ MM.1S-luc cells and closely monitored until the desired baseline bioluminescence was achieved, which occurred on day 30 post-inoculation. After a further measurement five days later, all mice with established disease, defined as lesions diffusely engrafted in the skeleton with an increase in bioluminescence between days 30 and 35, were randomized into three cohorts. Following daily intraperitoneal injection of vehicle control, 50 mg/kg (+)-JQ1, or 50 mg/kg (−)-JQ1 for 36 days we assessed the overall survival rate of the animals (Figure 2). Mice treated with vehicle control achieved a median survival of 24.5 days during the treatment period [n = 12], which was lengthened during (+)-JQ1 treatment. The median survival of the (+)-JQ1 treated group could not be determined since more than half of the animals survived to the pre-specified study endpoint of 36 days. The median survival of mice treated with (−)-JQ1 [n = 12] was 29 days. This compares to the original study which achieved a median survival of 22 days with vehicle control [n = 10] that was prolonged to 35 days with (+)-JQ1 treatment [n = 9]. Furthermore, the median survival rates for vehicle treated mice are in agreement with control conditions in other studies reporting survival during treatment with this xenograft model (Mitsiades et al., 2004; Tai et al., 2014). To compare the survival distributions a log-rank (Mantel-Cox) test was performed. As outlined in the Registered Report (Kandela et al., 2015), we planned to conduct two comparisons using the Bonferroni correction to adjust for multiple comparisons making the significance threshold 0.025. The planned comparison between (+)-JQ1 treatment and vehicle control was statistically significant (uncorrected p=0.024, corrected p=0.047); however, the planned comparison between (+)-JQ1 and (−)-JQ1 treatment was not statistically significant (uncorrected p=0.093, corrected p=0.187). Interestingly, this result suggests the (−)-JQ1 enantiomer might possess pharmacological activity even though relative MYC expression remained largely unchanged following treatment in vitro (Figure 1). This could be due to how (−)-JQ1 interacts with and is metabolized with other molecules in vivo compared to (+)-JQ1, thus leading to unexpected effects (Chhabra et al., 2013; Nguyen et al., 2006).

Figure 2

Download asset Open asset

Overall survival in JQ1-treated MM.1S-luc orthotopic xenograft model.

Kaplan-Meier plot of overall survival during the pre-specified study period of 36 days. Female SCID-beige mice with established MM.1S-luc orthotopic xenografts were randomized to daily IP injections of 50 mg/kg (+)-JQ1, 50 mg/kg (−)-JQ1, or vehicle control. Number of mice: n = 12 for each group. Log-rank (Mantel-Cox) test of (+)-JQ1 treatment compared to vehicle control; uncorrected p=0.024, *a priori* Bonferroni adjusted alpha level = 0.025; (Bonferroni corrected p=0.047). Log-rank (Mantel-Cox) test of (+)-JQ1 compared to (−)-JQ1; uncorrected p=0.093, *a priori* Bonferroni adjusted alpha level = 0.025; (Bonferroni corrected p=0.187). Additional details for this experiment can be found at https://osf.io/pnvtd/.

https://doi.org/10.7554/eLife.21253.003

In addition to monitoring overall survival, we assessed tumor burden, as measured by bioluminescence imaging, during the first 22 days of treatment (Figure 3). Mice treated with vehicle control or (+)-JQ1 on day 22 reached an average bioluminescence of 2.73×10⁸ [n = 7, SD = 6.01×10⁸] and 9.90×10⁷ [n = 11, SD = 1.81×10⁸], respectively. This compares to an average bioluminescence of 1.85×10¹⁰ [n = 10, SD = 1.01×10¹⁰] in vehicle control treated mice and 5.52×10⁹ [n = 9, SD = 2.25×10⁹] in (+)-JQ1 treated mice reported in the original study (Delmore et al., 2011). Finally, tumor burden at day 22 in mice treated with (−)-JQ1 reached an average bioluminescence of 7.13×10⁷ [n = 9, SD = 5.87×10⁷]. To test if (+)-JQ1 treatment decreased the burden of disease, we performed a one-way ANOVA on the day 22 bioluminescence signal (natural log-transformed), which was not statistically significant, F(2,24) = 1.126, p=0.341. Additionally, as outlined in the Registered Report (Kandela et al., 2015), we planned to conduct two comparisons using Fisher's least significant difference (LSD) test. The planned comparison of tumor burden from mice treated with (+)-JQ1 compared to vehicle control was not statistically significant (t(24) = 1.303, p=0.205), nor was the planned comparison between (+)-JQ1 treatment and (−)-JQ1 (t(24) = 1.221, p=0.234). These results should take into consideration that some mice were euthanized due to disease progression before the prespecified tumor burden analysis endpoint of 22 days (Figure 3—figure supplement 1), which did not occur in the original study. This led to a sample size imbalance due to the preferential death of mice in the vehicle control group (5 of 12) and (-)-JQ1 group (3 of 12) compared to the (+)-JQ1 group (1 of 12). The higher rate of early death in the vehicle control and (-)-JQ1 groups is reflective of the increased overall survival observed in (+)-JQ1 treated mice (Figure 2). Besides leading to information loss, if the mice that were euthanized ended up having high tumor burdens, then the group averages reported in this replication attempt would be distorted, limiting the opportunity to detect a statistically significant difference (Laajala et al., 2012). This could be due to biological variability of the kinetics of engraftment and growth of systemically injected tumor cells, as well as differences in the details between the original experimental design and this replication attempt that were not known or accounted for.

Figure 3 with 1 supplement see all

Download asset Open asset

Tumor burden in JQ1-treated MM.1S-luc orthotopic xenograft model.

Female SCID-beige mice were orthotopically xenografted after intravenous injection with MM.1S-luc cells. Following detection of established disease (diffusely engrafted in the skeleton with an increase in bioluminescence), mice were randomly assigned to receive daily IP injections of 50 mg/kg (+)-JQ1, 50 mg/kg (−)-JQ1, or vehicle control (VEH). (A) Representative whole-body bioluminescence images of mice bearing MM.1S tumors 22 days after the start of the indicated treatment. (B) Line graph of tumor burden, as measured by whole-body bioluminescence, of tumor bearing mice during the course of the indicated treatment. Means reported and error bars represent s.e.m. The number of mice per condition at start of treatment (n = 12 for each group) and at day 22 ((+)-JQ1 = 11, (−)-JQ1 = 9, Vehicle = 7). One-way ANOVA on a natural log transformed day 22 bioluminescence signal; F(2,24) = 1.126, p=0.341. The pairwise contrast between (+)-JQ1 treatment and Vehicle; Fisher’s LSD test; t(24) = 1.303, p=0.205 with *a priori* alpha level = 0.05. The pairwise contrast between (+)-JQ1 and (−)-JQ1 treatment; Fisher’s LSD test; t(24) = 1.221, p=0.234 with *a priori* alpha level = 0.05. Additional details for this experiment can be found at https://osf.io/pnvtd/.

https://doi.org/10.7554/eLife.21253.004

There are a number of factors that can affect the assessment of tumor growth in vivo using bioluminescence imaging. The depth and location of the tumor as well as the thickness or color of the animal’s skin can alter the bioluminescent signal (Baba et al., 2007), as well as the choice of anesthesia used (Keyaerts et al., 2012). The route of administration and the timing of imaging after D-luciferin injections can also impact the signal. While this study injected D-luciferin by intraperitoneal injection, same as the original study, this route of injection can lead to variations in the signal due to variations in the rate of absorption across the peritoneum to reach the luciferase expressing cells (Close et al., 2011). This makes intravenous (Keyaerts et al., 2008) or subcutaneous (Khalil et al., 2013) administration of D-luciferin attractive alternatives. Further, the signal, and thus the perception of tumor burden, can also vary during longitudinal monitoring since the time course of the bioluminescence signal after D-luciferin injection changes with days after the luciferase expressing cells were inoculation (Inoue et al., 2010).

Meta-analyses of original and replicated effects

We performed a meta-analysis using a random-effects model to combine each of the effects described above as pre-specified in the confirmatory analysis plan (Kandela et al., 2015). To provide a standardized measure of the effect, a common effect size was calculated for each effect from the original and replication studies. Cohen’s dz is the standardized difference between the two correlated measurements using the standard deviation of the difference scores. While, Cohen's d is the standardized difference between two independent means using the pooled sample standard deviation. The hazard ratio (HR) is the ratio of the probability of a particular event, in this case death, in one group compared to the probability in another group.

There were two comparisons made with the qRT-PCR data evaluating MYC expression in (+)-JQ1 treated MM.1S-luc cells (Figure 4A). The comparison between (+)-JQ1 at 1 hr to (+)-JQ1 at 0 hr resulted in dz = 17.41, 95% CI [6.02, 28.41] in this study, which compares to dz = 2.48, 95% CI [−0.59, 5.79] for the data estimated a priori from Figure 3B in the original study assuming two biological repeats as stated in the methods (Delmore et al., 2011). A meta-analysis of these two effects resulted in dz = 8.99, 95% CI [−5.51, 23.50], which was not statistically significant, (p=0.224). The comparison between (+)-JQ1 at 8 hr to (+)-JQ1 at 0 hr resulted in dz = 11.22, 95% CI [3.84, 18.35] in this study, while the original study was dz = 2.41, 95% CI [−0.61, 5.63], assuming two biological repeats (Delmore et al., 2011). A meta-analysis of these two effects resulted in dz = 6.20, 95% CI [−2.35, 14.75], which was not statistically significant (p=0.155). For both comparisons the original and replication results are consistent when considering the direction of the effect, however the point estimates of the replication effect sizes were not within the confidence intervals of the original results, or vice versa. Further, the Cochran's Q test for heterogeneity was statistically significant in both meta-analyses (0 hr vs 1 hr, p=0.0089; 0 hr vs 8 hr, p=0.022), which along with a large confidence intervals around the weighted average effect sizes from the meta-analyses suggests heterogeneity between the original and replication studies. The calculation of the confidence intervals of the above study effect sizes utilize the number of biological repeats as the sample size and not the number of technical replicates. This is because while technical replicates are necessary to evaluate reliability of an assay, it is not independently repeated data and should not be used to properly test scientific hypotheses (Vaux et al., 2012).

Figure 4

Download asset Open asset

Meta-analyses of each effect.

Effect size and 95% confidence interval are presented for (Delmore et al., 2011), this replication study (RP:CB), and a random effects meta-analysis of those two effects. Sample sizes used in Delmore et al. (2011) and this replication attempt are reported under the study name. (A) Relative *MYC* expression in MM.1S-luc cells treated with 500 nM (+)-JQ1 for 8 hr compared to 0 hr (meta-analysis p=0.224), and the comparison between (+)-JQ1 treatment for 1 hr to 0 hr (meta-analysis p=0.155). (B) HR for mice treated daily with 50 mg/kg (+)-JQ1 compared to vehicle control (meta-analysis p=0.0112). (C) Tumor burden, determined by bioluminescence at day 22, in mice bearing MM.1S-luc tumors and treated daily with 50 mg/kg (+)-JQ1 or vehicle control (meta-analysis p=0.110). Additional details for this experiment can be found at https://osf.io/9snja/.

https://doi.org/10.7554/eLife.21253.006

A comparison of the overall survival distributions for mice treated with (+)-JQ1 or vehicle control resulted in a HR of 3.75, 95% CI [1.19, 11.81] for this study, compared to the original study which resulted in a calculated HR of 25.93, 95% CI [5.48, 122.58]. A meta-analysis of these two effects resulted in a HR of 13.12, 95% CI [1.79, 96.02] was statistically significant (p=0.0112). This indicates that the null hypothesis, that the overall survival between mice treated with (+)-JQ1 or vehicle control are not different, can be rejected (Figure 4B). Both results are consistent when considering the direction of the effect, however the point estimate of the replication effect size was not within the confidence interval of the original result, or vice versa.

The comparison of tumor burden, as determined by bioluminescence at day 22, between (+)-JQ1 or vehicle control treated mice bearing MM.1S-luc tumors resulted in d = 0.56, 95% CI [−0.42, 1.52] in this study, which can be compared to d = 2.30, 95% CI [1.10, 3.46] calculated from the data reported in Figure 7D of the original study (Delmore et al., 2011). A meta-analysis of these effects resulted in d = 1.39, 95% CI [−0.31, 3.09], which was not statistically significant (p=0.110) (Figure 4C). Both results are consistent when considering the direction of the effect, however the point estimate of the replication effect size was not within the confidence interval of the original result, or vice versa. The large confidence intervals of the meta-analysis along with a statistically significant Cochran’s Q test (p=0.035), suggests heterogeneity between the original and replication studies.

This direct replication provides an opportunity to understand the present evidence of these effects. Any known differences, including reagents and protocol differences, were identified prior to conducting the experimental work and described in the Registered Report (Kandela et al., 2015). However, this is limited to what was obtainable from the original paper and through communication with the original authors, which means there might be particular features of the original experimental protocol that could be critical, but unidentified. So while some aspects, such as cell line, strain and sex of mice, number of cells injected, and the drug dose was maintained, others were unknown or not easily controlled for. These include variables such as cell line genetic drift (Hughes et al., 2007; Kleensang et al., 2016), as well as subclonal drift in heterogeneous stable cells (Shearer and Saunders, 2015), circadian biological responses to therapy (Fu and Kettner, 2013), the microbiome of recipient mice (Macpherson and McCoy, 2015), housing temperature in mouse facilities (Kokolus et al., 2013), and differing compound potency resulting from different stock solutions (Kannt and Wieland, 2016). Whether these or other factors influence the outcomes of this study is open to hypothesizing and further investigation, which is facilitated by direct replications and transparent reporting.

Materials and methods

As described in the Registered Report (Kandela et al., 2015), we attempted a replication of the experiments reported in Figures 3B and 7C–E of (Delmore et al., 2011). A detailed description of all protocols can be found in the Registered Report (Kandela et al., 2015). Additional detailed experimental notes, data, and analysis are available on the Open Science Framework (OSF) (RRID:SCR_003238) (https://osf.io/7zqxp/; Aird et al., 2016).

Cell culture

Request a detailed protocol

MM.1S-luc cells (Dr. Andrew Kung, Columbia University Medical Center), expressing luciferase and neomycin after retroviral transduction with a pMMP-LucNeo vector and G418-selection, were previously described (Mitsiades et al., 2004). Cells were maintained in RPMI 1640 with 2 mM L-glutamine (Life Technologies, cat# 11875–093) supplemented with 10% Fetal Bovine Serum (FBS) (Life Technologies, cat# 16000–044), 100 U/ml penicillin, 50 $μ$ g/ml streptomycin, and 200 $μ$ g/ml G-418 sulfate (Life Technologies, cat# 10131–035) at 37°C in a humidified atmosphere at 5% CO₂. Quality control data for the MM.1S cell line is available at https://osf.io/r38p3/. This includes results confirming the cell line was free of mycoplasma contamination and common mouse pathogens. Additionally, STR DNA profiling of the cell line was performed and cells were confirmed to be the indicated cell line when queried against STR profile databases.

Therapeutic compounds

Request a detailed protocol

5 mg each of (+)-JQ1 and (−)-JQ1 (JQ1 enantiomer set, EMD Millipore, cat# 500586; in vitro studies, (+)-JQ1 enantiomer lot# D00147531, (−)-JQ1 enantiomer lot# D00147532; in vivo studies, (+)-JQ1 enantiomer lot# 2630921, (−)-JQ1 enantiomer lot# 2639083) was dissolved in 1.094 ml DMSO (ATCC, 4-x-5) to generate a 10 mM stock, which was filter sterilized, aliquoted, and stored at −20°C until use for in vitro experiments. For in vivo applications, one vial (5 mg) each of (+)-JQ1 and (−)-JQ1 was dissolved in 1 ml of 5% (w/v) dextrose in sterile water as pre-specified in the Registered Report, but after extensive vortexing, the solution remained milky unless filter sterilized through a 0.22 $μ$ m filter. The formulation was changed to diluting 5 mg of each compound in 80 µl of DMSO (Sigma-Aldrich, cat# D8418) and then further diluting to 1 ml by addition of 920 $μ$ l of 10% (w/v) 2-Hydroxypropyl- $β$ -cyclodextrin (HP- $β$ -CD) (Sigma-Aldrich, cat# H107-5G) dropwise while vortexing to give a final concentration of 5 mg/ml. Additional details are available at: http://bradner.dfci.harvard.edu/JQ1-userguide-updated%20sept%202011.pdf. The final concentration of DMSO was 8% (v/v). The vehicle was 10% HP- $β$ -CD with 8% DMSO (v/v). Aliquots were stored at −20°C until use with no repeated cycles of freeze-thaws.

Gene expression analysis

Request a detailed protocol

Cells were seeded at 8×10⁵ cells per 35 mm tissue culture dish in 1.8 ml growth medium. The next day 10 mM stock solutions of (+)-JQ1 and (−)-JQ1 were diluted 1:2000 in growth medium to 5 $μ$ M and 0.2 ml was added to medium in each dish to give a final concentration of 500 nM. For vehicle control, DMSO was diluted 1:2000 in growth medium (0.05% DMSO). After 0, 1, or 8 hr, cells were centrifuged and each plate was lysed in 200 $μ$ l TRI Reagent following manufacturer’s instructions. Purified RNA was dissolved in 20 $μ$ l RNase-free water and RNA concentration and purity was determined using an Agilent 2100 Bioanalyzer (quality control data available at https://osf.io/d2a5w/). Total RNA was reverse transcribed into cDNA using a First-Strand cDNA Synthesis Kit (GE Healthcare, cat# GE27-9261-01) following manufacturer’s instructions. Reaction volume was 21 $μ$ l and consisted of 7 $μ$ l first-strand reaction mix, 1 $μ$ l random hexamers (0.2 µg pd[N]6), 1 $μ$ l DTT (200 mM), and 12 $μ$ l RNA diluted in water. An initial attempt with 1 $μ$ g total RNA was tried, but the qRT-PCR results indicated that this was too high, and was reduced to 0.1 $μ$ g total RNA for all subsequent assays. A negative control contained 12 $μ$ l of water with no RNA template was also included. qRT-PCR was performed using a real-time PCR system (Applied Biosystems, cat# 7900HT). Reaction volumes were 20 $μ$ l and consisted of 10 $μ$ l 2X TaqMan Gene Expression Master Mix (Life Technologies, cat# 4369016), 1 $μ$ l 20X TaqMan Gene Expression Assay (Applied Biosystems, cat# 4331182) for either MYC (Hs00905030_m1) or GAPDH (Hs02758991_g1), 2 $μ$ l cDNA template, and 7 $μ$ l water. A negative control contained 9 $μ$ l water with no cDNA template was also included. Reactions were performed in technical triplicate with the following parameters: 2 min/50°C, 10 min/95°C, and 40 cycles of 15 s/95°C and 1 min/60°C. MYC transcript levels were normalized to GAPDH levels in each sample using the $Δ Δ$ Ct method.

Animals

All animal procedures were approved by the Northwestern University IACUC# IS00000556 and were in accordance with the Northwestern University’s policies on the care, welfare, and treatment of laboratory animals. No blinding occurred during the experiments.

Six-week old female SCID-beige mice (Charles River, strain code 250) were housed in cages up to five mice and offered Certified Rodent Diet (Harlan Teklad, cat # 7912) ad libitum. The animal room was set to maintain between 68–75 °F, a relative humidity of 30–70%, a minimum of 15 room air changes per hour, and a 12 hr light/dark cycle, which was interrupted for study-related activities.

Animals were checked twice within the first day after MM.1S tumor cell inoculation for mortality, abnormalities, and signs of pain or distress. Cage-side observations were done daily. Body weight (scale: Scout Pro O’Haus, Parsippany, New Jersey; model # SP202) was recorded weekly after inoculation and daily after JQ1 treatments.

MM.1S inoculation

Request a detailed protocol

After one week of acclimation, female SCID-beige mice were inoculated intravenously (IV) via the lateral tail vein with 2×10⁶ MM.1S-luc cells suspended in 200 µl Phosphate Buffered Saline (PBS) (Sigma-Aldrich, cat# D8537). For inoculation, cells were washed 2X with PBS and resuspended in PBS at 10⁷ cells/ml. Immediately prior to inoculation, D-luciferin (Promega, cat# P1042) was added to a separate dish of cells in culture and imaged in a Xenogen IVIS Spectrum (Caliper Life Sciences, Hopkinton, Massachusetts) to confirm expression of the luciferase construct (image available at https://osf.io/4a2jw/).

A pilot study was conducted to establish the timescale of tumor progression that would achieve a similar tumor burden, as measured by whole-body bioluminescence (~2×10⁶), as observed in the original study after an initial attempt to engraft mice was unsuccessful in detecting bioluminescence even though disease progression (hind limb paralysis) was observed. For the pilot study, five mice were inoculated with MM.1S cells. Eight days after inoculation mice were injected intraperitoneally (IP) with 75 mg/kg of D-luciferin in 150 $μ$ l PBS and imaged 15 min later under isoflurane anesthesia in a Xenogen IVIS Spectrum. To monitor tumor progression, imaging was repeated at day 13 and 15 after inoculation and thereafter three times a week until day 57. The target bioluminescence of ~2×10⁶ was achieved at day 27 after inoculation. Pilot study and initial attempt data available at https://osf.io/jse9q/.

A total of thirty-eight female SCID-beige mice were inoculated with MM.1S cells with the aim of randomizing mice into different treatment groups after the target bioluminescence of ~2×10⁶ was achieved. Fourteen days after inoculation mice were injected intraperitoneally (IP) with 75 mg/kg of D-luciferin in 150 $μ$ l PBS and imaged 30 min later under isoflurane anesthesia in a Xenogen IVIS Spectrum. To monitor tumor progression, imaging was repeated at day 21, 27, and 30 after inoculation until the mean bioluminescence reached the target of ~2×10⁶, which was observed on day 30. Similar to the original study, imaging was repeated five days later and the difference in bioluminescence (day 35 minus day 30) in each mouse was calculated. Thirty-six mice showed established disease (lesions diffusely engrafted in the skeleton with an increase in bioluminescence between day 30 and 35) and were randomized into three groups using the alternating serpentine method. Randomization was consistent with even distribution across all groups (one-way ANOVA: F(2,33) = 0.027, p=0.973). Mice were injected daily with (+)-JQ1, (−)-JQ1, or vehicle control for 36 days. Tumor progression was monitored at day 1, 7, 22, and 29 of treatment, and survival was monitored until day 36.

JQ1 administration

Request a detailed protocol

Mice were weighed daily and injected IP daily with (+)-JQ1 or (−)-JQ1 (5 mg/ml) in a volume of 10 $μ$ l/g body weight to achieve a final dose of 50 mg/kg. Vehicle (10% HP- $β$ -CD, 8% DMSO) was injected IP in a volume of 10 $μ$ l/g body weight.

IVIS imaging

Request a detailed protocol

Images were acquired in a Xenogen IVIS Spectrum at a medium binning level (8) and a 22.6 cm field of view. Acquisition times were set to auto-exposure, and ranged from 60 to 0.5 s depending on the intensity of luminescence. Living Image software (RRID:SCR_014247), version 4.3, was used for quantitative analysis of the bioluminescence signal. Image analysis files are available at https://osf.io/jse9q/.

Statistical analysis

Request a detailed protocol

Statistical analysis was performed with R software (RRID:SCR_001905), version 3.2.3 ( R Core Team, 2016). All data, csv files, and analysis scripts are available on the OSF (https://osf.io/7zqxp/). Confirmatory statistical analysis was pre-registered (https://osf.io/ng3st/) before the experimental work began as outlined in the Registered Report (Kandela et al., 2015). Data were checked to ensure assumptions of statistical tests were met. When described in the results, the Bonferroni correction, to account for multiple testings, was applied to the alpha error or the p-value. The Bonferroni corrected value was determined by dividing the uncorrected value by the number of tests performed. For the alpha error this resulted in an a priori significance threshold of 0.025 (0.05/2). Although the Bonferroni method is conservative, it was accounted for in the power calculations to ensure sample size was sufficient. In cases where the number of groups were three and the sample sizes were evenly distributed among the groups, Fisher's LSD test was performed resulting in an a priori significance threshold of 0.05. For the qRT-PCR data analysis, we performed a mixed-design analysis of variance (ANOVA) on normalized MYC expression with time (0 hr, 1 hr, and 8 hr) as the within-subjects factor and treatment ((+)-JQ1, (−)-JQ1, or vehicle) as the between-subjects factor with an a priori significance threshold of 0.05. A meta-analysis of a common original and replication effect size was performed with a random effects model and the metafor R package (Viechtbauer, 2010) (available at https://osf.io/9snja/). The original study data presented in Figure 3B was extracted a priori from the published figure by determining the mean and upper/lower error values for each data point, while the raw data pertaining to Figure 7D,E were shared by the original authors. The summary data was published in the Registered Report (Kandela et al., 2015) and was used in the power calculations to determine the sample size for this study.

Deviations from registered report

Request a detailed protocol

The vehicle formulation for in vivo applications was changed from what is listed in the Registered Report. The Registered Report listed it as 5% dextrose in water (D5W) while the replication experiment used 10% HP- $β$ -CD with 8% DMSO (v/v). The vehicle listed in the original study was indicated as D5W, however 10% HP- $β$ -CD, 8% DMSO is described here as a vehicle which improves the solubility of the compound. Additional materials and instrumentation not listed in the Registered Report, but needed during experimentation are also listed.

An initial attempt to inoculate 33 animals with MM.1S-luc cells as outlined in the Registered Report was unsuccessful in detecting bioluminescence even though disease progression (hind limb paralysis) was observed and the MM.1S-luc cells had a strong luminescent signal prior to injection. MM.1S-luc cells were grown under antibiotic selection to enrich for luciferase expressing cells and a pilot study was designed to closely monitor tumor progression and establish the timescale required to achieve a similar tumor burden as that observed in the original study. Following this modification, which is reported here, the number of mice to inoculate was increased to 38 and monitored until the desired bioluminescence was achieved, which occurred on day 30 post-inoculation, after which the remaining aspects of the protocol were carried out.

References

1. Aird F
2. Kandela I
3. Mantis C
4. Iorns E
5. Denis A
6. Williams SR
7. Perfito N
8. Errington TM
(2016) Study 19: replication of delmore et al., 2011 (Cell)
Open Science Framework.

https://doi.org/10.17605/OSF.IO/7ZQXP
- Google Scholar
1. Alghamdi S
2. Khan I
3. Beeravolu N
4. McKee C
5. Thibodeau B
6. Wilson G
7. Chaudhry GR
(2016) BET protein inhibitor JQ1 inhibits growth and modulates WNT signaling in mesenchymal stem cells
Stem Cell Research & Therapy 7:22.

https://doi.org/10.1186/s13287-016-0278-3
- PubMed
- Google Scholar
1. Amorim S
2. Stathis A
3. Gleeson M
4. Iyengar S
5. Magarotto V
6. Leleu X
7. Morschhauser F
8. Karlin L
9. Broussais F
10. Rezai K
11. Herait P
12. Kahatt C
13. Lokiec F
14. Salles G
15. Facon T
16. Palumbo A
17. Cunningham D
18. Zucca E
19. Thieblemont C
(2016) Bromodomain inhibitor OTX015 in patients with lymphoma or multiple myeloma: a dose-escalation, open-label, pharmacokinetic, phase 1 study
The Lancet Haematology 3:e196–e204.

https://doi.org/10.1016/S2352-3026(16)00021-1
- PubMed
- Google Scholar
(2016) BET Bromodomain Inhibitors enhance efficacy and disrupt resistance to AR antagonists in the treatment of prostate cancer
Molecular Cancer Research 14:324–331.

https://doi.org/10.1158/1541-7786.MCR-15-0472
- PubMed
- Google Scholar
1. Azab AK
2. Runnels JM
3. Pitsillides C
4. Moreau AS
5. Azab F
6. Leleu X
7. Jia X
8. Wright R
9. Ospina B
10. Carlson AL
11. Alt C
12. Burwick N
13. Roccaro AM
14. Ngo HT
15. Farag M
16. Melhem MR
17. Sacco A
18. Munshi NC
19. Hideshima T
20. Rollins BJ
21. Anderson KC
22. Kung AL
23. Lin CP
24. Ghobrial IM
(2009) CXCR4 inhibitor AMD3100 disrupts the interaction of multiple myeloma cells with the bone marrow microenvironment and enhances their sensitivity to therapy
Blood 113:4341–4351.

https://doi.org/10.1182/blood-2008-10-186668
- PubMed
- Google Scholar
1. Baba S
2. Cho SY
3. Ye Z
4. Cheng L
5. Engles JM
6. Wahl RL
(2007)
How reproducible is bioluminescent imaging of tumor cell growth? Single time point versus the dynamic measurement approach

Molecular Imaging 6:315–322.
- PubMed
- Google Scholar
1. Bid HK
2. Phelps DA
3. Xaio L
4. Guttridge DC
5. Lin J
6. London C
7. Baker LH
8. Mo X
9. Houghton PJ
(2016) The bromodomain BET Inhibitor JQ1 suppresses tumor angiogenesis in models of childhood sarcoma
Molecular Cancer Therapeutics 15:1018–1028.

https://doi.org/10.1158/1535-7163.MCT-15-0567
- PubMed
- Google Scholar
1. Chaidos A
2. Caputo V
3. Gouvedenou K
4. Liu B
5. Marigo I
6. Chaudhry MS
7. Rotolo A
8. Tough DF
9. Smithers NN
10. Bassil AK
11. Chapman TD
12. Harker NR
13. Barbash O
14. Tummino P
15. Al-Mahdi N
16. Haynes AC
17. Cutler L
18. Le B
19. Rahemtulla A
20. Roberts I
21. Kleijnen M
22. Witherington JJ
23. Parr NJ
24. Prinjha RK
25. Karadimitris A
(2014) Potent antimyeloma activity of the novel bromodomain inhibitors I-BET151 and I-BET762
Blood 123:697–705.

https://doi.org/10.1182/blood-2013-01-478420
- PubMed
- Google Scholar
(2015) Inhibition of bromodomain and extra-terminal proteins (BET) as a potential therapeutic approach in haematological malignancies: emerging preclinical and clinical evidence
Therapeutic Advances in Hematology 6:128–141.

https://doi.org/10.1177/2040620715576662
- PubMed
- Google Scholar
(2013) A review of drug isomerism and its significance
International Journal of Applied and Basic Medical Research 3:16–8.

https://doi.org/10.4103/2229-516X.112233
- PubMed
- Google Scholar
1. Close DM
2. Xu T
3. Sayler GS
4. Ripp S
(2011) In vivo bioluminescent imaging (BLI): noninvasive visualization and interrogation of biological processes in living animals
Sensors 11:180–206.

https://doi.org/10.3390/s110100180
- PubMed
- Google Scholar
1. da Motta LL
2. Ledaki I
3. Purshouse K
4. Haider S
5. De Bastiani MA
6. Baban D
7. Morotti M
8. Steers G
9. Wigfield S
10. Bridges E
11. Li JL
12. Knapp S
13. Ebner D
14. Klamt F
15. Harris AL
16. McIntyre A
(2016) The BET inhibitor JQ1 selectively impairs tumour response to hypoxia and downregulates CA9 and angiogenesis in triple negative breast cancer
Oncogene pp. 1–11.

https://doi.org/10.1038/onc.2016.184
- PubMed
- Google Scholar
1. Delmore JE
2. Issa GC
3. Lemieux ME
4. Rahl PB
5. Shi J
6. Jacobs HM
7. Kastritis E
8. Gilpatrick T
9. Paranal RM
10. Qi J
11. Chesi M
12. Schinzel AC
13. McKeown MR
14. Heffernan TP
15. Vakoc CR
16. Bergsagel PL
17. Ghobrial IM
18. Richardson PG
19. Young RA
20. Hahn WC
21. Anderson KC
22. Kung AL
23. Bradner JE
24. Mitsiades CS
(2011) BET bromodomain inhibition as a therapeutic strategy to target c-Myc
Cell 146:904–917.

https://doi.org/10.1016/j.cell.2011.08.017
- PubMed
- Google Scholar
1. Errington TM
2. Iorns E
3. Gunn W
4. Tan FE
5. Lomax J
6. Nosek BA
(2014) An open investigation of the reproducibility of cancer biology research
eLife 3:e04333.

https://doi.org/10.7554/eLife.04333
- PubMed
- Google Scholar
1. Filippakopoulos P
2. Qi J
3. Picaud S
4. Shen Y
5. Smith WB
6. Fedorov O
7. Morse EM
8. Keates T
9. Hickman TT
10. Felletar I
11. Philpott M
12. Munro S
13. McKeown MR
14. Wang Y
15. Christie AL
16. West N
17. Cameron MJ
18. Schwartz B
19. Heightman TD
20. La Thangue N
21. French CA
22. Wiest O
23. Kung AL
24. Knapp S
25. Bradner JE
(2010) Selective inhibition of BET bromodomains
Nature 468:1067–1073.

https://doi.org/10.1038/nature09504
- PubMed
- Google Scholar
1. Fong CY
2. Gilan O
3. Lam EY
4. Rubin AF
5. Ftouni S
6. Tyler D
7. Stanley K
8. Sinha D
9. Yeh P
10. Morison J
11. Giotopoulos G
12. Lugo D
13. Jeffrey P
14. Lee SC
15. Carpenter C
16. Gregory R
17. Ramsay RG
18. Lane SW
19. Abdel-Wahab O
20. Kouzarides T
21. Johnstone RW
22. Dawson SJ
23. Huntly BJ
24. Prinjha RK
25. Papenfuss AT
26. Dawson MA
(2015) BET inhibitor resistance emerges from leukaemia stem cells
Nature 525:538–542.

https://doi.org/10.1038/nature14888
- PubMed
- Google Scholar
1. French CA
(2016) Small-Molecule Targeting of BET Proteins in Cancer
Advances in Cancer Research 131:21–258.

https://doi.org/10.1016/bs.acr.2016.04.001
- PubMed
- Google Scholar
1. Fu L
2. Kettner NM
(2013) The circadian clock in cancer development and therapy
Progress in Molecular Biology and Translational Science 119:221–282.

https://doi.org/10.1016/B978-0-12-396971-2.00009-9
- PubMed
- Google Scholar
1. Hughes P
2. Marshall D
3. Reid Y
4. Parkes H
5. Gelber C
(2007) The costs of using unauthenticated, over-passaged cell lines: how much more data do we need?
BioTechniques 43:575–586.

https://doi.org/10.2144/000112598
- PubMed
- Google Scholar
1. Inoue Y
2. Kiryu S
3. Watanabe M
4. Tojo A
5. Ohtomo K
(2010) Timing of imaging after D-Luciferin injection affects the longitudinal assessment of tumor growth using in vivo bioluminescence imaging
International Journal of Biomedical Imaging 2010:1–6.

https://doi.org/10.1155/2010/471408
- Google Scholar
(2015) Registered report: BET bromodomain inhibition as a therapeutic strategy to target c-Myc
eLife 4:e07072.

https://doi.org/10.7554/eLife.07072
- PubMed
- Google Scholar
1. Kannt A
2. Wieland T
(2016) Managing risks in drug discovery: reproducibility of published findings
Naunyn-Schmiedeberg's Archives of Pharmacology 389:353–360.

https://doi.org/10.1007/s00210-016-1216-8
- PubMed
- Google Scholar
1. Keyaerts M
2. Remory I
3. Caveliers V
4. Breckpot K
5. Bos TJ
6. Poelaert J
7. Bossuyt A
8. Lahoutte T
(2012) Inhibition of firefly luciferase by general anesthetics: effect on in vitro and in vivo bioluminescence imaging
PLoS One 7:e30061.

https://doi.org/10.1371/journal.pone.0030061
- PubMed
- Google Scholar
(2008) Dynamic bioluminescence imaging for quantitative tumour burden assessment using IV or IP administration of D: -luciferin: effect on intensity, time kinetics and repeatability of photon emission
European Journal of Nuclear Medicine and Molecular Imaging 35:999–1007.

https://doi.org/10.1007/s00259-007-0664-2
- PubMed
- Google Scholar
1. Khalil AA
2. Jameson MJ
3. Broaddus WC
4. Chung TD
5. Golding SE
6. Dever SM
7. Rosenberg E
8. Valerie K
(2013) Subcutaneous administration of D-luciferin is an effective alternative to intraperitoneal injection in bioluminescence imaging of xenograft tumors in nude mice
ISRN Molecular Imaging 2013:689279.

https://doi.org/10.1155/2013/689279
- PubMed
- Google Scholar
1. Kleensang A
2. Vantangoli MM
3. Odwin-DaCosta S
4. Andersen ME
5. Boekelheide K
6. Bouhifd M
7. Fornace AJ
8. Li HH
9. Livi CB
10. Madnick S
11. Maertens A
12. Rosenberg M
13. Yager JD
14. Zhaog L
15. Hartung T
(2016) Genetic variability in a frozen batch of MCF-7 cells invisible in routine authentication affecting cell function
Scientific Reports 6:28994.

https://doi.org/10.1038/srep28994
- PubMed
- Google Scholar
1. Kokolus KM
2. Capitano ML
3. Lee CT
4. Eng JW
5. Waight JD
6. Hylander BL
7. Sexton S
8. Hong CC
9. Gordon CJ
10. Abrams SI
11. Repasky EA
(2013) Baseline tumor growth and immune control in laboratory mice are significantly influenced by subthermoneutral housing temperature
PNAS 110:20176–20181.

https://doi.org/10.1073/pnas.1304291110
- PubMed
- Google Scholar
1. Kumar K
2. Raza SS
3. Knab LM
4. Chow CR
5. Kwok B
6. Bentrem DJ
7. Popovic R
8. Ebine K
9. Licht JD
10. Munshi HG
(2015) GLI2-dependent c-MYC upregulation mediates resistance of pancreatic cancer cells to the BET bromodomain inhibitor JQ1
Scientific Reports 5:9489.

https://doi.org/10.1038/srep09489
- Google Scholar
1. Kurimchak AM
2. Shelton C
3. Duncan KE
4. Johnson KJ
5. Brown J
6. O'Brien S
7. Gabbasov R
8. Fink LS
9. Li Y
10. Lounsbury N
11. Abou-Gharbia M
12. Childers WE
13. Connolly DC
14. Chernoff J
15. Peterson JR
16. Duncan JS
17. O’Brien S
(2016) Resistance to BET bromodomain inhibitors is mediated by kinome reprogramming in ovarian cancer
Cell Reports 16:1273–1286.

https://doi.org/10.1016/j.celrep.2016.06.091
- PubMed
- Google Scholar
(2012) Improved statistical modeling of tumor growth and treatment effect in preclinical animal studies with highly heterogeneous responses in vivo
Clinical Cancer Research 18:4385–4396.

https://doi.org/10.1158/1078-0432.CCR-11-3215
- PubMed
- Google Scholar
1. Lee DU
2. Katavolos P
3. Palanisamy G
4. Katewa A
5. Sioson C
6. Corpuz J
7. Pang J
8. DeMent K
9. Choo E
10. Ghilardi N
11. Diaz D
12. Danilenko DM
(2016) Nonselective inhibition of the epigenetic transcriptional regulator BET induces marked lymphoid and hematopoietic toxicity in mice
Toxicology and Applied Pharmacology 300:47–54.

https://doi.org/10.1016/j.taap.2016.03.013
- PubMed
- Google Scholar
1. Macpherson AJ
2. McCoy KD
(2015) Standardised animal models of host microbial mutualism
Mucosal Immunology 8:476–486.

https://doi.org/10.1038/mi.2014.113
- PubMed
- Google Scholar
(2004) Inhibition of the insulin-like growth factor receptor-1 tyrosine kinase activity as a therapeutic strategy for multiple myeloma, other hematologic malignancies, and solid tumors
Cancer Cell 5:221–230.

https://doi.org/10.1016/S1535-6108(04)00050-9
- PubMed
- Google Scholar
1. Montenegro RC
2. Clark PGK
3. Howarth A
4. Wan X
5. Ceroni A
6. Siejka P
7. Nunez-Alonso GA
8. Monteiro O
9. Rogers C
10. Gamble V
11. Burbano R
12. Brennan PE
13. Tallant C
14. Ebner D
15. Fedorov O
16. O'Neill E
17. Knapp S
18. Dixon D
19. Müller S
20. O’Neill E
(2014) BET inhibition as a new strategy for the treatment of gastric cancer
Oncotarget 7:28.

https://doi.org/10.18632/oncotarget.9766
- Google Scholar
(2006)
Chiral drugs: an overview

International Journal of Biomedical Science 2:85–100.
- PubMed
- Google Scholar
Software
1. R Core Team
(2016)
R: A Language and Environment for Statistical Computing

R Foundation for Statistical Computing, Vienna, Austria.
1. Rathert P
2. Roth M
3. Neumann T
4. Muerdter F
5. Roe JS
6. Muhar M
7. Deswal S
8. Cerny-Reiterer S
9. Peter B
10. Jude J
11. Hoffmann T
12. Boryń ŁM
13. Axelsson E
14. Schweifer N
15. Tontsch-Grunt U
16. Dow LE
17. Gianni D
18. Pearson M
19. Valent P
20. Stark A
21. Kraut N
22. Vakoc CR
23. Zuber J
(2015) Transcriptional plasticity promotes primary and acquired resistance to BET inhibition
Nature 525:543–547.

https://doi.org/10.1038/nature14898
- PubMed
- Google Scholar
1. Shearer RF
2. Saunders DN
(2015) Experimental design for stable genetic manipulation in mammalian cell lines: lentivirus and alternatives
Genes to Cells 20:1–10.

https://doi.org/10.1111/gtc.12183
- PubMed
- Google Scholar
1. Shu S
2. Lin CY
3. He HH
4. Witwicki RM
5. Tabassum DP
6. Roberts JM
7. Janiszewska M
8. Huh SJ
9. Liang Y
10. Ryan J
11. Doherty E
12. Mohammed H
13. Guo H
14. Stover DG
15. Ekram MB
16. Peluffo G
17. Brown J
18. D'Santos C
19. Krop IE
20. Dillon D
21. McKeown M
22. Ott C
23. Qi J
24. Ni M
25. Rao PK
26. Duarte M
27. Wu SY
28. Chiang CM
29. Anders L
30. Young RA
31. Winer EP
32. Letai A
33. Barry WT
34. Carroll JS
35. Long HW
36. Brown M
37. Liu XS
38. Meyer CA
39. Bradner JE
40. Polyak K
41. Jin Huh S
42. D’Santos C
43. Brown M
(2016) Response and resistance to BET bromodomain inhibitors in triple-negative breast cancer
Nature 529:413–417.

https://doi.org/10.1038/nature16508
- PubMed
- Google Scholar
1. Siu KT
2. Ramachandran J
3. Yee AJ
4. Eda H
5. Santo L
6. Panaroni C
7. Mertz JA
8. Sims III RJ
9. Cooper MR
10. Raje N
(2016) Preclinical activity of CPI-0610, a novel small-molecule bromodomain and extra-terminal protein inhibitor in the therapy of multiple myeloma
Leukemia.

https://doi.org/10.1038/leu.2016.355
- Google Scholar
(2016) Clinical response of carcinomas harboring the BRD4-NUT oncoprotein to the targeted bromodomain inhibitor OTX015/MK-8628
Cancer Discovery 6:492–500.

https://doi.org/10.1158/2159-8290.CD-15-1335
- PubMed
- Google Scholar
1. Tai YT
2. Landesman Y
3. Acharya C
4. Calle Y
5. Zhong MY
6. Cea M
7. Tannenbaum D
8. Cagnetta A
9. Reagan M
10. Munshi AA
11. Senapedis W
12. Saint-Martin JR
13. Kashyap T
14. Shacham S
15. Kauffman M
16. Gu Y
17. Wu L
18. Ghobrial I
19. Zhan F
20. Kung AL
21. Schey SA
22. Richardson P
23. Munshi NC
24. Anderson KC
(2014) CRM1 inhibition induces tumor cell cytotoxicity and impairs osteoclastogenesis in multiple myeloma: molecular mechanisms and therapeutic implications
Leukemia 28:155–165.

https://doi.org/10.1038/leu.2013.115
- PubMed
- Google Scholar
(2012) Replicates and repeats—what is the difference and is it significant?
EMBO Reports 13:291–296.

https://doi.org/10.1038/embor.2012.36
- Google Scholar
1. Viechtbauer W
(2010) Conducting Meta-Analyses in R with the metafor Package
Journal of Statistical Software 36:1–48.

https://doi.org/10.18637/jss.v036.i03
- Google Scholar
1. Wadhwa E
2. Nicolaides T
(2016) Bromodomain inhibitor review: Bromodomain and extra-terminal family protein inhibitors as a potential new therapy in central nervous system tumors
Cureus 8:e620.

https://doi.org/10.7759/cureus.620
- PubMed
- Google Scholar
1. Yao W
2. Yue P
3. Khuri FR
4. Sun SY
(2015) The BET bromodomain inhibitor, JQ1, facilitates c-FLIP degradation and enhances TRAIL-induced apoptosis independent of BRD4 and c-Myc inhibition
Oncotarget 6:34669–34679.

https://doi.org/10.18632/oncotarget.5785
- PubMed
- Google Scholar
1. Zhang Y
2. Moschetta M
3. Huynh D
4. Tai YT
5. Zhang Y
6. Zhang W
7. Mishima Y
8. Ring JE
9. Tam WF
10. Xu Q
11. Maiso P
12. Reagan M
13. Sahin I
14. Sacco A
15. Manier S
16. Aljawai Y
17. Glavey S
18. Munshi NC
19. Anderson KC
20. Pachter J
21. Roccaro AM
22. Ghobrial IM
(2014) Pyk2 promotes tumor progression in multiple myeloma
Blood 124:2675–2686.

https://doi.org/10.1182/blood-2014-03-563981
- PubMed
- Google Scholar
1. Zhang Z
2. Ma P
3. Jing Y
4. Yan Y
5. Cai MC
6. Zhang M
7. Zhang S
8. Peng H
9. Ji ZL
10. Di W
11. Gu Z
12. Gao WQ
13. Zhuang G
(2016) BET bromodomain inhibition as a therapeutic strategy in ovarian cancer by downregulating FoxM1
Theranostics 6:219–230.

https://doi.org/10.7150/thno.13178
- PubMed
- Google Scholar

Article and author information

Author details

Fraser Aird

Developmental Therapeutics Core, Northwestern University, Evanston, United States

Contribution
FA, Acquisition of data, Drafting or revising the article

Competing interests
FA: Developmental Therapeutics Core is a Science Exchange associated lab.
Irawati Kandela

Developmental Therapeutics Core, Northwestern University, Evanston, United States

Contribution
IK, Acquisition of data, Drafting or revising the article

Competing interests
IK: Developmental Therapeutics Core is a Science Exchange associated lab.
Christine Mantis

Developmental Therapeutics Core, Northwestern University, Evanston, United States

Contribution
CM, Acquisition of data, Drafting or revising the article

Competing interests
CM: Developmental Therapeutics Core is a Science Exchange associated lab.
Reproducibility Project: Cancer Biology

Developmental Therapeutics Core, Northwestern University, Evanston, United States

Contribution
RP:CB, Analysis and interpretation of data, Drafting or revising the article
For correspondence
1. tim@cos.io
2. nicole@scienceexchange.com
Competing interests
RP:CB: EI, NP: Employed by and hold shares in Science Exchange Inc.The other authors declare that no competing interests exist.
1. Elizabeth Iorns, Science Exchange, Palo Alto, United States
2. Alexandria Denis, Center for Open Science, Charlottesville, Virginia
3. Stephen R Williams, Center for Open Science, Charlottesville, Virginia
4. Nicole Perfito, Science Exchange, Palo Alto, California
5. Timothy M Errington, Center for Open Science, Charlottesville, Virginia

Funding

Laura and John Arnold Foundation

Reproducibility Project: Cancer Biology

The funder had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Acknowledgements

The Reproducibility Project: Cancer Biology would like to thank Dr. Andrew Kung (Memorial Sloan Kettering Cancer Center) for sharing critical reagents and data, specifically the MM.1S-luc cells. We would also like to thank the following companies for generously donating reagents to the Reproducibility Project: Cancer Biology; American Type and Tissue Collection (ATCC), Applied Biological Materials, BioLegend, Charles River Laboratories, Corning Incorporated, DDC Medical, EMD Millipore, Harlan Laboratories, LI-COR Biosciences, Mirus Bio, Novus Biologicals, Sigma-Aldrich, and System Biosciences (SBI).

Ethics

Animal experimentation: All animal procedures were approved by the Northwestern University IACUC# IS00000556 and were in accordance with the Northwestern University's policies on the care, welfare, and treatment of laboratory animals.

Copyright

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.