1. Genes and Chromosomes
Download icon

Registered report: Systematic identification of genomic markers of drug sensitivity in cancer cells

  1. John P Vanden Heuvel
  2. Jessica Bullenkamp
  3. Reproducibility Project: Cancer Biology Is a corresponding author
  1. Indigo Biosciences, State College, United States
  2. Penn State University, United States
  3. King's College London, United Kingdom
Registered Report
Cited
0
Views
878
Comments
0
Cite as: eLife 2016;5:e13620 doi: 10.7554/eLife.13620

Abstract

The Reproducibility Project: Cancer Biology seeks to address growing concerns about the reproducibility in scientific research by conducting replications of selected experiments from a number of high-profile papers in the field of cancer biology. The papers, which were published between 2010 and 2012, were selected on the basis of citations and Altmetric scores (Errington et al., 2014). This Registered Report describes the proposed replication plan of key experiments from “Systematic identification of genomic markers of drug sensitivity in cancer cells” by Garnett and colleagues, published in Nature in 2012 (Garnett et al., 2012). The experiments to be replicated are those reported in Figures 4C, 4E, 4F, and Supplemental Figures 16 and 20. Garnett and colleagues performed a high throughput screen assessing the effect of 130 drugs on 639 cancer-derived cell lines in order to identify novel interactions for possible therapeutic approaches. They then tested this approach by exploring in more detail a novel interaction they identified in which Ewing’s sarcoma cell lines showed an increased sensitivity to PARP inhibitors (Figure 4C). Mesenchymal progenitor cells (MPCs) transformed with the signature EWS-FLI1 translocation, the hallmark of Ewing’s sarcoma family tumors, exhibited increased sensitivity to the PARP inhibitor olaparib as compared to MPCs transformed with a different translocation (Figure 4E). Knockdown mediated by siRNA of EWS-FLI1 abrogated this sensitivity to olaparib (Figure 4F). The Reproducibility Project: Cancer Biology is a collaboration between the Center for Open Science and Science Exchange, and the results of the replications will be published by eLife.

https://doi.org/10.7554/eLife.13620.001

Introduction

In their 2012 Nature paper, Garnett and colleagues implemented a large-scale high throughput in vitro screen designed to assess interactions between drugs and cancer-derived human cell lines (Garnett et al., 2012). This study leveraged a collection of over 600 cell lines screened across 130 drugs, with the aim to uncover new interactions between known cancers and known drugs in order to identify new potential therapeutic avenues using extant drugs. They captured a large number of known gene-drug interactions of clinically active drugs and identified several novel gene–drug associations. The ability to accurately capture a large number of known clinically relevant drug response biomarkers as well as preferential cancer type sensitivities known to occur in the clinic, such as decreased sensitivity to BRAF inhibitors in BRAF mutant colorectal cancers relative to melanomas, demonstrated the effectiveness of this large-scale pharmacogenomic approach. A similar approach of interrogating a large panel of human cancer cell lines of diverse lineages to predict drug sensitivity was conducted and reported by Barretina and colleagues at the same time (Barretina et al., 2012).

Garnett and colleagues identified an unexpected highly significant association between the EWS-FLI1 translocation and sensitivity to the PARP inhibitor olaparib (Garnett et al., 2012). The EWS-FLI1 translocation is a defining cytogenetic characteristic of Ewing’s sarcoma family tumors (ESFTs). ESFTs are highly malignant tumors that occur in the bone and soft tissue, usually in children. The translocation event combines part of the EWS protein to a member of the ETS transcription factor family; in 90% of cases, this is FLI1. This creates a novel transcription factor, EWS-FLI1, whose oncogenic actions and mechanisms are still being fully explored. The translocation event is thought to be the initiating event for the development of ESFTs (Erkizan et al., 2010; Lessnick and Ladanyi, 2012).

PARP1 has diverse functions in chromatin modification, mitosis and cell death, but it is most well studied in the context of DNA repair and transcriptional regulation (Sonnenblick et al., 2014). PARP1 is a key component of single stranded break (SSB) repair; however, loss of PARP1 activity can be compensated for through DNA repair via homologous recombination (HR). This makes PARP1 an interesting therapeutic target in the context of malignancies with deficient HR, such as BRCA1 and BRCA2 mutant breast and ovarian cancers. In these cancers, loss of PARP activity results in synthetic lethality; with both SSB and HR impaired, the accumulation of DNA damage eventually kills the tumor cells (Jiang et al., 2015; Lord et al., 2015; Sonnenblick et al., 2014). PARP inhibitors (PARPi), such as olaparib, are now at the forefront of treatment for breast and ovarian cancers, as well as other malignancies (Feng et al., 2015).

In Figure 4C, a predicted interaction between Ewing’s sarcoma cells and the PARP inhibitor olaparib was tested. PARP inhibitors target BRCA-deficient cells that rely on alternative DNA damage repair pathways involving PARP. A panel of cell lines representing Ewing’s sarcoma, a BRCA-deficient line, as well as other osteosarcomas and cancers of soft tissue and epithelium were treated with a range of concentrations of olaparib. The concentration of olaparib required to reduce colony formation by 90% or more was much less for Ewing’s sarcoma cells (on par with the concentration required for the BRCA-deficient cell line) than for the non-Ewing’s sarcoma cell lines. This experiment will be replicated in Protocol 1.

In Figure 4E, the hypothesis that mouse mesenchymal progenitor cells (MPCs) that had been transformed with the EWS-FLI1 translocation would confer sensitivity to olaparib was tested. The sensitivity of these cells to olaparib were compared to MPCs transformed with a related translocation (FUS-CHOP) as well as to SK-N-MC cells, which have the EWS-FLI1 translocation endogenously. Treatment with olaparib did not inhibit the viability of the FUS-CHOP transformed MPCs, but did inhibit the viability of the SK-N-MC cells. Olaparib also inhibited the viability of the EWS-FLI1 transformed MEFs compared to the FUS-CHOP translocation. This experiment will be replicated in Protocol 2.

In Figure 4F, the effects of EWS-FLI1 depletion on a cell line carrying the translocation endogenously was tested. A673 cells were transfected with siRNAs targeting EWS-FLI1, which resulted in a partial rescue of sensitivity to olaparib compared to control siRNA transfected cells. This experiment will be replicated in Protocol 3.

A paper published at the same time as Garnett and colleagues’ work also confirmed that Ewing’s sarcoma cell lines were sensitive to treatment with PARP inhibitors (Brenner et al., 2012). In a previous paper, Brenner and colleagues reported that in prostate cancer PARP was a cofactor for wild-type ETS transcription factors, which makes up one half of the defining translocation-based fusion transcription factor of Ewing’s sarcoma, and that PARPi treatment of ETS positive prostate cancers disrupted their growth (Brenner et al., 2011; Legrand et al., 2013). Based on this finding, they examined the role of PARP1 and PARPi in Ewing’s sarcoma. Using immunoprecipitation, they detected a direct interaction between the EWS-FLI1 fusion transcription factor and PARP1 (Brenner et al., 2012). Further, they reported that transforming a cell line (in this case, PC3 cells) with the EWS-FLI1 translocation conferred sensitivity to treatment with olaparib, and that siRNA mediated knockdown of EWS-FLI1 inhibited transwell migration of ESFT derived cell lines, but not osteosarcoma cell lines (Brenner et al., 2012). Multiple groups have also reported the unique sensitivity of EWS-FLI1 carrying Ewing’s sarcoma derived cell lines to olaparib (Lee et al., 2013; Norris et al., 2014; Ordóñez et al., 2015). Additional work then demonstrated that, similar to breast and ovarian cancers harboring BRCA1/2 mutations, Ewing’s sarcomas may also have defects in DNA repair mechanisms, rendering them sensitive to PARP inhibition (Stewart et al., 2014). This has led to the start of clinical trials treating Ewing’s sarcoma patients with combination therapies targeting multiple DNA damage pathways and PARP inhibition. Results from a small scale nonrandomized phase II human trial failed to show clinical efficacy in patients with metastatic and/or recurrent Ewing sarcoma treated with only olaparib (Choy et al., 2014), but other trials are underway to explore the efficacy of PARP inhibition in combination with chemotherapy.

Materials and methods

Unless otherwise noted, all protocol information and references were derived from the original paper or information obtained directly from the authors.

Protocol 1: Colony formation assay of Ewing’s sarcoma cell lines with olaparib

This experiment assesses the sensitivity of Ewing’s sarcoma cell lines to the PARP inhibitor olaparib. A colony formation assay will be performed with Ewing’s sarcoma, osteosarcoma, and BRCA2-deficient and BRCA-proficient cells treated with a range of olaparib concentrations to determine the effective concentration (number of colonies reduced by at least 90%). This protocol replicates the experiment reported in Figure 4C and Supplemental Figure 16.

Sampling

  • The experiment will be performed with two replicates and each experiment will use 5 Ewing’s sarcoma cell lines and 7 osteosarcoma cell lines for a power of 82%.

    • See Power calculations for details.

  • The experiment will use the following cell lines:

    • Ewing’s sarcoma cell lines:

      • A673

      • TC-71

      • SK-N-MC

      • CHLA-9

      • CHLA-10

    • Osteosarcoma cell lines:

      • U-2-OS

      • SJSA-1

      • SAOS-2

      • HOS

      • MG-63

      • 143B

      • G-292

    • BRCA2-deficient cell line: [positive control]

      • DoTc2-4510

    • BRCA-proficient cell line: [negative control]

      • MES-SA

  • Each cell line will be treated with the following conditions:

    • Vehicle (DMSO)

    • 0.1 µM olaparib

    • 0.32 µM olaparib

    • 1 µM olaparib

    • 3.2 µM olaparib

    • 10 µM olaparib

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
OlaparibInhibitorSelleck ChemicalsS1060Source shared during communication with authors.
DMSOChemicalSigma Aldrich472301Source shared during communication with authors.
Phosphate buffered saline (PBS)BufferGibco-Life Technologies10010-023Source shared during communication with authors.
Giemsa stainChemicalSigma AldrichG5637Source shared during communication with authors.
MethanolChemicalFisher ScientificBP1105-4Source shared during communication with authors.
DoTc2-4510 cellsCell lineATCCCRL-7920Original source not specified.
MES-SA cellsCell lineATCCCRL-1976Original source not specified
U-2-OS cellsCell lineATCCHTB-96Original source not specified.
SAOS-2 cellsCell lineATCCHTB-85Original source not specified.
SJSA-1 cellsCell lineATCCCRL-2098Original source not specified.
HOS cellsCell lineATCCCRL-1543Original source not specified.
MG-63 cellsCell lineATCCCRL-1427Original source not specified.
143B cellsCell lineATCCCRL-8303Replaces osteosarcoma cells used originally;
see Known Differences.
G-292 cells, clone A141B1Cell lineATCCCRL-1423
A673 cellsCell lineATCCCRL-1598Replaces the ES cells used originally;
see Known Differences
SK-N-MC cellsCell lineATCCHTB-10
TC-71 cells2Cell lineChildren’s Oncology Group Cell
Culture and Xenograft Repository
 
CHLA-10 cells1Cell lineChildren’s Oncology Group Cell
Culture and Xenograft Repository
CHLA-9 cells3Cell lineChildren’s Oncology Group Cell
Culture and Xenograft Repository
Iscove’s modified DMEM (IMDM)Cell cultureLife Technologies12440-053Not originally included.
L-glutamineCell cultureLife Technologies25030-081Not originally included.
Insulin-Transferrin-Selenium (ITS)Growth factorLonza17-838ZNot originally included.
McCoy’s 5A Medium ModifiedCell cultureATCC30-2007Not originally included.
Fetal bovine serum (FBS)Cell cultureValley BiomedicalBS3032Original source not specified.
RPMI 1640 mediumCell cultureATCC30-2001Original source not specified.
Eagle’s Minimum Essential
Media (EMEM)
Cell cultureATCC30-2003Originally not specified.
5-bromo-2’-deoxyuridineNucleosideSigmaB5002Not originally included.
MEM Eagle with Earle’s BSSCell cultureLonza12-125FNot originally included.
DMEM – High GlucoseCell cultureGE-HealthcareE15-883Shared during communication with authors.
DMEM/F12Cell cultureLife Technologies11320-033Original source not specified.

Procedure

Notes:

  • All cell lines will be sent for STR profiling and mycoplasma testing.

  • A673 cells are maintained in DMEM with 10% FBS.

  • SAOS-2 are maintained in McCoy’s 5A Medium Modified supplemented with 15% FBS.

  • CHLA-10 cells and TC-71 are maintained in IMDM supplemented with 20% FBS, 4 mM L-glutamine, 5 µg/ml insulin, 5 µg/ml transferrin and 5 ng/ml selenium

  • DoTc2-4510 cells are maintained in DMEM/F12 with 5% FBS.

  • U-2-OS cells, HOS cells and G-292 cells are maintained in McCoy’s 5A Medium Modified supplemented with 10% FBS.

  • MG-63 cells are maintained in EMEM supplemented with 10% FBS.

  • 143B cells are maintained in Minimum essential medium (Eagle) in Earle's BSS with 0.015 mg/ml 5-bromo-2'-deoxyuridine, 90%; FBS, 10%.

  • SJSA-1 cells and SK-N-MC cells are maintained in RPMI 1640 medium supplemented with 10% FBS.

  • MES-SA cells are maintained in McCoy’s 5A Medium Modified supplemented with 10% FBS.

  • All cells kept at 37°C and 5% CO2.

  • Olaparib is stored as a 10 mM stock in DMSO at -80°C. Each aliquot is subjected to no more than 5 freeze-thaw cycles.

  1. Plate cells at low density in 6 well culture plates.

    1. Seed 2,000 cells per well in 2 ml of appropriate medium.

    2. Plate 6 wells per cell line in duplicate plates.

      1. Each cell line undergoes 6 treatments (see Sampling section above).

      2. Label one plate A and one plate B for each cell line.

    3. Let cells adhere overnight.

  2. The following day treat cells with varying concentrations of drug:

    1. Vehicle (DMSO at 0.1% v/v)

    2. 0.1 µM olaparib

    3. 0.32 µM olaparib

    4. 1 µM olaparib

    5. 2 µM olaparib

    6. 10 µM olaparib

  3. Replace media and drug every 3-4 days.

  4. After 7 to 21 days, when sufficient colonies are visible in the DMSO controls, fix cells for quantification.

    1. Stain cells once sufficient numbers of colonies are visible in DMSO wells.

      1. Sufficient colonies means at least 100 colonies, ideally over 200 colonies, are present in the vehicle treated wells for each cell line.

      2. DoTc2-24510 cells were cultured for about 12 days in the original study.

    2. Wash cells once in PBS.

    3. Fix in ice-cold methanol for 30 min while gently shaking at room temperature.

    4. Remove methanol and add Giemsa stain at 1:20 dilution in deionized water. Incubate for 4 hr at room temperature shaking or overnight at 4° shaking.

    5. 4 hr later, or the following day, rinse cells with water and air dry.

  5. Take brightfield images of plates and manually quantify the number of colonies, blinded, in each well from each plate.

  6. Determine and record the concentration at which colony formation was reduced by >90% compared to DMSO controls for each plate.

Deliverables

  • Data to be collected:

    • Images of all plates

    • Colony counts of each well

    • Graph of each cell line and the concentration of olaparib required to reduce colony formation by >90% compared to DMSO controls. (Compare to Figure 4C)

Confirmatory analysis plan

  • Statistical Analysis of the Replication Data:

    • Wilcoxon-Mann-Whitney test for ordinal data of the effective concentration of olaparib to reduce the colonies by at least 90% in Ewing’s sarcoma compared to osteosarcoma cell lines. Perform for each group (A or B) of replicate plates.

  • Meta-analysis of original and replication attempt effect sizes:

    • This replication attempt will perform the statistical analysis listed above, compute the effect sizes (for each independent attempt), compare them against the reported effect size in the original paper and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.

Known differences from the original study

  • The replication attempt will only examine Ewing’s sarcoma and osteosarcoma derived cell lines, with the BRCA2-deficient cell line as a positive control, and will not include the remaining cell types (soft tissue and epithelial).

  • Due to the inability to obtain any of the Ewing’s sarcoma cell lines used originally, and in consultation with the original authors, the replication attempt will use A673, TC-71, CHLA-9, SK-N-MC and CHLA-10 cells. The cell lines all carry the critical EWS/FLI1 translocation. The cells used in the original study were ES1, ES6, ES7, ES8, and MHH-ES-1.

  • Similarly, the replication attempt will use U-2-OS, SJSA-1, SAOS-2, HOS, MG-63, 143B, and G-292 cells. 143B and G-292 cells were not used in the original study and CAL-72, HuO-3N1, and NY cells that were used in the original study will not be included in this replication attempt.

  • All known differences are listed in the materials and reagents section above with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control

The cell lines used in this experiment will undergo STR profiling to confirm identity and will be sent for mycoplasma testing to ensure there is no contamination. The DMSO concentration, although not originally reported, will be kept at a low percentage to avoid toxicity. All data obtained from the experiment will be made publicly available, either in the published manuscript or as an open access dataset available on the OSF (https://osf.io/nbryi/).

Protocol 2: Olaparib sensitivity in cells transformed with the EWS-FLI1 rearrangement

This experiment assesses if sensitivity to PARP inhibitors is due to the presence of the EWS-FLI1 rearrangement. Mouse mesenchymal progenitor cells (MPCs) transformed with EWS-FLI1, or the related liposarcoma-associated translocation FUS-CHOP, will be analyzed for cellular viability after olaparib treatment. This protocol replicates the experiment reported in Figure 4E.

Sampling

  • The experiment will be repeated three times for a power of 99%.

    • See Power calculations for details.

  • The experiment will use three cell lines:

    • EWS-FLI1 transformed MPCs

    • FUS-CHOP transformed MPCs

    • SK-N-MC cells

      • These cells harbor the endogenous EWS-FUS1 translocation

  • Each cell line will be treated with the following conditions in technical triplicate:

    • No treatment [additional]

    • Vehicle (DMSO)

    • 0.39 µM olaparib

    • 0.78 µM olaparib

    • 1.56 µM olaparib

    • 3.13 µM olaparib

    • 6.25 µM olaparib

    • 12.5 µM olaparib

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
EWS-FLI1 transformed mouse
mesenchymal progenitor cells (MPCs)
Cell lineAuthorsN/AProvided by the Stamenkovic lab
FUS-CHOP transformed mouse
mesenchymal progenitor cells (MPCs)
Cell lineAuthorsN/AProvided by the Stamenkovic lab
SK-N-MC cellsCell lineATCCHTB-10Source shared during
communication with authors.
OlaparibInhibitorSelleck ChemicalsS1060Source shared during
communication with authors.
DMSOChemicalSigmaD8418Source shared during
communication with authors.
4% formaldehydeChemicalUSB19943Source shared during
communication with authors.
Syto60 fluorescent nucleic acid stainChemicalInvitrogenS11342Catalog # shared during
communication with authors.
FBSCell cultureValley BiomedicalBS3032Original source not specified.
RPMI 1640 mediumCell cultureATCC30-2001Original source not specified.
Fluorescent plate readerEquipmentLiCorSource shared during
communication with authors.
DMEM, low glucose,
GlutaMAX supplement, pyruvate
Cell cultureGibco21885-025Shared during
communication with authors.
MCDB 201 medium, with trace elements,
L-glutamine and 30 mM HEPES; powder
Cell cultureSigmaM6770Shared during
communication with authors.
Ascorbic acid-2-phosphateCell cultureSigmaA8960Shared during
communication with authors.
DexamethasoneChemicalSigmaD8893Shared during
communication with authors.
Linoleic acid-BSAChemicalSigmaL9530Shared during
communication with authors.
Insulin, transferrin,
sodium selenite supplement
Growth factorRoche (Sigma)1074547Shared during
communication with authors.
Dialyzed FCSCell cultureSigmaF0392Shared during
communication with authors.
EGF; humanGrowth factorSigmaE9644Shared during
communication with authors.
PDGF-BB, ratGrowth factorR&D Systems520-BB-050Shared during
communication with authors.
Penicillin-Streptomycin; 100XCell cultureSigmaP4333Original source not specified.
Leukemia inhibitory factor (LIF);
human; 10 µg/ml
Growth factorSigmaL5283Shared during communication with authors.
Replaces LIF generated from CHO LIF720D cells.
Fibronectin; 0.1% in PBSChemicalSigmaF1141Shared during
communication with authors.

Procedure

  • All cell lines will be sent for STR profiling and mycoplasma testing.

  • SK-N-MC cells are maintained in RPMI-1640 with 10% FBS.

  • MPCs are maintained in DMEM:MCDB (60:40) supplemented with 100 µM ascorbic acid-2-phosphate, 1 nM dexamethasone, 0.2 mg/ml linoleic acid-BSA, 5 µg/ml insulin, 5 µg/ml transferrin, 5 ng/ml sodium selenite, 2% dialyzed FCS, 10 ng/ml human EGF, 10 ng/ml rat PDGF-BB, 1X penicillin/streptomycin, and 10 ng/ml LIF. Coat culture dishes for cells with fibronectin (0.0001% in PBS) for 3 hr at 37˚C (or 4˚C overnight) before plating. Additional details available at: https://osf.io/2vxnj/?view_only=7c9fb185e4c64ae78660cad92083aaa1

  • All cells are kept at 37°C and 5% CO2.

  • Olaparib is stored as a 10 µM stock in DMSO at -80°C. Each aliquot is subjected to no more than 5 freeze-thaw cycles.

  1. Determine seeding density of each cell line so cells will be in the growth phase at the end of the assay (~70% confluency):

    1. Plate 500 – 1.6x104EWS-FLI1 transformed MPCs, FUS-CHOP transformed MPCs, and SK-N-MC cells in 96 well plates with 100 µl of appropriate medium in technical triplicate. Seed three plates for measurements at 48, 72, and 96 hr after seeding. Incubate overnight.

    2. 48 hr after seeding fix cells in 4% paraformaldehyde (PFA) for 30 min at 37°C.

      1. Stain cells with 1 µM Syto60 fluorescent nuclear dye, diluted in PBS, for 1 hr following manufacturer’s instructions.

        1. Wash out excess Syto60 prior to signal reading.

      2. Measure fluorescent signal intensity with a fluorescent plate reader.

    3. 24 hr later (72 hr after seeding) fix and stain cells with Cyto60 as described above and measure fluorescent signal intensity.

    4. 24 hr later (96 hr after seeding) fix and stain cells with Cyto60 as described above and measure fluorescent signal intensity.

      1. Use seeding density for each cell line that results in sub-confluency (~70%) at the end of the assay and where the signal is still in the linear range.

  2. Seed cells at density determined in step 1 above in 96-well plates and let grow overnight.

    1. Seed 21 wells per cell line.

      1. Each cell line will be treated with 7 concentrations of drug in technical triplicate (see Sampling section above).

    2. Seed additional wells in technical triplicate per cell line for measurements at 24, 48, and 72 hr after treatment to test for proliferation of cells (no-treatment condition).

  3. The next day, treat cells with a range of concentrations of olaparib.

    1. No-treatment [additional]

    2. Vehicle (DMSO at 0.1% v/v)

    3. 0.39 µM olaparib

    4. 0.78 µM olaparib

    5. 1.56 µM olaparib

    6. 3.13 µM olaparib

    7. 6.25 µM olaparib

    8. 12.5 µM olaparib

  4. Incubate for 24, 48, or 72 hr.

    1. Medium does not need to be changed during this period.

    2. No-treatment wells are incubated for 24, 48, or 72 hr.

    3. Olaparib or vehicle treated wells are incubated for 72 hr.

  5. After 24, 48, or 72 hr fix cells in 4% PFA for 30 min at 37°C.

  6. Stain cells with 1 µM Syto60 fluorescent nuclear dye, diluted in PBS, for 1 hr following manufacturer’s instructions.

    1. Wash out excess Syto60 prior to signal reading.

  7. Measure fluorescent signal intensity with a fluorescent plate reader.

    1. Excitation wavelength: 630 nm

    2. Emission wavelength: 694 nm

  8. Repeat steps 2–7 independently two additional times.

Deliverables

  • Data to be collected:

    • Raw data of fluorescent readout for all wells

    • Graph of normalized readings for each drug concentration compared to vehicle only control (Compare to Figure 4E)

Confirmatory analysis plan

  • Statistical Analysis of the Replication Data:Note: At the time of analysis, we will perform the Shapiro-Wilk test and generate a quantile-quantile plot to assess the normality of the data. We will also perform Levene’s test to assess homoscedasticity. If the data appears skewed we will perform the appropriate transformation in order to proceed with the proposed statistical analysis. If this is not possible we will perform the planned comparisons using the equivalent non-parametric test.

    • One way ANOVA on IC50 values of olaparib, determined by spline interpolation, of each cell line with the following planned comparisons using Fisher’s LSD test:

      • EWS-FLI1 transformed MPCs vs. FUS-CHOP transformed MPCs

      • FUS-CHOP transformed MPCs vs. SK-N-MC cells

  • Meta-analysis of original and replication attempt effect sizes:

    • This replication attempt will perform the statistical analysis listed above, compute the effect sizes, compare them against the reported effect size in the original paper and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.

Known differences from the original study

  • Commercially available LIF will be used in place of LIF generated from CHO LIF720D cells, as suggested by the original authors.

  • All known differences are listed in the materials and reagents section above with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control

The cell lines used in this experiment will undergo STR profiling to confirm identity and will be sent for mycoplasma testing to ensure there is no contamination. The DMSO concentration, although not originally reported, will be kept at a low percentage to avoid toxicity. The seeding density of each cell line will be empirically determined prior to conducting the replicates so cells will be still be in the growth phase at the end of the assay. Measurements will be taken at 24, 48, and 72 hr after seeding from cells not treated with drug to test for proliferation of cells during the assay. All data obtained from the experiment will be made publicly available, either in the published manuscript or as an open access dataset available on the OSF (https://osf.io/nbryi/).

Protocol 3: Olaparib sensitivity after depletion of EWS-FLI1 from A673 cells

This experiment assesses the sensitivity of PARP inhibitors to the presence of the EWS-FLI1 rearrangement. EWS-FLI1 specific siRNA will be used to deplete the fusion mRNA from A673 cells, which harbor the translocation endogenously, and cell viability after olaparib treatment will be assessed. This protocol replicates the experiment reported in Figure 4F and Supplemental Figure 20.

Sampling

  • The experiment will be repeated three times for a minimum power of 80%.

    • See Power calculations for details.

  • The experiment has 2 cohorts:

    • Cohort1: siControl transfected A673 cells

    • Cohort 2: siEF1 transfected A673 cells

  • Each cohort will be treated with the following conditions to assess cell viability in technical triplicate:

    • Untreated

    • 100 uM olaparib or DMSO equivalent

    • 33.33 uM olaparib or DMSO equivalent

    • 11.11 uM olaparib or DMSO equivalent

    • 3.704 uM olaparib or DMSO equivalent

    • 1.235 uM olaparib or DMSO equivalent

    • 0.412 uM olaparib or DMSO equivalent

    • 0.137 uM olaparib or DMSO equivalent

    • 0.046 uM olaparib or DMSO equivalent

    • 0.015 uM olaparib or DMSO equivalent

  • Each cohort will be treated with the following conditions for qRT-PCR analysis:

    • 1.3 µM olaparib or DMSO equivalent

  • Quantitative RT-PCR performed in technical triplicate for the following genes:

    • EWS-FLI1

    • RPLP0 (internal control)

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
A673 cellsCell lineATCCCRL-1598Source shared during
communication with authors.
OlaparibInhibitorSelleck ChemicalsS1060Source shared during
communication with authors.
DMSOChemicalSigmaD2650Source shared during
communication with authors.
siEF1Nucleic acidQiagenCustom order5'-GGCAGCAGAACCCUUCUUACG-3’
siCT control siRNANucleic acidQiagenSI03650318Catalog number shared during
communication with authors.
Cell Titer 96 Aqueous One
Solution Cell Proliferation Assay
Reporter assayPromegaG3582
DMEM - High GlucoseCell cultureGE-HealthcareE15-883Shared during
communication with authors.
FBSCell cultureValley BiomedicalBS3032Original source not specified.
O-MEMCell cultureGibco31985-062Shared during
communication with authors.
96 well tissue culture test platesLabwareTPP92096Source shared during
communication with authors.
Lipofectamine RNAiMAXCell cultureLife Technologies13778-150Shared during
communication with authors.
High-capacity cDNA reverse
transcription kit
KitApplied Biosystems4368814Shared during
communication with authors.
NucleoSpin RNA II kitKitMachery-Nagel740955.50Shared during
communication with authors.
Power SYBR Green PCR mastermixKitApplied Biosystems4367659Shared during
communication with authors.
qPCR machineEquipmentABI/PRISM7500Shared during
communication with authors.
EWS-FLI1 primersNucleic acidSynthesis left to the discretion of the
replicating lab and recorded later
Sequence shared during
communication with authors.
RPLP0 primersNucleic acidSequence shared during
communication with authors.
GloMax Multi+ Detection
System (spectrophotometer)
EquipmentPromega9311-011Shared during communication with authors.
Replaces BMG FLUOstar OPTIMA microplate reader.

Procedure

Notes:

  • All cell lines will be sent for STR profiling and mycoplasma testing.

  • A673 cells are maintained in DMEM with 10% FBS.

  • All cells are kept at 37°C and 5% CO2.

  • Olaparib is stored as a 10 mM stock in DMSO at -80°C. Each aliquot is subjected to no more than 5 freeze-thaw cycles.

  • siRNA stocks kept at 20 µM; final siRNA concentration is 25 nM.

  1. Seed cells for assays:

    1. For cell viability assay, plate 5000 A673 cells per well in 64 µl medium without antibiotics in a 96-well plate.

      1. Seed enough cells for each condition to be performed in technical triplicate.

    2. For qRT-PCR, plate 3x104 A673 cells per well of a 24 well plate in medium without antibiotics.

      1. This is a similar seeding density as the 96 well plate.

  2. Immediately transfect cells with 25 nM siControl or siEF1 siRNAs using Lipofectamine RNAiMAX with the cells in suspension. The following directions prepare enough transfection mixture for one 96-well plate. The amounts will be scaled accordingly to account for the plates used for the qRT-PCR analysis.

    1. Mix 12.17 µl of 20 µM siRNA stock with 962.1 µl of OptiMEM.

    2. Mix 18.26 µl of Lipfectamine RNAiMAX with 956 µl OptiMEM.

    3. Gently mix the two solutions together and incubate for 12 min at room temperature.

    4. Add 16 µl of transfection mixture per well to appropriate wells.

  3. Immediately after siRNA transfection, treat cells with varying concentrations of olaparib or vehicle (DMSO).

    1. See Sampling section above for details; include untreated cells and cells treated with vehicle only

    2. Prepare a 500 µM stock of Olaparib by adding 30 µl of 10 mM stock to 570 µl of DMEM.

    3. Prepare a stock of DMSO by adding 30 µl of DMSO to 570 µl of DMEM.

      1. These will be used for the vehicle treated cells.

    4. For cell viability assay, dilute olaparib and DMSO in DMEM by three-fold serial dilution as outlined:

      Experimental wells
      ControlOlaparib
      (µM)
      Background
      No drug10033.3311.113.7041.2350.4120.1370.0460.015No cells
      DMSO (µL used in olaparib dilution)
      10.3330.1110.0370.0120.0040.0015x10-42x10-4
      Vehicle only wells
      DMSO (µL/well, no olaparib)
      10.3330.1110.0370.0120.0040.0015x10-42x10-4
      1. Add 20 µl of each dilution to appropriate wells. Final volume per well is 100 µl.

    5. For qRT-PCR, treat cells with 1.3 µM olaparib or equivalent volume of DMSO.

      1. Dilute 500 µM stock of olaparib or stock of DMSO to create 6.5 µM (5X working solution) in DMEM. Add to plate to achieve 1.3 µM olaparib or equivalent volume of DMSO (0.013%).

  4. Incubate cells for 72 hr.

    1. Medium does not need to be changed during this time period.

  5. Measure cell viability by using the Cell Titer 96 well aqueous one assay according to the manufacturer’s instructions.

    1. Add 20 µl Cell Titer 96 Aqueous solution reagent per well containing 100 µl medium.

    2. Incubate plate at 37°C in humidified 5% CO2 for 4 hr.

    3. Record absorbance at 490 nm using a BMG FLUOstar OPTIMA microplate reader.

      1. Subtract average background (no cell) wells from each treated (olaparib or DMSO) well.

      2. Normalize values to corresponding untreated (no drug or vehicle) wells for each cohort.

      3. Determine IC50 value for each cohort using normalized olaparib values.

  6. qRT-PCR to confirm knockdown of EWS-FLI1 expression:

    1. Extract RNA with the NuceloSpin RNA II kit according to manufacturer’s instructions.

      1. Record A260/A280 and A260/A230 ratios.

    2. Synthesize cDNA using 1 µg of RNA and the High-capacity cDNA reverse transcription kit according to the manufacturer’s instructions.

    3. Perform qPCR using POWER SYBR Green PCR mastermix according to the manufacturer’s instructions in technical triplicate.

      1. Primers:

        1. EWS-FLI1(forward):

          1. 5'-GCCAAGCTCCAAGTCAATATAGC-3'

        2. EWS-FLI1(reverse):

          1. 5'-GAGGCCAGAATTCATGTTATTGC-3'

        3. RPLP0(forward): Internal Control

          1. 5'-GAAACTCTGCATTCTCGCTTC-3'

        4. RPLP0(reverse): Internal Control

          1. 5'-GGTGTAATCCGTCTCCACAG-3'

      2. Reaction conditions run on an ABI PRISM 7500.

        1. 95°C for 10 min

        2. 40 cycles of:

          1. 95°C for 15 s

          2. 60°C for 1 min

        3. Dissociation curve

      3. Analyze with 7500 SDS software or equivalent.

    4. Calculate relative EWS-FLI1 expression for each sample using RPLP0 as internal standard.

  7. Repeat independently two additional times.

Deliverables

  • Data to be collected:

    • Raw absorbance values for all wells.

    • Graph of absorbance corrected values for all concentrations of olaparib or DMSO normalized to untreated controls (as seen in Figure 4F).

    • IC50 values for each cohort using normalized olaparib values.

    • Raw and normalized qRT-PCR data (as seen in Supplemental Figure 20).

Confirmatory analysis plan

  • Statistical Analysis of the Replication Data:Note: At the time of analysis, we will perform the Shapiro-Wilk test and generate a quantile-quantile plot to assess the normality of the data. We will also perform Levene’s test to assess homoscedasticity. If the data appears skewed we will perform the appropriate transformation in order to proceed with the proposed statistical analysis. If this is not possible we will perform the planned comparisons using the equivalent non-parametric test.

    • o Viability assay:

      • Unpaired two-tailed t-test of olaparib IC50 values of siControl transfected cells compared to siEF1 transfected cells.

    • qRT-PCR:

      • Two-way ANOVA of siControl and siEF1 transfected cells treated with or without olaparib with the following planned comparisons using the Bonferroni correction:

        • siControl transfected cells treated with DMSO compared to siEF1 transfected cells treated with DMSO.

        • siControl transfected cells treated with olaparib compared to siEF1 transfected cells treated with olaparib.

  • Meta-analysis of original and replication attempt effect sizes:

    • This replication attempt will perform the statistical analysis listed above, compute the effect sizes, compare them against the reported effect size in the original paper and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.

Known differences from the original study

  • All known differences are listed in the materials and reagents section above with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control

The cell line used in this experiment will undergo STR profiling to confirm identity and will be sent for mycoplasma testing to ensure there is no contamination. The sample purity (A260/280 ratio) of the isolated RNA from each sample will be reported. All data obtained from the experiment will be made publicly available, either in the published manuscript or as an open access dataset available on the OSF (https://osf.io/nbryi/).

Power calculations

For additional details on power calculations, please see analysis scripts and associated files on the Open Science Framework: https://osf.io/j9bnk/

Protocol 1

Summary of original data estimated from graph reported in Figure 4C

Cell typeCell lineEffective concentration (µM)
Ewing’s
sarcoma
ES11
ES61
ES70.32
ES81
MHH-ES-10.32
OsteosarcomaCAL-7210
HOS1
HuO-3N13.2
MG-633.2
NY3.2
SAOS-23.2
SJSA-110
U-2-OS10
BRCA2-deficientDoTc2-45100.32

Test family

  • Wilcoxon-Mann-Whitney test (ordinal data): alpha error = 0.05

Power calculations

Group 1Group 2Effect size
(Cliff’s delta)
A priori powerGroup 1
sample size
Group 2
sample size
Ewing’s sarcomaOsteosarcoma0.9250081.8%57

Protocol 2

Summary of original data reported in Figure 4E (shared by authors)

Cell lineConcentration of olaparib (µM)MeanSDN
EWS-FLI1 transformed MPCs010.063
0.390.590.053
0.780.530.093
1.560.440.053
3.130.340.053
6.250.240.043
12.50.220.043
FUS-CHOP transformed MPCs010.093
0.391.060.013
0.781.030.063
1.561.110.083
3.130.980.093
6.250.590.073
12.50.450.043
SK-N-MC010.043
0.390.660.043
0.780.660.093
1.560.500.013
3.130.400.043
6.250.300.053
12.50.250.033

IC50 values of olaparib, determined by spline interpolation.

Calculations performed with R software, version 3.2.2 (R Development Core Team, 2015).

Cell lineMeanSDN
EWS-FLI1 transformed MPCs1.05020.53633
FUS-CHOP transformed MPCs7.79631.30243
SK-N-MC1.54490.05053

Test family

  • Two-tailed t test, Wilcoxon-Mann-Whitney test, Fisher’s LSD: alpha error = 0.05

Power Calculations performed with G*Power software, version 3.1.7 (Faul et al., 2007).

Group 1Group 2Effect size dA priori powerGroup 1
sample size
Group 2
sample size
EWS-FLI1
transformed MPCs
FUS-CHOP transformed MPCs6.7734383.8%12121
SK-N-MCFUS-CHOP transformed MPCs6.7828383.8%12121
  1. 1 3 samples per group will be used as a minimum making the power 99.9%.

Test family

  • Due to the large difference in variance, these parametric tests are only used for comparison purposes. The sample size is based on the non-parametric tests listed above.

  • ANOVA: Fixed effects, omnibus, one-way: alpha error = 0.05.

Power Calculations performed with G*Power software, version 3.1.7 (Faul et al., 2007).

ANOVA F test statistic and partial ηperformed with R software, version 3.2.2 (R Development Core Team, 2015).

GroupsF test
statistic
Partial η2Effect size fA priori
power
Total sample size
EWS-FLI1 transformed MPCs,
FUS-CHOP transformed MPCs,
and SK-N-MC
F(2,6) = 64.060.955264.6209799.9%6(3 groups)
  1. 1 9 total samples (3 per group) will be used as a minimum.

Test family

  • Due to the large difference in variance, these parametric tests are only used for comparison purposes. The sample size is based on the non-parametric tests listed above.

  • Two-tailed t test, difference between two independent means, Fisher’s LSD: alpha error = 0.05

Power Calculations performed with G*Power software, version 3.1.7 (Faul et al., 2007).

Group 1Group 2Effect size dA priori powerGroup 1 sample sizeGroup 2 sample size
EWS-FLI1 transformed MPCsFUS-CHOP transformed MPCs6.7734389.9%12121
SK-N-MCFUS-CHOP transformed MPCs6.7828389.9%12121
  1. 1 3 samples per group will be used as a minimum making the power 99.9%.

Protocol 3

Viability assay

Summary of original data reported in Figure 4F (shared by authors)

siRNAConcentration of olaparib (µM)
or volume of DMSO (µl)
MeanSDN
siControl
(DMSO treatment)
0 µl97.33600.953913
2x10-4 µl102.2033.700133
5x10-4 µl100.0880.902263
0.001 µl94.86281.300223
0.004 µl100.0953.847433
0.012 µl107.6341.053703
0.037 µl110.3784.415613
0.111 µl111.4670.681913
0.333 µl104.5011.984003
1.000 µl107.9051.611843
siControl
(olaparib treatment)
0 µM102.6642.822013
0.0152 µM95.99211.180483
0.046 µM83.18892.809893
0.1371 µM81.83702.939763
0.411 µM72.40563.100303
1.234 µM54.90262.745233
3.70 µM16.06363.509153
11.11 µM1.280320.610003
33.33 µM-1.455271.641013
100 µM2.282312.394273
siEF1
(DMSO treatment)
0 µl99.09711.134363
2x10-4 µl99.63971.215983
5x10-4 µl95.36220.451153
0.001 µl90.45994.319343
0.004 µl94.31790.868963
0.012 µl95.17522.350643
0.037 µl96.38371.394193
0.111 µl96.75761.134673
0.333 µl95.47621.384973
1.000 µl97.23651.248393
siEF1
(olaparib treatment)
0 µM100.9033.870043
0.0152 µM97.90234.770673
0.046 µM95.48535.476873
0.1371 µM93.97132.339653
0.411 µM89.44304.970933
1.234 µM76.63322.4365453
3.70 µM45.03961.674733
11.11 µM18.78151.784363
33.33 µM11.75413.752203
100 µM11.79972.227733

IC50 values of olaparib, determined by four-parameter log-logistic function.

Calculations performed with R software, version 3.2.2 (R Development Core Team, 2015).

Cell lineMeanSDN
siControl1.351910.06843
siEF12.745610.17153

Test family

  • Two-tailed t test, Wilcoxon-Mann-Whitney test: alpha error = 0.05

Power Calculations performed with G*Power software, version 3.1.7 (Faul et al., 2007).

Group 1Group 2Effect size dA priori powerGroup 1 sample sizeGroup 2 sample size
siControlsiEF110.6749498.7%12121
  1. 1 3 samples per group will be used as a minimum making the power 99.9%.

Test family

  • Due to the large difference in variance, these parametric tests are only used for comparison purposes. The sample size is based on the non-parametric tests listed above.

  • Two-tailed t test, difference between two independent means: alpha error = 0.05

Power Calculations performed with G*Power software, version 3.1.7 (Faul et al., 2007).

Group 1Group 2Effect size dA priori powerGroup 1 sample sizeGroup 2 sample size
siControlsiEF110.6749499.6%12121
  1. 1 3 samples per group will be used as a minimum making the power 99.9%.

qRT-PCR

Summary of original data estimated from graph reported in Supplemental Figure 20.

TreatmentsiRNAMeanSDN
DMSOsiControl10022.93
siEF14.750.8383
1.3 µM olaparibsiControl90.514.53
siEF17.260.8383

Test family

  • Two-tailed t test, Wilcoxon-Mann-Whitney test, Bonferroni’s correction: alpha error = 0.025

Power Calculations performed with G*Power software, version 3.1.7 (Faul et al., 2007).

Group 1Group 2Effect size dA priori powerGroup 1 sample sizeGroup 2 sample size
siControl cells treated with DMSOsiEF1 cells treated with DMSO5.8771398.3%33
siControl cells treated with olaparibsiEF1 cells treated with olaparib8.0910899.9%33

Test family

  • Due to the large difference in variance, these parametric tests are only used for comparison purposes. The sample size is based on the non-parametric tests listed above.

  • ANOVA: Fixed effects, special, main effects and interactions: alpha error = 0.05.

Power Calculations performed with G*Power software, version 3.1.7 (Faul et al., 2007).

ANOVA F test statistic and partial η2performed with R software, version 3.2.2 (R Development Core Team, 2015).

GroupsF test statisticPartial η2Effect size fA priori powerTotal sample size
A673 cells transfected with
siControl or siEF1 and treated
with DMSO or olaparib
F(1,8) = 129.85 (main effect: siRNA)0.941964.0287799.2%161
(4 groups)
  1. 1 12 samples (3 per group) will be used based on the planned comparisons making the power 99.9%.

Test family

  • Due to the large difference in variance, these parametric tests are only used for comparison purposes. The sample size is based on the non-parametric tests listed above.

  • 2 tailed t test, difference between two independent means, Bonferroni’s correction: alpha error = 0.025

Power Calculations performed with G*Power software, version 3.1.7 (Faul et al., 2007).

Group 1Group 2Effect size dA priori powerGroup 1 sample sizeGroup 2 sample size
siControl cells
treated with DMSO
siEF1 cells treated
with DMSO
5.8771399.1%33
siControl cells
treated with olaparib
siEF1 cells treated
with olaparib
8.0910880.6%12121
  1. 1 3 samples per group will be used based on the other comparions making the power 99.9%.

References

  1. 1
  2. 2
  3. 3
  4. 4
    Stereotactic ablative body radiation therapy for tumors in the lung in octogenarians: a retrospective single institution study
    1. NA Cannon
    2. P Iyengar
    3. H Choy
    4. R Timmerman
    5. J Meyer
    6. JE Butrynski
    7. DC Harmon
    8. JA Morgan
    9. S George
    (2014)
    BMC Cancer, 14, 10.1186/1471-2407-14-971.
  5. 5
  6. 6
  7. 7
    An open investigation of the reproducibility of cancer biology research
    1. TM Errington
    2. E Iorns
    3. W Gunn
    4. FE Tan
    5. J Lomax
    6. BA Nosek
    (2014)
    eLife, 3, 10.7554/eLife.04333.
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
    Radiochemical assay of glutathione S-epoxide transferase and its enhancement by phenobarbital in rat liver in vivo
    1. J Marniemi
    2. MG Parkki
    3. AJ Legrand
    4. S Choul-Li
    5. C Spriet
    6. T Idziorek
    (1975)
    Biochemical Pharmacology, 24, 10.1371/journal.pone.0055883.
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
    R: A language and environment for statistical computing. R Foundation for Statistical Computing
    1. R Development Core Team
    (2015)
    R: A language and environment for statistical computing. R Foundation for Statistical Computing.
  18. 18
  19. 19

Decision letter

  1. Joaquín M Espinosa
    Reviewing Editor; University of Colorado School of Medicine, United States

In the interests of transparency, eLife includes the editorial decision letter and accompanying author responses. A lightly edited version of the letter sent to the authors after peer review is shown, indicating the most substantive concerns; minor comments are not usually included.

Thank you for submitting your work entitled "Registered report: Systematic identification of genomic markers of drug sensitivity in cancer cells" for consideration by eLife. Your article has been reviewed by three peer reviewers, and the evaluation has been overseen by a Reviewing Editor and Charles Sawyers as the Senior Editor. The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

Summary:

This Registered report aims at assessing the reproducibility of the results of a large-scale pharmacogenomic study published in Nature in 2012. The study leverages a collection of over 600 cell lines screened across 138 drugs.

One of the major hurdles in assessing the reproducibility of this study is its scale. In the registered report the authors propose to reproduce a single finding that was presented in one of the main figures of the original report. While this finding is indeed an important part of the original study it does not embody a large part of the original conclusions: The original paper demonstrated that a drug screen performed across a large collection of cell lines was able to capture accurately a large number of known clinically relevant drug response biomarkers as well as preferential cancer type sensitivities known to occur in the clinic (for example that BRAF mutant colorectal cancers are less responsive to BRAF inhibitor than melanomas). It also showed that genomic modeling of the drug response could yield biological insights into drug mechanism of action (Elastic Net analysis outputs). This was distinct from previous smaller scale efforts (and also valuable for other uses) that had been reported prior to the original report (NCI60 results, for example). The reviewers encourage this Reproducibility Report after addressing the following concerns:

Essential revisions:

1) The Introduction of the registered report should better reflect this challenge and the breadth of the results presented in the original report.

For example: "To confirm their high throughput approach Garnett and colleagues explored one novel interaction…". This is similarly an over simplification. Several results "Confirmed" the high throughput approach capability of capturing clinically and biologically relevant drug responses: Chiefly a large number of known gene-drug interactions for clinically active drugs.

End of Introduction: It should be further clarified that the clinical results reported for Olaparib in Ewing's sarcoma correspond to single agent olaparib only. Combinations with olaparib are currently explored but no results have been made public.

Technical aspects:

2) Protocol 1; Procedure; 1: Authors should employ duplicate plates.

3) Protocol 1; Procedure; 4a: The assay (fix and stain) should be stopped when the control (untreated plates) contain at least 100 colonies (ideally over 200).

4) Protocol 1; Known differences from the original study: It is not clear why the authors have chosen not to include all the controls that were presented in the original study. Positive and negative control in a consistent disease background (breast) of BRCA1/2 deficient and proficient cell lines are important to show that the assay is correctly capturing differential sensitivity to PARP inhibitors across genotypes.

5) Protocol 2, first paragraph: As PARP inhibitor sensitivity relies on replication/proliferation of the cells mechanistically it will be important to show that all cell lines are in good health and proliferative in no drug condition. This is particularly important for MPCs that can be more challenging than average cell lines to maintain in culture. The protocol should include a test for appropriate proliferation.

6) Protocol 2; Procedure; 1: Related to the point above: All assays should include a measurement of proliferation to show that drug treatment occurred while cells were replicating since PARP inhibitor sensitivity depends on replication. Furthermore, differential replication across lines can yield over or underestimate of sensitivity.

7) Protocol 2; Procedure; 6: What level of knock down would be deemed sufficient to declare that the gene expression was affected but no biological effect observed?

8) It is unfortunate that some of the cell lines originally used in Garnett et al. were not available in this Reproducibility Project. However, it will be very useful to the scientific community that this Reproducibility Project and the related reagents and cell lines will be made available for other researchers to reproduce this work.

9) Regarding the statistical analyses, the authors should be aware that repeating the experiment twice or three times on the same cell lines will not give completely independent results which may impact on the results. However, it seems that the power calculations (at least for protocol 1) are conducted using only one run of the experiment. Secondly the power calculation assumes that the observed results will be as strong as those seen in Garnett et al. Even for a real finding, this may be optimistic due to the large number of tests carried out by Garnett and the "winner's curse", i.e. the fact that the most striking findings in a multiple testing context tend to be upwardly biased.

https://doi.org/10.7554/eLife.13620.003

Author response

Essential revisions:

1) The Introduction of the registered report should better reflect this challenge and the breadth of the results presented in the original report.

For example: "To confirm their high throughput approach Garnett and colleagues explored one novel interaction…". This is similarly an over simplification. Several results "Confirmed" the high throughput approach capability of capturing clinically and biologically relevant drug responses: Chiefly a large number of known gene-drug interactions for clinically active drugs.

Thank you for this suggestion. We have revised the beginning of the Introduction to highlight the large undertaking as well as the utility of the approach and the accurate capture of known associations. The sentence in the first paragraph of the Introduction has also been revised.

End of Introduction: It should be further clarified that the clinical results reported for Olaparib in Ewing's sarcoma correspond to single agent olaparib only. Combinations with olaparib are currently explored but no results have been made public.

We have revised this line to reflect this important aspect.

Technical aspects:

2) Protocol 1; Procedure; 1: Authors should employ duplicate plates.

We intended to perform the experiment in duplicate and have revised the manuscript to better reflect this.

3) Protocol 1; Procedure; 4a: The assay (fix and stain) should be stopped when the control (untreated plates) contain at least 100 colonies (ideally over 200).

Thank you for clarifying the sufficient number of colonies in the vehicle treated plates. We have revised the manuscript to reflect this.

4) Protocol 1; Known differences from the original study: It is not clear why the authors have chosen not to include all the controls that were presented in the original study. Positive and negative control in a consistent disease background (breast) of BRCA1/2 deficient and proficient cell lines are important to show that the assay is correctly capturing differential sensitivity to PARP inhibitors across genotypes.

The DoTc2-4510 cell line (uterus tissue that is mutant for BRCA2 and wild-type for BRCA1) was already included as a positive control and we have added the MES-SA cell line (uterus tissue that is wild-type for BRCA1 and BRCA2) as a negative control in the revised Registered Report.

5) Protocol 2, first paragraph: As PARP inhibitor sensitivity relies on replication / proliferation of the cells mechanistically it will be important to show that all cell lines are in good health and proliferative in no drug condition. This is particularly important for MPCs that can be more challenging than average cell lines to maintain in culture. The protocol should include a test for appropriate proliferation.

Thank you for this comment. We have included additional measurements at 24 and 48 hr after treatment (in addition to the originally planned 72 hr measurement) for cells not treated with drug. This will occur for the seeding optimization (Step 1) and for each replicate of the assay (Steps 2-7).

6) Protocol 2; Procedure; 1: Related to the point above: All assays should include a measurement of proliferation to show that drug treatment occurred while cells were replicating since PARP inhibitor sensitivity depends on replication. Furthermore, differential replication across lines can yield over or underestimate of sensitivity.

Thank you for this comment. We have added a test for proliferation for each biological replicate in the revised manuscript.

7) Protocol 2; Procedure; 6: What level of knock down would be deemed sufficient to declare that the gene expression was affected but no biological effect observed?

We do not have information about what level of knock down is necessary in order to observe a biological effect in this assay. The original study does not indicate a threshold, but does report the level of knock down (Supplemental Figure 20). Whether the replication attempt is capable of achieving the same degree of knockdown is important to consider as is the ability to observe a biological effect. Instead of declaring a threshold of knock down we plan to compare our gene expression levels to those of the original study as well as the viability assay results. This will allow the research community to assess this question.

8) It is unfortunate that some of the cell lines originally used in Garnett et al. were not available in this Reproducibility Project. However, it will be very useful to the scientific community that this Reproducibility Project and the related reagents and cell lines will be made available for other researchers to reproduce this work.

We agree and for any reagent or cell line not already available to the research community through a commercial supplier or repository we will work to make these valuable reagents available for other researchers.

9) Regarding the statistical analyses, the authors should be aware that repeating the experiment twice or three times on the same cell lines will not give completely independent results which may impact on the results. However, it seems that the power calculations (at least for protocol 1) are conducted using only one run of the experiment. Secondly the power calculation assumes that the observed results will be as strong as those seen in Garnett et al. Even for a real finding, this may be optimistic due to the large number of tests carried out by Garnett and the "winner's curse", i.e. the fact that the most striking findings in a multiple testing context tend to be upwardly biased.

We agree that while repeating the experiment multiple times on the same cell lines does not give complete independence. However, we plan to perform the experiment on several independent samples derived from the same population of cells. While not complete independence, it is as much as can be obtained for a given cell line based experiment – similar to using multiple mice all of which have the same inbred genetic background. However, for Protocol 1 the cell line is the biological replicate, opposed to the random sample from a given cell line. The power calculations were conducted to determine how many Ewing’s sarcoma cell lines and how many osteosarcoma cell lines are necessary to conduct the proposed test. Since each cell line is independent of each other, the number of cell lines of a given disease type were determined to achieve at least 80% power.

Regarding the approach used for the power calculations, we agree there are approaches one could take to guard against inflated effect sizes, such as utilizing the 95% confidence interval of the effect size. However, the Reproducibility Project: Cancer Biology is designed to conduct replications that have 80% power to detect the point estimate of the originally reported effect size. While this has the limitation of being underpowered to detect smaller effects than what is originally reported, this standardizes the approach across all studies to be designed to detect the originally reported effect size with at least 80% power.

https://doi.org/10.7554/eLife.13620.004

Article and author information

Author details

  1. John P Vanden Heuvel

    1. Indigo Biosciences, State College, United States
    2. Veterinary and Biomedical Sciences, Penn State University, University Park, PA, United States
    Contribution
    JPVH, Drafting or revising the article
    Competing interests
    JPVH: Indigo Biosciences is a Science Exchange associated laboratory
  2. Jessica Bullenkamp

    King's College London, London, United Kingdom
    Contribution
    JB, Drafting or revising the article
    Competing interests
    No competing interests declared.
  3. Reproducibility Project: Cancer Biology

    Contribution
    RP:CB, Conception and design, Drafting or revising the article
    For correspondence
    tim@cos.io
    Competing interests
    RP:CB: EI, FT, NP, and JL are employed and hold shares in Science Exchange, Inc
    1. Elizabeth Iorns, Science Exchange, Palo Alto, United States
    2. William Gunn, Mendeley, London, United Kingdom
    3. Fraser Tan, Science Exchange, Palo Alto, United States
    4. Joelle Lomax, Science Exchange, Palo Alto, United States
    5. Nicole Perfito, Science Exchange, Palo Alto, United States
    6. Timothy Errington, Center for Open Science, Charlottesville, United States

Funding

Laura and John Arnold Foundation

  • Reproducibility Project: Cancer Biology

The funder had no role in study design or the decision to submit the work for publication.

Acknowledgements

The Reproducibility Project: Cancer Biology core team would like to thank the original authors, in particular Dr. Cyril Benes and Dr. Ivan Stamenkovic, for generously sharing critical information as well as reagents to ensure the fidelity and quality of this replication attempt. We thank Courtney Soderberg at the Center for Open Science for assistance with statistical analyses. We would also like to thank the following companies for generously donating reagents to the Reproducibility Project: Cancer Biology; American Type and Tissue Collection (ATCC), Applied Biological Materials, BioLegend, Charles River Laboratories, Corning Incorporated, DDC Medical, EMD Millipore, Harlan Laboratories, LI-COR Biosciences, Mirus Bio, Novus Biologicals, Sigma-Aldrich, and System Biosciences (SBI).

Reviewing Editor

  1. Joaquín M Espinosa, Reviewing Editor, University of Colorado School of Medicine, United States

Publication history

  1. Received: December 8, 2015
  2. Accepted: May 25, 2016
  3. Version of Record published: June 23, 2016 (version 1)

Copyright

© 2016, Vanden Heuvel et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 878
    Page views
  • 149
    Downloads
  • 0
    Citations

Article citation count generated by polling the highest count across the following sources: PubMed Central, Scopus, Crossref.

Comments

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Download citations (links to download the citations from this article in formats compatible with various reference manager tools)

Open citations (links to open the citations from this article in various online reference manager services)

Further reading

  1. MJ Garnett et al.
    1
  2. Curated by Roger Davis et al.
    Collection Updated

    Investigating reproducibility in preclinical cancer research.

    1. Biophysics and Structural Biology
    2. Genes and Chromosomes
    Matthew Allan Sdano et al.
    Research Article