1. Human Biology and Medicine
Download icon

Registered report: RAF inhibitors prime wild-type RAF to activate the MAPK pathway and enhance growth

  1. Ajay Bhargava
  2. Steven Pelech
  3. Ben Woodard
  4. John Kerwin
  5. Nimet Maherali
  6. Reproducibility Project: Cancer Biology Is a corresponding author
  1. Shakti BioResearch, United States
  2. Kinexus Bioinformatics Corporation, Canada
  3. University of Maryland, United States
  4. Harvard Stem Cell Institute, United States
Registered Report
Cited
0
Views
736
Comments
0
Cite as: eLife 2016;5:e09976 doi: 10.7554/eLife.09976

Abstract

The Reproducibility Project: Cancer Biology seeks to address growing concerns about reproducibility in scientific research by conducting replications of selected experiments from a number of high-profile papers in the field of cancer biology. The papers, which were published between 2010 and 2012, were selected on the basis of citations and Altmetric scores (Errington et al., 2014). This Registered Report describes the proposed replication plan of key experiments from 'RAF inhibitors prime wild-type RAF to activate the MAPK pathway and enhance growth' by Hatzivassiliou and colleagues, published in Nature in 2010 (Hatzivassiliou et al., 2010). Hatzivassiliou and colleagues examined the paradoxical response of RAF-WT tumors to treatment with RAF inhibitors. The key experiments being replicated include Figure 1A, in which the original authors demonstrated that treatment of a subset of BRAFWT tumor cell lines with RAF small molecule inhibitors resulted in an increase in cell viability, Figure 2B, which reported that RAF inhibitor activation of the MAPK pathway was dependent on CRAF but not BRAF, and Figure 4A, where the dimerization of BRAF and CRAF was modulated by the RAF inhibitor PLX4720, but not GDC-0879. The Reproducibility Project: Cancer Biology is a collaboration between the Center for Open Science and Science Exchange, and the results of the replications will be published by eLife.

https://doi.org/10.7554/eLife.09976.001

Introduction

Mutations activating the H/K/N-RAS>B/C-RAF>MEK1/2>ERK1/2 signaling pathways are commonly found in many types of cancer, making members of this pathway promising drug targets. Several small molecule inhibitors have been created that target the three RAF isoforms. However, early observations using these drugs noted a surprising paradox; while drugs targeting CRAF were able to inhibit CRAF activity in vitro, they paradoxically activated the MEK1/2>ERK1/2 signaling modules in vivo. This activation was not due to direct activation of signaling components downstream of RAF (Hall-Jackson et al., 1999a; 1999b).

Hatzivassiliou and colleagues found that RAF inhibitors, while effective in blocking signaling in BRAFV600E mutant (MT) cancer cell lines, paradoxically increased cell proliferation in BRAFWT cancer cell lines (Hatzivassiliou et al., 2010). Their findings were published along with two other reports demonstrating similar results (Heidorn et al., 2010; Poulikakos et al., 2010) and provided a key insight into the mechanism of paradoxical RAF activation in BRAFWT cells, showing that it depended on drug-induced dimerization of wild-type (WT) RAF isoforms, specifically CRAF.

In Figure 1A, Hatzivassiliou and colleagues treated 19 cancer cell lines, comprising 4 BRAFV600E mutant lines, 7 RAF/RAS-WT lines, and 8 KRAS-MT lines, with varying concentrations of two RAF inhibitors and calculated the IC50 value for each drug in each cell line. They found that, although cancer cell lines carrying the BRAFV600E mutation were susceptible to the RAF inhibitors, BRAFWT cell lines were not. This experiment will be replicated in Protocol 1.

To elucidate whether CRAF or BRAF contributed to MEK signaling in RAF-treated KRAS-mutant cells, the authors used inducible shRNA cell lines to test whether BRAF or CRAF were necessary for the activation of MEK1/2 in HCT116 cells, which are KRAS mutant. As reported in Figure 2B, silencing CRAF reversed MEK activation upon treatment with the RAF inhibitors GDC-0879 and PLX4720. This experiment will be replicated in Protocol 2.

To test whether inhibitor priming was mediated by the inhibitors’ conformational effects on the RAF kinase domain, the authors assayed BRAF-CRAF heterodimerization through a series of immunoprecipitation assays coupled with or without RAF inhibitors. In Figure 4A, they reported that the CRAF kinase domain forms a stable complex with the BRAF kinase domain when inhibitors are not present. However, in the presence of the RAF inhibitor PLX4720, this CRAF-BRAF heterodimer kinase domain interaction is destabilized. In the presence of DGC-0879, the CRAF-BRAF interaction is stabilized. This experiment will be replicated in Protocol 3.

Hatzivassiliou’s work was published along with two companion papers; Heidorn and colleagues showed that drugs targeting BRAFV600E caused dimerization with CRAF and pathway activation (Heidorn et al., 2010), while Poulikakos and colleagues observed that paradoxical RAF activation only occurred in the context of BRAFWT (Poulikakos et al., 2010). In a subsequent study, Poulikakos and colleagues showed that dimerization was a critical factor in allowing a variant of BRAFV600E to demonstrate enhanced activity as compared to BRAFV600E (Poulikakos et al., 2011). Work by Joseph and colleagues confirmed the findings of Hatzivassiliou and colleagues that BRAFV600E cell lines were sensitive to treatment with the BRAF inhibitor PLX4720, while RAS mutant/BRAFWT or RAS/RAF WT cell lines were not, and that MEK1/2>ERK1/2 signaling was activated in these BRAFWT lines (Joseph et al., 2010). Lee and colleagues assayed a panel of BRAFV600E, NRAS mutant, or BRAF/NRAS WT cell lines by treating them with PLX4720. They observed that PLX4720 inhibited ERK signaling in BRAFV600E cells, but they did not observe paradoxical MEK1/2>ERK1/2 activation in BRAFWT lines. They attributed this effect to a lower percentage of serum in their culture conditions as compared to those used in previous studies. They then examined colony formation to assess drug effects on cell survival, and saw strong growth inhibition exclusively in BRAFV600E cells (Lee et al., 2010). Paradoxical activation was also observed in BRAFWT cells by Carnahan and colleagues and by Halaban and colleagues (Carnahan et al., 2010; Halaban et al., 2010). A year later, Kaplan and colleagues published corroborating evidence that PLX4720 paradoxically activated MEK1/2>ERK1/2 signaling in BRAFWT cells. They also confirmed that silencing of CRAF abrogated this activation of MEK1/2>ERK1/2 signaling (Kaplan et al., 2011).

Materials and methods

Unless otherwise noted, all protocol information were derived from the original paper, references from the original paper, or information obtained directly from the authors. An asterisk (*) indicates data or information provided by the Reproducibility Project: Cancer Biology core team. A hashtag (#) indicates information provided by the replicating lab. All references to Figures refer to the original study.

Protocol 1: Assessing cell viability of a panel of cancer cell lines treated with RAF and MEK inhibitors

This protocol describes the treatment of a panel of human cancer cell lines with or without mutations in BRAF or RAS with drugs targeting RAF and MEK and assessing cell viability. This experiment is a replication of Figure 1A.

Sampling

  • This experiment will be repeated four times.

    • See Power calculations for details.

  • Each experiment consists of three cohorts:

    • Cohort 1: Cell lines treated with a range of concentrations of PLX4720

      • 20 µM

      • 10 µM

      • 5 µM

      • 2.5 µM

      • 1.25 µM

      • 0.625 µM

      • 0.313 µM

      • 0.156 µM

      • 0.078 µM

    • Cohort 2: Cell lines treated with a range of concentrations of GDC-0879

      • 20 µM

      • 10 µM

      • 5 µM

      • 2.5 µM

      • 1.25 µM

      • 0.625 µM

      • 0.313 µM

      • 0.156 µM

      • 0.078 µM

    • Cohort 3: Cell lines treated with a range of concentrations of PD0325901

      • 20 µM

      • 10 µM

      • 5 µM

      • 2.5 µM

      • 1.25 µM

      • 0.625 µM

      • 0.313 µM

      • 0.156 µM

      • 0.078 µM

  • Each cohort consists of three cell lines:

    • A375 cells

      • BRAFV600E mutant

    • MeWo cells

      • RAFWT/RASWT

    • HCT116

      • RAS mutant

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
A375CellsATCCCRL-1619DMEM + 10% FBS
MeWoCellsATCCHTB-65EMEM + 10% FBS
HCT116CellsATCCCCL-247McCoy's 5a Medium
Modified + 10% FBS
PLX4720InhibitorSymansisSY-PLX4720
PD0325901InhibitorSymansisSY-PD0325901
GDC-0879InhibitorSelleckchemS1104Replaces Genentech and
Array BioPharma source
Fluorescent plate readerEquipmentBioTek FLx800
15-cm cell culture platesEquipmentCorning430599Original unspecified
96-well platesMaterialsCorning3903
DMEMMediumATCC30–2002Original unspecified
EMEMMediumATCC30–2003Original unspecified
McCoy's 5a medium modifiedMediumATCC30–2007Original unspecified
Fetal bovine serum (FBS)ReagentATCC30–2020Original unspecified
DMSOReagentFisherD128–500Original unspecified
Cell Titer Glo kitReagentPromegaG7570

Procedure

Notes:

  • A375 cells are maintained in DMEM supplemented with 10% FBS.

  • MeWo cells are maintained in EMEM supplemented with 10% FBS.

  • HCT116 cells are maintained in McCoy's 5a Medium Modified supplemented with 10% FBS.

    • All cell lines are kept at 37°C/5% CO2.

  • All cell lines will be sent for STR profiling and mycoplasma testing

  1. Expand cell lines as needed in 15-cm plates.

  2. Determine range of detection of replicating lab’s plate reader:

    1. Plate 500 – 1.6 × 104 A375, MeWo, and HCT116 cells in quadruplicate wells in a 96 well plate with 100 µl of appropriate medium. Incubate 5 days.

      1. Plate medium alone (no cells).

      2. 500 cells/well

      3. 1000 cells/well

      4. 2000 cells/well

      5. 4000 cells/well

      6. 8000 cells/well

      7. 16,000 cells/well

    2. Five days later measure cell viability with the Cell Titer Glo kit according to manufacturer’s instructions.

      1. Plot relative luminescence to cells/well.

      2. Use seeding density for each cell line that give sub-confluency in 5 days and where the signal is still in the linear range at the end of the assay.

  3. Seed cells, at density determined in Step 2 above, in quadruplicate wells (technical replicates) in 96-well plates and incubate overnight. Note: Information in Steps 2 through 3 is derived from Hoeflich and colleagues (Hoeflich et al., 2009).

    1. Include control wells that contain media but no cells to control for background luminescence.

    2. Also include wells with cells that will remain untreated (no DMSO).

  4. The next day, treat cells with varying doses of each drug diluted in DMSO first and then add fresh media (to avoid excess DMSO toxicity, keep final DMSO percentage below 0.2%).

    1. Perform serial dilutions of drugs in DMSO in a 96-well plate.

      1. Drug doses (twofold dilution series, see Sampling section for concentrations)

      2. Vehicle control (DMSO)

    2. Further dilute compounds in fresh growth media.

    3. Replace media in wells with new media containing appropriate drug concentrations.

    4. d. As controls, include wells of untreated cells and wells of cells treated only with vehicle.

  5. Incubate cells for 4 days.

    1. Do not replace media during this incubation.

  6. Measure cell viability with the Cell Titer Glo kit according to the manufacturer’s instructions.

    1. Record luminescence.

    2. For each treated well, subtract average background luminescence calculated from media-only wells. The background luminescence defines 0% viability (baseline).

    3. Normalize luminescence to the average of the vehicle treated cells. The vehicle-treated cells define 100% viability (top).

    4. Fit data to a four-parameter curve, with the top and baseline held constant at 100% and 0% each, and calculate the absolute EC50 value (where the curve crosses 50% viability) for each drug treatment of each cell line.

      1. Only report EC50 values that can be accurately estimated. Otherwise report as >20 µM or <0.078 µM.

  7. Repeat Steps 3–6 independently three additional times.

Deliverables

  • Data to be collected:

    • All raw luminescence values

    • Luminescence values adjusted to compensate for background luminescence

    • Luminescence values normalized to vehicle treated cells.

    • EC50 values for each cell line and each drug treatment (as seen in Figure 1A)

Confirmatory analysis plan

  • Statistical analysis of the replication data:

    • n/a

  • Meta-analysis of original and replication attempt effect sizes:

    • The replication data will be presented as a mean with 95% confidence intervals and will include the original data point, calculated directly from the graph, as a single point on the same plot for comparison.

Known differences from the original study

  • All known differences are listed in the 'Materials and reagents' section above with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

  • While the original experiment examined nineteen cell lines, the replication will be restricted to three cell lines; A375, representing the BRAFV600E mutant lines, MeWo, representing the RAFWT/RASWT cell lines, and HCT116, representing the RAS mutant cell lines. HCT116 is used in a subsequent experiment described in Protocol 2.

  • The replicating lab will plate the cells in a 96-well plate as opposed to a 384-well plate.

Provisions for quality control

All data obtained from the experiment - raw data, data analysis, control data, and quality control data - will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/0hezb/).

  • STR profiling and mycoplasma detection results

Protocol 2: Assessing CRAF and BRAF roles in drug-dependent activation of MEK

This protocol describes the treatment of HCT116 (KRAS-MT) cells expressing doxycycline-inducible shRNAs against BRAF and CRAF, and treatment with RAF inhibitors followed by Western blot examination of activation of MEK. This experiment is a replication of Figure 2B.

Sampling

  • This experiment will be repeated seven times for a final power of ≥80%.

    • The original data presented is qualitative (representative images). In order to determine an appropriate number of replicates to perform initially, we have estimated the sample sizes required based on a range of potential variance.

    • See Power calculations for details.

  • The experiment consists of two cohorts:

    • Cohort 1: HCT116 cells with dox-inducible shRNA against BRAF

    • Cohort 2: HCT116 cells with dox-inducible shRNA against CRAF

  • Each cohort will receive the following treatments:

    • No dox:

      • DMSO

      • 0.1 µM PLX4720

      • 1 µM PLX4720

      • 10 µM PLX4720

    • Dox:

      • DMSO

      • 0.1 µM PLX4720

      • 1 µM PLX4720

      • 10 µM PLX4720

    • No dox:

      • DMSO

      • 0.1 µM GDC-0879

      • 1 µM GDC-0879

      • 10 µM GDC-0879

    • Dox:

      • DMSO

      • 0.1 µM GDC-0879

      • 1 µM GDC-0879

      • 0 µM GDC-0879

  • Lysates from each treatment are probed for:

    • phospho-MEK 1/2

    • total MEK 1/2

    • BRAF

    • CRAF

    • Actin (additional loading control)

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
HCT116 cells expressing doxycycline-inducible
shRNA directed against BRAF
CellsProvided by original authors
HCT116 cells expressing doxycycline-inducible
shRNA directed against CRAF
CellsProvided by original authors
DoxycyclineDrugAlfa AesarJ67043-AEOriginal unspecified
PLX4720 RAF inhibitorDrugSymansisSY-PLX4720
GDC-0879InhibitorSelleckchemS1104Replaces Genentech
and Array BioPharma
source
DMSOReagentSigmaD2650-5X5MLOriginal unspecified
McCoy's 5a MediumMediumATCC30-2007
Fetal bovine serumReagentSeradigm1400-500GOriginal unspecified
15-cm cell culture platesMaterialsCorning430599Original unspecified
Protease inhibitor mixture; Complete MiniReagentRoche Applied Science04693159001
Phosphatase inhibitor mixReagentPierce78420
SDS-PAGE (4–20%) Tris-GlycineMaterialsBioRad456–1094Original unspecified
Nitrocellulose membraneMaterialsBioRad170–4158Original unspecified
ECL detection reagentsReagentsAmershamRPN2232
Rabbit phospho MEK (pMEK) 1/2 (Ser217/221) antibodyAntibodyCell Signaling Technology91211:1000
Rabbit MEK 1/2 antibody (clone 47E6)AntibodyCell Signaling Technology91261:1000
Mouse anti-BRAF antibodyAntibodySanta Cruz Biotechnologysc-52841:1000
Goat Anti-Mouse IgG-HRPAntibodyPierce314321:5,000–1:200,000
Goat Anti-Rabbit IgG-HRPAntibodyPierce314601:5,000–1:200,000
Mouse anti-CRAF antibodyAntibodyBD6101511:1000
Mouse anti-ß-actin (HRP conjugate) (clone 8H10D10)AntibodyCell Signaling Technology12262Not originally used
Kaleidoscope prestained standards protein ladderReagentBioRad161–0324Original unspecified
4–15% Mini-PROTEAN TGX Stain-Free SDS-PAGE gelMaterialsBioRad456–8085Original unspecified
Trans-Blot Turbo Mini Nitrocellulose transfer packsEquipmentBioRad170–4158Original unspecified

Procedure

Notes:

  • Some information derived from Hoeflich and colleagues (Hoeflich et al., 2006).

  • HCT116 cells are maintained in McCoy’s 5a Medium modified supplemented with 10% FBS at 37°C/5% CO2.

  • All cells will be sent for STR profiling and mycoplasma testing.

  1. Determine if concentration of doxycycline to induce knockdown of BRAF and CRAF in HCT116 cells needs to be optimized. Before beginning experiment, perform protocol details as outlined below, without drug treatment (Step 4) and only analyzing expression of BRAF, CRAF, and actin (Step 8). Perform with at least 1 well per group (with and without dox treatment for each cell line).

    1. Normalize BRAF and CRAF to actin levels.

    2. If level of depletion is not similar to reported levels in Figure 2B, further optimize conditions, such as increasing concentration of dox.

    3. Once conditions of knockdown are optimized, use for all replicates of experimental procedure.

  2. Seed #5x104 cells/well in 24-well plates for treatment.

    1. Seed 16 wells per cohort.

      1. 8 wells will be treated with dox.

      2. 8 wells will remain untreated.

  3. Induce shRNA expression of appropriate wells by treatment with 2 mg/ml dox for 3 days.

    1. This condition will be checked in Step 1 of this protocol and optimized if needed. Once optimized, use that condition for all replicates.

  4. Treat appropriate cells with varying concentrations of PLX4720 or GDC-0879 for 1 hr.

    1. Treat cells with varying doses of each drug diluted in DMSO first and then fresh media added (to avoid excess DMSO toxicity, keep final DMSO percentage below 0.2%).

    2. In each set of 8 wells (dox treated and untreated), treat as follows:

      1. Well 1: DMSO

      2. Well 2: 0.1 µM PLX4720

      3. Well 3: 1 µM PLX4720

      4. Well 4: 10 µM PLX4720

      5. Well 5: DMSO

      6. Well 6: 0.1 µM GDC-0879

      7. Well 7: 1 µM GDC-0879

      8. Well 8: 10 µM GDC-0879

  5. Lyse cells and harvest protein:

    1. Rinse cells in ice cold PBS.

    2. Lyse cells in ice cold lysis buffer: 0.5% NP40, 20 mM Tris pH7.5, 137 mM NaCl, 10% glycerol, 1 mM EDTA plus protease inhibitor mixture-complete mini, and phosphatase inhibitor mix.

    3. Spin lysate at 12,000xg for 5 min at 4°C.

      1. Transfer lysate to fresh tube after spinning.

    4. Quantify protein by the BCA method.

  6. Separate proteins by SDS-PAGE:

    1. #Adjust sample to 1.5 µg/µL with 2X Lammeli Buffer/H2O.

    2. #Boil sample for 5 min at >90°C.

      1. Load #10–20 µg of protein per lane on a #4–15% SDS-PAGE gel.

      2. Run alongside a size marker ladder.

      3. Transfer to nitrocellulose membrane using a #Trans-Blot Turbo Mini according to the manufacturer’s instructions.

        1. #Run at 25 V, 1 A for 30 min.

        2. *Confirm protein transfer by Ponceau staining.

  7. Block membrane in 5% non-fat dried milk in TBST (20 mM Tris pH 7.5, 136 mM NaCl, 0.1% Tween-20).

  8. Incubate membrane at 4°C overnight with primary antibodies #diluted in 5% milk in TBST:

    1. Mouse anti-BRAF; 1:1000 dilution; 86 kDa

    2. Mouse anti-CRAF; 1:1000 dilution; 74 kDa

    3. Rabbit anti-pMEK 1/2; 1:1000 dilution; 45 kDa

    4. Rabbit anti-total MEK 1/2; 1:1000 dilution; 45 kDa

    5. Mouse anti-ß-Actin-HRP; 1:1000 dilution; 42 kDa

      1. Run one gel/membrane per antibody; do not strip and reprobe membranes for multiple antibodies.

      2. Note: Actin serves as a loading control to ensure equal loading of lanes (additional).

  9. #Wash membranes 3 x 5 min in TBST.

  10. Incubate with HRP-conjugated secondary antibodies #diluted 1:20,000 in 5% milk in TBST for 1 hr at room temperature.

  11. Visualize bands with ECL detection kit according to manufacturer’s protocol.

    1. Quantify band intensity.

    2. For each drug and dose in each cell line (treated with or without dox), normalize pMEK values to total MEK values.

  12. Repeat Steps 2–11 independently six additional times.

Deliverables

  • Data to be collected:

    • Images of whole gel, including ladder, of shRNA optimization (Step 1).

    • Images of whole gel, including ladder (compare to Figure 2B).

    • Quantification of band intensities; phospho-protein levels normalized to total protein levels.

Confirmatory analysis plan

  • Statistical analysis of the replication data:

    • Compare band intensities across all groups.

      • Four-way ANOVA (2 x 2 x 2 x 4 factorial) of the normalized pMEK values for each cell line (with or without dox), drug (PLX4720 or GDC-0879), and dose (0, 0.1, 1, and 10 µM) followed by:

        • Two-way interaction contrast of normalized pMEK values from BRAF and CRAF shRNA cell lines (with or without dox) across varying doses of GDC-0879 with the following Bonferroni corrected comparisons:

          • BRAF shRNA cell line with dox compared to without dox (across varying doses of GDC-0879)

          • CRAF shRNA cell line with dox compared to without dox (across varying doses of GDC-0879)

        • Two-way interaction contrast of normalized pMEK values from BRAF and CRAF shRNA cell lines (with or without dox) across varying doses of PLX4720 with the following Bonferroni corrected comparisons:

          • BRAF shRNA cell line with dox compared to without dox (across varying doses of PLX4720)

          • CRAF shRNA cell line with dox compared to without dox (across varying doses of PLX4720)

  • Meta-analysis of original and replication attempt effect sizes:

    • The replication data will be presented as a mean with 95% confidence intervals and will include the original data point, calculated directly from the representative image, as a single point on the same plot for comparison.

Known differences from the original study

  • All known differences are listed in the 'Materials and reagents' section above with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

  • The replication attempt will use actin as an additional loading control not used in the original study.

Provisions for quality control

All data obtained from the experiment - raw data, data analysis, control data, and quality control data - will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/0hezb/).

  • STR profiling and mycoplasma detection results.

  • Induced shRNA knockdown conditions will be checked, and optimized if needed, prior to proceeding with the experiment.

  • Image of Ponceau staining confirming protein transfer.

  • Protein loading will be confirmed using actin.

Protocol 3: Biochemical heterodimerization assay with recombinant RAF proteins in the presence or absence of RAF inhibitors

This protocol describes how to perform immunoprecipitation and Western blot analysis with recombinant CRAF and BRAF kinase domains in the presence or absence of the RAF inhibitors PLX4720 or GDC-0879. Wild-type BRAF and BRAFV600E kinase domains will be tested in the presence of wild-type CRAF. This experiment is a replication of Figure 4A.

Sampling

  • This experiment will be repeated six times for a final power of ≥80%.

    • The original data presented are qualitative (representative images). In order to determine an appropriate number of replicates to perform initially, we have estimated the sample sizes required based on a range of potential variance. We will also determine sample size post hoc.

    • See Power calculations for details.

  • Each experiment consists of two cohorts:

    • Cohort 1: CRAF + BRAFWT

    • Cohort 2: CRAF + BRAFV600E

  • Each cohort is incubated with CRAF and treated for 1 hr with:

    • DMSO

    • 10 µM of PLX4720

    • 10 µM GDC-0879

    • 1 mM AMP-PCP

  • Include a sample of CRAF, without BRAF, treated with DMSO (negative control)

  • Immunoprecipitate CRAF from each sample and probe for CRAF and BRAF.

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
pFBHTc delta N 1-416 BRAFPlasmidMRC-PPUDU586Made at Genentech and
Array BioPharma.
Communication with authors.
pFBHTc delta N 1-416 BRAFV600EPlasmidMRC-PPUDU603
Recombinant glutathione
S-transferase (GST)-CRAF
(Y340D/Y341D) kinase domain
ProteinInvitrogenPV3805Original catalog # not specified
Assay bufferChemicalSpecific brand information
will be left up to the
discretion of the replicating
lab and recorded later
DMSOChemical
PLX4720InhibitorSymansisSY-PLX4720
GDC-0879InhibitorSelleckchemS1104Replaces Genentech and Array
BioPharma source
Adenylylmethylenediphosphonate (AMP-PCP)ChemicalSigmaM7510Original catalog # not specified
Rabbit anti-GSTAntibodyCell Signaling Technology2622
Protein A agarose beadsChromatographyMilliporeIP02Original catalog # not specified
SDS-PAGE gelWestern materialsPrepared in replicating labOriginal unspecified
Precision Plus Protein All Blue StandardsReagentBioRad161–0393Original unspecified
Ponceau stainReagentSigmaP7170Not originally included
Nitrocellulose membraneMaterialPall CorporationPN 66485Original unspecified
Mouse anti-CRAF (clone 53)AntibodyBD Biosciences610151
Mouse anti-BRAFAntibodySigmaWH0000673M1
Goat anti-mouse IgG-HRPAntibodyInvitrogen Molecular ProbesA11029Replaces Alexa Fluor 488
goat anti-mouse IgG
from Invitrogen, cat# A11029
ECL Plus ReagentDetection assayLumigenPS-3Replaces TyphoonTM Scanner
from Amersham Bioscience
Fluor-S Max ScannerInstrumentBioRad

Procedure

  1. Express 6His-BRAFWT and 6His-BRAFV600E kinase domains (417–766) from pFBHTc delta N 1-416 BRAF and pFBHTc delta N 1-416 BRAFV600E vectors, respectively, using baculovirus cells, following lab standard procedures. Affinity purify proteins using a nickel column using lab standard procedures.

  2. Add 500 nM 6His-BRAFWT kinase domain or 6His-BRAFV600E kinase domain and 500 nM GST-CRAF kinase domain to assay buffer.

    1. Assay buffer: 25 mM HEPES, pH 7.4, 10 mM MgCl2, 0.01% (v/v) Triton X-100, and 2 mM DTT.

    2. Four samples of GST-CRAF + BRAFWT

    3. Four samples of GST-CRAF + BRAFV600E

    4. One sample with GST-CRAF alone.

  3. Incubate samples for 1 hr at room temperature in the presence of a fixed concentration of compound or vehicle:

    1. DMSO

    2. 10 µM of PLX4720

    3. 10 µM GDC-0879

    4. 1 mM AMP-PCP

      1. For all compounds, keep final DMSO concentration to 0.25%

  4. Immunoprecipitate GST-CRAF proteins with rabbit anti-GST antibody and protein A agarose beads following manufacturer’s instructions.

  5. Separate proteins by SDS-PAGE:

    1. Boil samples in SDS-Sample Buffer at 100°C for 5 min.

    2. Load samples on a SDS-PAGE gel.

      1. Electrophorese all samples on gel alongside a size marker ladder.

      2. Perform electrophoresis at 30 mA for 55 min, monitoring the dye front until it comes off the gel.

    3. Transfer to nitrocellulose membrane at 250 mA per gel for 1 hr.

      1. *Confirm protein transfer by Ponceau staining.

  6. Block membrane in 5% non-fat dried milk in TBST (20 mM Tris-HCl pH 7.5, 136 mM NaCl, 0.1% Tween-20) as recommended by manufacturer.

  7. Incubate membrane at 4°C overnight with antibodies against:

    1. Mouse anti-CRAF; 1:1000; 66 kDa

    2. Mouse anti-BRAF; 1:1000; 44 kDa

  8. Incubate with secondary antibody in 1X TBS for 1 hr at room temperature as recommended by manufacturer.

    1. Rinse the membrane at least three times with TBST.

  9. Detect secondary antibody and visualize bands with an imager.

    1. Quantify band intensity.

    2. For each sample, normalize BRAF IP values to GST-CRAF IP values.

    3. For each BRAF variant, normalize compound treatment to DMSO.

  10. Repeat experiment independently five additional times.

Deliverables

  • Data to be collected:

    • Full gel images with ladder positions marked

    • Quantification of band intensities

      • Raw measurements as well as normalized band intensities

    • Graph of mean band intensities across replicates

Confirmatory analysis plan

  • Statistical analysis of the replication data:

    • Bonferonni corrected one-sample t-tests of normalized BRAFWT values (normalized to GST-CRAF and then DMSO) of the following conditions compared to 1 (DMSO):

      • PLX4720

      • GDC-0879

      • AMP-PCP

    • Bonferonni corrected one-sample t-tests of normalized BRAFV600E values (normalized to GST-CRAF and then DMSO) of the following conditions compared to 1 (DMSO):

      • PLX4720

      • GDC-0879

      • AMP-PCP

  • Meta-analysis of original and replication attempt effect sizes:

    • The replication data will be presented as a mean with 95% confidence intervals and will include the original data point, calculated directly from the representative image, as a single point on the same plot for comparison.

  • Additional exploratory analysis:

    • Two-way ANOVA of BRAF values (normalized to GST-CRAF) from DMSO, PLX4720, GDC-0879, or AMP-PCP treated samples for each BRAF variant incubated with GST-CRAF with the following Bonferroni corrected comparisons:

      • BRAFWT treated with DMSO compared to BRAFWT treated with PLX4720

      • BRAFWT treated with DMSO compared to BRAFWT treated with GDC-0879

      • BRAFWT treated with DMSO compared to BRAFWT treated with AMP-PCP

      • BRAFV600E treated with DMSO compared to BRAFV600E treated with PLX4720

      • BRAFV600E treated with DMSO compared to BRAFV600E treated with GDC-0879

      • BRAFV600E treated with DMSO compared to BRAFV600E treated with AMP-PCP

Known differences from the original study

  • All known differences are listed in the 'Materials and reagents' section above with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

  • The original data examined samples treated with AZ-628, a chemically unrelated ATP-competitive RAF inhibitor, which had a similar reported effect as GDC-0879. The replication will be restricted to examining only PLX4720 and GDC-0879, similar to the other experiments included in this replication attempt.

  • The replicating lab will use a modified version of their in-house Western Blot protocol with antibodies analyzed by an ECL detection system instead fluorescence based.

Provisions for quality control

All data obtained from the experiment - raw data, data analysis, control data, and quality control data - will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/0hezb/).

  • Image of Ponceau staining confirming protein transfer

Power calculations

Note: for details on the full set of power calculations, see https://osf.io/a4e75/.

Protocol 1

Summary of original data

  • Values estimated from original published figure.

Figure 1A: RAF inhibitors and MEK inhibitorEC50 (µM)
A375PLX47200.5
GDC-08790.3
PD0325901<0.0781
MeWoPLX4720>20
GDC-0879>20
PD0325901<0.0781
HCT116PLX4720>20
GDC-0879>20
PD03259010.18

Power calculations

  • Due to EC50 values, such as PLX4720 and GDC-0879 with MeWo and HCT116 cells, unable to be determined (i.e. they are above 20 µM), this replication attempt will not compare values, but instead report EC50 values that can be accurately estimated and compare them to the original reported values. If unable to obtain an EC50 estimate the highest or lowest dose, depending on the situation, will be reported, similar to the original report.

Protocol 2

Summary of original data reported in Figure 2B

  • Summary of data:

    • Band densities were obtained with Image Studio Lite (LiCOR) from published images.

    • Normalization was performed by dividing the phospho-band intensity by the total protein band intensity.

Figure 2B; assumed number of biological replicates = 3
hairpinDoxDrugDoseNormalized pMEK
band density
shBRAF-GDC-087900.1453
shBRAF-GDC-08790.10.8004
shBRAF-GDC-08791.00.1263
shBRAF-GDC-087910.00.0245
shBRAF+GDC-087900.0457
shBRAF+GDC-08790.10.6477
shBRAF+GDC-08791.00.1485
shBRAF+GDC-087910.00.0115
shBRAF-PLX472000.0795
shBRAF-PLX4720.10.1615
shBRAF-PLX4721.00.7760
shBRAF-PLX47210.01.0721
shBRAF+PLX47200.0861
shBRAF+PLX4720.10.1388
shBRAF+PLX4721.00.4522
shBRAF+PLX47210.00.9831
shCRAF-GDC-087900.2121
shCRAF-GDC-08790.10.7038
shCRAF-GDC-08791.00.1554
shCRAF-GDC-087910.00.0239
shCRAF+GDC-087900.1005
shCRAF+GDC-08790.10.1896
shCRAF+GDC-08791.00.0933
shCRAF+GDC-087910.00.0315
shCRAF-PLX472000.4166
shCRAF-PLX4720.10.7469
shCRAF-PLX4721.01.1387
shCRAF-PLX47210.01.2976
shCRAF+PLX47200.2945
shCRAF+PLX4720.10.2449
shCRAF+PLX4721.00.2983
shCRAF+PLX47210.00.2729

The original data does not indicate the error associated with multiple biological replicates. To identify a suitable sample size, power calculations were performed using different levels of relative variance using the values quantified from the reported image as the mean. At each level of variance the effect size was estimated and used to calculate the needed sample size to achieve at least 80% power with the indicated alpha error. The achieved power is reported.

Test family

  • Four-way ANOVA: Fixed effects, special, main effects and interactions, alpha error = 0.05

Power calculations

  • Performed with G*Power software, version 3.1.7 (Faul et al., 2007).

  • ANOVA F test statistic and partial η2 performed with R software, version 3.2.2 (R Core Team, 2015).

    • For a given relative variance, 10,000 simulations were run and the F statistic and partial η2 was calculated for each simulated data set.

GroupsVariance estimateF test statistic F(1,64)
(shRNA, Dox,
Drug interaction)
Partial η2Effect size fA priori powerTotal sample
size
(32 groups)
Normalized pMEK
in shBRAF or
shCRAF cells treated
with or without Dox
and varying doses
of GDC-0879 or
PLX4720
2%2513.610.972455.9406399.9%96
15%45.61150.396160.8099999.9%96
28%13.83180.167260.4481799.1%96
40%7.32030.096500.3268188.4%96

Test family

  • ANOVA: Fixed effects, special, main effects and interactions, alpha error = 0.05 for two-way interaction contrast.

Power calculations

GroupsVariance estimateF test statistic
F(1,64)
(shRNA, Dox)
Partial η2Effect size fA priori powerTotal sample
size
(32 groups)
Normalized pMEK
in shBRAF or shCRAF
cells treated with or
without Dox across
varying doses of
GDC-0879
2%327.180.836392.2610199.9%96
15%5.816510.083310.3014782.9%96
28%1.669280.025420.1615082.1%320
40%0.817950.012620.1130581.5%640
Normalized pMEK
in shBRAF or
shCRAF cells treated
with or without Dox
across varying doses
of PLX4720
2%7267.40.9912710.656199.9%96
15%129.1980.668731.4208299.9%96
28%37.07860.366830.7611599.9%96
40%18.16850.221110.5328199.9%96

Test family

  • ANOVA: Fixed effects, special, main effects and interactions, Bonferroni’s correction, alpha error = 0.025 for contrasts within each drug type

Power calculations

DrugGroup 1
across dose
Group 2
across dose
Variance
estimate
Effect
size f
A priori
power
Samples
per group
GDC-0879shBRAF (+ Dox)shBRAF (- Dox)2%1.7787499.9%3
15%0.2371784.6%6
28%0.1270581.2%19
40%0.0889480.4%38
shCRAF (+ Dox)shCRAF (- Dox)2%4.9762999.9%3
15%0.6635199.9%3
28%0.3554588.0%3
40%0.2488180.9%5
PLX4720shBRAF (+ Dox)shBRAF (- Dox)2%3.1381599.9%3
15%0.4184196.2%3
28%0.2241586.2%7
40%0.1569082.9%13
shCRAF (+ Dox)shCRAF (- Dox)2%18.208199.9%3
15%2.4277599.9%3
28%1.3005899.9%3
40%0.9104099.9%3
  • Based on these power calculations, we will then run the experiment seven times. Each time we will quantify band intensity. We will determine the standard deviation of band intensity across the biological replicates and combine this with the reported value from the original study to simulate the original effect size. We will use this simulated effect size to determine the number of replicates necessary to reach a power of at least 80%. We will then perform additional replicates, if required, to ensure that the experiment has more than 80% power to detect the original effect.

Protocol 3

Summary of original data reported in Figure 4A

  • Summary of data:

    • Normalization was performed by dividing the BRAF-band intensity by the captured GST-CRAF band intensity.

Figure 4A; assumed number of biological replicates = 3
BRAF variantDrugNormalized BRAF band density
WTDMSO1
WTAMP-PCP0.2
WTGDC-08792.7
WTPLX47200.2
V600EDMSO1
V600EAMP-PCP1.4
V600EGDC-08791.3
V600EPLX47201.3

The original data does not indicate the error associated with multiple biological replicates. To identify a suitable sample size, power calculations were performed using different levels of relative variance using the values quantified from the reported image as the mean. At each level of variance, the effect size was estimated and used to calculate the needed sample size to achieve at least 80% power with the indicated alpha error. The achieved power is reported.

Test family

  • Two-tailed t-test: Difference from constant (one sample case), Bonferroni’s correction, alpha error = 0.00833.

Power calculations

Group 1 (constant)Group 2Variance
estimate
Effect
size d
A priori
power
Samples
per group
WT BRAF (DMSO)WT BRAF (AMP-PCP)2%200.0099.9%3
15%26.66799.9%3
28%14.28699.4%3
40%10.00091.8%3
WT BRAF (DMSO)WT BRAF (GDC-0879)2%31.48299.9%3
15%4.197584.2%4
28%2.248782.0%6
40%1.574184.7%9
WT BRAF (DMSO)WT BRAF (PLX4720)2%200.0099.9%3
15%26.66799.9%3
28%14.28699.4%3
40%10.00091.8%3
Sensitivity calculationsDetectable effect
size d
A prior
power
Samples
per group
BRAFV600E (DMSO)BRAFV600E (AMP-PCP)2%8.019480.0%3
15%3.958180.0%4
28%2.197380.0%6
40%1.491780.0%9
BRAFV600E (DMSO)BRAFV600E (GDC-0879)2%8.019480.0%3
15%3.958180.0%4
28%2.197380.0%6
40%1.491780.0%9
BRAFV600E (DMSO)BRAFV600E (PLX4720)2%8.019480.0%3
15%3.958180.0%4
28%2.197380.0%6
40%1.491780.0%9
  • Based on these power calculations, we will then run the experiment six times. Each time we will quantify band intensity. We will determine the standard deviation of band intensity across the biological replicates and combine this with the reported value from the original study to simulate the original effect size. We will use this simulated effect size to determine the number of replicates necessary to reach a power of at least 80%. We will then perform additional replicates, if required, to ensure that the experiment has more than 80% power to detect the original effect.

References

  1. 1
  2. 2
    An open investigation of the reproducibility of cancer biology research
    1. TM Errington
    2. E Iorns
    3. W Gunn
    4. FE Tan
    5. J Lomax
    6. BA Nosek
    (2014)
    eLife, 3, 10.7554/eLife.04333.
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
    R: a language and environment for statistical computing
    1. R Core Team
    (2015)
    R Foundation for Statistical Computing, http://www.R-project.org/.

Decision letter

  1. Roger Davis
    Reviewing Editor; Howard Hughes Medical Institute & University of Massachusetts Medical School, United States

In the interests of transparency, eLife includes the editorial decision letter and accompanying author responses. A lightly edited version of the letter sent to the authors after peer review is shown, indicating the most substantive concerns; minor comments are not usually included.

Thank you for submitting your work entitled "Registered Report: RAF inhibitors prime wild-type RAF to activate the MAPK pathway and enhance growth" for peer review at eLife. Your submission has been favorably evaluated by Sean Morrison (Senior editor) and five reviewers, one of whom is a member of our Board of Reviewing Editors.

The reviewers have discussed the reviews with one another and the Reviewing editor has drafted this decision to help you prepare a revised submission.

This is a proposal to replicate a study by Malek and co-workers that demonstrated a role for Raf inhibitors to regulate CRAF/BRAF dimerization, activate the MAPK pathway, and increase tumor growth. The conclusions of this study have been widely accepted in the field. Nevertheless, this replication study is worthwhile. Three experiments will be replicated: Figure 1A, 2A and 2B.

There are two major concerns with the proposed replication study:

1) The restriction of the analysis to the PLX drug in each study. The authors should examine both the GDC-0879 and PLX-4720 drugs because the binding modes of these molecules are different.

2) As noted in the Registered Report Introduction, paradoxical RAF activation was described previously. The advance provided by the Nature paper was to identify a mechanism. The proposed replication studies (Figure 1A, 2A and 2B) do not really address the mechanistic conclusions of the paper. Consequently, the registered report should be expanded, for example – to include data showing drug-induced CRAF/BRAF dimerization (Figure 4A).

Additional comments:

A) The authors state that GDC-0879 and PLX-4720 have similar effects; however, as described in this paper and other work, these 2 compounds have very different binding modes to BRAF based on the structural studies described by Hatzivassiliou et al. This results in different functional effects as shown in Figure 2C and Figures 4A-C. For instance, the PLX-4720 molecule exhibited significantly less CRAF activation compared to GDC-0879, exhibited far less BRAF/CRAF dimerization, and overall had a weaker effect on paradoxical activation compared to GDC-0879 (requiring much higher doses of drug). Based on these significant differences, both drugs should be used to replicate the study.

B) No information is provided as to how the cell viability dose response (EC50) data will be fit. The data presented in Figure 1A reports an absolute EC50 value and not a relative EC50. Also the highest dose of drug tested was 20 μM and compounds which are shown at 20 μM simply did not reach >50% inhibition at the highest dose tested (20 μM). This is apparent in Supplementary Figure 2 where the highest dose of GDC-0879 tested is 20 μM and little effect is seen at that dose. Hence it is more accurate to compare EC50 values in BRAF-WT cell lines selected to EC50>20 μM rather than the absolute 20 μM value listed in the subsection “Power calculations”. Also as described in point 1, GDC-0879 should also be used in these studies for appropriate comparisons.

C) The authors propose starting dosing PLX-4720 in the cellular viability studies at 160 μM; however, this compound is has decreased solubility in media or water. In order to conduct such a study, the authors need to first demonstrate that PLX-4720 is soluble in the media tested at this dose. The authors also do not provide information on whether the compound dilutions are conducted in DMSO first and then media is added. Due to solubility issues, the compound should first be serially diluted in DMSO and then media added, ensuring again the final DMSO concentration is not >0.2%.

D) The cellular viability experiments are being conducted in 96-well plates (16,000 cells per well) and not 384-well plates and the seeding densities selected (14,000 cells per well) are not optimized. The authors need to identify optimal seeding densities for each given cell line ensuring that over the 4-day period of time the cells are in a logarithmic growth phase. The growth rates of cell lines will vary depending on both the media used as well as differences in serum lots, incubator temperature and other effects.

E) For Protocols 2 and 3: Again both PLX-4720 and GDC-0879 should be included when replicating these data given the different mechanisms of these inhibitors. As can be seen in Figure 2B, the data with these 2 compounds are not identical. The outcome of this experiment is also highly dependent on the degree of BRAF or CRAF knockdown upon Dox treatment. In order to appropriately replicate the results, the authors first need to ensure that with 2 mg/ml dox after 3 days BRAF and CRAF is depleted to similar levels as that shown in Figure 2B. If they are not, the authors need to optimize conditions further to ensure that the degree of BRAF and CRAF depletion is similar. The stable sh-inducible knockdown cell lines are pools and not clonal and hence optimization of knockdown conditions will be required as growth conditions will vary in a different laboratory with different lots of doxycyclin, serum, etc. Moreover, total MEK should be utilized to normalize pMEK levels for data analysis. Protein loading should be confirmed using actin and not BRAF or CRAF protein levels, especially given the experiment involves depleting cells of BRAF and CRAF.

F) As the variance increases, the sample size calculated increases – for example, the estimated sample size increases from 11 to 13 to 17 when the variance increases from 15% to 28% to 40%. Where do the investigators plan to stop their assumed variance and select a sample size? Please clarify.

G) Just before Protocol 2 description, the authors say that they will perform additional replicates to ensure that the experiment has more than 80% power. This gives the impression that the authors will continue to add more samples until they reach statistical significance. Please clarify.

https://doi.org/10.7554/eLife.09976.002

Author response

This is a proposal to replicate a study by Malek and co-workers that demonstrated a role for Raf inhibitors to regulate CRAF/BRAF dimerization, activate the MAPK pathway, and increase tumor growth. The conclusions of this study have been widely accepted in the field. Nevertheless, this replication study is worthwhile. Three experiments will be replicated: Figure 1A, 2A and 2B.

There are two major concerns with the proposed replication study: 1) The restriction of the analysis to the PLX drug in each study. The authors should examine both the GDC-0879 and PLX-4720 drugs because the binding modes of these molecules are different.We agree that both drugs should be added to the planned replication. The revised manuscript includes both compounds.

2) As noted in the Registered Report Introduction, paradoxical RAF activation was described previously. The advance provided by the Nature paper was to identify a mechanism. The proposed replication studies (Figure 1A, 2A and 2B) do not really address the mechanistic conclusions of the paper. Consequently, the registered report should be expanded, for example

to include data showing drug-induced CRAF/BRAF dimerization (Figure 4A).We agree and have included Figure 4A in the revised manuscript. We are also removing Figure 2A from the replication attempt.

Additional comments: A) The authors state that GDC-0879 and PLX-4720 have similar effects; however, as described in this paper and other work, these 2 compounds have very different binding modes to BRAF based on the structural studies described by Hatzivassiliou et al. This results in different functional effects as shown in Figure 2C and Figures 4A-C. For instance, the PLX-4720 molecule exhibited significantly less CRAF activation compared to GDC-0879, exhibited far less BRAF/CRAF dimerization, and overall had a weaker effect on paradoxical activation compared to GDC-0879 (requiring much higher doses of drug). Based on these significant differences, both drugs should be used to replicate the study.

We agree that both drugs should be added to the planned replication. The revised manuscript includes both compounds.

B) No information is provided as to how the cell viability dose response (EC50) data will be fit. The data presented in Figure 1A reports an absolute EC50 value and not a relative EC50. Also the highest dose of drug tested was 20 μM and compounds which are shown at 20 μM simply did not reach >50% inhibition at the highest dose tested (20 μM). This is apparent in Supplementary Figure 2 where the highest dose of GDC-0879 tested is 20 μM and little effect is seen at that dose. Hence it is more accurate to compare EC50 values in BRAF-WT cell lines selected to EC50>20 μM rather than the absolute 20 μM value listed in the subsection “Power calculations”. Also as described in point 1, GDC-0879 should also be used in these studies for appropriate comparisons.Thank you for raising these important points. It was not clear from the original paper if the EC50 values were absolute or relative. Additionally, we have included how the EC50 value will be determined: specifically, vehicle treated cells define 100% viability and the media-only wells define 0% viability, with the absolute EC50 calculated as the concentration at which the curve crosses 50% viability when the top (100%) and baseline (0%) are held constant when fitting the four-parameter curve.

We agree that the values for PLX4720 and GDC-0879 are above 20 µM for RAFWT/RASWT and RASMUTANT cell lines, however the concentration at which they do reach 50% inhibition is unknown (or not possible). The initial approach was to use 20 µM as a conservative estimate of what that value might be. However, as it is unlikely to be reached without additional complications, as raised by the next comment (c) below, we are reframing the approach to perform the cellular viability experiments. We will perform the experiment exactly as originally reported. If a curve is unable to be fit, as was the case for the original report, the EC50 will be reported as >20 µM. Only EC50 values that can be accurately estimated will be reported and compared to the original reported values.

Finally, we have included the GDC-0879 compound in the revised manuscript.

C) The authors propose starting dosing PLX-4720 in the cellular viability studies at 160 μM; however, this compound is has decreased solubility in media or water. In order to conduct such a study, the authors need to first demonstrate that PLX-4720 is soluble in the media tested at this dose. The authors also do not provide information on whether the compound dilutions are conducted in DMSO first and then media is added. Due to solubility issues, the compound should first be serially diluted in DMSO and then media added, ensuring again the final DMSO concentration is not >0.2%.Thank you for raising this important point. First, we have removed any additional concentrations that were used in the original assay in the revised manuscript. As a result, this will potentially lead to reporting EC50 values as >20 µM, or <0.078 µM, similar to Hatzivasiliou et al., 2010. Only EC50 values that can be accurately estimated will be reported and compared to the original reported values. Second, we have clarified the methodology to reflect how the compound dilutions are conducted first in DMSO and then media is added as described in Hoeflich et al., 2009, which was referenced in Hatzivassiliou et al., 2010.

D) The cellular viability experiments are being conducted in 96-well plates (16,000 cells per well) and not 384-well plates and the seeding densities selected (14,000 cells per well) are not optimized. The authors need to identify optimal seeding densities for each given cell line ensuring that over the 4-day period of time the cells are in a logarithmic growth phase. The growth rates of cell lines will vary depending on both the media used as well as differences in serum lots, incubator temperature and other effects.We agree and have added an optimization protocol that will identify the seeding density of each cell to ensure the cells are in a logarithmic growth phase at the end of the time period. The results of the optimization protocol will define the seeding density used in the cellular viability experiments with the compounds.

E) For Protocols 2 and 3: Again both PLX-4720 and GDC-0879 should be included when replicating these data given the different mechanisms of these inhibitors. As can be seen in Figure 2B, the data with these 2 compounds are not identical. The outcome of this experiment is also highly dependent on the degree of BRAF or CRAF knockdown upon Dox treatment. In order to appropriately replicate the results, the authors first need to ensure that with 2 mg/ml dox after 3 days BRAF and CRAF is depleted to similar levels as that shown in Figure 2B. If they are not, the authors need to optimize conditions further to ensure that the degree of BRAF and CRAF depletion is similar. The stable sh-inducible knockdown cell lines are pools and not clonal and hence optimization of knockdown conditions will be required as growth conditions will vary in a different laboratory with different lots of doxycyclin, serum, etc. Moreover, total MEK should be utilized to normalize pMEK levels for data analysis. Protein loading should be confirmed using actin and not BRAF or CRAF protein levels, especially given the experiment involves depleting cells of BRAF and CRAF.

We agree that both drugs should be added to the planned replication. The revised manuscript includes both compounds. We also agree and have included an optimization step in the protocol for inducing knockdown of BRAF and CRAF by Dox treatment. A range of dox will be used to determine if 2 mg/ml dox is sufficient to knockdown BRAF and CRAF to levels similar to what was reported. If necessary, the concentration of dox will be adjusted prior to conducting the experiment.

We have pMEK levels normalized to total MEK, which is the DV used in the statistical analysis.

We agree that BRAF and CRAF are not ideal loading controls and have included actin as an additional measure not included in the original report.

F) As the variance increases, the sample size calculated increases

for example, the estimated sample size increases from 11 to 13 to 17 when the variance increases from 15% to 28% to 40%. Where do the investigators plan to stop their assumed variance and select a sample size? Please clarify.At the beginning of each protocol the starting sample size is defined (for each group), as well as at the end of each power calculation section. The range of variance is used to provide some guidance about the anticipated scope of the effort. Because we do not know the variance in the originally reported result, or what variance the replication might obtain, this approach provides a way to select an appropriate starting point, and thus minimum sample size. As stated at the end of each power calculation we will then use the variation from the minimum sample to perform a power calculation in order to identify if more samples are needed. This is further explained in response to the question below.

G) Just before Protocol 2 description, the authors say that they will perform additional replicates to ensure that the experiment has more than 80% power. This gives the impression that the authors will continue to add more samples until they reach statistical significance. Please clarify.

In order to perform a proper power calculation to determine minimum sample size, both the difference between means and variance estimates are needed. Because we do not have the original observed variance for Figure 2B, we have performed power calculations on a range of potential effect sizes (using the originally reported ‘mean’ values and a range of variances to calculate Cohen’s d). This method acts as a guide for selecting a starting, minimum, sample size. After performing this minimum, pre-defined sample size we will use the variance from these replication samples together with the means from the original report to determine if additional biological replicates are needed. The additional samples will not be added until statistical significance is reached for the replication attempt, but rather additional samples will be added until the estimated original effect size (calculated using the originally reported ‘mean’ value and the replication variance) can be detected with at least 80% power.

https://doi.org/10.7554/eLife.09976.003

Article and author information

Author details

  1. Ajay Bhargava

    Shakti BioResearch, Woodbridge, United States
    Contribution
    AB, Drafting or revising the article
    Competing interests
    AB: Shakti Bioresearch is a Science Exchange associated lab.
  2. Steven Pelech

    Kinexus Bioinformatics Corporation, Vancouver, Canada
    Contribution
    SP, Drafting or revising the article
    Competing interests
    No competing interests declared.
  3. Ben Woodard

    Biotechnology Research and Education Program, University of Maryland, College Park, United States
    Contribution
    BW, Drafting or revising the article
    Competing interests
    BW: Biotechnology Research and Education Program is a Science Exchange associated lab.
  4. John Kerwin

    Biotechnology Research and Education Program, University of Maryland, College Park, United States
    Contribution
    JK, Drafting or revising the article
    Competing interests
    JK: Biotechnology Research and Education Program is a Science Exchange associated lab.
  5. Nimet Maherali

    Harvard Stem Cell Institute, Cambridge, United States
    Contribution
    NM, Drafting or revising the article
    Competing interests
    No competing interests declared.
  6. Reproducibility Project: Cancer Biology

    Contribution
    RP:CB, Conception and design; Drafting or revising the article
    For correspondence
    fraser@scienceexchange.com
    Competing interests
    RP:CB: EI, FT, JL, and NP are employed by and hold shares in Science Exchange Inc.
    1. Elizabeth Iorns, Science Exchange, Palo Alto, United States
    2. William Gunn, Mendeley, London, United Kingdom
    3. Fraser Tan, Science Exchange, Palo Alto, United States
    4. Joelle Lomax, Science Exchange, Palo Alto, United States
    5. Nicole Perfito, Science Exchange, Palo Alto, United States
    6. Timothy Errington, Center for Open Science, Charlottesville, United States

Funding

Laura and John Arnold Foundation

  • Reproducibility Project: Cancer Biology

The Reproducibility Project: Cancer Biology is funded by the Laura and John Arnold Foundation, provided to the Center for Open Science in collaboration with Science Exchange. The funder had no role in study design or the decision to submit the work for publication.

Acknowledgements

The Reproducibility Project: Cancer Biology core team would like to thank the original authors, in particular Shiva Malek, for generously sharing critical information as well as reagents to ensure the fidelity and quality of this replication attempt. We thank Courtney Soderberg at the Center for Open Science for assistance with statistical analyses and Catherine Sutter at Kinexus Bioinformatics and Stephen Williams at the Center for Open Science for review and helpful comment on the manuscript. We would also like to thank the following companies for generously donating reagents to the Reproducibility Project: Cancer Biology; American Type Culture Collection (ATCC), Applied Biological Materials, BioLegend, Charles River Laboratories, Corning Incorporated, DDC Medical, EMD Millipore, Harlan Laboratories, LI-COR Biosciences, Mirus Bio, Novus Biologicals, Sigma-Aldrich, and System Biosciences (SBI).

Reviewing Editor

  1. Roger Davis, Reviewing Editor, Howard Hughes Medical Institute & University of Massachusetts Medical School, United States

Publication history

  1. Received: July 9, 2015
  2. Accepted: January 6, 2016
  3. Version of Record published: February 16, 2016 (version 1)

Copyright

© 2016, Bhargava et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 736
    Page views
  • 135
    Downloads
  • 0
    Citations

Article citation count generated by polling the highest count across the following sources: PubMed Central, Scopus, Crossref.

Comments

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Download citations (links to download the citations from this article in formats compatible with various reference manager tools)

Open citations (links to open the citations from this article in various online reference manager services)

Further reading

  1. G Hatzivassiliou et al.
    1
    1. Cancer Biology
    Curated by Roger Davis et al.
    Collection Updated

    Investigating reproducibility in preclinical cancer research.

    1. Human Biology and Medicine
    2. Neuroscience
    Bingshuo Li et al.
    Tools and Resources