Registered report: RAF inhibitors prime wild-type RAF to activate the MAPK pathway and enhance growth

The Reproducibility Project: Cancer Biology seeks to address growing concerns about reproducibility in scientific research by conducting replications of selected experiments from a number of high-profile papers in the field of cancer biology. The papers, which were published between 2010 and 2012, were selected on the basis of citations and Altmetric scores (Errington et al., 2014). This Registered Report describes the proposed replication plan of key experiments from 'RAF inhibitors prime wild-type RAF to activate the MAPK pathway and enhance growth' by Hatzivassiliou and colleagues, published in Nature in 2010 (Hatzivassiliou et al., 2010). Hatzivassiliou and colleagues examined the paradoxical response of RAF-WT tumors to treatment with RAF inhibitors. The key experiments being replicated include Figure 1A, in which the original authors demonstrated that treatment of a subset of BRAFWT tumor cell lines with RAF small molecule inhibitors resulted in an increase in cell viability, Figure 2B, which reported that RAF inhibitor activation of the MAPK pathway was dependent on CRAF but not BRAF, and Figure 4A, where the dimerization of BRAF and CRAF was modulated by the RAF inhibitor PLX4720, but not GDC-0879. The Reproducibility Project: Cancer Biology is a collaboration between the Center for Open Science and Science Exchange, and the results of the replications will be published by eLife. DOI: http://dx.doi.org/10.7554/eLife.09976.001


Introduction
Mutations activating the H/K/N-RAS>B/C-RAF>MEK1/2>ERK1/2 signaling pathways are commonly found in many types of cancer, making members of this pathway promising drug targets. Several small molecule inhibitors have been created that target the three RAF isoforms. However, early observations using these drugs noted a surprising paradox; while drugs targeting CRAF were able to inhibit CRAF activity in vitro, they paradoxically activated the MEK1/2>ERK1/2 signaling modules in vivo. This activation was not due to direct activation of signaling components downstream of RAF (Hall-Jackson et al., 1999a;1999b).
Hatzivassiliou and colleagues found that RAF inhibitors, while effective in blocking signaling in BRAF V600E mutant (MT) cancer cell lines, paradoxically increased cell proliferation in BRAF WT cancer cell lines (Hatzivassiliou et al., 2010). Their findings were published along with two other reports demonstrating similar results (Heidorn et al., 2010;Poulikakos et al., 2010) and provided a key insight into the mechanism of paradoxical RAF activation in BRAF WT cells, showing that it depended on drug-induced dimerization of wild-type (WT) RAF isoforms, specifically CRAF.
In Figure 1A, Hatzivassiliou and colleagues treated 19 cancer cell lines, comprising 4 BRAF V600E mutant lines, 7 RAF/RAS-WT lines, and 8 KRAS-MT lines, with varying concentrations of two RAF inhibitors and calculated the IC 50 value for each drug in each cell line. They found that, although cancer cell lines carrying the BRAF V600E mutation were susceptible to the RAF inhibitors, BRAF WT cell lines were not. This experiment will be replicated in Protocol 1.
To elucidate whether CRAF or BRAF contributed to MEK signaling in RAF-treated KRAS-mutant cells, the authors used inducible shRNA cell lines to test whether BRAF or CRAF were necessary for the activation of MEK1/2 in HCT116 cells, which are KRAS mutant. As reported in Figure 2B, silencing CRAF reversed MEK activation upon treatment with the RAF inhibitors GDC-0879 and PLX4720. This experiment will be replicated in Protocol 2.
To test whether inhibitor priming was mediated by the inhibitors' conformational effects on the RAF kinase domain, the authors assayed BRAF-CRAF heterodimerization through a series of immunoprecipitation assays coupled with or without RAF inhibitors. In Figure 4A, they reported that the CRAF kinase domain forms a stable complex with the BRAF kinase domain when inhibitors are not present. However, in the presence of the RAF inhibitor PLX4720, this CRAF-BRAF heterodimer kinase domain interaction is destabilized. In the presence of DGC-0879, the CRAF-BRAF interaction is stabilized. This experiment will be replicated in Protocol 3.
Hatzivassiliou's work was published along with two companion papers; Heidorn and colleagues showed that drugs targeting BRAF V600E caused dimerization with CRAF and pathway activation (Heidorn et al., 2010), while Poulikakos and colleagues observed that paradoxical RAF activation only occurred in the context of BRAF WT . In a subsequent study, Poulikakos and colleagues showed that dimerization was a critical factor in allowing a variant of BRAF V600E to demonstrate enhanced activity as compared to BRAF V600E (Poulikakos et al., 2011). Work by Joseph and colleagues confirmed the findings of Hatzivassiliou and colleagues that BRAF V600E cell lines were sensitive to treatment with the BRAF inhibitor PLX4720, while RAS mutant/BRAF WT or RAS/RAF WT cell lines were not, and that MEK1/2>ERK1/2 signaling was activated in these BRAF WT lines (Joseph et al., 2010). Lee and colleagues assayed a panel of BRAF V600E , NRAS mutant, or BRAF/NRAS WT cell lines by treating them with PLX4720. They observed that PLX4720 inhibited ERK signaling in BRAF V600E cells, but they did not observe paradoxical MEK1/2>ERK1/2 activation in BRAF WT lines. They attributed this effect to a lower percentage of serum in their culture conditions as compared to those used in previous studies. They then examined colony formation to assess drug effects on cell survival, and saw strong growth inhibition exclusively in BRAF V600E cells (Lee et al., 2010). Paradoxical activation was also observed in BRAF WT cells by Carnahan and colleagues and by Halaban and colleagues (Carnahan et al., 2010;Halaban et al., 2010). A year later, Kaplan and colleagues published corroborating evidence that PLX4720 paradoxically activated MEK1/2>ERK1/2 signaling in BRAF WT cells. They also confirmed that silencing of CRAF abrogated this activation of MEK1/2>ERK1/2 signaling (Kaplan et al., 2011).

Materials and methods
Unless otherwise noted, all protocol information were derived from the original paper, references from the original paper, or information obtained directly from the authors. An asterisk (*) indicates data or information provided by the Reproducibility Project: Cancer Biology core team. A hashtag (#) indicates information provided by the replicating lab. All references to Figures refer to the original study.

Protocol 1: Assessing cell viability of a panel of cancer cell lines treated with RAF and MEK inhibitors
This protocol describes the treatment of a panel of human cancer cell lines with or without mutations in BRAF or RAS with drugs targeting RAF and MEK and assessing cell viability. This experiment is a replication of Figure 1A.

Sampling
. This experiment will be repeated four times.
. See Power calculations for details. . Each experiment consists of three cohorts: . Cohort 1: Cell lines treated with a range of concentrations of PLX4720 . 20 mM . 10 mM . Deliverables . Data to be collected: . All raw luminescence values . Luminescence values adjusted to compensate for background luminescence . Luminescence values normalized to vehicle treated cells.
. EC 50 values for each cell line and each drug treatment (as seen in Figure 1A)

Confirmatory analysis plan
. Statistical analysis of the replication data: . n/a . Meta-analysis of original and replication attempt effect sizes: . The replication data will be presented as a mean with 95% confidence intervals and will include the original data point, calculated directly from the graph, as a single point on the same plot for comparison.
Known differences from the original study . All known differences are listed in the 'Materials and reagents' section above with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.
. While the original experiment examined nineteen cell lines, the replication will be restricted to three cell lines; A375, representing the BRAF V600E mutant lines, MeWo, representing the RAF WT /RAS WT cell lines, and HCT116, representing the RAS mutant cell lines. HCT116 is used in a subsequent experiment described in Protocol 2.
. The replicating lab will plate the cells in a 96-well plate as opposed to a 384-well plate.

Provisions for quality control
All data obtained from the experiment -raw data, data analysis, control data, and quality control data -will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/0hezb/). .

STR profiling and mycoplasma detection results
Protocol 2: Assessing CRAF and BRAF roles in drug-dependent activation of MEK This protocol describes the treatment of HCT116 (KRAS-MT) cells expressing doxycycline-inducible shRNAs against BRAF and CRAF, and treatment with RAF inhibitors followed by Western blot examination of activation of MEK. This experiment is a replication of Figure 2B.

Sampling
. This experiment will be repeated seven times for a final power of !80%. . The original data presented is qualitative (representative images). In order to determine an appropriate number of replicates to perform initially, we have estimated the sample sizes required based on a range of potential variance.
. See Power calculations for details. . The experiment consists of two cohorts: . Cohort 1: HCT116 cells with dox-inducible shRNA against BRAF . Cohort 2: HCT116 cells with dox-inducible shRNA against CRAF . Each cohort will receive the following treatments: . No dox: Lysates from each treatment are probed for: . . Some information derived from Hoeflich and colleagues (Hoeflich et al., 2006).
. All cells will be sent for STR profiling and mycoplasma testing.
1. Determine if concentration of doxycycline to induce knockdown of BRAF and CRAF in HCT116 cells needs to be optimized. Before beginning experiment, perform protocol details as outlined below, without drug treatment (Step 4) and only analyzing expression of BRAF, CRAF, and actin (Step 8). Perform with at least 1 well per group (with and without dox treatment for each cell line). a. Normalize BRAF and CRAF to actin levels. b. If level of depletion is not similar to reported levels in Figure 2B, further optimize conditions, such as increasing concentration of dox. c. Once conditions of knockdown are optimized, use for all replicates of experimental procedure. 2. Seed # 5x10 4 cells/well in 24-well plates for treatment.
a. Seed 16 wells per cohort. i. 8 wells will be treated with dox. ii. 8 wells will remain untreated. 3. Induce shRNA expression of appropriate wells by treatment with 2 mg/ml dox for 3 days.
a. This condition will be checked in Step 1  ii. Note: Actin serves as a loading control to ensure equal loading of lanes (additional). 9. # Wash membranes 3 x 5 min in TBST. 10. Incubate with HRP-conjugated secondary antibodies # diluted 1:20,000 in 5% milk in TBST for 1 hr at room temperature. 11. Visualize bands with ECL detection kit according to manufacturer's protocol.
a. Quantify band intensity. b. For each drug and dose in each cell line (treated with or without dox), normalize pMEK values to total MEK values. 12. Repeat Steps 2-11 independently six additional times.

Deliverables
. Data to be collected: . Images of whole gel, including ladder, of shRNA optimization (Step 1). . Images of whole gel, including ladder (compare to Figure 2B). . Quantification of band intensities; phospho-protein levels normalized to total protein levels.

Confirmatory analysis plan
. Statistical analysis of the replication data: . Compare band intensities across all groups. . Four-way ANOVA (2 x 2 x 2 x 4 factorial) of the normalized pMEK values for each cell line (with or without dox), drug (PLX4720 or GDC-0879), and dose (0, 0.1, 1, and 10 mM) followed by: . Two-way interaction contrast of normalized pMEK values from BRAF and CRAF shRNA cell lines (with or without dox) across varying doses of GDC-0879 with the following Bonferroni corrected comparisons: . BRAF shRNA cell line with dox compared to without dox (across varying doses of GDC-0879) . CRAF shRNA cell line with dox compared to without dox (across varying doses of GDC-0879) . Two-way interaction contrast of normalized pMEK values from BRAF and CRAF shRNA cell lines (with or without dox) across varying doses of PLX4720 with the following Bonferroni corrected comparisons: . BRAF shRNA cell line with dox compared to without dox (across varying doses of PLX4720) . CRAF shRNA cell line with dox compared to without dox (across varying doses of PLX4720) . Meta-analysis of original and replication attempt effect sizes: . The replication data will be presented as a mean with 95% confidence intervals and will include the original data point, calculated directly from the representative image, as a single point on the same plot for comparison.
Known differences from the original study . All known differences are listed in the 'Materials and reagents' section above with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.
. The replication attempt will use actin as an additional loading control not used in the original study.

Provisions for quality control
All data obtained from the experiment -raw data, data analysis, control data, and quality control data -will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/0hezb/).
. STR profiling and mycoplasma detection results. . Induced shRNA knockdown conditions will be checked, and optimized if needed, prior to proceeding with the experiment.
. Image of Ponceau staining confirming protein transfer.
. Protein loading will be confirmed using actin.

Protocol 3: Biochemical heterodimerization assay with recombinant RAF proteins in the presence or absence of RAF inhibitors
This protocol describes how to perform immunoprecipitation and Western blot analysis with recombinant CRAF and BRAF kinase domains in the presence or absence of the RAF inhibitors PLX4720 or GDC-0879. Wild-type BRAF and BRAF V600E kinase domains will be tested in the presence of wildtype CRAF. This experiment is a replication of Figure 4A.

Sampling
. This experiment will be repeated six times for a final power of !80%. . The original data presented are qualitative (representative images). In order to determine an appropriate number of replicates to perform initially, we have estimated the sample sizes required based on a range of potential variance. We will also determine sample size post hoc.
. See Power calculations for details. . Each experiment consists of two cohorts: . Cohort 1: CRAF + BRAF WT . Cohort 2: CRAF + BRAF V600E . Each cohort is incubated with CRAF and treated for 1 hr with: . DMSO . 10 mM of PLX4720 . 10 mM GDC-0879 . 1 mM AMP-PCP . Include a sample of CRAF, without BRAF, treated with DMSO (negative control) . Immunoprecipitate CRAF from each sample and probe for CRAF and BRAF. . Data to be collected:

Materials and reagents
. Full gel images with ladder positions marked . Quantification of band intensities . Raw measurements as well as normalized band intensities . Graph of mean band intensities across replicates Confirmatory analysis plan . Statistical analysis of the replication data: . Bonferonni corrected one-sample t-tests of normalized BRAF WT values (normalized to GST-CRAF and then DMSO) of the following conditions compared to 1 (DMSO): . PLX4720 . GDC-0879 . AMP-PCP . Bonferonni corrected one-sample t-tests of normalized BRAF V600E values (normalized to GST-CRAF and then DMSO) of the following conditions compared to 1 (DMSO): . PLX4720 . GDC-0879 . AMP-PCP . Meta-analysis of original and replication attempt effect sizes: . The replication data will be presented as a mean with 95% confidence intervals and will include the original data point, calculated directly from the representative image, as a single point on the same plot for comparison.
. Additional exploratory analysis: . Two-way ANOVA of BRAF values (normalized to GST-CRAF) from DMSO, PLX4720, GDC-0879, or AMP-PCP treated samples for each BRAF variant incubated with GST-CRAF with the following Bonferroni corrected comparisons: . Known differences from the original study . All known differences are listed in the 'Materials and reagents' section above with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.
. The original data examined samples treated with AZ-628, a chemically unrelated ATP-competitive RAF inhibitor, which had a similar reported effect as GDC-0879. The replication will be restricted to examining only PLX4720 and GDC-0879, similar to the other experiments included in this replication attempt.
. The replicating lab will use a modified version of their in-house Western Blot protocol with antibodies analyzed by an ECL detection system instead fluorescence based.

Provisions for quality control
All data obtained from the experiment -raw data, data analysis, control data, and quality control data -will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/0hezb/).

Protocol 2
Summary of original data reported in Figure 2B . Summary of data: . Band densities were obtained with Image Studio Lite (LiCOR) from published images. . Normalization was performed by dividing the phospho-band intensity by the total protein band intensity. Continued on next page The original data does not indicate the error associated with multiple biological replicates. To identify a suitable sample size, power calculations were performed using different levels of relative variance using the values quantified from the reported image as the mean. At each level of variance the effect size was estimated and used to calculate the needed sample size to achieve at least 80% power with the indicated alpha error. The achieved power is reported.

Test family
. Four-way ANOVA: Fixed effects, special, main effects and interactions, alpha error = 0.05

Power calculations
. Performed with G*Power software, version 3.1.7 (Faul et al., 2007). . ANOVA F test statistic and partial h 2 performed with R software, version 3.2.2 (R Core Team, 2015).
. For a given relative variance, 10,000 simulations were run and the F statistic and partial h 2 was calculated for each simulated data set. . Based on these power calculations, we will then run the experiment seven times. Each time we will quantify band intensity. We will determine the standard deviation of band intensity across the biological replicates and combine this with the reported value from the original study to simulate the original effect size. We will use this simulated effect size to determine the number of replicates necessary to reach a power of at least 80%. We will then perform additional replicates, if required, to ensure that the experiment has more than 80% power to detect the original effect.

Protocol 3
Summary of original data reported in Figure 4A . Summary of data: . Normalization was performed by dividing the BRAF-band intensity by the captured GST-CRAF band intensity. The original data does not indicate the error associated with multiple biological replicates. To identify a suitable sample size, power calculations were performed using different levels of relative variance using the values quantified from the reported image as the mean. At each level of variance, the effect size was estimated and used to calculate the needed sample size to achieve at least 80% power with the indicated alpha error. The achieved power is reported.

Power calculations
. Performed with G*Power software, version 3.1.7 (Faul et al., 2007). Continued on next page