Registered report: Kinase-dead BRAF and oncogenic RAS cooperate to drive tumor progression through CRAF

The Reproducibility Project: Cancer Biology seeks to address growing concerns about reproducibility in scientific research by conducting replications of selected experiments from a number of high-profile papers in the field of cancer biology. The papers, which were published between 2010 and 2012, were selected on the basis of citations and Altmetric scores (Errington et al., 2014). This Registered Report describes the proposed replication plan of key experiments from "Kinase-dead BRAF and oncogenic RAS cooperate to drive tumor progression through CRAF" by Heidorn and colleagues, published in Cell in 2010 (Heidorn et al., 2010). The experiments to be replicated are those reported in Figures 1A, 1B, 3A, 3B, and 4D. Heidorn and colleagues report that paradoxical activation of the RAF-RAS-MEK-ERK pathway by BRAF inhibitors when applied to BRAFWT cells is a result of BRAF/CRAF heterodimer formation upon inactivation of BRAF kinase activity, and occurs only in the context of active RAS. The Reproducibility Project: Cancer Biology is a collaboration between the Center for Open Science and Science Exchange, and the results of the replications will be published by eLife. DOI: http://dx.doi.org/10.7554/eLife.11999.001


Introduction
The RAS-RAF-MEK-ERK signaling pathway is routinely disregulated in many forms of cancer. Activating mutations in BRAF are found in almost half of all melanomas, and of these mutations, almost 90% involve a valine to glutamic acid transition at position 600 (BRAF V600E ) (Solit and Rosen 2014). The therapeutic effect of drugs that target this form of BRAF have proved less efficacious than expected, due to an unexpected effect in cells that are BRAF WT ; in these cells, drugs that target BRAF paradoxically activate rather than repress downstream signaling (Hall-Jackson et al., 1999a;Hall-Jackson et al., 1999b). In their 2010 paper, Heidorn and colleagues examined the mechanism of action behind this paradoxical activation of MEK/ERK signaling. Heidorn and colleagues first observed that paradoxical activation occurred only in the context of BRAF WT and activated RAS, an observation confirmed by two other groups (Hatzivassiliou et al., 2010;Poulikakos et al., 2010). Dissecting the mechanism, they reported that the formation of BRAF/CRAF heterodimers was necessary for pathway activation, and formation of those heterodimers required active RAS signaling.
In Figure 1A, Heidorn and colleagues examined pathway activation in response to a range of drugs. The inhibitors, sorafenib, which targets and represses both BRAF and CRAF, PLX4720, which is highly selective for and inhibits the activity of BRAF V600E . 885-A, which specifically targets and inhibits BRAF, and the MEK inhibitor PC184352 were examined. As expected, all four drugs blocked MEK/ERK activation in BRAF V600E A375 cells. However, in cells with active RAS, such as D04 cells (BRAF WT /NRAS Q61L ), MEK/ERK signaling was not repressed by PLX4720 or 885-A. This paradoxical activation in BRAF WT cells was also observed by several other groups (Carnahan et al., 2010;Joseph et al., 2010;Lee et al., 2010;Kaplan et al., 2011). This experiment will be replicated in Protocol 1. Previous work had shown that activated RAS in melanoma signals through CRAF, while normal signaling in healthy melanocytes is accomplished through BRAF (Dumaz et al., 2006). To determine if CRAF was required for paradoxical pathway activation, Heidorn and colleagues treated D04 cells with siRNAs targeting NRAS and CRAF. Knockdown of either NRAS or CRAF abrogated activation of MEK/ERK by 885-A, as seen in Figure 1B. This experiment will be replicated in Protocol 2. The necessity of CRAF also explains the lack of activation upon treatment with sorafenib observed in Figure 1A; since sorafenib inhibits both BRAF and CRAF, it does not result in pathway activation.
Since activated RAS is known to drive heterodimerization of BRAF and CRAF (Weber et al., 2001), Heidorn and colleagues also tested if drug binding drove heterodimerization of BRAF and CRAF, and if this heterodimerization was dependent on active RAS signaling. In Figure 3A, they transfected D04 cells with a mutant version of CRAF that was unable to bind to RAS (CRAF R89L ). Immunoprecipitation experiments showed that while CRAF WT was able to bind to BRAF in the presence of activated RAS, CRAF R89L was unable to bind to BRAF. This key experiment will be replicated in Protocol 3. The authors showed that BRAF binds to CRAF but only in the presence of WT RAS, not oncogenic RAS. In Figure 3B, myc-tagged BRAF or myc-tagged mutant BRAF ( R188L BRAF) were transfected into D04 cells and treated with either DMSO(-) or 885-A(+). The authors show that mutant of BRAF ( R188L BRAF) does not bind to CRAF even in the presence of 885-A, which induces RAS activity.
After confirming that drug binding to BRAF drove BRAF binding to CRAF, Heidorn and colleagues tested a kinase dead version of BRAF (BRAF D594A ) ( Figure 4D). Interestingly, this version of BRAF still bound to CRAF, indicating that it is not drug binding per se, but inhibition of BRAF activity, that drives BRAF binding to CRAF and paradoxical activation of MEK/ERK. This key experiment will be replicated in Protocol 4. Packer and colleagues extended the work of Heidorn and colleagues to examine if other more broadly targeted tyrosine kinase inhibitors were also able to paradoxically activate the RAS-RAF pathway. They observed paradoxical pathway activation in D04 cells after treatment with imatinib, nilotinib, dasatinib, and the BRAF inhibitor SB590885. As in Heidorn et al., paradoxical activation only occurred in cells with BRAF WT and required active RAS, as knockdown of NRAS abrogated the effect. Interestingly, while Heidorn and colleagues reported that knockdown of CRAF alone was able to block paradoxical activation, Packer and colleagues reported that only combined knockdown of BRAF and CRAF was able to block paradoxical activation (Packer et al., 2011). Work by Rebocho and colleagues and by Kaplan and colleagues aligned with Heidorn's findings that silencing of CRAF alone was able to abrogate paradoxical activation Rebocho and Marais 2012). Packer and colleagues also reported that BRAF/CRAF heterodimerization was dependent upon RAS by demonstrating that CRAF R89L was unable to form heterodimers with BRAF (Packer et al., 2011).
Activation of NRAS signaling appears to be a key step in acquired drug resistance, supporting the hypothesis that paradoxical activation can only occur in the context of active RAS signaling. Su and colleagues derived a drug-resistant BRAF V600E melanoma cell line by growing A375 cells in the presence of vemurafenib (PLX4032, a BRAF V600E inhibitor). Interestingly, drug resistance was dependent on expression of CRAF, and the resistant lines that emerged had acquired an activating mutation in KRAS (Su et al., 2012). Nazarian and colleagues also observed the acquisition of activating mutations in NRAS when they derived PLX4032-resistant cell lines (Nazarian et al., 2010). Lidsky and colleagues also showed that increased levels of NRAS were key to vemurafenib resistance, although they did not observe any activating mutations in their resistant cell lines (Lidsky et al., 2014).

Materials and methods
Unless otherwise noted, all protocol information was derived from the original paper, references from the original paper, or information obtained directly from the authors. An asterisk (*) indicates data or information provided by the Reproducibility Project: Cancer Biology core team. A hashtag (#) indicates information provided by the replicating lab.
Protocol 1: Treatment of BRAF mutant cells with various RAF inhibitors and assessment of activation of ERK This protocol describes how to treat NRAS mutant D04 cells and NRAS wild-type cells also carrying the BRAF V600E mutation with various BRAF inhibitors and assess ERK phosphorylation by Western blot, as reported in Figure 1A.

Sampling
. The experiment will be performed independently at least three times for a final power of at least 80%. The original data is qualitative, thus to determine an appropriate number of replicates to initially perform, sample sizes based on a range of potential variance was determined.
. See Power calculations for details. . Each experiment consists of eight cohorts: . . Raw images of whole gels with ladders included (as reported in Figure 1A). . Densitometric quantification of all bands.

Confirmatory analysis plan
. Statistical Analysis of the Replication Data: Note: At the time of analysis, we will perform the Shapiro-Wilk test and generate a quantile-quantile plot to assess the normality of the data. We will also perform Levene's test to assess homoscedasticity. If the data appears skewed, we will perform a transformation in order to proceed with the proposed statistical analysis. If this is not possible, we will perform the equivalent non-parametric test listed.
. Two-way ANOVA on normalized pERK values (to ERK1/2) in A375 or D04 cells treated with PD184352, sorafenib, SB590885, or vehicle (DMSO) with the following planned contrasts with the Bonferroni correction: . Normalized pERK band intensity in A375 cells: . Vehicle treatment vs. all three drug treatments (PD184352, sorafenib, and SB590885) . Normalized pERK band intensity in D04 cells: . Vehicle treatment vs. PD184352 and SB590885 treatments . Vehicle treatment vs. sorafenib treatment . Meta-analysis of original and replication attempt effect sizes: . The replication data (mean and 95% confidence interval) will be plotted with the original quantified data value displayed as a single point on the same plot for comparison.

Known differences from the original study
The replication attempt will use D04 and A375 cells and will exclude MM415, MM485, and WM852 cells. It will also exclude the drug PLX4720 and will replace 885-A with its analogue SB590885. The original authors suggest they have found similar results with this analogue (personal communication with Dr. Dhomen). All known differences, if any, are listed in the 'Materials and reagents' section above with the originally used item listed in the comments section. The comments section also lists if the source of original item was not specified. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control
All data obtained from the experiment -raw data, data analysis, control data, and quality control data -will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/b1aw6/). Cells will be sent for mycoplasma testing confirming lack of contamination and STR profiling confirming cell line authenticity. The transfer efficiency during the Western blot procedure will be monitored by Ponceau staining.

Protocol 2: Treatment of NRAS or CRAF silenced D04 cells with SB590885 and assessment of MEK and ERK phosphorylation
This protocol describes treatment of D04 cells transfected with siRNAs targeting NRAS or CRAF with SB590885 and assessment of those cells for activation of MEK and ERK by Western blot, as reported in Figure 1B.

Sampling
. The experiment will be performed independently at least four times for a final power of at least 80%. The original data is qualitative, thus to determine an appropriate number of replicates to initially perform, sample sizes based on a range of potential variance was determined. Deliverables . Data to be collected: . Protein quantification results from Bradford assay. . Images of Ponceau stained membranes. . Images of whole gels with ladder (as reported in Figure 1B). . Densitometric quantification of all bands.

Confirmatory analysis plan
. Statistical Analysis of the Replication Data: Note: At the time of analysis, we will calculate Pearson's r to check for correlation between the dependent variables, a scatter plot to assess linearity, and a Box's M test to check for equality of covariance matrices. We will also perform the Shapiro-Wilk test and generate a quantile-quantile plot to assess the normality of the data. We will perform Levene's test to assess homoscedasticity. If the data appears skewed, we will perform a transformation in order to proceed with the proposed statistical analysis. If this is not possible, we will perform the equivalent non-parametric test.
. One-way MANOVA comparing the differences between SB590885 treatment and vehicle treatment of normalized band intensities for pMEK and pERK levels in D04 cells transfected with siRNA for NRAS, CRAF, or control with the following Bonferroni-corrected comparisons: . Difference in normalized ppMEK levels between SB590885 and vehicle treatment: . Control siRNA compared to NRAS siRNA. . Control siRNA compared to CRAF siRNA. . Difference in normalized ppERK levels between SB590885 and vehicle treatment: . Control siRNA compared to NRAS siRNA. . Control siRNA compared to CRAF siRNA . Meta-analysis of original and replication attempt effect sizes: . The replication data (mean and 95% confidence interval) will be plotted with the original quantified data value displayed as a single point on the same plot for comparison.

Known differences from the original study
The replication will replace 885-A with its analogue SB590885. The original authors suggest they have found similar results with this analogue (personal communication with Dr. Dhomen). All known differences, if any, are listed in the 'Materials and reagents' section above with the originally used item listed in the comments section. The comments section also lists if the source of original item was not specified. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control
Cells will be sent for mycoplasma testing confirming lack of contamination and STR profiling confirming cell line authenticity. The transfer efficiency during the Western blot procedure will be monitored by Ponceau staining. The membrane will be imaged after stripping to confirm and measure background. All data obtained from the experiment -raw data, data analysis, control data, and quality control data -will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/b1aw6/).

Protocol 3: Immunoprecipitation of CRAF from SB590885 treated D04 cells expressing myc-tagged CRAF WT or CRAF R89L
This protocol describes how to immunoprecipitate myc-tagged CRAF WT or CRAF R89L , a mutant form that cannot bind RAS, from D04 cells and probe the pulldown for BRAF, as reported in Figure 3A.

Sampling
. The experiment will be performed independently at least three times for a final power of at least 80%. The original data is qualitative, thus to determine an appropriate number of replicates to initially perform, sample sizes based on a range of potential variance was determined.
. See Power calculations for details. . Each experiment consists of six cohorts: . Deliverables . Data to be collected: . Protein quantification results from Bradford assay. . Images of Ponceau stained membranes. . Transfection QC images of whole gels with ladder. . Images of whole gels with ladder (as reported in Figure 3A). . Densitometric quantification of all bands.

Confirmatory analysis plan
. Statistical Analysis of the Replication Data: Note: At the time of analysis, we will perform the Shapiro-Wilk test and generate a quantile-quantile plot to assess the normality of the data. We will also perform Levene's test to assess homoscedasticity. If the data appears skewed, we will perform a transformation in order to proceed with the proposed statistical analysis. If this is not possible, we will perform the equivalent non-parametric test listed.
. Two-way ANOVA comparing normalized IP BRAF (to IP a myc) band intensity in D04 cells transfected with Myc-CRAF WT vector or Myc-CRAF R89L vector with or without SB590885 drug treatment, and the following Bonferroni-corrected comparisons: . Normalized IP BRAF band intensity in cells with Myc-CRAF WT vector with SB590885 treatment vs. vehicle treatment.
. Normalized IP BRAF band intensity in cells with Myc-CRAF R89L vector with SB590885 treatment vs. vehicle treatment.
. Meta-analysis of original and replication attempt effect sizes: . The replication data (mean and 95% confidence interval) will be plotted with the original quantified data value displayed as a single point on the same plot for comparison.

Known differences from the original study
The transfection method using Nucleofectin Solution V and electroporation will be replaced with a lipid-based method using Effectene Transfection Reagent, and protocol will be changed according to Manufacturer's instructions. This difference in transfection protocol might lead to differences in expression that could lead to differences in results. The replication will replace 885-A with its analogue SB590885. The original authors suggest they have found similar results with this analogue (personal communication with Dr. Dhomen). All known differences, if any, are listed in the 'Materials and reagents' section above with the originally used item listed in the comments section. The comments section also lists if the source of original item was not specified. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control
Cells will be sent for mycoplasma testing confirming lack of contamination and STR profiling confirming cell line authenticity. Transfection will be confirmed with Western blot. The transfer efficiency during the Western blot procedure will be monitored by Ponceau staining. All data obtained from the experiment -raw data, data analysis, control data, and quality control data -will be made publicly available, either as a published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/b1aw6/).

Protocol 4: Immunoprecipitation of BRAF from SB590885 treated D04 cells expressing myc-tagged BRAF WT or BRAF R188L
This protocol describes how to immunoprecipitate myc-tagged BRAF WT or BRAF R188L , a mutant form that cannot bind RAS, from D04 cells and probe the pulldown for CRAF, as reported in Figure  3B.

Sampling
. The experiment will be performed independently at least three times for a final power of at least 80%. The original data is qualitative, thus to determine an appropriate number of replicates to initially perform, sample sizes based on a range of potential variance was determined.
. See Power calculations for details. . Each experiment consists of six cohorts: . Deliverables . Data to be collected: . Protein quantification results from Bradford assay. . Images of Ponceau stained membranes. . Transfection QC images of whole gels with ladder. . Images of whole gels with ladder (as reported in Figure 3A). . Densitometric quantification of all bands.

Confirmatory analysis plan
. Statistical Analysis of the Replication Data: Note: At the time of analysis, we will perform the Shapiro-Wilk test and generate a quantile-quantile plot to assess the normality of the data. We will also perform Levene's test to assess homoscedasticity. If the data appears skewed, we will perform a transformation in order to proceed with the proposed statistical analysis. If this is not possible, we will perform the equivalent non-parametric test listed.
. Two-way ANOVA comparing normalized IP CRAF (to IP a myc) band intensity in D04 cells transfected with Myc-BRAF WT vector or Myc-BRAF R188L vector with or without SB590885 drug treatment, and the following Bonferroni-corrected comparisons: . Normalized IP CRAF band intensity in cells with Myc-BRAF WT vector with SB590885 treatment vs. vehicle treatment.
. Normalized IP CRAF band intensity in cells with Myc-BRAF R188L vector with SB590885 treatment vs. vehicle treatment.
. Meta-analysis of original and replication attempt effect sizes: . The replication data (mean and 95% confidence interval) will be plotted with the original quantified data value displayed as a single point on the same plot for comparison.

Known differences from the original study
The transfection method using Nucleofectin Solution V and electroporation will be replaced with a lipid-based method using Effectene Transfection Reagent, and protocol will be changed according to Manufacturer's instructions. The replication will replace 885-A with its analogue SB590885. The original authors suggest they have found similar results with this analogue (personal communication with Dr. Dhomen). All known differences, if any, are listed in the 'Materials and reagents' section above with the originally used item listed in the comments section. The comments section also lists if the source of original item was not specified. All differences have the same capabilities as the original and are not expected to alter the experimental design.
Cells will be sent for mycoplasma testing confirming lack of contamination and STR profiling confirming cell line authenticity. Transfection will be confirmed with Western blot. The transfer efficiency during the Western blot procedure will be monitored by Ponceau staining. All data obtained from the experiment -raw data, data analysis, control data, and quality control data -will be made publicly available, either as a published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/b1aw6/).

Protocol 5: Expression of BRAF kinase dead mutant in D04 cells and its effect on CRAF binding
This protocol describes how to transiently express myc-tagged BRAF WT or BRAF D59A in D04 cells and assess CRAF binding by immunoprecipitation and blotting, as reported in Figure 4D.

Sampling
. The experiment will be performed independently at least three times for a minimum power of 80%. The original data is qualitative, thus to determine an appropriate number of replicates to initially perform, sample sizes based on a range of potential variance was determined.
. See Power Calculations for details. . Each experiment consists of three cohorts: . Deliverables . Data to be collected: . Protein quantification results from Bradford assay. . Images of Ponceau stained membranes. . Images of whole gels (as reported in Figure 4D). . Densitometric quantification of all bands. . Any data pertaining to cell growth conditions optimization, if performed.

Confirmatory analysis plan
. Statistical Analysis of the Replication Data: . A two sample Welch's t-test comparing normalized IP CRAF (using IP myc-BRAF band intensity) in D04 cells transfected with Myc-BRAF WT vector vs. Myc-BRAF D594A vector . Meta-analysis of original and replication attempt effect sizes: . The replication data (mean and 95% confidence interval) will be plotted with the original quantified data value displayed as a single point on the same plot for comparison.

Known differences from the original study
All known differences, if any, are listed in the 'Materials and reagents' section above with the originally used item listed in the comments section. The comments section also lists if the source of original item was not specified. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control
Cells will be sent for mycoplasma testing confirming lack of contamination and STR profiling confirming cell line authenticity. Transfection will be confirmed with Western blot. The transfer efficiency during the Western blot procedure will be monitored by Ponceau staining. All data obtained from the experiment -raw data, data analysis, control data, and quality control data -will be made publicly available, either as a published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/b1aw6/). Cells will be sent for mycoplasma testing confirming lack of contamination and STR profiling confirming cell line authenticity.

Power calculations
For additional details on power calculations, please see analysis scripts and associated files on the Open Science Framework: https://osf.io/eaktg/ Protocol 1 Summary of original data . The original data presented is qualitative (images of Western blots). We used Image Studio Lite (LICOR) to perform densitometric analysis of the presented bands. We then performed a priori power calculations with a range of assumed standard deviations to determine the number of replicates to perform.
For A375  . Based on these power calculations, we will run the experiment three times. Each time, we will quantify band intensity. We will determine the standard deviation of band intensity across the biological replicates and combine this with the effect size from the original study to calculate the number of replicates necessary to reach a power of at least 80%. We will then perform additional replicates, if required, to ensure the experiment has more than 80% power to detect the original effect.

Protocol 2
Summary of original data . The original data presented is qualitative (images of Western blots). We used Image Studio Lite (LICOR) to perform densitometric analysis of the presented bands. We then performed a priori power calculations with a range of assumed standard deviations to determine the number of replicates to perform.
. Note: band intensity quantified from the image reported in Figure 1B:

Test family
. Due to the lack of raw original data, we are unable to perform power calculations using a MANOVA. We are determining sample size using two one-way ANOVAs.
. Two, one-way ANOVAs (Bonferroni corrected) on the difference in the normalized band intensity for pMEK and pERK separately in transfected cells treated with 885-A minus DMSO followed by Bonferroni corrected comparisons for the following groups: . . Based on these power calculations, we will run the experiment four times. Each time, we will quantify band intensity. We will determine the standard deviation of band intensity across the biological replicates and combine this with the effect size from the original study to calculate the number of replicates necessary to reach a power of at least 80%. We will then perform additional replicates, if required, to ensure the experiment has more than 80% power to detect the original effect.

Protocol 3
Summary of original data . The original data presented is qualitative (images of Western blots). We used Image Studio Lite (LICOR) to perform densitometric analysis of the presented bands. We then performed a priori power calculations with a range of assumed standard deviations to determine the number of replicates to perform.
. Note: band intensity quantified from the image reported in Figure 3A: . Based on these power calculations, we will run the experiment three times. Each time, we will quantify band intensity. We will determine the standard deviation of band intensity across the biological replicates and combine this with the effect size from the original study to calculate the number of replicates necessary to reach a power of at least 80%. We will then perform additional replicates, if required, to ensure the experiment has more than 80% power to detect the original effect.

Protocol 4
Summary of original data . The original data presented is qualitative (images of Western blots). We used Image Studio Lite (LICOR) to perform densitometric analysis of the presented bands. We then performed a priori power calculations with a range of assumed standard deviations to determine the number of replicates to perform.
. Note: band intensity quantified from the image reported in Figure 3B:  (Faul et al., 2007) . 2% variance: . ANOVA: Fixed effects, special, main effects, and interactions; alpha error = 0.05 . Based on these power calculations, we will run the experiment three times. Each time, we will quantify band intensity. We will determine the standard deviation of band intensity across the biological replicates and combine this with the effect size from the original study to calculate the number of replicates necessary to reach a power of at least 80%. We will then perform additional replicates, if required, to ensure the experiment has more than 80% power to detect the original effect.

Protocol 5
Summary of original data . The original data presented is qualitative (images of Western blots). We used Image Studio Lite (LICOR) to perform densitometric analysis of the presented bands. We then performed a priori power calculations with a range of assumed standard deviations to determine the number of replicates to perform.
. Note: band intensity quantified from the image reported in Figure 4D . The band intensities for two groups were beyond the dynamic range for intensity calculation: . IP myc-tagged BRAF in cells transfected with the BRAF mutant (D594A): In this case, we used the value for band intensity of IP myc-tagged BRAF in cells transfected with wild type BRAF as an estimate. Since the band for wild type BRAF transfected cells was less intense, this underestimates the effect size, so we are likely overestimating the sample size required.  1 The BRAF group SD was used as the divisor.
. Based on these power calculations, we will run the experiment three times. Each time, we will quantify band intensity. We will determine the standard deviation of band intensity across the biological replicates and combine this with the effect size from the original study to calculate the number of replicates necessary to reach a power of at least 80%. We will then perform additional replicates, if required, to ensure the experiment has more than 80% power to detect the original effect.