Abstract

The Reproducibility Project: Cancer Biology seeks to address growing concerns about reproducibility in scientific research by conducting replications of 50 papers in the field of cancer biology published between 2010 and 2012. This Registered report describes the proposed replication plan of key experiments from ‘Coadministration of a tumor-penetrating peptide enhances the efficacy of cancer drugs’ by Sugahara and colleagues, published in Science in 2010 (Sugahara et al., 2010). The key experiments being replicated include Figure 2 and Supplemental Figure 9A. In Figure 2, Sugahara and colleagues presented data on the tumor penetrance of doxorubicin (DOX) when co-administered with the peptide iRGD, as well as the effect of co-treatment of DOX and iRGD on tumor weight and cell death. In Supplemental Figure 9A, they tracked body weight of mice treated with DOX and iRGD to provide evidence that iRGD does not increase known DOX toxicity. The Reproducibility Project: Cancer Biology is a collaboration between the Center for Open Science and Science Exchange, and the results of the replications will be published by eLife.

DOI: http://dx.doi.org/10.7554/eLife.06959.001

Replication Study

  1. Replication Study: Coadministration of a tumor-penetrating peptide enhances the efficacy of cancer drugs

    1. Christine Mantis
    2. Irawati Kandela
    3. Fraser Aird
    4. Reproducibility Project: Cancer Biology
    eLife 2017;6:e17584

Original article

  1. Coadministration of a Tumor-Penetrating Peptide Enhances the Efficacy of Cancer Drugs

    1. KN Sugahara
    2. T Teesalu
    3. PP Karmali
    4. VR Kotamraju
    5. L Agemy
    6. DR Greenwald
    7. E Ruoslahti
    Science 2010;328:1031-1035

Main text

Introduction

αVβ3 integrin is a marker of tumor blood vessels and is targeted by the family of RGD peptides that mimic its natural ligand. This family of small peptides has been widely shown to promote targeting of a variety of therapeutics to tumor blood vessels (for review see Danhier et al., 2012; Feron, 2010). Sugahara and colleagues previously presented data showing that a novel cyclized form of this peptide, iRGD, coupled to a motif that binds to the Neuropilin-1 receptor, helped increase tissue penetrance beyond the vasculature of anti-cancer drugs when it was directly conjugated to those chemotherapies (Sugahara et al., 2009; Feron, 2010). In their 2010 study, they showed that iRGD can increase penetrance simply through co-administration with therapies, including peptide-based therapeutics, small molecule drug compounds, and nanoparticle-based therapeutics. This replication attempt will focus on the finding that iRGD co-treatment with doxorubicin (DOX) in orthotopic xenograft prostate tumors increased drug penetrance and accumulation, increased TUNEL staining for apoptotic cells, and decreased tumor volume and weight.

In Figure 2B, Sugahara and colleagues showed increased tissue permeability of DOX when co-injected with iRGD by demonstrating increased accumulation of DOX in the prostate when intravenously co-injected with iRGD. They documented little change in DOX penetrance in non-prostate tissues of the mouse. This figure is evidence of the first key point for replication: that iRGD increases tissue penetrance of the co-injected drug. This experiment will be replicated in Protocol 3.

In Figure 2C, Sugahara and colleagues showed how treatment with varying doses of DOX, with or without iRGD, affected tumor weight. They showed that treatment with 1 mg/kg DOX with iRGD caused a decrease in tumor weight similar to treatment with 3 mg/kg DOX alone. Addition of iRGD to 3 mg/kg DOX further decreased tumor weight. In Supplemental Figure 9A, they also showed that the overall body weight of mice treated with DOX and iRGD did not change as compared to mice treated with DOX alone, indicating that DOX-related weight gain was not exacerbated by the addition of iRGD. These experiments will be replicated in Protocol 4.

In Figure 2D, Sugahara and colleagues showed that treatment with DOX in combination with iRGD increased the number of TUNEL positive (i.e., apoptotic) tumor cells as compared to DOX alone. They also showed that iRGD did not increase the number of TUNEL positive cells in the heart, indicating that DOX-related cardiotoxicity is not exacerbated by co-treatment with iRGD. This experiment will be replicated in Protocol 5.

To date, the closest direct replication of these experiments has been performed by Akashi and colleagues, who tested the effects of co-administration of iRGD with the drug gemcitabine to target xenograft models of pancreatic cancer. Relative tumor volume in two pancreatic cancer cell line-derived xenografts treated with gemcitabine in combination with iRGD was significantly reduced when compared to tumor volume from xenografts treated with gemcitabine alone. Additionally, Akashi and colleagues assessed compound penetration by co-administering iRDG and the dye Evans Blue. They showed that Evans Blue dye penetrated further into tumor tissue when co-administered with iRGD than alone (Akashi et al., 2014). Additionally, Gu and colleagues showed that co-administration of iRGD with encapsulated paclitaxel increased nanoparticle extravasation across the blood–brain barrier, and co-administration of paclitaxel nanoparticles and iRGD led to an increase in mean survival of mice carrying intracranial gliomas (Gu et al., 2013). Furthermore, Pang and colleagues extended the work done in Sugahara and colleagues' 2010 paper by showing that binding of iRDG through an additional cysteine residue to plasma albumin prolonged the half-life of iRGD and increased tumor penetrance (Pang et al., 2014). In 2011, Ruoslahti and colleagues extended their findings to show that iRDG was also effective in increasing tumor penetrance of a novel therapeutic consisting of a nanoparticle-encapsulated peptide that homed to tumor vasculature and disrupted mitochondrial membranes, causing cell death; this co-treatment increased the survival of mice bearing glioblastomas as compared to mice treated with the nanoparticle alone (Agemy et al., 2011).

Materials and methods

Unless otherwise noted, all protocol information was derived from the original paper, references from the original paper, or information obtained directly from the authors. An asterisk (*) indicates data or information provided by the Reproducibility Project: Cancer Biology core team. A hashtag (#) indicates information provided by the replicating lab.

Protocol 1: synthesis of iRDG

This summary describes the synthesis of the peptide iRDG based on information from Sugahara and colleagues (Sugahara et al., 2009, 2010). The iRGD peptide (H-Cys-Arg-Gly-Asp-Lys-Gly-Pro-Asp-Cys-NH2) will be chemically synthesized using Fmoc (9-fluorenylmethoxy carbonyl) chemistry sequence. The fully synthesized crude peptide will be cleaved from the resin and cleaned with Trifluoroacetic acid (TFA). The crude peptides will be then diethyl ether precipitated, drained, and washed. The peptides will then be amide blocked on the C terminus and cyclized by a disulfide bridge between C1 and C9. The peptides will be isolated and purified by high-performance liquid chromatography (HPLC). Fractions of greater than 95% purity will be used for the investigation. The purity and molecular weight of the peptide will be confirmed by matrix-assisted laser desorption ionization (MALDI)-time of flight mass spectrometry. Exact synthesis specifications were not originally specified; the lab will follow standard procedures for synthesis.

All data obtained from the experiment—synthesis specifications, materials, raw data, data analysis, control data, and quality control data including the MALDI-TOF mass spectrometry data—will be made publicly available, either in the published manuscript or as an open access data set available on the Open Science Framework (https://osf.io/xu1g2/).

Protocol 2: generation of mice bearing orthotopic prostate tumors

This protocol describes how to generate mice that carry orthotopic 22Rv1 human prostate tumor xenografts.

Sampling

  • Number of mice required:

    1. For Protocol 3: 10 mice.

    2. For Protocols 4 and 5: 21 mice.

    3. Total: 31 mice.

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
22Rv1 human prostate cellsCellsATCCCRL-2505
DMEM—high glucoseMediumSigma–AldrichD6429Original unspecified
FBSReagentInvitrogen16,000Original unspecified
Penicillin/StreptomycinReagentSigma–AldrichP4333Original unspecified
Male 4- to 6-week-old athymic BALB/c nude miceMiceHarlan laboratoriesOrder code 069(nu)/070(nu/+)
4-0 coated vicryl sutureMaterialsEthiconJ835GOriginal unspecified
Poly(vinylpyrrolidone)–Iodine complexReagentSigma–AldrichPVP-1Original unspecified
buprenonorphineReagentLeft to the discretion of the replicating lab and recorded laterOriginal unspecified
isofluraneAnestheticOriginal unspecified
Procedure

Note:

  • 22Rv1 prostate cells are maintained in DMEM with 10% fetal bovine serum and penicillin/streptomycin at 37°C/5% CO2.

    1. Resuspend 1 × 106 22Rv1 cells per 10 µl for each injection.

    2. Inject cells orthotopically into the prostate glands of male 4- to 6-week-old athymic BALB/c nude mice#.

      • a. Anesthetize mice using 2% isoflurane.

      • b. Prepare skin for incision with povidone-iodine.

        • i. Use sterile drapes, gloves, and instruments.

      • c. Make a 5 mm midline incision over the bladder.

      • d. Grasp the bladder with blunt forceps and move aside to expose ventral prostate gland.

      • e. Inject cells (10 µl) into ventral prostate gland.

      • f. Close incision with sutures.

      • g. Inject mice subcutaneously with 0.1 mg/kg buprenonorphine immediately post operatively.

      • h. Observe mice until they are awake, ambulatory, and drinking.

      • i. Check mice again at the end of the day (around 5 pm).

      • j. The next morning, inject mice subcutaneously with 0.1 mg/kg buprenonorphine.

      • k. Remove sutures on Day 7.

Deliverables

  • Data to be collected:

    1. Mouse health records (age of mice at time of injection).

  • Samples delivered for further analysis:

    1. Orthotopic tumor bearing mice for use in Protocols 2–4.

Confirmatory analysis plan

  • Statistical analysis of the replication data:

    1. None applicable.

  • Meta-analysis of original and replication attempt effect sizes:

    1. Not applicable.

Known differences from original study:

  • The xenograft injection protocol is from the replicating lab. The original protocol was not specified.

Provisions for quality control

The cell lines used in this experiment will undergo STR profiling to confirm their identity and will be sent for mycoplasma testing to ensure there is no contamination. Additionally, cells used for xenograft injection will be screened against a Rodent Pathogen Panel to ensure no contamination prior to injection. All data obtained from the experiment—raw data, data analysis, control data, and quality control data—will be made publicly available, either in the published manuscript or as an open access data set available on the Open Science Framework (https://osf.io/xu1g2/).

  • A lab with experience in prostate gland tumor xenografts will perform the experiment.

Protocol 3: quantifying the amount of Dox present in tumor tissue and major organs in mice treated with Dox with or without iRGD

This protocol describes how to treat mice bearing human 22Rv1 prostate tumors from Protocol 2 with DOX and/or iRGD, harvest the tumors and assess DOX penetrance by measuring absorbance at 490 nm (OD490), as seen in Figure 2B.

Sampling

  • This experiment will analyze at least 3 mice per group for a final power of 97.2%.

    1. See power calculations section for details.

  • The experiment consists of two cohorts:

    1. Cohort 1: mice treated with DOX and PBS.

      • A. N = 4.

        • To buffer against unexpected mouse deaths, 4 mice bearing tumors will be treated.

    2. Cohort 2: mice treated with DOX and iRGD.

      • A. N = 4.

        • To buffer against unexpected mouse deaths, 4 mice bearing tumors will be treated.

    3. Cohort 3: untreated mice.

      • A. N = 2.

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
Orthotopic tumor bearing miceMiceFrom Protocol 2
Doxorubicin hydrochlorideDrugSigma–AldrichD1515Original unspecified
iRGDPeptideFrom Protocol 1
30 G needle/syringe for IV injection of drugs (step 3)MaterialsBD Biosciences305106
Bovine Serum Albumin (BSA)ReagentSigma–AldrichA2153Original unspecified
ChloroformReagentSigma–AldrichC2432Original unspecified
Isopropyl alcoholReagentLeft to the discretion of the replicating lab and recorded laterOriginal unspecified
15-ml conical tubesMaterialsCorning Life Sciences352095Original unspecified
Dulbecco's Phosphate Buffered Saline (PBS)ReagentSigma–AldrichD8537Original unspecified
Animal feeding needle, 24 G, L × diam. 1 in. × 1.25 mm, ballMaterialSigma–AldrichCAD7900Original unspecified
18 G syringe needlesMaterialSigma–AldrichZ192554Original unspecified
Sodium dodecyl sulfate solution (10%)ReagentSigma–Aldrich71,736Original unspecified
Sulfuric acid (H2S04)ReagentSigma–Aldrich339741Original unspecified
8453 UV-Vis spectrophotometerInstrumentAgilentOriginal unspecified
DMEM—high glucoseMediumSigma–AldrichD6429Original unspecified
Procedure

  1. Generate tumor bearing mice as per Protocol 2.

  2. Allow tumors to grow for 2 weeks from time of injection.

  3. Inject mice with drugs in combination:

    • a. On day of injection, randomly assign the 10 mice into the two treatment groups and the untreated group.

      • i. Assign each mouse a number 1 through 10.

      • ii. After mice have been assigned numbers, enter the treatment labels (4 labels as Negative control, 4 labels as Experimental, and 2 labels as Untreated), and randomize 10 subjects into 1 block using www.randomization.com. Record seed number.

    • b. Negative control: inject mice intravenously with 10 mg/kg Dox suspended in 100 µl PBS.

    • c. Experimental: inject mice intravenously with 10 mg/kg Dox and 4 µmol/kg iRGD suspended in 100 µl PBS.

    • d. Untreated mice receive no injections.

  4. 1 hour later, sacrifice mice and excise tissues:

    • a. Deeply anesthetize the mice with isoflurane.

    • b. Perfuse through the heart with PBS + 1% BSA#.

      • i. Place the deeply anesthetized mouse in a heated cage for 10 min.

      • ii. Secure the mouse in the supine position by taping the paws to a Styrofoam work surface.

      • iii. Make an incision through the skin with surgical scissors along the thoracic midline from just beneath the xiphoid process to the clavicle. Make two additional skin incisions from the xiphoid process along the base of the ventral ribcage laterally.

      • iv. Reflect the two flaps of skin rostrally and laterally making sure to expose the thoracic field completely.

      • v. Grasp the cartilage of the xiphoid process with blunt forceps and raise it slightly to insert pointed scissors. Cut through the thoracic musculature and ribcage between the breastbone and medial rib insertion points and extend the incision rostrally to the level of the clavicles.

      • vi. Separate the diaphragm from the chest wall on both sides with scissors.

      • vii. Pin with 18 G needles the reflected ribcage laterally to expose the heart.

      • viii. Gently grasp the pericardial sac with blunt forceps and tear it fully.

      • ix. Secure the beating heart with blunt forceps and make a 1–2 mm incision in the left ventricle. Immediately insert a 24 G × 25.4 mm animal feeding needle. The tip is bulbous and will not damage the heart. Thread the feeding needle into the base of the aortic arch using a dissecting microscope. Clamp the needle base to the left ventricle above the incision site using a hemostat.

      • x. Cut the right atrium with scissors and at the first sign of blood flow, begin infusion of DMEM containing 1% BSA (stage 1 perfusate).

        1. Use gravity driven perfusion at a rate of 3 mls per minute.

      • xi. Continue perfusing the body until the fluid exiting the right atrium is entirely clear.

      • xii. Ensure that organs of interest become pale; if an organ does not become pale, exclude the organ from further analysis.

    • c. Excise prostate tumor tissue, liver, spleen, pancreas, heart, lung, kidneys, and brain.

  5. Homogenize each tissue separately in 1% sodium dodecyl sulfate and 1 mM H2SO4 in water.

    • a. Homogenize 100-mg tissue in 0.5 ml solution using a mortar and pestle#.

  6. Add 2 ml of chloroform:isopropyl alcohol (1:1, vol/vol).

  7. Vortex in 15-ml conical tubes and run through freeze/thaw cycles.

    • a. #Place tubes on dry ice for 1 min to freeze, then thaw for 5 min in a 25°C waterbath.

  8. Centrifuge samples at 14,000×g for 15 min at #room temperature.

    • a. Store samples at 4°C until ready for Step 9.

    • b. For the two untreated mice, combine their samples to create the blank reference for each tissue; that is, combine tumor with tumor, liver with liver, etc.

  9. Measure the OD490 of the organic phase (the lowest phase).

    • a. For each measurement, blank with appropriate tissue homogenate (tumor, liver, spleen, etc) from the untreated control mice.

    • b. Calculate the fold change in Dox level from mice treated with iRGD by dividing by the absorbance reading of mice treated with Dox alone.

    • c. Graph the fold change by tissue.

Deliverables

  • Data to be collected:

    1. Raw readings of OD490 absorbance of each sample.

    2. Graph of DOX accumulation with iRGD or PBS per organ (compare to Figure 2B).

Confirmatory analysis plan

  • Statistical analysis of the replication data:

    1. At the time of analysis, we will perform the Shapiro–Wilk test and generate a quantile–quantile (q–q) plot to attempt to assess the normality of the data and also perform Levene's test to assess homoscedasiticity. If the data appear skewed, we will attempt a transformation in order to proceed with the proposed statistical analysis listed below and possibly perform the appropriate non-parametric test.

      • A. Compare the level of Dox + iRGD to level of Dox alone in tumor tissue.

        • Unpaired two-tailed Student's t-test.

          1. Original analysis.

  • Meta-analysis of original and replication attempt effect sizes:

    1. This replication attempt will perform the statistical analysis listed above, compute the effects sizes, compare them against the reported effect size in the original paper, and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.

Known differences from original study

  • Details noted with a hashtag (#) were provided by the replicating lab.

Provisions for quality control

Tissue homogenate from untreated mice will be used to blank the spectrophotometer. Mice will be randomly assigned to treatment groups. All data obtained from the experiment—raw data, data analysis, control data and quality control data—will be made publicly available, either in the published manuscript or as an open access data set available on the Open Science Framework (https://osf.io/xu1g2/).

  • A lab with experience in prostate gland tumor xenografts will perform the experiment.

Protocol 4: effect of Dox alone or Dox in combination with iRGD on tumor growth and total body weight

This protocol describes how to treat mice bearing human 22Rv1 prostate tumors from Protocol 2 with DOX and/or iRGD, monitor body weight and then assess tumor weight, as seen in Figure 2C and Supplemental Figure 9A.

Sampling

  • This experiment will analyze at least 6 mice per group for a final power of 93.5%.

    1. See power calculations section for details.

  • The experiment consists of three cohorts:

    1. Cohort 1: mice treated with PBS alone.

      • A. N = 7.

        • To buffer against unexpected mouse deaths, 7 mice bearing tumors will be treated.

    2. Cohort 2: mice treated with 1 mg/kg Dox and PBS.

      • A. N = 7.

        • To buffer against unexpected mouse deaths, 7 mice bearing tumors will be treated.

    3. Cohort 3: mice treated with 1 mg/kg Dox and 4 µmol/kg iRGD.

      • A. N = 7.

        • To buffer against unexpected mouse deaths, 7 mice bearing tumors will be treated.

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
Orthotopic tumor bearing miceMiceFrom Protocol 2
Doxorubicin hydrochlorideDrugSigma–AldrichD1515Vehicle is PBS
iRGDPeptideFrom Protocol 1Vehicle is PBS
ParaformaldehydeReagentSigma–Aldrich158127Original unspecified
ParaffinReagentSpecific brand left to the discretion of the replicating lab and will be recorded laterOriginal unspecified
Bovine Serum Albumin (BSA)ReagentSigma–AldrichA2153Original unspecified
Dulbecco's Phosphate Buffered Saline (PBS)ReagentSigma–AldrichD8537Original unspecified
Procedure

  1. Generate tumor bearing mice as per Protocol 2.

  2. Allow tumors to grow for 2 weeks from time of injection.

  3. Inject mice with drugs according to their cohort; this is Day 0.

    • a. On day of injection, randomly assign the 21 mice into the three treatment groups.

      • i. Assign each mouse a number 1 through 21.

      • ii. After mice have been assigned numbers, enter the treatment labels (Cohort 1, Cohort 2, and Cohort 3) and randomize 3 subjects into 7 blocks using www.randomization.com. Record seed number.

    • b. Cohort 1: mice treated with PBS alone.

    • c. Cohort 2: mice treated with 1 mg/kg Dox and PBS.

    • d. Cohort 3: mice treated with 1 mg/kg Dox and 4 µmol/kg iRGD.

  4. Repeat injection every other day for 24 days.

  5. Weigh mice every 4 days, starting on Day 0.

  6. After 24 days of treatment, harvest tissue.

    • a. Perfuse mice as outlined in Protocol 3 Steps 4a–b.

    • b. Excise prostate tumor tissue and heart tissue.

  7. Weigh tumor tissue.

  8. Process, embed, and section tissue.

    • a. Fix tumor and heart tissue in 4% paraformaldehyde overnight at 4°C.

    • b. Cut each tumor and each heart in half.

    • c. #Dehydrate tissue and infiltrate with paraffin.

    • d. #Embed in paraffin.

      • i. Use one half of each tumor or heart to perform the sectioning below. Hold the other half in reserve in case more sections are needed later.

    • e. Cut at least 7 5-µm thick sections spaced throughout the tumor or heart halves and mount on glass slides (i.e., the sections should not be serial).

      • i. Tumor and heart sections to be used in Protocol 5.

Deliverables

  • Data to be collected:

    1. Record of the drug treatment regimen and weight of each tumor for each mouse.

    2. Raw values for mouse body weight at time points during treatment.

    3. Graph of tumor weight by drug treatment in grams (compare to Figure 2C).

    4. Graph of change in body weight as a percentage of body weight on day 0 (compare to Supplemental Figure 9A).

  • Samples delivered for further analysis.

    1. Tumor and heart tissue processed to sections for further analysis (see Protocol 5).

Confirmatory analysis plan

  • Statistical analysis of the replication data:

    1. At the time of analysis, we will perform the Shapiro–Wilk test and generate a quantile–quantile (q–q) plot to attempt to assess the normality of the data and also perform Levene's test to assess homoscedasiticity. If the data appear skewed, we will attempt a transformation in order to proceed with the proposed statistical analysis listed below and possibly perform the appropriate non-parametric test.

      • A. Tumor weights in each cohort (as seen in Figure 2C).

        • One-way ANOVA followed by Fisher's LSD t-tests for the following comparisons:

          1. 1 mg/kg Dox vs 1 mg/kg Dox and 4 µmol/kg iRGD.

      • A. Body weight shift (as seen in Supplemental Figure 9A).

        • One-way ANOVA on Day 24 time points.

          1. As seen in the original analysis.

        • Additional analysis: one-way ANOVA of calculated area under the curve of mouse body weight from each cohort followed by Fisher's LSD corrected t-tests for the following comparison:

          1. 1 mg/kg Dox vs 1 mg/kg Dox and 4 µmol/kg iRGD.

  • Meta-analysis of original and replication attempt effect sizes:

    1. This replication attempt will perform the statistical analysis listed above, compute the effects sizes, compare them against the reported effect size in the original paper and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.

Known differences from original study

  • The replication study will be restricted to examining the following groups:

    1. No dox/no peptide.

    2. 1 mg/kg Dox.

    3. 1 mg/kg Dox/iRGD.

  • The tumor tissue will be embedded in paraffin for paraffin sectioning rather than in OCT for cryosectioning.

Provisions for quality control

Mice will be randomly assigned to treatment groups. All data obtained from the experiment—raw data, data analysis, control data, and quality control data—will be made publicly available, either in the published manuscript or as an open access data set available on the Open Science Framework (https://osf.io/xu1g2/).

  • A lab with experience in prostate gland tumor xenografts will perform the experiment.

Protocol 5: assessment of TUNEL staining of tumor and heart tissue after drug treatment

This protocol describes how to assess cell death via TUNEL staining in prostate tumors derived from 22Rv1 xenografts treated with DOX and/or iRGD, as seen in Figure 2D.

Sampling

  • This protocol uses tissues derived from Protocol 4.

    1. This experiment will analyze 6 tumors per group, for a final power of 88.8%.

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
Sections of tumor and heart tissueTissueFrom Protocol 4
In Situ Cell Death Detection Kit PODReagentRoche Applied Science11684817910
MicroscopeInstrumentOlympusBX40
Scanscope scannerInstrumentAperioCM-1
ImageJSoftwareNIH
Procedure

Note: This protocol uses tumor and heart tissues derived from Protocol 4.

  1. Perform TUNEL staining of tumor and heart tissue sections with the In Situ Cell Death Detection Kit POD according to the manufacturer's instructions.

    • a. Include manufacturer's recommended controls:

      • i. Positive control: incubate with label solution instead of with TUNEL reaction mixture.

      • ii. Negative control: incubate with micrococcal nuclease prior to labeling procedure.

    • b. Stain a total of 7 slides for each tissue; 5 for analysis, one negative control, one positive control.

  2. Scan the stained sections with a Scanscope CM-1 scanner and quantify areas of TUNEL positive staining with ImageJ software.

    • a. *Image 5 random fields at 40× per section and image 5 sections per tumor and per heart.

    • b. If sections are unable to be imaged due to autofluorescence or damage during the staining procedure, take images, and exclude from analysis with indicated reason.

Deliverables

  • Data to be collected:

    1. All images taken for all tumors and treatment groups, including control images.

    2. Raw numbers from ImageJ analysis for each field and section of each tumor and heart.

    3. Determine the ratio of TUNEL staining as fold change relative to staining in the tumors treated with PBS only. Graph by treatment regimen.

Confirmatory analysis plan

  • Statistical analysis of the replication data:

    1. At the time of analysis, we will perform the Shapiro–Wilk test and generate a quantile–quantile (q–q) plot to attempt to assess the normality of the data and also perform Levene's test to assess homoscedasiticity. If the data appear skewed, we will attempt a transformation in order to proceed with the proposed statistical analysis listed below and possibly perform the appropriate non-parametric test.

      • A. Compare TUNEL positive area in tumor across all conditions.

        • Bonferroni corrected one-way ANOVA followed by Bonferroni corrected t-tests for the following comparisons:

          1. Mice treated with 1 mg/kg Dox and PBS vs mice treated with 1 mg/kg Dox and 4 µmol/kg iRGD.

      • A. Compare TUNEL positive area in heart across all conditions.

        • Bonferroni corrected one-way ANOVA followed by Bonferroni corrected t-tests for the following comparisons:

          1. Mice treated with 1 mg/kg Dox and PBS vs mice treated with 1 mg/kg Dox and 4 µmol/kg iRGD.

  • Meta-analysis of original and replication attempt effect sizes:

    1. This replication attempt will perform the statistical analysis listed above, compute the effects sizes, compare them against the reported effect size in the original paper, and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.

Known differences from original study

  • The replication study will be restricted to examining the following groups:

    1. No dox/no peptide.

    2. 1 mg/kg Dox/no peptide.

    3. 1 mg/kg Dox/iRGD.

Provisions for quality control

Manufacturer recommended positive and negative controls will be used when performing TUNEL staining of tumor and heart tissues. All data obtained from the experiment—raw data, data analysis, control data, and quality control data—will be made publicly available, either in the published manuscript or as an open access data set available on the Open Science Framework (https://osf.io/xu1g2/).

  • A lab with experience in prostate gland tumor xenografts will perform the experiment.

Power calculations

Protocol 1

  • Not applicable.

Protocol 2

  • Not applicable.

Protocol 3
Summary of original data

  • Note: values estimated from original graph.

Figure 2B: DOX accumulation fold free doxMeanSEMSDN
TumorDOX10.480.833
DOX + iRGD7.151.051.823
LiverDOX10.61.043
DOX + iRGD1.510.671.163
SpleenDOX10.40.693
DOX + iRGD0.360.721.253
PancreasDOX10.691.203
DOX + iRGD0.150.090.163
HeartDOX10.570.993
DOX + iRGD0.210.290.503
LungDOX10.671.163
DOX + iRGD0.770.661.143
KidneyDOX10.120.213
DOX + iRGD10.020.033
BrainDOX10.190.333
DOX + iRGD0.440.230.403
  • Stdev was calculated using formula SD = SEM*(SQRT n).

Test family

  • Unpaired two-tailed Student's t-test comparing tumor free Dox to tumor Dox + iRGD.

    1. As seen in the original analysis.

Power calculations

Group 1 vsGroup 2Effect sizeA priori powerGroup 1 sample sizeGroup 2 sample size
Dox alone in tumor tissueDox + iRGD in tumor tissue4.34800097.2%33
Protocol 4
Summary of original data

  • Note: values estimated from original graph.

Figure 2C: Tumor weightMeanSEMSDN
Free DoxPeptide
NoneNone1.190.070.2210
1 mg/kgNone0.8170.0930.2910
1 mg/kgiRGD0.350.020.0610
  • Stdev was calculated using formula SD = SEM*(SQRT n).

Test family

  • One-way ANOVA followed by Fisher's LSD t-tests for the following comparison:

    1. Mice treated with 1 mg/kg Dox and PBS vs mice treated with 1 mg/kg Dox and iRGD.

Power calculations

One-way ANOVA
F (2, 27)p-valuePartial η2Effect size fA priori powerSample size per group
39.045<0.00010.743081.70066594.1%*3*
  • Due to power calculations for Figure 2D, we will be using 6 tumors per group, for an achieved power of 99.9%.

T-test
Group 1 vsGroup 2Effect size dA priori powerGroup 1 sample sizeGroup 2 sample size
1 mg/kg Dox and PBS1 mg/kg Dox and iRGD2.23014086.9%*5*5*
  • Due to power calculations for Figure 2D, we will be using 6 tumors per group, for an achieved power of 93.5%.

Summary of original data

  • Note: values estimated from original graph

Supp. Figure 9AMean body weight shift (%)SEMSDN
PBSDay 01000010
Day 4100.6391.5324.84460937510
Day 8100.9581.5644.94580226110
Day 12102.2982.4897.87090909610
Day 16104.51.5324.84460937510
Day 20105.5852.2026.96333540810
Day 24105.9041.5644.94580226110
1 mg/kg DoxDay 01000010
Day 4101.2130.9573.02629972110
Day 8103.2871.4854.69598232510
Day 12103.3351.4844.69282004810
Day 16105.3942.0596.51112970210
Day 20105.5852.2026.96333540810
Day 24106.7342.3467.41870339110
1 mg/kg dox + iRGDDay 01000010
Day 499.2032.2026.96333540810
Day 899.9681.1813.73464991710
Day 1299.841.3094.13942145710
Day 16101.6921.4384.54735527510
Day 20102.5531.5644.94580226110
Day 24103.321.6285.14818803110
Test family

  • One-way ANOVA on Day 24 time points.

    1. As seen in the original analysis.

  • Additional analysis: one-way ANOVA of calculated area under the curve of mouse body weight from each cohort followed by Fisher's LSD corrected t-tests for the following comparison:

    1. 1 mg/kg Dox vs 1 mg/kg Dox and 4 µmol/kg iRGD.

Sensitivity calculations

  • F statistic and partial η2 performed with R software (3.1.2) (R Core team, 2014).

  • Sensitivity calculations performed using G*power software (Faul et al., 2007).

  • α = 0.05.

  • Sample size derived from Figure 2D (Protocol 5) power calculations.

  • One way ANOVA on day 24 time points.

One-way ANOVA
F (2, 27)Partial η2Reported effect size fDetectable effect size fA priori powerSample size per group
0.04870.0035970.0600830.81128280.0%6
  • Additional analysis: area under the curve performed with R software (3.1.2) (R Core team, 2014).

Area under the curve
MeanSDN
PBS2467.728127.768710
1 mg/kg dox2488.724118.395710
1 mg/kg dox + iRGD2419.664107.618610
One-way ANOVA
F (2, 27)Partial η2Reported effect size fDetectable effect size fA priori powerSample size per group
0.89690.0622970.2577510.81128280.0%6
Fisher's LSD corrected t-test
Group 1Group 2Reported effect size dDetectable effect size dA priori powerSample size per group
1 mg/kg dox1 mg/kg dox + iRGD0.6104181.79554180.0%6
Protocol 5
Summary of original data

  • Note: Values estimated from original graph.

Figure 2D: TUNEL staining of tumor and heartMeanSEMSDN
Free DoxPeptideTissue
NoneNoneTumor10.10.3210
Heart10.270.8510
1 mg/kgNoneTumor1.40.170.5410
Heart1.650.280.8910
1 mg/kgiRGDTumor2.580.20.6310
Heart1.70.310.9810
  • Stdev was calculated using formula SD = SEM*(SQRT n).

Test family

  • Bonferroni corrected one-way ANOVAs followed by Bonferroni's corrected t-tests for the following comparison:

    1. Tumor tissue from mice treated with 1 mg/kg Dox and PBS vs mice treated with 1 mg/kg Dox and iRGD.

Power calculations

  • F statistic and partial η2 performed with R software (3.1.2) (R Core Team, 2014).

  • Power calculations performed using G*power software (Faul et al., 2007).

  • α = 0.025.

  • Sample size calculations performed for tumor tissue and sensitivity calculations for heart tissue.

One-way ANOVA
F (2, 27)Partial η2Reported effect size fDetectable effect size fA priori powerSample size per group
Tumor
25.5960.6546981.376959n/a89.5%*4*
Heart
1.84850.1204340.3700330.90863980.0%6
  • With a sample size of 6 per group, achieved power is 99.4%.

Bonferroni corrected t-tests
Group 1 vsGroup 2Reported effect size dDetectable effect size dA priori powerGroup 1 sample sizeGroup 2 sample size
Tumor
1 mg/kg Dox alone1 mg/kg Dox + iRGD2.298170n/a88.8%66
Heart
1 mg/kg Dox alone1 mg/kg Dox + iRGD0.0550482.04418580.0%66

References

Acknowledgements

The Reproducibility Project: Cancer Biology core team would like to thank the following companies for generously donating reagents to the Reproducibility Project: Cancer Biology; American Type Culture Collection (ATCC), BioLegend, Cell Signaling Technology, Charles River Laboratories, Corning Incorporated, DDC Medical, EMD Millipore, Harlan Laboratories, LI-COR Biosciences, Mirus Bio, Novus Biologicals, Sigma–Aldrich, and System Biosciences (SBI).

Decision letter

Duojia Pan, Reviewing editor, Howard Hughes Medical Institute, Johns Hopkins University School of Medicine, United States

eLife posts the editorial decision letter and author response on a selection of the published articles (subject to the approval of the authors). An edited version of the letter sent to the authors after peer review is shown, indicating the substantive concerns or comments; minor concerns are not usually shown. Reviewers have the opportunity to discuss the decision before the letter is sent (see review process). Similarly, the author response typically shows only responses to the major concerns raised by the reviewers.

Thank you for sending your work entitled “Registered report: Coadministration of a Tumor-Penetrating Peptide Enhances the Efficacy of Cancer Drugs” for consideration at eLife. Your article has been favorably evaluated by Stylianos Antonarakis (Senior editor), a Reviewing editor, and two reviewers, one of whom, Erkki Ruoslahti, has agreed to share his identity.

The Reviewing editor and the reviewers discussed their comments before we reached this decision, and the Reviewing editor has assembled the following comments to help you prepare a revised submission.

Overall, the reviewers have found the experimental plan and statistical analysis to be reasonable. One particular concern is the number of animals. While they are sufficient for one experiment, ideally one would like to see the experiment to be repeated at least once. The reviewers have suggested the following corrections/improvements to be considered in a revision:

Reviewer #1:

This is a reasonable plan, but a few issues should be dealt with:

1) It would be appropriate to cite Agemy et al. PNAS 108:17450-17455 (2011) PMCID: PMC3198371 in the Introduction, as it reports iRGD work partially conducted at the Salk Institute.

2) The peptide sequence in the subsection headed “Protocol 1: Synthesis of iRDG” is incorrect.

3) The iRGD peptide sequence in the subsection headed “Protocol 1: Synthesis of iRDG” is shown with unblocked N- and C-termini. The original paper is silent on whether they were blocked or not, but in the subject work, Sugahara et al. used iRGD that was amide-blocked at the C-terminus. Although the unblocked peptide is also known to be active, it would be safer to block the C-terminus.

4) The tumor line Sugahara et al. used is 22Rv1 (ATCC #CRL-2505). Here it is said to be RRv1 and the ATCC number given corresponds to 22Rv1.

5) Protocol 3, item 4b: In the protocol, the mice are first euthanized but later on the mice are said to have a beating heart. The perfusion should be done under deep anesthesia with beating heart, not after euthanasia. Also, gravity driven perfusion at about 3 ml/min should be used, and the solution should be DMEM containing 1% BSA (not heparinzed saline). Besides making sure that fluid exciting the right atrium becomes clear, one should see a change in blood content of organs; if an organ does not become pale, that tissue should not be used for further analyses.

6) The most serious flaw in the plan concerns the number of mice to be used. They do not allow for any repeat experiments. To rely on one experiment in a study that will presumably be highly publicized and could make or break careers does not seem appropriate.

Reviewer #2:

This protocol plans to replicate some of the key findings of Sugahara and colleagues regarding the benefits of DOX + iRGD in relation to prostate tumor in mice.

The power calculations are fine.

Given that the proposed work requires a small number of mice, it may be worthwhile to consider nonparametric analysis methods in addition to the proposed t test and ANOVA approaches during data analyses.

The investigators plan to use mixed effects model ANOVA as an additional exploratory analysis in protocol 3. It is not clear why this analysis is needed. It is possible that a mixed effects model would be stretching the data too far due to the small number of mice. My suggestion would be to remove this mixed effects analysis plan (unless there is a compelling reason to pursue this).

DOI: http://dx.doi.org/10.7554/eLife.06959.002

Author response

Comments

If your username is different from your full name, we require you to identify yourself within the comment itself. Comments are checked by a moderator (and/or an eLife editor) before they appear. Comments should be constructive, relevant to the article, conform to our terms and conditions, and include any pertinent competing interests.