Design of the HPV-automated visual evaluation (PAVE) study: Validating a novel cervical screening strategy

  1. Silvia de Sanjosé  Is a corresponding author
  2. Rebecca B Perkins
  3. Nicole Campos
  4. Federica Inturrisi
  5. Didem Egemen
  6. Brian Befano
  7. Ana Cecilia Rodriguez
  8. Jose Jerónimo
  9. Li C Cheung
  10. Kanan Desai
  11. Paul Han
  12. Akiva P Novetsky
  13. Abigail Ukwuani
  14. Jenna Marcus
  15. Syed Rakin Ahmed
  16. Nicolas Wentzensen
  17. Jayashree Kalpathy-Cramer
  18. Mark Schiffman
  19. On behalf of the PAVE Study Group
  1. Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, United States
  2. ISGlobal, Spain
  3. University Chobanian and Avedisian School of Medicine/Boston Medical Center, United States
  4. Center for Health Decision Science, Harvard T.H. Chan School of Public Health, United States
  5. Information Management Services Inc, United States
  6. Department of Epidemiology, University of Washington School of Public Health, United States
  7. Division of Cancer Control and Population Sciences, National Cancer Institute, National Institutes of Health, United States
  8. Westchester Medical Center/New York Medical College, United States
  9. Feinberg School of Medicine at Northwestern University, United States
  10. Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, United States
  11. Harvard Graduate Program in Biophysics, Harvard Medical School, Harvard University, United States
  12. Massachusetts Institute of Technology, United States
  13. Geisel School of Medicine at Dartmouth, Dartmouth College, United States
  14. University of Colorado Anschutz Medical Campus, United States

Abstract

Background:

The HPV-automated visual evaluation (PAVE) Study is an extensive, multinational initiative designed to advance cervical cancer prevention in resource-constrained regions. Cervical cancer disproportionally affects regions with limited access to preventive measures. PAVE aims to assess a novel screening-triage-treatment strategy integrating self-sampled HPV testing, deep-learning-based automated visual evaluation (AVE), and targeted therapies.

Methods:

Phase 1 efficacy involves screening up to 100,000 women aged 25–49 across nine countries, using self-collected vaginal samples for hierarchical HPV evaluation: HPV16, else HPV18/45, else HPV31/33/35/52/58, else HPV39/51/56/59/68 else negative. HPV-positive individuals undergo further evaluation, including pelvic exams, cervical imaging, and biopsies. AVE algorithms analyze images, assigning risk scores for precancer, validated against histologic high-grade precancer. Phase 1, however, does not integrate AVE results into patient management, contrasting them with local standard care.

Phase 2 effectiveness focuses on deploying AVE software and HPV genotype data in real-time clinical decision-making, evaluating feasibility, acceptability, cost-effectiveness, and health communication of the PAVE strategy in practice.

Results:

Currently, sites have commenced fieldwork, and conclusive results are pending.

Conclusions:

The study aspires to validate a screen-triage-treat protocol utilizing innovative biomarkers to deliver an accurate, feasible, and cost-effective strategy for cervical cancer prevention in resource-limited areas. Should the study validate PAVE, its broader implementation could be recommended, potentially expanding cervical cancer prevention worldwide.

Funding:

The consortial sites are responsible for their own study costs. Research equipment and supplies, and the NCI-affiliated staff are funded by the National Cancer Institute Intramural Research Program including supplemental funding from the Cancer Cures Moonshot Initiative. No commercial support was obtained. Brian Befano was supported by NCI/ NIH under Grant T32CA09168.

eLife assessment

This important study will provide evidence about a novel screen-triage-treat strategy for cervical cancer prevention. The trial will generate convincing evidence regarding the efficacy, effectiveness, cost-effectiveness, feasibility and acceptability in a range of geographically spread low-resource settings. The strategy should contribute to improving access to cervical cancer prevention to vulnerable women with low access to health care, and, therefore, at the highest risk of cervical cancer.

https://doi.org/10.7554/eLife.91469.3.sa0

Introduction

Global burden of cervical cancer

Cervical cancer causes substantial morbidity and mortality worldwide, with approximately 600,000 incident cases and 340,000 deaths each year (Singh et al., 2023). Globally, cervical cancer is caused by persistent infection with one of ~13 carcinogenic human papillomavirus (HPV) types (Schiffman et al., 2007). Cervical cancer rates vary greatly worldwide due to uneven access to effective preventive measures; nearly 85% of cervical cancer cases and almost 90% of cervical cancer deaths occur in low- and middle-income countries (LMIC) (IARC, 2022; Ferlay et al., 2020). The World Health Organization (WHO) has called for the global elimination of cervical cancer, based on an advanced understanding of the natural history of the causal carcinogenic types of cervical HPV infection and the existence of effective preventive technologies, including prophylactic HPV vaccination and cervical screening (Schiffman et al., 2007; World Health Organization, 2021; WHO, 2020). However, translation of the HPV-based prevention methods has not yet occurred in many LMICs.

While prophylactic vaccination will eventually decrease cervical cancer rates (Lei et al., 2020) if high uptake can be achieved in LMIC, the maximum potential health benefits of vaccinating adolescents today will not be achieved for 40 years. However, the rapid implementation of a broad, effective cervical screening campaign for adult women in the highest-burden areas will advance cancer control by 20 years (Figure 1). The U.S. Cancer Moonshot initiative for Accelerated Control of Cervical Cancer has supported development of the new screening methods (Perkins et al., 2023).

Timing and deaths averted with one-time prevention campaigns: vaccination only, screening only, or both.

Projection of the relative timing of health benefits, measured as deaths averted, accrued by vaccination and/or screening applied through one-time campaigns. Three scenarios were examined: (1) a one-time screening campaign providing effective management for approximately 25% of 30- to 49-year-old women in 2027 (i.e. 20 birth cohorts) (green line); (2) vaccinating 90% of 9- to 14-year-old girls in 2027 (i.e. six birth cohorts) with a bivalent HPV16/18 vaccination (orange line); and (3) both a screening campaign and human papillomavirus (HPV) vaccination for respective birth cohorts in 2027 (blue line). We considered cervical cancer deaths averted over the lifetime of cohorts subject to the intervention, and conservatively assumed that deaths averted due to screening would only occur after age 50, to account for prevalent cancers. Projections were developed for the ~65 low- and middle-income countries (LMIC) with age-standardized cervical cancer incidence greater than 10 per 100,000 women (Perkins et al., 2023). For each country, we assumed that, in the absence of any intervention, the number of cervical cancer deaths for each 5 year age group would apply each year for the lifetime of the selected birth cohorts (Perkins et al., 2023). We conservatively and crudely assumed that screening and management would avert 25% of cervical cancer deaths (equivalent to screening uptake of 40% of eligible women, with 62.5% of screen-positive women receiving appropriate management) beginning at age 50 years. For vaccination cohorts, we assumed that a bivalent HPV16/18 vaccine (i.e. against the genotypes responsible for 70% of cervical cancers) with 90% uptake would avert 63% of cervical cancer deaths. While data on the costs of implementing novel screening strategies and single-dose HPV vaccination for female adolescents are forthcoming from the HPV-automated visual evaluation (PAVE) consortium and single-dose vaccination studies, we crudely assumed a single vaccine dose cost US$4.5031 with an average financial delivery cost per dose (i.e. per fully immunized girl) of US$7 (Akumbom et al., 2022). We assumed a bundled financial cost per woman screened of US$15, including a low-cost rapid HPV genotyping assay with triage and treatment of screen-positive women. According to our projections, the number of interventions needed to avert one cervical cancer death was similar for HPV vaccination and screening (i.e. 278 for HPV vaccination; 293 for screening). A one-time screening campaign for women aged 30–49 years in the selected countries yielded a financial cost of ~US$2.5 billion to avert ~570,000 deaths, or US$4,400 per death averted. On a similar order of magnitude, a one-time single-dose bivalent HPV vaccination campaign of girls aged 9–14 years in the same countries would cost ~US$2.0 billion and avert ~640,000 deaths, or US$3,200 per death averted. Of note, these ballpark estimates are undiscounted and do not account for cancer treatment cost offsets. We also did not consider demographic changes over the lifetime of intervention cohorts, nor did we consider the indirect benefits of vaccination or prevention of other HPV-related cancers.

Screening using HPV testing

The detection of carcinogenic cervical/vaginal HPV DNA is currently the most sensitive screening method to distinguish those with an appreciable risk of precancer or cancer from those at low risk (Demarco et al., 2022; Castle et al., 2012). The WHO currently recommends using either screen-treat or screen-triage-treat strategies. HPV testing is preferred to using visual inspection with acetic acid (VIA) as the primary screening method where resources permit (World Health Organization, 2021). There is a growing consensus that to achieve broad screening coverage, HPV testing of self-collected cervicovaginal specimens would be optimal for many populations (IARC, 2022; World Health Organization, 2021; Perkins et al., 2023; Arbyn et al., 2018). The results from meta-analyses comparing the performance of self-collected to clinician-collected samples, using PCR-based HPV detection, showed similar sensitivity and specificity for the detection of cervical precancer (Arbyn et al., 2018). As of 2022, seven LMICs were recommending HPV self-collection (Serrano et al., 2022).

There is broad scientific agreement that HPV testing is the preferred screening method due to its high negative predictive value and reproducibility. Treatment of all HPV-positive women with thermal ablation (i.e. screen-treat strategy) is an option under current WHO guidelines (World Health Organization, 2021); however, this may substantially overtreat infections, only the minority of which would progress to cancer (Demarco et al., 2020). To make the best use of limited resources and concentrate on those at the highest risk, triage strategies are critical to determine which HPV-positive women are at higher risk of cervical cancer. Triage with cytology or dual stain, as used in high-resource settings, is unlikely to be a feasible solution in the majority of low-resource settings. Cervical visual examination using visual techniques, including VIA, are often used as triage methods. However, these techniques are subject to human error, have low accuracy for precancer, and require continuous training and quality control measures (Sankaranarayanan et al., 2003; Catarino et al., 2018). HPV genotyping is a newer, more accurate method of triage, as genotype carcinogenicity varies predictably across populations (Sankaranarayanan et al., 2009; Qiao et al., 2008). HPV16 is the most carcinogenic, followed by HPV18/45, followed by HPV31/33/35/52/58, followed by HPV39/51/56/59/68 (Guan et al., 2012). AVE using an Artificial Intelligence (AI) algorithm shows promise as a relatively simple and fast triage method that could be used in conjunction with HPV genotyping to generate a highly accurate composite triage test (Desai et al., 2020; Desai et al., 2022b).

HPV-AVE (PAVE) strategy

The U.S. National Cancer Institute (NCI) is currently leading a consortium of a multi-centric study designed to evaluate a novel cervical screening and triage strategy for resource-limited settings, including settings with high HIV prevalence, as part of a global strategy to reduce cervical cancer burden. The PAVE strategy aims to target cervical precancer accurately and affordably by (1) self-sampled HPV screening; (2) triage among HPV-positive participants by combination of extended genotyping and visual evaluation assisted by deep-learning-based AVE; and (3) treatment using thermal ablation or excision Large Loop Excision of the Transformation Zone (LLETZ). PAVE utilizes the concept of risk-based management, defined as patient management determined by their risk of precancer/cancer to minimize overtreatment in low-risk patients and concentrate treatment resources on high-risk patients (Figure 2). This manuscript describes the study protocol, structure, and logic of the PAVE strategy.

Risk-based HPV-automated visual evaluation (PAVE) screen-triage-treat strategy provides risk stratification to assist in the management of screening participants.

hrHPV: refers to those human papillomavirus (HPV) types considered as having a high potential capacity to induce cervical cancer when the infection is persistent over time. It includes HPV16, 18, 45, 31, 33, 45, 52, 58, 39, 51, 56, 59, and 68.

Methods

The PAVE study has two phases: efficacy (2023–2024) and effectiveness (planned to begin in 2024 or 2025). The efficacy phase aims to refine and validate the screen-triage portion of the protocol. The effectiveness phase will research the introduction of the PAVE strategy into clinical practice.

Phase 1: Efficacy

Setting: Study design and locations

The study aims to recruit up to 100,000 women in nine countries: Brazil, Cambodia, Dominican Republic, El Salvador, Eswatini, Honduras, Malawi, Nigeria, and Tanzania (Figure 3 and Figure 4). Criteria for study site selection included: (a) existing screening programs, (b) willingness to research self-sampled HPV for screening, (c) capacity to run the HPV test (d) availability of pathology services to process biopsies, and (e) access to treatment services including ablation, excision, and cancer treatment. Outreach and recruitment activities under the protocol include awareness campaigns to inform the eligible female population in the catchment area. Research protocol details including recruitment strategies, number of visits, and institutional review board approval are under the control of individual study sites. The PAVE project is integrated into the screening activities at all study sites, and at select sites is also integrated into other ongoing research studies.

Map of HPV-automated visual evaluation (PAVE) study sites.
Schematic of HPV-automated visual evaluation (PAVE) protocol elements.

Ethical and regulatory aspects

This multi-centric study is designed to function as a consortium. All ethical oversight of recruitment and clinical data collection will be done by the local sites under the guidance of their own Institutional Review Boards and will follow local guidelines. All participants will have an informed consent for participation in the study and can drop their participation at any time during the process. All study documents written in languages other than English are officially translated into English for study records. Compiled analysis of de-identified data by NCI research staff for study purposes is deemed non-human subjects research by NIH.

Protocol overview (Date of protocol latest review: September 24, 2023).

In-country elements: Patient enrollment and data collection

The steps of the PAVE protocol include (1) determination of study eligibility, (2) informed consent, (3) self-sampled HPV testing, (4) cervical image collection and biopsy collection for those testing HPV-positive, and (5) treatment as indicated per local protocols (Figure 4).

Enrollment

Eligibility criteria are: individuals with a cervix aged 30–49 years (general population) or 25–49 years if living with HIV (WLHIV), not currently pregnant, and able to understand study risks, benefits, and alternatives and provide informed consent in their native language. Those eligible for and interested in study participation undergo informed consent per local protocols. Those who choose to enroll provide basic demographic information (age, parity, HIV status if known). Pregnant women, or those with no cervix or having been treated for a cervical cancer are excluded.

At each study site, we anticipate ~1–2% prevalence of histopathologic precancers (HPV-positive adjudicated CIN2+/AIS), yielding about 100–200 precancers and a very few cancers out of a sample of 10,000 women. Combined across all countries, we anticipate up to 1000–2000 precancers.

HPV self-collection

Participants self-collect a vaginal sample for HPV testing using a FLOQSwab (Copan) following instruction by study personnel. Self-samples (FLOQSwab) are delivered dry for testing. All sites intend to use the ScreenFire HPV risk-stratification (RS) assay (ScreenFire) (U.S. patent 11091799, Atila Biosystems Inc, Sunnyvale, CA, U.S.) (Desai et al., 2022a; Inturrisi et al., 2024). However, other HPV tests may be acceptable alternatives if they can provide genotyping information in the following groups: HPV16, HPV18/45, HPV31/33/35/52/58, and HPV39/51/56/59/68.

HPV tests are run onsite or in local laboratories in a few days, with results returned to women quickly per local protocols. Women screening HPV-negative are informed of their results, reassured about their low subsequent risk of cervical cancer, and their participation in the study ends at most sites. The exception is El Salvador, at which 5% of those screening HPV-negative undergo colposcopic examination. On average, approximately 80% of participants will screen HPV-negative, but this varies by study population.

Triage of HPV-positive results: Image collection and biopsy

Women with HPV-positive results undergo a speculum exam with an application of 5% acetic acid. Cervical images will be collected by a trained study provider using a dedicated device (Iris, Liger Medical LLC, Lehi, UT, U.S.). Local clinicians also record their VIA assessment (negative, positive, suspicion of cancer) or colposcopy impression (normal, low-grade, high-grade, or more severe).

Following image capture, pathology specimens are collected. Biopsies will be collected from up to four acetowhite areas for each participant. If no acetowhite areas are seen, then an endocervical sample (using curette or brush) or cytology will be collected. Sites using colposcopy will collect punch biopsies per standard practice. Sites using VIA will use Softbiopsy/SoftECC brush biopsy (Histologics LLC, Anaheim, CA, U.S.), a device that is simpler to learn and perform and is associated with lower bleeding risk. All women that have an HPV-positive test are expected to have a histologic diagnosis (biopsy, endocervical sample (ECC), and/or excisional tissue diagnosis). HPV-positive women with a negative triage evaluation initially, but who are later identified by PAVE activities to have a precancer defined for study purposes as HPV-positive CIN2, CIN3 or AIS CIN2+, will be flagged and the clinical sites will be notified to permit ‘safety net’ recall for adequate management.

Treatment

Treatment is provided for women meeting criteria per local protocols: VIA-positive or suspicion of cancer in sites using VIA, CIN2 + on biopsy and/or high-grade colposcopy impression in sites using colposcopy/biopsy, or HPV-positive and acetowhite changes or HPV16, 18/45 positive or HPV-positive for sites using screen-treat protocols. For sites using VIA, treatment decisions for those screening VIA-positive will be based on the standard of care, most commonly an adaptation of the WHO visual assessment for treatment (VAT) criteria for the use of ablation. For sites using LLETZ, treatment decisions will follow local protocols. Table 1 describes screening, triage, biopsy, and treatment protocols for each site.

Table 1
Site-specific primary triage and treatment protocols Biopsy and treatment protocols.
SitePrimary Screening TestTriage methodStaff taking biopsiesBiopsy InstrumentTreatment thresholdPrimary Treatment
Dominican RepublicScreenFire HPV, CytologyColposcopyGynecologistsBiopsy forcepsCIN2+ +biopsyAblation or LLETZ
MalawiScreenFire HPVVIANursesSoftbrushVIA-positiveAblation or LLETZ
NigeriaScreenFire HPVColposcopyGeneral Doctors,
Gynecologists,
GYN oncologists
Biopsy forcepsHPV-positiveAblation or LLETZ
BrazilScreenFire HPVConventional cytology, liquid-based cytology, and colposcopyGynecologistsBiopsy forcepsCIN2 + biopsy or high-grade colpo impressionLLETZ
CambodiaScreenFire HPVColposcopy (mobile colposcope)Nurse midwives, General Doctors,
Gynecologists
SoftbrushVIA-positive or
HPV16,18/45 positive
Ablation or LLETZ
EswatiniScreenFire HPV & VIAVIANurse midwivesSoftbrushVIA-positiveAblation
El SalvadorScreenFire HPV, Cytology, CareHPVColposcopy and VATGeneral DoctorsBiopsy forcepsHPV-positiveAblation
TanzaniaScreenFire HPVVIAGynecologists, nurses, nurse midwivesBiopsy forceps, SoftbrushVIA-positiveAblation
HondurasScreenFire HPVColposcopy and VATNurse midwives, General Doctors,Biopsy forcepsVIA-positiveAblation

Note to Table 1: Prior to the PAVE study, El Salvador screened with primary HPV screening (clinician-collected CareHPV), Brazil and DR screened with cytology, and Cambodia, Eswatini, Honduras, Malawi, Nigeria, and Tanzania screened with VIA. All sites are introducing self-sampled HPV testing with ScreenFire as part of the PAVE protocol. In El Salvador, women are continuing to screen initially with both ScreenFire and CareHPV. Triage testing is performed in all women with HPV-positive results. In El Salvador, triage testing is also performed on 5% of those testing HPV-negative.

Central elements: Data management, HPV testing, AVE algorithm development, quality assurance, statistical analysis

Data management

Study data including demographic survey information, HPV test results (negative/positive, genotype for positive results), VIA or colposcopy impression, and local pathology results (cytology, biopsy, and/or excisional specimen) are collected using District Health Information System 2 (DHIS2), Redcap or WEMA platforms. The collected data are associated with the corresponding images obtained during the triage visits. To ensure confidentiality, all personal identification information is removed from datasets before transferring outside the country of origin. De-identified datasets are securely transferred to a common server handled by the NGO specialized in country adaptation of DHIS2 information systems Enlace Hispano Americano de Salud (EHAS) and from there, compiled data are transferred to Information Management Services (IMS), the NCI data management contractor, for data storage and analytic support during the course of the study. The noteworthy element in this study design element is that data rights (and residual biospecimens) will remain with the study site partners within their countries. The data on loan for PAVE analyses will be stored securely and data can be withdrawn and destroyed at any time by study site partners. This arrangement is important to many aspects of international data and biospecimen sharing.

Figure 5 illustrates the various steps involved in the PAVE consortium, from data collection to data analysis. This outline is for those sites utilizing the DHIS2 app, where pre-defined data is collected from women who agree to participate. It is important to note that any individual information collected for clinical management purposes will not be uploaded to the study server.

Flowchart of data sharing in the HPV-automated visual evaluation (PAVE) study sites using the DHIS2 during the efficacy phase.

The study server incorporates the ITEA app, which facilitates the transfer of data to the NGO EHAS. At EHAS, the data undergoes a thorough check to ensure appropriate study identification. Additionally, if necessary, cervical images of the participants are linked to the core database. Once the verification process is approved, the data is sent to the National Cancer Institute- Information Management System (NCI-IMS) for preparation before analysis.

During this step, data verification and cleaning procedures are performed to ensure the accuracy and quality of the data. Subsequently, the images, as well as the HPV and histology assessments (when available), are forwarded to Prof Jayashree Kalpathy-Cramer’s team for AI evaluation. The results, in the form of data scores, are then transmitted back to NCI-IMS for comprehensive analysis of the consortium data.

Note: Sites using other apps like RedCap will extract the Core Data, that includes the information listed at the beginning of this section from their files, and forward it to EHAS. Images will follow the same procedure.

HPV testing/typing

The ScreenFire HPV test is a new assay designed to detect the 13 high-risk HPV (hrHPV) genotypes grouped into the four risk groups described above, and specifically engineered to provide risk stratification by HPV genotype based on carcinogenicity. ScreenFire includes an internal control for sample quality guidance. The ScreenFire HPV assay is an isothermal, multiplex nucleic acid amplification method that uses 3NT technology to reduce false positivity and increase assay performance. ScreenFire can be run on 1–96 samples per batch, requires basic pipetting skills, and takes approximately 2.5–3 hr in total including sample preparation, pipetting, and DNA amplification with readout.

Validation

ScreenFire was compared against reference research HPV DNA assays in 2078 stored specimens. Overall concordance for both viral types was >90%, and sensitivity for CIN3 + was 94.2%, similar to Linear Array (93.1%) and TypeSeq (95.9%), indicating excellent performance (Inturrisi et al., 2024). Regulatory approval will require additional comparisons in screening settings. Additional clinical studies will be nested within the PAVE protocol. In El Salvador, 5% of HPV-negative women will undergo colposcopic evaluation. Comparison of ScreenFire to other WHO pre-qualified HPV tests (CareHPV and Abbott) may also be performed on subsets of patients.

Development of AVE algorithm using clinical images

Cervical images will be transferred via the digital camera to a secure server using a specially designed script. Images and non-PHI data will be shared and downloaded on a loan basis to the PAVE AI team at NCI and the AI collaborators to train the AVE algorithm. Because images from the Iris device have not been used previously with the AI algorithm, a pilot phase will take place to retrain the AI-based algorithm. Due to the similarities of the Iris device to previously tested digital cameras, we expect to be able to retrain the algorithm successfully as done in our prior work (Parham et al., 2023).

During the PAVE study, four AI algorithms are being developed and evaluated: (1) cervix detector, (2) image quality classifier, (3) disease classifier to identify precancer, and (4) treatability/SCJ classifier.

  1. Cervix detector. This algorithm is designed to display a bounding box on the screen, aiding healthcare providers ensure that the cervix is centralized within the image. This feature simplifies the process of locating the cervix within the digital image, enhancing the efficiency and accuracy of image collection.

  2. Image quality classifier. This algorithm aims to identify images that may be unsuitable for accurate disease assessment due to factors like obstruction or inadequate visual sharpness. By flagging such images, it helps ensure that only good-quality images are used when training the algorithm. If shown to be useful, the image quality classifier could provide feedback in real-time to clinicians when taking images to ensure adequate image quality.

  3. Disease classifier. This algorithm aims to visually distinguish precancerous changes from lesser abnormalities, and classifies cervical images as normal, indeterminate, or precancer+. AVE results are assessed on repeatability (correct classification of replicate images of the same patient) and accuracy (correct classification of AVE based on histopathology, as well as minimization of extreme misclassification of normal as precancer or vice versa). Accuracy is defined as a correct classification of participants to <precancer or precancer+. Precancer + is rigorously defined to include HPV-positive histologic CIN3, AIS, cancer, and CIN2 diagnoses confirmed by expert pathologic review and positive for the eight most carcinogenic HPV genotypes. CIN2 is qualified because although CIN2 is the threshold for treatment in most clinical practices worldwide, it is a less reproducible pathologic diagnosis and may regress without treatment, especially when associated with lower carcinogenicity HPV genotypes (Lee et al., 2018; Kylebäck et al., 2022).

    The current prototype algorithm is the result of several years of development. Earlier algorithms were limited by poor repeatability, misclassification including grave errors (i.e. cases with precancer called normal or vice versa), overfitting, and lack of portability (defined as the ability of the algorithm to accurately classify images from different image capture devices and study settings other than the dataset on which it was trained) (Desai et al., 2022b). Additional techniques have been applied to develop the prototype version of the AVE algorithm that will be refined and tested in the PAVE study (Ahmed et al., 2023a), resulting in improved reliability and consistency of model predictions across repeat images from the same woman (Lemay et al., 2022). The disease classifier algorithm is trained using histology as the truth standard for defining the presence or absence of precancer. The outcome definition for the purpose of training a three-class ordinal classification algorithm includes normal (HPV-positive with histology normal, HPV-negative, HPV-negative cervicitis), indeterminate (low-grade HPV-related abnormalities, CIN1), and precancer+, as determined by histology among HPV-positives (defined above). To ensure portability, the algorithm will undergo external validation using datasets distinct from the training set of images. Early data indicate that while our algorithm may function among patients across diverse geographies (Ahmed et al., 2023b) a dedicated device may be needed because AVE fails to transfer without retraining (Parham et al., 2023). The images collected in PAVE will be used to refine and externally validate the prototype AVE algorithm (Lemay et al., 2022). We test the repeatability, accuracy, and calibration of the model before the algorithm is tested in clinical settings during the effectiveness phase (Egemen et al., 2023).

  4. Treatability/SCJ classifier. This algorithm is being developed to classify the SCJ as fully visible, partially visible, or not visible, using the expert colposcopic impression as the truth standard. The goal is to assist providers in determining treatment eligibility. SCJ visibility might be critical (i.e. necessary although not sufficient) for eligibility for thermal ablation procedures.

Pathology quality assurance

Pathology readings are performed locally with centralized quality assurance on a subset of cases. Histotechnology adequacy via slide review from all participating laboratories includes assessment of specimen preparation, staining adequacy, and clarity/readability of scanned images by the assigned referent NCI study pathologist in collaboration with pathologists at each study site. Local pathologists involved in the PAVE project are asked to complete a performance competency review which includes providing diagnoses on 20 standardized cases. Issues with either slide preparation or interpretation were addressed via videoconference between the NCI expert pathologist and local laboratories.

To assure histopathology reading standardization, the following cases will be reviewed by an expert gynecologic pathologist, making use of a Motic whole slide scan review collection and transfer: (1) all cases with histology CIN2 + or high-grade squamous intraepithelial lesion (HSIL); (2) all HPV16 + with <CIN2 pathology; (3) all cases read as precancer + on AVE; and (4) 5% of biopsies read as normal. Results will be classified for the study purposes as normal, low-grade (<CIN2), high-grade (CIN2, CIN3), adenocarcinoma in situ (AIS), or invasive cervical cancer (squamous, adenosquamous, adenocarcinoma, or other).

Statistical analysis

The primary objective of the PAVE protocol is to compare the sensitivity at a given specificity level of the PAVE approach for triaging HPV-positive women to current SOC (HPV testing without genotyping triaged by VIA or colposcopic impression) (Figure 6). The PAVE screen-triage-treat protocol combines the three-class classification label (normal, indeterminate, precancer+) provided by the AVE algorithm with the four hrHPV risk group strata to create 12 strata of risk of precancer (Table 2).

Theoretical approach to compare the PAVE strategy (which is HPV and AVE combined strategy. in red) and the standard of care (SOC) screening and triage outcome (which commonly is VIA, in blue).

This figure represents a hypothetical example showing how testing for HPV and using AVE as triage (PAVE) and SOC will be compared. Under a specific specificity value (which will be determined by the SOC), we will compare the difference between the sensitivities of PAVE and SOC. In this example, we had three categories for human papillomavirus (HPV) genotype groups and three categories for automated visual evaluation (AVE) (normal-indeterminate-precancer/cancer), and in total nine PAVE categories. We estimate that about half of the CIN3 + cases will be positive for HPV16, and about 10–15% will be HPV18/45 positive, and that the remaining two other channels will have a 20% prevalence within cases.

Table 2
HPV-AVE risk strata.
AVE Risk Classification
HPV risk groupPrecancer+IndeterminateNormal
HPV16HighestHighHigh
HPV18/45HighHighHigh
HPV31/33/35/52/58HighMediumMedium
HPV39/51/56/59/68HighMediumLow
NegativeLowest

The PAVE protocol will be compared to SOC VIA or colposcopy screen-triage protocols using the visual impressions recorded during the study. In SOC scenarios, participants are classified as positive or negative in the HPV test and as normal or abnormal in visual evaluations (e.g. VIA negative or positive, colposcopic impression less than high-grade or high-grade+) (Table 3).

Table 3
Risk strata for participants with an HPV-positive women and visual Standard of Care (SOC) as the triage (i.e. VIA, colposcopy) test.
SOC classification*
HPV test resultPositive/High gradeNormal
PositiveHighLow
NegativeLowest
  1. Note: Participants with a negative test for HPV will not have a VIA nor colposcopy assessment. see Table 1 for SOC at individual PAVE sites.

The strata from Table 2 will be plotted as a ROC-like curve of sensitivity versus (1-specificity). The ROC curve for each study site will yield an area under the curve (AUC) for both the PAVE and the SOC strategies.

The 12-stratum AVE risk score will be compared to a visual assessment of SOC: VIA (negative or positive) or colposcopic impression (less than high grade vs. high grade+) (Egemen et al., 2023). At each site, we will compare the sensitivity of the two approaches, at the (1-specificity) value produced by SOC visual evaluation. We will compile these values for the consortium and calculate the average (weighted by study size). We hypothesize that the difference of two sensitivities (PAVE minus SOC) conducted on the overall consortium data will be significantly greater than zero (the null hypothesis of no difference in sensitivity), indicating that PAVE detects more precancer than SOC triage at the same level of specificity. Where available, the analysis will be stratified by HIV status. Throughout the study, we will be checking for the consistency of results at the different steps across sites (e.g. performance of HPV results, quality of the images, histopathology reports). We will assess the reproducibility of the PAVE strategy across different sites by measuring the variability of the AUC values.

The list of variables collected in addition to HPV results and images is available on request.

Additional analyses that may be performed during the efficacy phase include:

  • Development of treatability algorithm: In addition to defining SCJ visibility, an AI algorithm to assess whether a lesion fulfills WHO ablation criteria is in development. If AI algorithm development is successful, the output will be compared to the VIA or colposcopy assessment of eligibility for ablation or referral for surgical management. Accuracy of the three-class classification label provided by the AVE treatability algorithm (treatable, uncertain, not treatable) will be compared to the against a truth label based on the evaluation of three experts.

  • Impact of HIV status on PAVE algorithm: We will evaluate the impact of HIV on the PAVE performance for the accuracy in the detection of precancer+ by comparing the accuracy in women with and without HIV, with some consideration of the role of effective ART (Parham et al., 2023).

Phase 2: Effectiveness

During the efficacy phase, the PAVE algorithm is undergoing evaluation and development, and clinicians will not be provided with HPV genotyping, AI algorithm outputs, or risk strata. When the efficacy phase is complete, if the PAVE algorithm outperforms the SOC, we will begin the effectiveness phase. Sites that chose to participate in the effectiveness phase will test the following screen-treat-triage protocol. Screening: self-sampled HPV testing with genotyping (ScreenFire or equivalent test). Triage of HPV-positive individuals: image collection using the Iris device, upon which the AI algorithms have been installed. The AI algorithms will guide the clinician in taking high-quality images (cervix identifier, image quality classifier) and then provide a disease classification score of normal, indeterminate, or precancer+ and an SCJ visibility assessment of fully visible, partially visible, or not visible. The clinician will then enter the HPV genotyping test result and the device will output a risk category using the strata in Table 2 (lowest, low, medium, high, highest). The risk category and SCJ visibility assessment are designed as clinical management tools to aid clinicians in determining which patients are most likely to benefit from treatment, and among those needing treatment, whether ablation can be considered. This phase will assess the feasibility and acceptability of the PAVE strategy in clinical practice.

Cost-effectiveness analysis

To inform decision makers designing cervical cancer prevention programs in resource-limited settings, we will analyze the cost-effectiveness (i.e. cost per precancer detected) and affordability (the impact on a payer’s budget) of the PAVE strategy at several sites. Micro-costing efforts to estimate the cost of all resource ingredients used for a screening episode are underway with technical assistance provided to study sites. A microsimulation model of genotype-specific HPV natural history and cervical carcinogenesis is being developed specifically for evaluation of novel biomarker triage tests, including AVE (Campos et al., 2021). By adapting this natural history model to setting-specific HPV prevalence patterns (by genotype and age); overlaying screening, triage, and treatment strategies; and using setting-specific healthcare delivery data inputs for uptake, adherence to management, and costs, we will evaluate the cost per precancer detected by the PAVE strategy relative to SOC. We will explore the health and economic implications of applying different management approaches (e.g. triage, treatment, and follow-up) to different risk strata, depending on health system capacity.

The development of the microsimulation model and the micro-costing tools for the PAVE consortium will serve as the basis for estimating the real-world costs and health benefits of implementing novel screening and management strategies. These tools can be adapted to different settings, with refinement of management algorithms, health care delivery variables, and cost estimates as implementation and scale-up occur. An early exercise to approximate the potential costs and benefits of a highly effective screening campaign delivered to women aged 30–49 years in the ~65 highest burden LMIC (Ferlay et al., 2020; GAVI, 2023; Akumbom et al., 2022; World Bank, 2023; Figure 1) and an HPV vaccination program delivered to girls ages 9–14 years found that the number of screening or adolescent HPV vaccinations needed to avert one cervical cancer death was similar for each intervention (i.e. 293 for screening; 278 for vaccination). Assuming a bundled cost of US$15 per woman screened and managed appropriately, a one-time screening campaign that achieves 40% uptake and ~60% adherence to recommended treatment for screen-positive women yielded a financial cost of ~US$2.5 billion to avert ~570,000 deaths, or US$4400 per death averted. On a similar order of magnitude, a one-time single-dose bivalent HPV vaccination campaign achieving 90% coverage of girls aged 9–14 years in the same countries (US$4.50 vaccine cost; US$7 delivery cost) would cost ~US$2.0 billion and avert ~640,000 deaths, or US$3200 per death averted. Of note, these ballpark estimates are undiscounted and do not account for cancer treatment cost offsets. While data are not yet available on the costs of implementing novel highly effective screening strategies for adult women and single-dose HPV vaccination for female adolescents, these data are forthcoming from the PAVE consortium and single-dose vaccination studies. Refining these cost and effectiveness estimates, and obtaining country-specific data, is a high priority and a critical component of the PAVE consortium objectives.

Stakeholder knowledge and attitudes regarding cervical cancer prevention and screening interventions in the PAVE study

Effective dissemination and implementation of the PAVE strategy in the future will require clear and consistent strategies for communicating information to healthcare providers and patients. The field of HPV and screening is rapidly evolving and constantly being enriched with new scientific information. However, this influx of information can sometimes lead to unclear or conflicting messages, which in turn may diminish the effectiveness of interventions aimed at improving screening rates and optimizing management strategies. Furthermore, both healthcare providers and patients can differ in their knowledge and perceptions of cervical cancer risk, tolerance of risk, and their personal values and attitudes regarding cervical cancer prevention and screening, all of which can influence the uptake of prevention strategies such as the PAVE strategy. To address these challenges, it will be necessary to provide healthcare providers with accurate, consistent information about the latest scientific advances and guidelines, and as well as training and tools that can help them effectively inform and engage patients in cervical cancer prevention and screening.

To address these needs and prepare for broader dissemination and implementation of the PAVE strategy, the Communication and Retention Workgroup is conducting a mixed-methods pilot study utilizing both qualitative interviews and survey questionnaires administered to scientific experts as well as key stakeholders—patients and healthcare providers—at four participating sites (Brazil, El Salvador, Nigeria, Tanzania). The specific objectives of this pilot study are to explore stakeholders’ knowledge, perceptions, and attitudes regarding cervical cancer prevention and screening, and their preferences for information and participation in decision-making. The study will generate evidence to enable the future development of effective, ethical strategies for engaging eligible members of LMIC communities in cervical cancer prevention and screening, including using the PAVE strategy.

Discussion

Timeline and next steps

At the time of this writing, the study has been initiated in nine countries. It is expected that a preliminary interim analysis on the efficacy of the PAVE strategy will be completed by the early 2024, and field recruitment will be completed by end of 2024. The efficacy phase is designed to assess the validity of the PAVE protocol by refining the AVE protocol using histopathology specimens and demonstrate the superiority of PAVE for detecting precancer and minimizing unnecessary referrals compared to the SOC. Following the initial efficacy phase, the effectiveness phase is planned to examine additional factors including program feasibility, acceptability, and cost-effectiveness.

In the Figure 7, a schematic timeline of the project from proof of principle to the estimated final efficacy and effectiveness steps.

Estimated timeline of HPV-automated visual evaluation (PAVE) validation.

Regulatory considerations

The rapid advancement of AI in healthcare has prompted significant ethical and regulatory discussions. Global ethical principles specific to AI in healthcare are in development and are rapidly evolving, regulatory authorities such as the WHO and the U.S. Food and Drug Administration (FDA) are responsible for developing guidelines that ensure the safe, effective, and appropriate use of AI technologies in healthcare and therapeutic development. In addition to complying with existing ethical principles in medicine, AI solutions must demonstrate scientific validity, ensuring their effectiveness and reliability. It is imperative that AI technologies do not perpetuate discrimination or bias, or exclude specific population segments. One major concern is the potential creation of a permanent digital identity, linked to individuals' health and personal data, without obtaining proper consent (WHO, 2021, FDA, 2023).

The Iris device, used in PAVE, is an example of an AI-based system in healthcare. It will incorporate AI algorithms that aim to generate scores evaluating image quality, presence of precancerous lesions, and visibility of the SCJ based on digital cervix images. These scores provide additional information to assist healthcare providers in making informed decisions. To arrive at a comprehensive decision, the provider will need to integrate the HPV information, gynecological examination findings, scoring system outputs generated from the AI algorithms, and the patient’s medical history. Considering its functionality, the AI algorithm used in the PAVE system, based on FDA specifications, can be considered an assistant to medical management offering the joint evaluation of the HPV data and the AVE outcome. As WHO is actively evaluating screening and triage approaches, it is expected that clearer regulatory guidance on the use of the algorithms will be available before their implementation (WHO, 2021; FDA, 2023).

Competition and commercial aspects

The PAVE strategy utilizes certain products of significant commercial importance. The selection of each device or assay is based on factors such as accuracy to achieve its purpose (i.e. high sensitivity of the ScreenFire HPV test to detect precancer, high-quality image capture by the Iris device) and affordability. It is important to note that NCI PAVE collaborators have no commercial interests that could create appearance of conflict of interest. But for collaborators does not preclude future commercial developments. The presence of competition with similar assays is desirable as long as they can provide similar qualities by commertial partners. If the PAVE strategy is proven to be more accurate than the SOC for screening and triage, there will likely be an increased demand for devices and assays. Procurement of these resources may present challenges and require effective communication with the respective Ministries of Health to ensure a smooth introduction in countries where the need is evident.

Dissemination

After the demonstration of efficacy and effectiveness, we expect to have a screen-treat-triage protocol fulfilling WHO principles that can be used at the discretion of local health authorities in LMIC for cervical cancer prevention programs. If the PAVE strategy is proven to outperform the local SOC, and be feasible, acceptable, and affordable for resource-limited settings, then countries may switch their current SOC to the PAVE strategy. De-implementation of existing strategies, such as cytology, colposcopy, and VIA, as well as implementation of self-sampled HPV with AVE triage will require buy-in from stakeholders and policymakers as well as substantial investment in educating and retraining the laboratory and clinical workforces. Cytology/colposcopy programs, though effective, are limited in scope and are costly to maintain, therefore, switching may be attractive to health ministers. However, laboratories will need funding for the purchase of equipment for running HPV testing and materials for self-sampling, and the cytotechnologist and pathologist workforce will be reduced. Countries with existing VIA programs will require significant introduction costs such as laboratory machinery and training of healthcare personnel, and recurrent costs including reagents and self-sampling materials. In all settings, program continuation beyond the initial study will require local governments and programs to address issues of procurement and implementation, as well as de-implementation of existing strategies.

In conclusion, the PAVE project will develop and validate a strategy using self-sampled HPV with genotyping and AVE to identify precancer in a large group of women from many different settings. The PAVE objective is to create an accurate, feasible, cost-effective screening and triage protocol for cervical cancer prevention in resource-limited settings. If proven effective, cost-effective, feasible, and acceptable, the strategy can have a major impact in reducing cervical cancer particularly among non-vaccinated adult women.

Appendix 1

PAVE Study Group

Brazil

Ana Ribeiro - ana-ribeiro.dantas@fiocruz.br, Tainá Raiol - taina.raiol@fiocruz.br, Center for Women’s Integrated Health, Oswaldo Cruz Foundation (Fiocruz), Brasília, DF, Brazil. MARCO Clinical and Molecular Research Center, University Hospital of Brasília/EBSERH, Federal District, Brazil

Cambodia

Te Vantha, MD, Director of Takeo Provincial Hospital, Cambodia. Thay Sovannara, MD, Medical Practitioner, Raffles Medical Group, Cambodia. Judith Norman, MD, Director of Women's Health, Mercy Medical Center, Cambodia, judynorman@gmail.com. Dr. Andrew T. Goldstein, Director, Gynecologic Cancers Research Foundation drg.cvvd@gmail.com.

Dominican Republic

Margaret M. Madeleine, MPH, PhD, Program in Epidemiology, Fred Hutchinson Cancer Centermmadelei@fredhutch.org. Yeycy Donastorg, MD, Instituto Dermatológico y Cirugía de la Piel "Dr. Huberto Bogaert Díaz", HIV Vaccine Trials Research Unit, Santo Domingo, Dominican Republic. ydonastorg@gmail.com

El Salvador

Miriam Cremer MD; Basic Health International, Pittsburgh, PA 15205, USA. Ob/Gyn and Women's Health Institute, Cleveland Clinic, Cleveland, OH 44195, USA miriam.cremer@gmail.com. Karla Alfaro, MD Basic Health International, El Salvador, kalfaro@basichealth.org

Honduras

Miriam Cremer MD; Basic Health International, Pittsburgh, PA 15205, USA. Ob/Gyn and Women's Health Institute, Cleveland Clinic, Cleveland, OH 44195, USA, miriam.cremer@gmail.com. Karla Alfaro, MD Basic Health International, El Salvador, kalfaro@basichealth.org. Jaqueline Figueroa, MD, Programa Nacional contra el Cáncer, Tegucigalpa, Honduras. jacqueline_figueroan@yahoo.com

Eswatini

Eyrun F. Kjetland, MD, PhD, Professor, Departments of Global Health and Infectious Diseases Ullevaal, Centre for imported and Tropical Diseases, Oslo University Hospital Ullevaal, Oslo, Norway; College of Health Sciences, Discipline of Public Health, Nelson Mandela School of Medicine, University of KwaZulu-Natal, Durban, South Africa; Centre for Bilharzia and Tropical Health Research (non-profit), BRIGHT Academy, Durban, South Africa e.f.kjetland@medisin.uio.no, Teresa Norris, Founder and President, HPV Global Action, tnorris@hpvglobalaction.org. Zeev Rosberger, PhD, Department of Oncology, Psychology and Psychiatry, McGill University, Montreal, Canada, zeev.rosberger@mcgill.ca. Amelie McFadyen, MA, Chief Executive Officer, HPV Global Action, ameliemcfadyen@hpvglobalaction.org. Marc Steben, MD, Ecole de Sante Publique, Université de Montréal; International society for STD research, marc@marcsteben.com

Malawi

Amna Haider, MD, Epidemiologist, Department of Epidemiology and Training, Epicentre, Dubai, UAE, amna.haider@epicentre.msf.org. George Kassim Chilinda, MD, Médecins Sans Frontières, Operational Centre Paris, Blantyre, Malawi, gchilinda@gmail.com. Henry B.K.Phiri, MD- Sexual and reproductive health department, Ministry of Health, Malawi, henryphiri06@gmail.com

Nigeria

Ajenifuja Kayode Olusegun, MD, Obafemi Awolowo University Teaching Hospital, Ile-Ife, Osun state Nigeria, ajenifujako@yahoo.com. Adepiti Clement Akinfolarin, MD, Obafemi Awolowo University Teaching Hospital, Ile-Ife, Osun state Nigeria, akinfolarindepiti@yahoo.co.uk. Adekunbiola Banjo, MD, College of Medicine University of Lagos, Lagos, aafbanjo@cmul.edu.ng. Moharson-Bello Imran, MD, College of Medicine, University of Ibadan, Oyo state, Nigeria, imranmorhasonbello@gmail.com. Oyinloye Temitope, MD, Obafemi Awolowo University Teaching Hospitals Complex, Ile-Ife, Osun state, Nigeria, projectcoordinator.itoju@gmail.com. Bola-Oyebamiji Sekinat, MD, College of Medicine, Osun state University, Osogbo, Osun state. Adeyemo Marydiya, MD, College of Medicine, Osun state University, Osogbo, Osun state

Tanzania

Karen Yeates-MD, MPH, Department of Medicine, Queen’s University, Kingston, Ontario, Canada, yeatesk@queensu.ca. Safina Yuma, MD, Cervical Cancer Focal Person, Ministry of Health, Tanzania, sychande@yahoo.com. Bariki Mchome, MD, Head, Reproductive Health Centre, Kilimanjaro Christian Medical Centre, Kilimanjaro, Tanzania, barikimchome@gmail.com. Alex Mremi, MD, Head, Department of Pathology, Kilimanjaro Christian Medical Centre, Kilimanjaro, Tanzania, alexmremi@gmail.com.

Data availability

In the Protocol phase of the PAVE study we do not report neither previously generated nor newly generated datasets. There is no code/software/algorithm available to be shared.

References

  1. Book
    1. World Health Organization
    (2021)
    WHO Guideline for Screening and Treatment of Cervical Pre-Cancer Lesions for Cervical Cancer Prevention
    Geneva: World Health Organization.

Peer review

Reviewer #1 (Public Review):

Summary: A description of a modern protocol for cervical screening that likely could be used in any country of the world, based on self-sampling, extended HPV genotypinng and AI-assisted visual inspection - which is probably the best available combination today.

Strengths: Modern, optimised protocol, designed for global use. Innovative.

Weaknesses: The protocol is not clear. I could not even find how many women were going to be enrolled, the timelines of the study, the statistical methods ("comparing" is not statistics) or the power calculations.

Tables 2 and 3 are too schematic - surely the authors must have an approximate idea of what the actual numbers are behind the green, red and yellow colors.

Figure 1 comparing screening and vaccination is somewhat misleading. They screen 20 birth cohorts but vaccinate only 5 birth cohorts. Furthermore, the theoretical gains of screening has not really been attained in any country in practise. Modelling can be a difficult task and the commentary does not provide any detail on how to evaluate what was done. It just seems unnecessary to attack vaccination as a motivation on why screening needs to be modernised.

https://doi.org/10.7554/eLife.91469.3.sa1

Reviewer #2 (Public Review):

Summary:

This manuscript describes the study protocol, structure and logic of the PAVE strategy. The PAVE study is a multicentric study to evaluate a novel cervical screen-triage-treat strategy for resource-limited settings as part of a global strategy to reduce cervical cancer burden. The PAVE strategy involves: (1) screening with self-sampled HPV testing; (2) triage of HPV-positive participants with a combination of extended genotyping and visual evaluation of the cervix assisted by deep-learning-based automated visual evaluation (AVE); and (3) treatment with thermal ablation or excision (Large Loop Excision of the Transformation Zone). The PAVE study has two phases: efficacy (2023-2024) and effectiveness (planned to begin in 2024-2025). The efficacy phase aims to refine and validate the screen-triage portion of the protocol. The effectiveness phase will examine few implementation of the PAVE strategy into clinical practice. In following phases implementation will further explored.

Strengths and weaknesses

The Pave Study develops and evaluates a novel strategy that combines HPV self-collection -that has been proven effective to increase screening coverage in different settings-, with genotyping and Automated Visual Evaluation as triage. The proposed strategy combined three key innovations to improve an important step in the cervical cancer care continuum. If the strategy is effective it will contribute to enhance cervical cancer prevention in low resource settings.

As authors mentioned, despite the existence of effective preventive technologies (e.g., HPV vaccine and HPV test) translation of the HPV prevention methods has not yet occurred in many Low-Middle-Income Countries. So, in this context, new screen-triage-treat strategies are needed and if PAVE strategy were effective, it could be a landmark for cervical cancer prevention.

The PAVE Study is a solid and important study that is aimed to be carried out in nine countries and recruit tens thousands of women. It is a study with a large and diverse sample that can provide useful information for the development of this new screen-triage-treat strategy. Another strength is the fact that the PAVE project is integrated into the screening activities placed in the selected countries that will allow to evaluate efficacy and effectiveness in real-word context.

The manuscript does not present results because its aim is to describe the study protocol, structure and logic of the PAVE strategy.

Phase 1 aims to evaluate efficacy of the strategy. Methods are well described and are consistent with the study aims.

Phase 2 aims to evaluate the implementation of the PAVE strategy in clinical practice. The inclusion of implementation evaluation in this type of studies is an important milestone in the field of cervical cancer prevention. It has been shown that many strategies that have proven to be effective in controlled studies face barriers when they are implemented in real life. In that sense, results of phase 2 are key to ensure the future implementation of the strategy.

https://doi.org/10.7554/eLife.91469.3.sa2

Reviewer #3 (Public Review):

Summary: Despite being preventable and treatable, cervical cancer remains the second most common cause of cancer death globally. This cancer, and associated deaths, occur overwhelmingly in low- and middle-income countries (LMIC), reflecting a lack of access to vaccination, screening and treatment services. Cervical screening is the second pillar in the WHO strategy to eliminate cervical cancer as a public health problem and will be critical in delivering early gains in cervical cancer prevention as the impact of vaccination will not be realized for several decades. However, screening strategies implemented in high income countries are not feasible or affordable in LMICs. This ambitious multi-center study aims to address these issues by developing and systematically evaluating a novel approach to cervical screening. The approach, based on primary screening with self-collected specimens for HPV testing, is focused on optimizing triage of people in whom HPV is detected, so that sensitivity for the detection of pre-cancer and cancer is maximized while treatment of people without pre-cancer or cancer is minimized.

Strengths:

The triage proposed for this study builds on the authors' previously published work in designing the ScreenFire test to appropriately group the 13 detected genotypes into four channels and to develop automated visual evaluation (AVE) of images of the cervix, taken by health workers.

The move from mobile telephone devices to a dedicated device to acquire and evaluate images, overcomes challenges previously encountered whereby updates of mobile phone models required retraining of the AVE algorithm.

The separation of the study into two phases, an efficacy phase in which screen positive people will be triaged and treated according to local standard of care and the performance of AVE will be evaluated against biopsy outcomes will be followed by the second phase in which the effectiveness, cost-effectiveness, feasibility and acceptability will be evaluated.

The setting in a range of low resource settings which are geographically well spread and reflective of where the global cancer burden is highest.

Weaknesses:

Potential ascertainment bias due to the lack of specified biopsy (such as small four quadrant biopsies or small biopsies across the transformation zone) when aceto-white areas are not identified. This has the potential lead to lead to an over-estimate of sensitivity of the triage approach, particularly in the setting of VIA as compared to colposcopy. While the authors specify endocervical sampling in this setting, using curette or brush (for cytology), this may not be as sensitive unless clinicians are experienced in endocervical curette procedures.

https://doi.org/10.7554/eLife.91469.3.sa3

Author response

The following is the authors’ response to the original reviews.

Reviewer #1 (Recommendations For The Authors):

It is not clear if the cost-effectiveness cited refers exactly to the PAVE protocol. No line item costings are given. As far as I know, the AmpFire test is very expensive (some 6 USD) and AI-assisted colposcopy has at least formerly been very expensive.

Response: As mentioned in the section on "Cost-effectiveness analysis," the cost-effectiveness results refer to "an early exercise to approximate the potential costs and benefits of a highly effective screening campaign delivered to women aged 30-49 years in the ~65 highest burden LMIC (Figure 1; Suppl Materials) and an HPV vaccination program delivered to girls aged 9-14 years". Because this modeling was intended to be a high-level approximation prior to the availability of micro-costing and use of a new microsimulation model reflecting the epidemiology of HPV in PAVE study sites, we used a bundled cost of US$15 per woman screened and managed appropriately, including the ~$6 cost of the ScreenFire test, triage with AVE for women with HPV positivity, and treatment based on risk stratification. Micro-costing and microsimulation model development for PAVE sites are ongoing alongside the study and will have the capability to reflect setting-specific differences in delivery costs, as well as different burdens of HPV and precancer. These refinements of costing and cost-effectiveness estimates are a high priority of the PAVE consortium

Reviewer #2 (Recommendations For The Authors):

As mentioned above, the description of phase 2 could be improved. I suggest that the inclusion of Implementation Science frameworks and tools could contribute to strengthening methods to measure implementation outcomes. Perhaps if the protocol and scope of the study allows it, I suggest that the authors evaluate the incorporation of the assessment of barriers and facilitators of implementation to inform future scaling up of the PAVE strategy. To do this, for example, some Implementation Science Frameworks, such as Conceptual Framework of Implementation Research (CFIR)1-2 could be useful. In addition, as the authors mentioned, future dissemination will need an effective communication strategy and to design it they will carry out a pilot study. The inclusion of CFIR framework or other similar framework, could contribute to identifying contextual factors that might affect implementation and contribute to designing an accurate implementation and dissemination strategy.

The authors also mentioned that if the PAVE strategy is effective, it could replace the current standard of care. This fact would lead to the need to carry out a des-implementation process. This process needs stakeholders' engagement and political will, among other contextual factors (e.g., human resources, organizational changes, etc.).Implementation of new strategies needs that implementers perceive it as acceptable, adaptable, compatible and with greater advantages than the usual practice. In this sense, the analysis of implementation outcomes guided by CFIR framework could play an important role in this future des-implementation process.

1. Damschroder, et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Sci 4, 50 (2009) https://doi.org/10.1186/1748-5908-4-50.

1. Damschroder, L.J., Reardon, C.M., Widerquist, M.A.O. et al. The updated Consolidated Framework for Implementation Research based on user feedback. Implementation Sci 17, 75 (2022). https://doi.org/10.1186/s13012-022-01245-0

Response: Phase 2 refers to limited aspects of PAVE implementation, mainly introducing the management algorithms and evaluating the acceptability by providers and patients. Based on preliminary results of PAVE in the efficacy analysis a more comprehensive implementation intervention is being planned.

Reviewer #3 (Recommendations For The Authors):

This is a very strong protocol and obviously the synthesis of many years' of work. I have some minor suggestions only.

The issue raised as a weakness could be addressed by specifying that biopsy adequacy is evaluated by the local histopathologist. Those cases that don't contain at least some stroma and only superficial strips of epithelium should probably be assessed as "unsatisfactory" and excluded from triage performance calculations.

While endocervical curettage is commonly performed in North America, resulting in good quality samples, there is considerable global variation in this practice. The procedure yielding high quality samples is usually somewhat painful due to the cervical dilation and may in fact be more painful than small biopsies.

Response: We are undertaking a thorough evaluation of histology assessment together with the on-site pathologists and an external expert reviewer. It is critical that the study material be of good quality and that the diagnosis be highly accurate as these elements are critical for patient management but also for an adequate training of the AI algorithm. We are recommending to use for endocervical sampling a soft tissue by Histologics that provides excellent material and it is reported to be less painful than regular curette. Pathologists are requested to verify the quality of the sampling of this approach.

The sentence starting at line 311 could add that, clinicians also record transformation type and/ or colposcopy adequacy.

Response: Added

The clinicians are reporting the VIA or the colposcopy impression and also the visibility of the SCJ.

The manuscript could be strengthened by specifying what will happen to people who have HPV detected and are triage negative. Will they be recalled for follow-up HPV test at around 12 months or some other interval?

Finally, will those who have been treated be recalled for a follow-up HPV test at around 12 months, particularly those treated with thermal ablation? Follow-up of people in whom HPV is detected, whether triage negative or positive (and treated) would strengthen the study and enhance participant safety. If this is already planned it would strengthen the manuscript to cover these aspects.

Response: The PAVE strategy runs under a Consortium agreement and thus we cannot dictate specific protocols for follow-up. We are very eager to promote an adequate follow-up for those with a triage test negative, but the monitoring of its implementation is beyond PAVE. All settings have under their guidelines a yearly follow-up for any woman receiving thermal ablation and shorter intervals for those getting LEEP (LLETZ).

https://doi.org/10.7554/eLife.91469.3.sa4

Article and author information

Author details

  1. Silvia de Sanjosé

    1. Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Rockville, United States
    2. ISGlobal, Barcelona, Spain
    Contribution
    Conceptualization, Data curation, Investigation, Methodology, Supervision, Validation, Writing – original draft, Writing – review and editing
    For correspondence
    desanjose.silvia@gmail.com
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-5909-676X
  2. Rebecca B Perkins

    University Chobanian and Avedisian School of Medicine/Boston Medical Center, Boston, United States
    Contribution
    Writing – review and editing, Methodology, Conceptualization, Writing – original draft
    Competing interests
    No competing interests declared
  3. Nicole Campos

    Center for Health Decision Science, Harvard T.H. Chan School of Public Health, Boston, United States
    Contribution
    Validation, Formal analysis, Methodology, Conceptualization, Writing – original draft
    Competing interests
    No competing interests declared
  4. Federica Inturrisi

    Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Rockville, United States
    Contribution
    Data curation, Investigation, Conceptualization, Writing – original draft
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-3661-0842
  5. Didem Egemen

    Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Rockville, United States
    Contribution
    Formal analysis, Methodology, Conceptualization, Writing – original draft
    Competing interests
    No competing interests declared
  6. Brian Befano

    1. Information Management Services Inc, Calverton, United States
    2. Department of Epidemiology, University of Washington School of Public Health, Seattle, United States
    Contribution
    Formal analysis, Investigation, Methodology, Writing – original draft
    Competing interests
    is an employee of Information Management Services Inc
  7. Ana Cecilia Rodriguez

    Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Rockville, United States
    Contribution
    Formal analysis, Methodology, Conceptualization, Writing – original draft
    Competing interests
    No competing interests declared
  8. Jose Jerónimo

    Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Rockville, United States
    Contribution
    Methodology, Writing – original draft
    Competing interests
    No competing interests declared
  9. Li C Cheung

    Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Rockville, United States
    Contribution
    Formal analysis, Data curation, Writing – original draft
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-1625-4331
  10. Kanan Desai

    Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Rockville, United States
    Contribution
    Data curation, Methodology, Conceptualization, Writing – original draft
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-8992-5944
  11. Paul Han

    Division of Cancer Control and Population Sciences, National Cancer Institute, National Institutes of Health, Rockville, United States
    Contribution
    Validation, Data curation, Methodology, Writing – original draft
    Competing interests
    No competing interests declared
  12. Akiva P Novetsky

    Westchester Medical Center/New York Medical College, Valhalla, United States
    Contribution
    Writing – review and editing, Data curation, Writing – original draft
    Competing interests
    No competing interests declared
  13. Abigail Ukwuani

    Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Rockville, United States
    Contribution
    Project administration, Writing – original draft
    Competing interests
    No competing interests declared
  14. Jenna Marcus

    Feinberg School of Medicine at Northwestern University, Chicago, United States
    Contribution
    Writing – review and editing, Writing – original draft
    Competing interests
    No competing interests declared
  15. Syed Rakin Ahmed

    1. Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Boston, United States
    2. Harvard Graduate Program in Biophysics, Harvard Medical School, Harvard University, Cambridge, United States
    3. Massachusetts Institute of Technology, Cambridge, United States
    4. Geisel School of Medicine at Dartmouth, Dartmouth College, Hanover, United States
    Contribution
    Software, Formal analysis, Methodology, Conceptualization, Writing – original draft
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-1615-8633
  16. Nicolas Wentzensen

    Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Rockville, United States
    Contribution
    Methodology, Conceptualization, Writing – original draft
    Competing interests
    No competing interests declared
  17. Jayashree Kalpathy-Cramer

    1. Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Boston, United States
    2. University of Colorado Anschutz Medical Campus, Aurora, United States
    Contribution
    Validation, Formal analysis, Data curation, Investigation, Writing – original draft
    Competing interests
    No competing interests declared
  18. Mark Schiffman

    Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Rockville, United States
    Contribution
    Validation, Data curation, Methodology, Conceptualization, Supervision
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-4625-2508

Funding

Intramural Research Program of the National Cancer Institute

  • Silvia de Sanjosé
  • Rebecca B Perkins
  • Nicole Campos
  • Federica Inturrisi
  • Didem Egemen
  • Brian Befano
  • Ana Cecilia Rodriguez
  • Jose Jerónimo
  • Li C Cheung
  • Kanan Desai
  • Paul Han
  • Abigail Ukwuani
  • Syed Rakin Ahmed
  • Nicolas Wentzensen
  • Mark Schiffman

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Acknowledgements

Farideh Almani for secreterial support.

Ethics

Informed consent has been obtained in each study site together with the corresponding ethical approval at each site.

Senior Editor

  1. Eduardo L Franco, McGill University, Canada

Reviewing Editor

  1. Talía Malagón, McGill University, Canada

Version history

  1. Sent for peer review: August 17, 2023
  2. Preprint posted: August 31, 2023 (view preprint)
  3. Preprint posted: October 16, 2023 (view preprint)
  4. Preprint posted: December 5, 2023 (view preprint)
  5. Version of Record published: January 15, 2024 (version 1)

Cite all versions

You can cite all versions using the DOI https://doi.org/10.7554/eLife.91469. This DOI represents all versions, and will always resolve to the latest one.

Copyright

This is an open-access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.

Metrics

  • 222
    Page views
  • 55
    Downloads
  • 0
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Silvia de Sanjosé
  2. Rebecca B Perkins
  3. Nicole Campos
  4. Federica Inturrisi
  5. Didem Egemen
  6. Brian Befano
  7. Ana Cecilia Rodriguez
  8. Jose Jerónimo
  9. Li C Cheung
  10. Kanan Desai
  11. Paul Han
  12. Akiva P Novetsky
  13. Abigail Ukwuani
  14. Jenna Marcus
  15. Syed Rakin Ahmed
  16. Nicolas Wentzensen
  17. Jayashree Kalpathy-Cramer
  18. Mark Schiffman
  19. On behalf of the PAVE Study Group
(2024)
Design of the HPV-automated visual evaluation (PAVE) study: Validating a novel cervical screening strategy
eLife 12:RP91469.
https://doi.org/10.7554/eLife.91469.3

Share this article

https://doi.org/10.7554/eLife.91469

Further reading

    1. Epidemiology and Global Health
    David Robert Grimes
    Research Advance Updated

    In biomedical science, it is a reality that many published results do not withstand deeper investigation, and there is growing concern over a replicability crisis in science. Recently, Ellipse of Insignificance (EOI) analysis was introduced as a tool to allow researchers to gauge the robustness of reported results in dichotomous outcome design trials, giving precise deterministic values for the degree of miscoding between events and non-events tolerable simultaneously in both control and experimental arms (Grimes, 2022). While this is useful for situations where potential miscoding might transpire, it does not account for situations where apparently significant findings might result from accidental or deliberate data redaction in either the control or experimental arms of an experiment, or from missing data or systematic redaction. To address these scenarios, we introduce Region of Attainable Redaction (ROAR), a tool that extends EOI analysis to account for situations of potential data redaction. This produces a bounded cubic curve rather than an ellipse, and we outline how this can be used to identify potential redaction through an approach analogous to EOI. Applications are illustrated, and source code, including a web-based implementation that performs EOI and ROAR analysis in tandem for dichotomous outcome trials is provided.

    1. Epidemiology and Global Health
    Qixin He, John K Chaillet, Frédéric Labbé
    Research Article

    The establishment and spread of antimalarial drug resistance vary drastically across different biogeographic regions. Though most infections occur in sub-Saharan Africa, resistant strains often emerge in low-transmission regions. Existing models on resistance evolution lack consensus on the relationship between transmission intensity and drug resistance, possibly due to overlooking the feedback between antigenic diversity, host immunity, and selection for resistance. To address this, we developed a novel compartmental model that tracks sensitive and resistant parasite strains, as well as the host dynamics of generalized and antigen-specific immunity. Our results show a negative correlation between parasite prevalence and resistance frequency, regardless of resistance cost or efficacy. Validation using chloroquine-resistant marker data supports this trend. Post discontinuation of drugs, resistance remains high in low-diversity, low-transmission regions, while it steadily decreases in high-diversity, high-transmission regions. Our study underscores the critical role of malaria strain diversity in the biogeographic patterns of resistance evolution.