Research Article

Epidemiology and Global Health

Impact of COVID-19 pandemic on breast and cervical cancer screening in Denmark: A register-based study

Center for Epidemiological Research, Nykøbing Falster Hospital, Denmark
Department of Public Health, University of Copenhagen, Denmark
Department of Gynecology and Obstetrics, Odense University Hospital, Denmark
Department of Breast Examinations, Copenhagen University Hospital Herlev/Gentofte, Denmark
Department of Pathology, Aarhus University Hospital, Denmark

Mar 21, 2023

https://doi.org/10.7554/eLife.81605

Version of Record

Accepted for publication after peer review and revision.

Download
Cite
Share
CommentOpen annotations (there are currently 0 annotations on this page).

195 views

Version of Record published: March 21, 2023 (This version)
Accepted: February 14, 2023
Preprint posted: January 11, 2023 (Go to version)
Received: July 4, 2022

1. Part of Collection
COVID-19: A Collection of Articles

Edited by Diane M Harper et al.
Further reading

Article
Figures and data
Abstract
Editor's evaluation
Introduction
Materials and methods
Results
Discussion
Data availability
References
Decision letter
Author response
Article and author information
Metrics

Abstract

Background:

Denmark was one of the few countries where it was politically decided to continue cancer screening during the COVID-19 pandemic. We assessed the actual population uptake of mammography and cervical screening during this period.

Methods:

The first COVID-19 lockdown in Denmark was announced on 11 March 2020. To investigate possible changes in cancer screening activity due to the COVID-19 pandemic, we analysed data from the beginning of 2017 until the end of 2021. A time series analysis was carried out to discover possible trends and outliers in the screening activities in the period 2017–2021. Data on mammography screening and cervical screening were retrieved from governmental pandemic-specific monitoring of health care activities.

Results:

A brief drop was seen in screening activity right after the first COVID-19 lockdown, but the activity quickly returned to its previous level. A short-term deficit of 43% [CI –49 to –37] was found for mammography screening. A short-term deficit of 62% [CI –65 to –58] was found for cervical screening. Furthermore, a slight, statistically significant downward trend in cervical screening from 2018 to 2021 was probably unrelated to the pandemic. Other changes, for example, a marked drop in mammography screening towards the end of 2021, also seem unrelated to the pandemic.

Conclusions:

Denmark continued cancer screening during the pandemic, but following the first lockdown a temporary drop was seen in breast and cervical screening activity.

Funding:

Region Zealand (R22-A597).

Editor's evaluation

Denmark was one of the few countries that did not suspend cancer screening in the early stage of the COVID-19 pandemic. This important study offers convincing evidence of how the pandemic impacted the use of breast and cervical cancer screening services. This article has a broad interest to public health researchers and health policy implementation.

https://doi.org/10.7554/eLife.81605.sa0

Introduction

For almost 2 years, COVID-19 has been a central part of daily life in Denmark. On 11 March 2020, when the WHO declared COVID-19 a global pandemic (WHO, 2020), the prime minister of Denmark announced a national lockdown (Danish Government, 2020). In many ways, the first lockdown in Denmark was similar to the handling of the pandemic in other countries with, among other restrictions, temporary suspension of non-urgent health care services. However, as opposed to the situation in, for example, the UK and Italy, the cancer screening programmes were not put on hold (Danish Health Authority, 2022a; Maringe et al., 2020; Vanni et al., 2020). By 22 January 2022, the last COVID-19 restrictions in Denmark were repealed.

Cancer screening in Denmark is free of charge, so even for citizens economically affected by COVID-19, payment should be no hindrance to participation. However, at the lockdown press conference, the prime minister and the head of the Danish Health Authority focused on the need to protect the health care system by avoiding unnecessary contact with general practitioners (GPs) and emergency departments. Some people could have interpreted this as a ‘stay home’ message from cancer screening as well (Wilson et al., 2021). The screening programmes could also have been affected by, for example, relocation of personnel to COVID-19 tasks and use of HPV-testing equipment for analysis of COVID-19 tests. For individual citizens existing barriers to participation in cancer screening could also have been enhanced during the pandemic, for example, difficulties in booking consultations at their GPs (Wilson et al., 2021), and new barriers could have arisen, such as concern about becoming infected with COVID-19 (Wilson et al., 2021); concern about unnecessary burdening of the health care system (Wilson et al., 2021); or simply by changes of priorities (Castanon et al., 2021). In the present study, we assessed the effect of the COVID-19 pandemic on the number of tests performed in mammography and cervical screening, nationally and by region.

Materials and methods

Setting

Request a detailed protocol

Health care in Denmark is financed through taxes, and cancer screening, assessment, and treatment of detected lesions are free of charge for the patient. Denmark has three national cancer screening programmes. First, biennial mammography screening for women aged 50–69 occurs at specialised hospital clinics (Danish Health Authority, 2019a). It is a stand-alone programme including only mammography screening taken in the public sector after invitation. In Denmark, opportunistic breast screening is minimal (Jensen et al., 2005). Second, third or fifth yearly cervical screening for women aged 23–64, where samples are collected primarily in the private sector by GPs or office-based gynaecologists (Danish Health Data Authority, 2020). It is an integrated programme including both cell samples collected after invitation, cell samples collected at the woman’s/doctor’s initiative, or collected as a control of a previous abnormal test. A unique screening initiative took place in 2017, where women above 70 had a one-time invitation to screening (Region Midtjylland, 2018), as this birth cohort has not previously been offered screening. Lastly, biennially screening for bowel cancer. As this programme is based on faecal immunological test kits sent to people’s home, the programme was not part of the Danish health care monitoring system from which we extracted data (Danish Health Authority, 2019b). The five Danish regions are responsible for the programmes, following the Danish Health Authority guidelines.

Health care activities are registered centrally. Based on these registered data, the Danish Health Authority closely monitored health care activities in hospitals and the private practice sector during the pandemic. From 10 June 2020 and up until 5 January 2022, activity data were published regularly in 13 reports (Danish Health Authority, 2022b). The present study was based on the activity data collected in this monitoring effort.

At mammography screening, two views are taken, a craniocaudal and a mediolateral oblique. The screens are read independently by two radiologists, now in one region by one radiologist and an AI-based system, and if inconsistent, a consensus decision is reached. The screening result is reported as normal or abnormal; in the latter case, the letter is combined with an invitation to clinical assessment. The result is communicated to the woman via her Digital Post for communication with public authorities. For simplicity, we used the term ‘mammography screen’ for the entire mammography examination.

At cervical screening, a cell sample is taken by the GP/gynaecologist, placed in a vial, and sent to the regional pathology laboratory for cytology examination, now also primary HPV test in an implementation trial. The result is communicated both to the woman and to the cell-sample collector. In cases of milder abnormalities, the woman is recommended cell-sample control. For more severe abnormalities, the woman is referred to a gynaecologist.

At the start of the lockdown on 13 March 2020, emphasis was, among other things, put on limiting the spread of COVID-19 to health care staff. For the GPs, the Organisation of General Practitioners in Danish called Praktiserende Lægers Organisation, PLO, specified guidelines dividing patient contacts into those needing consultation and those that could be handled by phone/web or could eventually be postponed (DSAM, 2022a). In the first GP guideline from 25 March 2020, cervical screening was recommended to be postponed. However, the Danish Health Authority intervened, and 9 days later, on 3 April 2020, a new GP guideline stated that cervical screening was not suspended (DSAM, 2022b). Furthermore, to compensate for the observed drop in mammography screening in the early phase of the pandemic, all five regions issued extra reminders to women who had cancelled their screening appointments in June and July 2020 (Danish Cancer Society, 2020a).

Throughout the COVID-19 lockdown, the Danish Cancer Society and news agencies regularly reported on the state of cancer-related activities, including trends in numbers of mammography screens and cell samples (Danish Cancer Society, 2020b; Danish Health Authority, 2022b). The Danish Cancer Society encouraged women who had skipped cervical screening to book a new appointment with their GP (Danish Cancer Society, 2020b).

Data

For mammography screening, the government retrieved from the National Patient Register (Landspatientsregistret [LPR]) using the activity codes DZ123A (mammography screening), DZ123AA (mammography screening under the Health Act), and DZ108A (now expired code), and the procedure code UXRC45 (mammography, screening) alone or combined with additional diagnosis codes ZPROON (normal test result) and ZPR01N (abnormal test result). The same data source and codes are used in the annual reporting from the Danish Quality Database on Mammography Screening (DKMS, 2021). Data on cervical screening were retrieved from the Danish Health Insurance Register using the combination of provider codes 80 (GP) and 07 (office-based gynaecologist) with payment codes 2102 and 4301 (cervical cell sample). The data source and codes differ from the ones used in the annual reporting from the Danish Quality Database on Cervical Cancer Screening (DKLS, 2022) where data are retrieved from the National Pathology Register. We retrieved data from 2017 to 2021, with the period before 13 March 2020 representing the pre-COVID-19 period.

All data were available in weekly aggregated form, using a standard ISO-week numbering system. The Danish Health Data Authority provided mammography data for 2017–2021. Cell-sample data for 2017–2019 and from weeks 34 to 52 in 2021 were provided by the Danish Health Data Authority, while we extracted cell-sample data from 2020 and weeks 1 to 33 in 2021 from eSundhed.dk (Danish Health Data Authority, 2022).

Data analysis

Request a detailed protocol

To identify a possible impact of the COVID-19 epidemic on cancer screening activity, we undertook a time series analysis. To easily identify seasonal patterns, especially moving holidays, for example, Easter, summer holidays, etc., data were split into periods of 4 weeks, called ‘months’ in the rest of this paper, making a total of 13 months per year, with week 53 of 2020, a leap year, included in month 13 (Supplementary file 1).

Seasonal and trend patterns were then identified through a Seasonal Decomposition of Time Series by Loess (STL decomposition) with robust fitting. The seasonal-adjusted time series, that is the original time series without the seasonal component, were then modelled with a negative binomial integer-valued GARCH model with a logarithmic link (Liboschik et al., 2017). Outliers and linear trends in the time series were identified through graphical inspection and analytical procedure. In particular, outliers were identified through the procedures described in Liboschik et al., 2017; here, outliners are named ‘interventions’and in López-de-Lacalle, 2019 using the ‘tso function’ in the tsoutlier package. Linear trends were found by searching for a statistically significant linear term in the regression model. Once a liner trend was identified, the start date was chosen as the date minimising the AIC of the model.

Estimated reductions in the number of screens were obtained from the coefficients of the relevant ‘intervention’ covariate of the model. The corresponding 95% confidence intervals are calculated from the coefficients’ standard error using normal approximation.

All analyses and plots were done using R ver. 4.1.1 (R Development Core Team, 2021), with the package collection tidyverse (Wickham et al., 2019) and the packages tscount (Liboschik et al., 2017) and tsoutlier (López-de-Lacalle, 2019).

Results

Mammography screening

On average, 287,284 mammography screenings were performed annually from 2017 to 2019, compared with 260,508 in 2020 and 270,303 in 2021 (Table 1).

Table 1

Number of mammography and cervical screens by year and region, Denmark 2017–2021.

Region	Mammography screening					Cervical screening
Region	Year 2017	Year 2018	Year 2019	Year 2020	Year 2021	Year 2017	Year 2018	Year 2019	Year 2020	Year 2021
Denmark	289,150	290,275	282,426	260,508	270,303	443.126	348,950	344,045	306,472	323,598
Capital	78,323	75,415	75,964	65,012	51,457	155,022	124,149	120,329	110,655	116,145
Central	67,282	68,117	66,895	63,043	69,719	97,775	77,200	84,099	72,006	75,967
North	30,835	31,033	30,415	31,978	26,532	43,818	34,435	33,719	29,293	32,288
South	70,396	69,468	68,271	64,108	72,876	88,540	67,896	64,972	57,108	59,738
Zealand	42,314	46,242	40,881	36,367	49,719	57,971	45,270	40,926	37,410	39,460

Source: Own calculations based on numbers provided by the Danish Health Data Authority.

The time trend analysis for mammography screening is illustrated in Figures 1—3. Figure 1 depicts the observed weekly number of mammography screening. Figure 2 shows the decomposition of the monthly data. In the seasonal component, one can clearly identify the prominent low peaks due to Easter, month 4, summer, month 8, and Christmas, month 13. In Figure 3, the fitted model is depicted together with the seasonal-adjusted data.

Figure 1

Download asset Open asset

Weekly number of mammography screens, Denmark 2017–2021.

Source: Own calculations based on numbers provided by The Danish Health Data Authority.

Figure 2

Download asset Open asset

Decomposition of time series in number of mammography screens into seasonal, trend, and irregular component, Denmark 2017–2021.

Source: Own calculations based on numbers provided by The Danish Health Data Authority.

Figure 3 with 1 supplement see all

Download asset Open asset

Seasonal-adjusted time series and fitted model of the number of mammography screens, Denmark 2017–2021.

Source: Own calculations based on numbers provided by The Danish Health Data Authority.

We found that from 24 February 2020 to 22 March 2020, there was a –22.6% [CI –30.1 to –14.4] reduction in the number of mammography screenings, and from 23 March 2020 to 19 April 2020, a –43.3% [CI –49.0 to –36.9] reduction (Figure 3 and Table 2). The activity decline was short-termed; however, it did not reach the previous levels in the first months after the first lockdown. There was a statistically significant decrease of –17.0% [CI –21.2 to –12.6] in the number of mammography screens from 16 August 2021 to 2 January 2022 (Figure 3). All percentages were with respect to the expected number in the absence of the outlier. Separating the numbers by region, the Capital Region and the much smaller Region North were responsible for the drop at the end of 2021 (Figure 3—figure supplement 1). In the Capital Region, from 19 June 2021 to 15 August 2021, the decrease was –34.0% [CI –43.4 to –23.1]; from 16 August 2021 to 12 September 2021 –67.2% [CI –72.1 to –61.3]; from 13 September 2021 to 10 October 2021 –85.3% [CI –88.1 to –81.9], and from 11 October 2021 to 2 January 2022 –41.3% [CI –49.5 to –31.8].

Table 2

Estimated changes in number of seasonally adjusted mammography screens.

Denmark and regions, 2017–2021. Reductions and increases are with respect to the average base line.

	Time period	Estimated change in percent
Denmark	24 February 2020 – 22 March 2020 23 March – 19 April 2020 16 August 2021 – 02 January 2022	–22.6% [CI –30.1 to –14.4] –43.3% [CI –49.0 to –36.9] –17.0% [CI –21.2 to –12.6]
Capital	30 December 2019 – 18 July 2021 23 March 2020 – 19 April 2020 19 July 2021 – 15 August 2021 16 August 2021 – 12 September 2021 13 September 2021 – 10 October 2021 11 November 2021 – 02 January 2022	–11.8% [CI –15.7 to –7.7] – (level shift) –43.5% [CI –51.6 to –34.1] –34.0% [CI –43.4 to –23.1] –67.2 [CI –72.1 to –61.3] –85.3 [CI –88.1 to –81.9] –41.3 [CI –49.5 to –31.8]
Central	24 February 2020 – 22 March 2020 23 March 2020 – 19 April 2020	–15.7% [CI –24.4 to –6.1] –38.2% [CI –44.7 to –30.9]
North	24 May 2021 – 02 January 2022	7.5% [CI 5.1–9.9]
South	23 March 2020 – 19 April 2020	–43.5% [CI –53.5 to –31.4]
Zealand	24 February 2020 – 19 April 2020 02 November 2020 – 02 January 2022	–51.5% [CI –58.5 to –43.3] 1.6% [CI 0.7–2.5]

Source: Own calculations based on numbers provided by The Danish Health Data Authority.

Cervical screening

On average, 346,498 cell samples were collected annually from 2018 to 2019, compared with 306,472 in 2020 and 325,598 in 2021 (Table 1).

The time series analysis for cervical screening is illustrated in Figures 4—6. Analysis was done the same way for mammography screening analysis. Overall, there was an annual, statistically significant reduction of –1.9% [CI –2.9 to –0.8] from 6 November 2017 to 2 January 2022 in the number of cell samples in Denmark (Table 3). The special initiative for screening elderly women was issued in 2017, which the model illustrated in Figures 4—6 and Table 1. There was a drop in activity of –30.7% [CI –36.5 to –24.3] from 24 February 2020 to 22 March 2020 of –61.9% [CI –65.2 to –58.2] from 23 March 2020 to 19 April 2020 (Table 3). Hereafter, activity returned to the previously (linearly decreasing) level. Unexpectedly, there was a drop in activity from 21 June 2021 to 18 July 2021 of –20.3% [CI –27.1 to –12.9] (Figure 5, Table 3).

Figure 4

Download asset Open asset

Time series of the weekly number of cervical, Denmark 2017–2021.

Source: Own calculations based on numbers provided by The Danish Health Data Authority.

Figure 5

Download asset Open asset

Decomposition of time series in number of cervical cancer screens into seasonal, trend, and irregular component, Denmark 2017–2021.

Source: Own calculations based on numbers provided by The Danish Health Data Authority.

Figure 6 with 1 supplement see all

Download asset Open asset

Seasonal-adjusted monthly time series and fitted model of the number of cervical cancer screens, Denmark 2017–2021.

Source: Own calculations based on numbers provided by The Danish Health Data Authority.

Table 3

Estimated changes in number of seasonally adjusted cervical cell samples.

Denmark and regions, 2018–2020. Reductions and increases are with respect to the average base line.

	Time period	Estimated change in percent
Denmark	6 November 2017 – 02 January 2022 24 February 2020 – 22 March 2020 23 March – 19 April 2020 21 June 2021 – 18 July 2021	–1.9% [CI –2.9 to –0.8] annual reduction –30.7% [CI –36.5 to –24.3] –61.9% [CI –65.2 to –58.2] –20.3% [CI –27.1 to –12.9]
Capital	24 February 2020 – 22 March 2020 23 March 2020 – 19 April 2020 21 June 2021 – 18 July 2021	–32.4% [CI –38.1 to –26.1] –57.9% [CI –61.7 to –53.7] –23.8% [CI –30.3 to –16.8]
Central	24 February 2020 – 22 March 2020 23 March 2020 – 19 April 2020 21 June 2021 – 18 July 2021	–31.0% [CI –37.6 to –23.8] –62.8% [CI –66.6 to –58.5] –24.0 [CI –31.2 to –16.0]
North	09 September 2019 – 02 January 2022 24 February 2020 – 22 March 2020 23 March 2020 – 19 April 2020 29 March 2021 –25 April 2021	–6.9% [CI –8.9 to –4.8] annual reduction –33.9% [CI –41.5 to –25.3] –61.1% [CI –65.9 to –55.5] 94.5% [CI 72.9–118.8]
South	01 January 2018 – 02 January 2022 23 March 2020 – 19 April 2020 21 June 2021 – 18 July 2021	–5.0% [CI –6.4 to –3.6] annual reduction –60.4% [CI –64.4 to –55.9] –19.6 [CI –27.6 to –10.8]
Zealand	23 March 2020 – 19 April 2020 01 March 2021 – 28 March 2021 21 June 2021 – 18 July 2021	–67.6% [CI –71.8 to –62.7] 31.9% [CI 16.0–50.0] –12.7% [CI –23.4 to –0.6]

Source: Own calculations based on numbers provided by The Danish Health Data Authority.

When the data from Denmark were split by region, the numbers from Capital Region, Region Zealand, and Region Central followed the national trend (Figure 6—figure supplement 1, Table 3). Region North had an annual reduction of –6.9% [CI –8.9 to –4.8] from 9 September 2019 to 2 January 2022, and Region South of –5.0% [CI –6.4 to –3.6] from 1 January 2018 to 2 January 2022 (Figure 6—figure supplement 1, Table 3). The drop around 1 July 2021 was seen in all regions except Region North.

Discussion

Main finding

A survey undertaken by the International Cancer Screening Network documented that Denmark was one of the few countries where the health authorities did not suspend cancer screening during the COVID-19 pandemic (Puricelli Perin et al., 2021). Here, we demonstrated that this policy was largely followed, as only a brief drop was seen in the screening activity following the lockdown of the society on 13 March 2020.

Our analysis also illustrated the difficulties of using time series analysis in assessing a causal association between the COVID-19 pandemic and screening activity, as activity patterns were also affected by other societal circumstances, such as the shortage of radiologists.

For cervical screening, except during the unique initiative for screening of elderly women in 2017, our data indicated a slight, statistically significant downward trend in cervical screening activity. During the last 10 years, Denmark has experienced a slight general decline in participation in cervical screening. The health authorities are aware of this tendency, and self-sampling kits for HPV testing are currently being rolled out as an alternative to clinician collected samples (DKLS, 2022). A more detailed analysis showed that this was limited to two of the five Danish regions. Further in-depth analyses are required to explain this phenomenon. Furthermore, in mid-2021, there was a temporal drop in activities in four out of five regions. It coincided with a national nurse strike, which is unlikely to be directly linked to the drop, as the strike did not involve nurses employed in GP/gynaecologist practices. It could have affected the minor part of the screening activity that the hospitals undertake.

For mammography screening, our data showed activity decline in relation to the first lockdown and that the pre-lockdown levels were reached after several month. This could be due to the issue of extra reminders for screening, but further research would be needed to analyse the association, for example, through a survey. Further, we saw a steep decrease in the screening activity during the second half of 2021, limited to two of the five Danish regions.

In Denmark, the same staff works in screening and diagnostic mammography, and the Capital Region decided to limit the number of screening invitations in August–October 2021 to 25% of the normal level to meet the time limits for diagnostic mammography set in the Danish breast cancer patient package. Incidentally, this was well captured by our model. During this period, the mammography service in the Capital Region underwent organisational changes, moved location, acquired new equipment, and implemented an AI reader.

In our analysis, we focused on the potential effect of the COVID-19 epidemic on breast and cervical screening activity. One can speculate that the pandemic might have affected waiting times for follow-up and treatment of detected lesions. To answer this question, further in-depth analysis of health care data would be needed. However, it should be stressed that reminder systems for follow-up of abnormal findings were in place also during the pandemic.

Strength and limitations

The main strength of this study was the usage of nationwide register data.

In Denmark, all health care activities are registered in national databases. During the COVID-19 pandemic, the Danish Health Authority used the Danish Health Insurance Register to monitor cervical screening. In Denmark, GPs and office-based gynaecologists are fee-for-service paid by the government, and these payments are registered in the Danish Health Insurance Register. Therefore, the completeness of the data depends on the screening providers’ use of correct payment codes and timely reporting. In the annual monitoring of cervical screening, undertaken by the regions and reported in DKLS, data are retrieved from the National Pathology Register, which covers all pathology specimens analysed in Denmark. Nationwide, there was a difference between the two datasets in 2021, with 323,598 cell samples in our dataset and 372,508 in DKLS, (Supplementary files 2). As the DKLS data are expected to be complete, 13% of cell samples from 2021 were thus missing in our data, but the time trends are expected to be the same.

The Danish Health Authority used the National Patient Register as a source for the COVID-19 monitoring reports. DKMS use the same data source. As our data are reported by year, and DKMS’ data are reported by screening invitation-round, the numbers are not directly comparable. However, when corrected for length of period, we had 5% more screens than DKMS, (Supplementary files 2). The deficit in DKMS data derived almost entirely from two of the five Danish regions, which might partly be due to different dates for data retrieval.

A further limitation was that we used the number of tests to analyse trends, not considering that the size of the screening target groups could vary slightly over the years. It should also be noted that it is impossible to make a causal conclusion based on a trend analysis.

Another limitation was that we reported on tests performed and not on women examined.

Finally, even though a number of tests have been carried out to check the validity of our analyses (i.e. analysis of (partial) autocorrelation functions, periodogram of Pearson residuals, non-randomised PIT histogram, marginal calibration plot, normality of residuals), our results should be further checked in different settings, such as different time series models and/or using data with higher time resolution (weekly, daily).

Other studies

During the first 3 months of the COVID-19 pandemic in Denmark, the number of cancer diagnoses dropped by one-third compared with previous years (Skovlund et al., 2020). Early during the lockdown, the Danish Cancer Society collected information from screening providers and reported decreased participation in mammography and cervical screening (Danish Cancer Society, 2020a; Danish Cancer Society, 2020b). According to the DKLS 2020 report, there was a drop in cervical screens by approximately 10% from 2019–2020 is suggested to be related to COVID-19 and the lockdown (Waldstrøm et al., 2022).

In the Netherlands, suspension of the breast cancer screening programme was associated with a substantial decrease in women diagnosed with breast cancer in the first weeks after suspension, and the number remained low until June 2020 (Dinmohamed et al., 2020). Australia experienced decreased capacity after resuming the breast screening program, only reaching 83% of the 2018 level (Feletto et al., 2020). This could eventually delay screening and result in disease progression (Davies et al., 2022; Wilson et al., 2021).

In the UK, Wilson et al. undertook a mixed-method study in August–November 2020, comprising an online survey and qualitative interviews. In the survey, 30% answered that they were less likely to attend cervical screening than before the corona pandemic lockdown, and 75% declared that they were worried about delay in cancer screening caused by COVID-19 (Wilson et al., 2021). Although reluctance to participate in cervical screening is not a new phenomenon, especially among young women and college graduates (Wenger et al., 2022), further cancellation related to the pandemic could potentially have long-term consequences, as participation in screening once is a predictor of participation in the future (Kotzur et al., 2020; Wilson et al., 2021). The pandemic could therefore in particular be expected to affect women scheduled to enter the screening program during the pandemic (Wilson et al., 2021). Moreover, vulnerable groups who have been advised to be careful during the pandemic would weigh the pros and cons of screening against the risk of COVID-19 infection (Walker et al., 2021).

Public health implications

Our analysis showed a steep, temporal drop in breast and cervical screening activity following the WHO declaration of the pandemic and the simultaneous lockdown of the Danish society, indicating that COVID-19 did have an imitated and temporary effect on breast and cervical cancer screening in Denmark. The focused policy of keeping screening going was a decisive factor. Political decisions were communicated from the Danish Health Authority to the five regions responsible for cancer screening. The Danish Cancer Society and the press also played an essential role in repeatedly reporting cancer screening status. The initial suspension in marts 2020 of cervical screening undertaken by GPs was revoked within 9 days, and extra reminders for mammography screening were issued when real-time monitoring data indicated a decrease in activity.

The very possibility of continuing screening during the pandemic was undoubtedly facilitated by the robustness of the health care system. During the period, March 2020–December 2021, 3550 persons in Denmark died with COVID-19 (Neergaard, 2022), constituting 3.6% out of a total number of 99,000 deaths in the period (StatBank Denmark, 2022). Life expectancy increased from 2018/2019 to 2019/2020 for both men and women but levelled off from 2019/2020 to 2020/2021; for men changing from 79.5 to 79.6 years, and for women slightly decreased from 83.6 to 83.4 years (StatBank Denmark, 2022). The COVID-19 lockdown period was nevertheless a period where health care resources were stretched far beyond the normal level. After a breakdown of negotiations on a new agreement, the nurses went on strike in June–July 2021, causing further delays in elective treatment, for example, hip replacement (TV2 News, 2022).

As handling cancer screening during an extraordinary situation like the COVID-19 pandemic depends on the local organisation and resources, the experiences from Denmark can probably not be generalised to all settings. Nevertheless, the example illustrated the potentials for health policy implementation in a high-income, welfare state with a publicly run health care system.

Conclusion

In Denmark, it was decided politically to continue mammography and cervical screening throughout the COVID-19 pandemic in 2020 and 2021. Our study showed that screening activity dropped suddenly at the time of the first lockdown of the society but recovered in the following months and went back to the pre-pandemic level.

Data availability

The Danish Health Data Authority has provided all data. All data generated or analysed during this study are included in the manuscript and supporting file. Dataset has been deposited at DataCite (https://search.datacite.org/) and with DOI:10.17894/ucph.b1860a53-63e3-45c7-970b-6482ab2947c7. All analyses and plots were done using R ver. 4.1.1 (R Core Team; 2021), with the package collection tidyverse (Wickham et al.; 2019) and the packages tscount (Liboschik et al.; 2017) and tsoutlier (ópez-de-Lacalle; 2019). The software and packages are publicly available and can be downloaded at https://cran.r-project.org/mirrors.html, https://cran.r-project.org/web/packages/tsoutliers/index.html and https://www.tidyverse.org/.

The following data sets were generated

1. Nonboe MH
(2022) Electronic Research Data Archive University of Copenhagen
Data.
https://doi.org/10.17894/ucph.b1860a53-63e3-45c7-970b-6482ab2947c7

References

1. Castanon A
2. Rebolj M
3. Burger EA
4. de Kok IMCM
5. Smith MA
6. Hanley SJB
7. Carozzi FM
8. Peacock S
9. O’Mahony JF
(2021) Cervical screening during the COVID-19 pandemic: optimising recovery strategies
The Lancet. Public Health 6:e522–e527.
https://doi.org/10.1016/S2468-2667(21)00078-5
- PubMed
- Google Scholar
Website
1. Danish Cancer Society
(2020a) Aflyste du din screening for brystkræft under coronakrisen? Sig ja tak til at få en ny tid! Kræft Bekæmp
Accessed July 3, 2020.

https://www.cancer.dk/nyheder/aflyste-du-din-screening-for-brystkraeft-under-coronakrisen-sig-ja-tak-til-at-faa-en-ny-tid/
Website
1. Danish Cancer Society
(2020b) Husk at bestille tid til den celleprøve, du aflyste under coronakrisen. kræft bekæmp
Accessed July 3, 2020.

https://www.cancer.dk/nyheder/husk-at-bestille-tid-til-den-celleproeve-du-aflyste-419/
Website
1. Danish Government
(2020) The prime minister: we will need to help each other: pressmeeting on coronavirus
Accessed March 11, 2020.

https://www.regeringen.dk/nyheder/2020/pressemoede-11-marts-i-spejlsalen/
Website
1. Danish Health Authority
(2019a) Breast cancer screening
Accessed January 27, 2022.

https://www.sst.dk/da/viden/screening/screening-for-kraeft/brystkraeft
Website
1. Danish Health Authority
(2019b) Bowel cancer screening
Accessed January 27, 2022.

https://www.sst.dk/da/Viden/Screening/Screening-for-kraeft/Tarmkraeft
Report
1. Danish Health Authority
(2022a) Håndtering af COVID-19: Beskrivelse af kritiske funktioner i sundhedsvæsenet under COVID-19
Danish Health Authority.

https://www.ft.dk/samling/20191/almdel/suu/spm/767/svar/1644761/2167278.pdf
- Google Scholar
Website
1. Danish Health Authority
(2022b) COVID-19: Monitoring of activity i the health care system in Denmark
Accessed January 12, 2022.

https://www.sst.dk/da/Udgivelser/2021/COVID-19-Monitorering-af-aktivitet-i-sundhedsvaesenet
Website
1. Danish Health Data Authority
(2020) Cervical cancer screening
Accessed January 10, 2022.

https://www.sst.dk/da/Udgivelser/2018/Screening-for-livmoderhalskraeft
Website
1. Danish Health Data Authority
(2022) Monitoring of the reopening of the healt care system
Accessed January 12, 2022.

https://www.esundhed.dk/Emner/Patienter-og-sygehuse/Monitorering-af-genaabningen-af-sundhedsvaesnet
1. Davies JM
2. Spencer A
3. Macdonald S
4. Dobson L
5. Haydock E
6. Burton H
7. Angelopoulos G
8. Martin-Hirsch P
9. Wood NJ
10. Thangavelu A
11. Hutson R
12. Munot S
13. Flynn M
14. Smith M
15. DeCruze B
16. Myriokefalitaki E
17. Sap K
18. Winter-Roach B
19. Macdonald R
20. Edmondson RJ
(2022) Cervical cancer and COVID-an assessment of the initial effect of the pandemic and subsequent projection of impact for women in England: a cohort study
BJOG 129:1133–1139.
https://doi.org/10.1111/1471-0528.17098
- PubMed
- Google Scholar
(2020) The impact of the temporary suspension of national cancer screening programmes due to the COVID-19 epidemic on the diagnosis of breast and colorectal cancer in the netherlands
Journal of Hematology & Oncology 13:147.
https://doi.org/10.1186/s13045-020-00984-1
- PubMed
- Google Scholar
Report
1. DKLS
(2022)
Danish Quality Database for Cervical Cancer Screening

DKLS Report 2021.
- Google Scholar
Report
1. DKMS
(2021)
Danish Quality Database for Mammography Screening

DKMS Report.
- Google Scholar
Website
1. DSAM
(2022a) Suggestion to limit the spread of disease in private practices - yellow-green list
Accessed April 8, 2022.

https://www.dsam.dk/files/366/gul_groen_liste_020420.pdf
Website
1. DSAM
(2022b) News on Coronavirus - DSAM - Danish Society for Flere nyheder vedr. coronavirus
Accessed February 14, 2022.

https://www.dsam.dk/forsiden/coronavirus/flere_nyheder_vedr_coronavirus/
1. Feletto E
2. Grogan P
3. Nickson C
4. Smith M
5. Canfell K
(2020) How has COVID-19 impacted cancer screening? adaptation of services and the future outlook in australia
Public Health Research & Practice 30:3042026.
https://doi.org/10.17061/phrp3042026
- PubMed
- Google Scholar
(2005) Do nonattenders in mammography screening programmes seek mammography elsewhere?
International Journal of Cancer 113:464–470.
https://doi.org/10.1002/ijc.20604
- PubMed
- Google Scholar
1. Kotzur M
2. McCowan C
3. Macdonald S
4. Wyke S
5. Gatting L
6. Campbell C
7. Weller D
8. Crighton E
9. Steele RJC
10. Robb KA
(2020) Why colorectal screening fails to achieve the uptake rates of breast and cervical cancer screening: a comparative qualitative study
BMJ Quality & Safety 29:482–490.
https://doi.org/10.1136/bmjqs-2019-009998
- PubMed
- Google Scholar
(2017) Tscount: an R package for analysis of count time series following generalized linear models
Journal of Statistical Software 82:1–51.
https://doi.org/10.18637/jss.v082.i05
- Google Scholar
Software
1. López-de-Lacalle J
(2019) Tsoutliers: detection of outliers in time series, version 0.6-8
CRAN.

https://CRAN.R-project.org/package=tsoutliers
1. Maringe C
2. Spicer J
3. Morris M
4. Purushotham A
5. Nolte E
6. Sullivan R
7. Rachet B
8. Aggarwal A
(2020) The impact of the COVID-19 pandemic on cancer deaths due to delays in diagnosis in England, UK: a national, population-based, modelling study
The Lancet. Oncology 21:1023–1034.
https://doi.org/10.1016/S1470-2045(20)30388-0
- PubMed
- Google Scholar
Website
1. Neergaard ASF
(2022) Danmarks Statistik: COVID-19: Næsten dobbelt så høj dødelighed for mænd
Accessed May 4, 2022.

https://www.dst.dk/da/Statistik/nyheder-analyser-publ/bagtal/2022/2022-05-04-doede-med-covid
(2021) Early assessment of the first wave of the COVID-19 pandemic on cancer screening services: the international cancer screening network COVID-19 survey
Preventive Medicine 151:106642.
https://doi.org/10.1016/j.ypmed.2021.106642
- PubMed
- Google Scholar
Software
1. R Development Core Team
(2021) R: A language and environment for statistical computing, version 1.1.1
R Foundation for Statistical Computing, Vienna, Austria.

https://www.r-project.org/foundation/
Website
1. Region Midtjylland
(2018) ScreeningsNyt
Accessed June 23, 2022.

https://www.screeningsnyt.com
(2020) Hidden morbidities: drop in cancer diagnoses during the COVID-19 pandemic in denmark
Acta Oncologica 60:20–23.
https://doi.org/10.1080/0284186X.2020.1858235
- PubMed
- Google Scholar
Website
1. StatBank Denmark
(2022) Statistikbanken
Accessed June 23, 2022.

https://www.statistikbanken.dk/20015
Website
1. TV2 News
(2022) Flere tusinder har fået udskudt deres operation: - Jeg venter på at blive undersøgt for cancer
Accessed June 23, 2022.

https://nyheder.tv2.dk/samfund/2021-08-11-flere-tusinder-har-faaet-udskudt-deres-operation-jeg-venter-paa-at-blive
1. Vanni G
2. Pellicciaro M
3. Materazzo M
4. Bruno V
5. Oldani C
6. Pistolese CA
7. Buonomo C
8. Caspi J
9. Gualtieri P
10. Chiaravalloti A
11. Palombi L
12. Piccione E
13. Buonomo OC
(2020) Lockdown of breast cancer screening for COVID-19: possible scenario
In Vivo 34:3047–3053.
https://doi.org/10.21873/invivo.12139
- PubMed
- Google Scholar
Website
1. Waldstrøm M
2. Serizawa R
3. Andersen B
4. Frandsen AP
5. Nielsen S
6. Jochumsen K
7. Bennetsen MH
8. Heje HN
9. Ørskov D
10. Laustsen MUB
11. Lynge E
12. Hall PB
13. Andersen ABT
14. Ibfelt EH
(2022) Dansk kvalitetsdatabase for livmoderhalskræftscreenning - resumé af årsrapport 2020. ugeskr Læg
Accessed May 6, 2022.

https://ugeskriftet.dk/nyhed/dansk-kvalitetsdatabase-livmoderhalskraeftscreening-resume-af-arsrapport-2020
1. Walker MJ
2. Meggetto O
3. Gao J
4. Espino-Hernández G
5. Jembere N
6. Bravo CA
7. Rey M
8. Aslam U
9. Sheppard AJ
10. Lofters AK
11. Tammemägi MC
12. Tinmouth J
13. Kupets R
14. Chiarelli AM
15. Rabeneck L
(2021) Measuring the impact of the COVID-19 pandemic on organized cancer screening and diagnostic follow-up care in ontario, canada: a provincial, population-based study
Preventive Medicine 151:106586.
https://doi.org/10.1016/j.ypmed.2021.106586
- PubMed
- Google Scholar
(2022) The impact of COVID-19 on routine medical care and cancer screening
Journal of General Internal Medicine 37:1450–1456.
https://doi.org/10.1007/s11606-021-07254-x
- PubMed
- Google Scholar
Website
1. WHO
(2020) WHO Director-General’s opening remarks at the media briefing on COVID-19
Accessed March 11, 2020.

https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19
1. Wickham H
2. Averick M
3. Bryan J
4. Chang W
5. McGowan L
6. François R
7. Grolemund G
8. Hayes A
9. Henry L
10. Hester J
11. Kuhn M
12. Pedersen T
13. Miller E
14. Bache S
15. Müller K
16. Ooms J
17. Robinson D
18. Seidel D
19. Spinu V
20. Takahashi K
21. Vaughan D
22. Wilke C
23. Woo K
24. Yutani H
(2019) Welcome to the tidyverse
Journal of Open Source Software 4:1686.
https://doi.org/10.21105/joss.01686
- Google Scholar
1. Wilson R
2. Quinn-Scoggins H
3. Moriarty Y
4. Hughes J
5. Goddard M
6. Cannings-John R
7. Whitelock V
8. Whitaker KL
9. Grozeva D
10. Townson J
11. Osborne K
12. Smits S
13. Robling M
14. Hepburn J
15. Moore G
16. Gjini A
17. Brain K
18. Waller J
(2021) Intentions to participate in cervical and colorectal cancer screening during the COVID-19 pandemic: a mixed-methods study
Preventive Medicine 153:106826.
https://doi.org/10.1016/j.ypmed.2021.106826
- PubMed
- Google Scholar

Decision letter

Nicolas Schlecht

Reviewing Editor; Roswell Park Comprehensive Cancer Center, United States
Eduardo L Franco

Senior Editor; McGill University, Canada
Douglas M Puricelli Perin

Reviewer; Leidos Biomedical Research Inc., United States
Johannes Berkhof

Reviewer; Amsterdam UMC Location VUmc, Netherlands

In the interests of transparency, eLife publishes the most substantive revision requests and the accompanying author responses.

Decision letter after peer review:

Thank you for submitting your article "Impact of COVID-19 pandemic on cancer screening in Denmark: A register-based study" for consideration by eLife. Your article has been reviewed by 3 peer reviewers, and the evaluation has been overseen by a Reviewing Editor and Eduardo Franco as the Senior Editor. The following individuals involved in the review of your submission have agreed to reveal their identity: Douglas M Puricelli Perin (Reviewer #1); Johannes Berkhof (Reviewer #2).

As is customary in eLife, the reviewers have discussed their critiques with one another and with the editors. What follows below is the Reviewing Editor's edited compilation of the essential and ancillary points provided by reviewers in their critiques and in their interaction post-review. Please submit a revised version that addresses these concerns directly. Although we expect that you will address these comments in your response letter, we also need to see the corresponding revision clearly marked in the text of the manuscript. Some of the reviewers' comments may seem to be simple queries or challenges that do not prompt revisions to the text. Please keep in mind, however, that readers may have the same perspective as the reviewers. Therefore, it is essential that you attempt to amend or expand the text to clarify the narrative accordingly.

Essential revisions:

1) The reviewers all concur this is a strong article that merits publication with suggested edits to improve clarity and presentation, which should be addressed individually.

2) Appropriateness of the time series model should also be discussed, including alternative approaches that might improve fit, such as using exact dates in line with important events of the pandemic.

Reviewer #1 (Recommendations for the authors):

In this paper, the authors assessed through a time series analysis of the total number of breast and cervical cancer screening tests, stratified by region, in Denmark. The completeness and reliability of the registry databases and adequate modeling tools are strengths of this study, while the reliance on a number of tests as the outcome measure without information about coverage and target populations presents as a weakness.

The authors adequately described the methods used in their research, which would allow researchers to conduct similar and more in-depth studies following these analyses utilizing these databases. The results were presented in a thorough manner, highlighting the immediate impact that the declaration of the COVID-19 pandemic had on people seeking health services, considering that mammography abruptly dropped by 43.3% and cervical cell samples 61.9% in the initial month of the pandemic.

Although I understand the authors' attempt to share the good news about the number of tests coming up after this initial shock, I would argue that they have not quickly recovered to pre-pandemic levels across all regions, especially considering the numbers from 2017 and 2018, a steep decrease in mammography in the second half of 2021, and a downward trend in cervical cancer screening, even if not fully explained by the pandemic.

This work adds to the general literature on cancer screening in the COVID-19 era, and even though we cannot directly infer it from the analysis, this work provides great insight into how a disruptive event such as a pandemic may prevent people from seeking health services during and long after the event, considering that Denmark's services did not stop and the health resources and infrastructure were readily available for cancer screening coupled with strong messaging from government and advocacy groups encouraging people to screen.

Thank you for the opportunity to review your paper as it contributes greatly to understanding the effects of the ongoing COVID-19 pandemic on cancer screening in a country that did not have its screening services interrupted. It is very illuminating, but not unexpected, to see the reduction in the number of breast and cervical cancer screening tests even if services were not interrupted and strongly promoted. I have a few suggestions that hopefully would help strengthen your article:

1. I suggest redefining your study purpose as described on page 3, lines 81-82: "In the present study, we assessed the population's cancer screening uptake during the COVID-19 pandemic in Denmark." It is my understanding that you did not assess patient uptake but looked at the number of tests instead, even though this could be understood as a proxy, so it would more appropriately describe what you did. I would also advise framing it around the research question that your paper is seeking to answer, which I understand as "have we seen effects of the COVID-19 pandemic on mammography and cervical screening considering the number of tests performed nationally and by region?" It would be best to specifically nominate breast and cervical cancer screening since you did not analyze bowel screening.

2. Your methods are thoroughly described and it was particularly helpful to have a brief description of the Danish databases and screening programs. I would like to focus on the bowel cancer screening programs, which you also describe in the methods even though you did not include it in the analysis. On page 4, lines 106-109, you state: "As only activities in the hospitals and the private sector were monitored, activities in the bowel cancer screening program were not included, as this program is based on self-sampled tests." I understand your explanation about why the bowel program was not included but I find that it would be more adequate to say that they were not included because the FIT kits are sent to the person's home instead, considering that self-sample tests could be performed in clinics. In addition, I still wonder about follow-up colonoscopies to positive FIT tests, as I would consider that part of the screening process and a second step in the bowel screening event. Would an analysis of follow-up colonoscopies have been feasible within the Danish databases? If so, I would suggest including them in this paper. Otherwise, I agree on keeping the focus on breast and cervical cancer screening, making that clear in the title and introduction, and would remove the description about the bowel program.

3. In the results, I would suggest strictly reporting the outcomes of the analyses and avoiding making explanatory comments, as you could better expand on those in the discussion. For example, on page 9, lines 199-202, you state: "The activity decline was short-termed; however, it did not reach the previous levels in the first months after the first lockdown. Pre-lockdown levels were reached only after the issue of additional extra reminders for examination." The number of mammographies did recover but it is not obvious to me looking at your data that the decline is short-termed, considering that there was another big drop in 2021 and that the numbers seem to still be going down (although not statistically significant). Also, it is not immediately obvious where the extra reminders fit in your results since you cannot directly attribute the catch-up in the number of tests to the reminders based on the time series analysis only. It would be appropriate to suggest an effect in the discussion, however, as you did, and maybe encourage researchers to look further into that through a survey, for example.

4. Finally, the discussion would benefit from more insights and details regarding the regional differences in the number of screening tests and the downward trend in cervical cancer screening.

On page 19, lines 295-300, you note that "A more detailed analysis showed that this was limited to two of the five Danish regions. We do not know the reason for this phenomenon. Furthermore, there were temporal drop-in activities in four out of five regions. It coincided with a national nurse strike, which is unlikely to be directly linked to the drop, as the strike did not involve nurses employed in GP/gynaecologist practices. It could have affected the minor part of the screening activity that the hospitals undertake." That definitely encourages further research but would you have any suggestions as to why? Could it be exacerbated by the lingering pandemic? Is this trend also seen for other health services within the same area?

Also, the large increase in number of mammography in Region South observed in 2020 and 2021 shown in Table 1 is not explained in the discussion. Was there a migration within Denmark? People moving to other regions considering the pandemic, wanting to be closer to family, now being able to work from home? These are questions that come to my mind as I look at your results.

One final comment, I would avoid a definite statement about COVID-19 not affecting cancer screening in Denmark. I would rephrase the statement between pages 22-23, lines 369-371 to say that the pandemic did have an effect even if temporarily, and I would limit that observation to breast and cervical cancer screening since you did not include an analysis of bowel cancer screening. And in your conclusion, on page 23, lines 396-398, I would not agree that your study shows that screening activity rapidly went back to pre-pandemic level, but that it did recover after a few months from a sudden and steep drop when the pandemic was declared.

Reviewer #2 (Recommendations for the authors):

This paper estimates the effect of the Covid19 lockdown on the number of screens in the breast cancer and cervical cancer screening program in Denmark, where the program was not stopped after the start of the pandemic. The authors give several reasons for a decline in the number of screens in the discussion of their paper, including concern about becoming infected, concern about unnecessary burdening on the health system, etc.

The main strengths of the paper are the high quality of the registry data and the advanced statistical approach for estimating the Covid19-related drop in the screening coverage. A statistical time series model was applied to estimate the change in coverage after the start of the pandemic, adjusted for seasonal variation. The statistical analyses are advanced, but the appropriateness of the statistical model has not been checked. Nevertheless, the model seems to provide strong evidence of a short, steep decline in coverage after the start of the Covid19 pandemic. The authors are very cautious in not overinterpreting the data, for instance, by repeatedly stating that it is difficult to draw causal conclusions based on time series data. A causal interpretation of the results can be strengthened by better connecting the breast cancer and cervical cancer screening coverage data. The two cancers were analyzed and discussed independently but a causal association seems plausible since for both cancers a sharp temporary decline in the number of screens was observed after the start of the pandemic. Moreover, separate time series models fitted to the different regions in Denmark showed a similar decline in coverage right after the start of the pandemic, which again strengthens the interpretation of the presented results. A limitation of the paper is that it does not provide information on the effect of Covid19 on follow-up procedures for screen-positive women.

The data are interesting and the statistical analyses are advanced. The following, mainly methodological comments can be raised.

1. The appropriateness of the GARCH model is not discussed. Why has the GARCH model been chosen and it is appropriate when adjusting the data for seasonal variation? The model does not do a very good job of fitting breast cancer screening data in region North (Figure S1): The decline in the time series data after the start of the pandemic was missed by the model. Therefore, it would be interesting to compare the fit of the GARCH model to other time series models.

2. How are the estimated reductions in the number of screens and their 95% confidence intervals measured?

3. The y-axes of some figures do not include the value 0. This gives an exaggerated picture of the reduction in the number of screens and makes it difficult to compare different figures.

4. Figure 2 is a nice decomposition of the breast cancer screening data. A similar figure might be included for cervical cancer.

5. The interpretation of the cervical cancer plot is incomplete. The strong increase in screens in 2017 is not mentioned in the Results section. Besides, the interpretation of the region-specific cervical cancer plots (S3) is incomplete. The reduction in all regions except region North in July 2021 is mentioned, but the large increase in region North in April 2021 is not mentioned.

6. The discussion about registries in Table 4 seems a bit technical. Maybe it should be moved to the Supplementary Appendix.

7. Did the Covid19 influence follow-up procedures in Denmark? If the number of screens remained the same but follow-up medical procedures (diagnostics + treatment) were postponed, will this not lead to long waiting times that could have been avoided by adjusting the number of screens downwards?

Reviewer #3 (Recommendations for the authors):

In this study, the authors aim to show how mammography screening and cervical cancer screening in Denmark were affected by the covid-19 pandemic. By using data from national registries and analyzing the data using a time-series approach, they find that a drop in attendance during the first lock-down period, although screening was not suspended. They also found that the screening attendance rapidly returned to the pre-pandemic level in 2020.

Strengths:

The use of data from national register data, i.e. data that covers the whole Danish population and is not suffering from selection bias is a major strength of this paper. The data analysis, using a seasonally adjusted time series, identifying seasonal trends and patterns are sound and appropriate methodology for this type of data. The authors have also handled problems with outliers by both analytical and visual inspection.

Weaknesses:

As the authors clearly state, it is not possible to draw causal conclusions from trend analysis. However, with the pandemic having so severe consequences, it is highly likely that the change in the observed data is a consequence of the pandemic. Although the study analyses data from national registries there are some missing data, but not to an extent that would change the conclusions in the paper.

There are two major weaknesses of the paper. Firstly, the trend analysis is very data-driven and the actual dates of the first lock-down are not explicitly taken into the model. The first lockdown was effectuated on March 13th, 2020, while the results presented found a decrease of 22.6% from February 24 to March 22nd, 2020, and a decrease of 43.3% from March 23rd to April 19th, 2020 compared to pre-pandemic levels. The 22.6% drop is then estimated for both pre and during the lockdown, and one would not expect a decrease in February due to the lockdown in March. A more "mix" approach forcing the dates of the actual lock-down in the data analysis would probably give a better estimate of the effect. The second weakness is that subsequent covid measures during 2020 and 2021 such as enforcing masks/face shields in the autumn of 2020, and the subsequent lockdown at the end of 2021 are not mentioned in the paper.

The abstracts lack information on the data analysis and the conclusion should rather focus on empirical findings rather than the Danish policy of continuing screening.

The authors should consider using the exact date of the lockdown in the data analysis rather than being fully data-driven, or at least trying to explain a reduction in screening activity before the lockdown. The comparisons in yearly screening, i.e. comparing 2019 with 2020, should rather be broken down more in line with important dates of the epidemic.

https://doi.org/10.7554/eLife.81605.sa1

Author response

Essential revisions:

1) The reviewers all concur this is a strong article that merits publication with suggested edits to improve clarity and presentation, which should be addressed individually.

Thank you for the positive comments.

2) Appropriateness of the time series model should also be discussed, including alternative approaches that might improve fit, such as using exact dates in line with important events of the pandemic.

For discussion on the statistical method, see Answer 1 under Reviewer 2 below.

Reviewer #1 (Recommendations for the authors):

In this paper, the authors assessed through a time series analysis of the total number of breast and cervical cancer screening tests, stratified by region, in Denmark. The completeness and reliability of the registry databases and adequate modeling tools are strengths of this study, while the reliance on a number of tests as the outcome measure without information about coverage and target populations presents as a weakness.

The authors adequately described the methods used in their research, which would allow researchers to conduct similar and more in-depth studies following these analyses utilizing these databases. The results were presented in a thorough manner, highlighting the immediate impact that the declaration of the COVID-19 pandemic had on people seeking health services, considering that mammography abruptly dropped by 43.3% and cervical cell samples 61.9% in the initial month of the pandemic.

Although I understand the authors' attempt to share the good news about the number of tests coming up after this initial shock, I would argue that they have not quickly recovered to pre-pandemic levels across all regions, especially considering the numbers from 2017 and 2018, a steep decrease in mammography in the second half of 2021, and a downward trend in cervical cancer screening, even if not fully explained by the pandemic.

This work adds to the general literature on cancer screening in the COVID-19 era, and even though we cannot directly infer it from the analysis, this work provides great insight into how a disruptive event such as a pandemic may prevent people from seeking health services during and long after the event, considering that Denmark's services did not stop and the health resources and infrastructure were readily available for cancer screening coupled with strong messaging from government and advocacy groups encouraging people to screen.

Thank you for these reflections. Please see our responses to the points below.

Thank you for the opportunity to review your paper as it contributes greatly to understanding the effects of the ongoing COVID-19 pandemic on cancer screening in a country that did not have its screening services interrupted. It is very illuminating, but not unexpected, to see the reduction in the number of breast and cervical cancer screening tests even if services were not interrupted and strongly promoted. I have a few suggestions that hopefully would help strengthen your article:

Thank you for your positive and very helpful comments.

1. I suggest redefining your study purpose as described on page 3, lines 81-82: "In the present study, we assessed the population's cancer screening uptake during the COVID-19 pandemic in Denmark." It is my understanding that you did not assess patient uptake but looked at the number of tests instead, even though this could be understood as a proxy, so it would more appropriately describe what you did. I would also advise framing it around the research question that your paper is seeking to answer, which I understand as "have we seen effects of the COVID-19 pandemic on mammography and cervical screening considering the number of tests performed nationally and by region?" It would be best to specifically nominate breast and cervical cancer screening since you did not analyze bowel screening.

Answer 1. We agree to specify the study purpose as follows, on page 3 line 81-83:

“In the present study, we assessed the effect of the COVID-19 pandemic on the number of tests performed in mammography and cervical screening, nationally and by region.”

2. Your methods are thoroughly described and it was particularly helpful to have a brief description of the Danish databases and screening programs. I would like to focus on the bowel cancer screening programs, which you also describe in the methods even though you did not include it in the analysis. On page 4, lines 106-109, you state: "As only activities in the hospitals and the private sector were monitored, activities in the bowel cancer screening program were not included, as this program is based on self-sampled tests." I understand your explanation about why the bowel program was not included but I find that it would be more adequate to say that they were not included because the FIT kits are sent to the person's home instead, considering that self-sample tests could be performed in clinics. In addition, I still wonder about follow-up colonoscopies to positive FIT tests, as I would consider that part of the screening process and a second step in the bowel screening event. Would an analysis of follow-up colonoscopies have been feasible within the Danish databases? If so, I would suggest including them in this paper. Otherwise, I agree on keeping the focus on breast and cervical cancer screening, making that clear in the title and introduction, and would remove the description about the bowel program.

Thank you for your suggestion. Sadly, the follow-up colonoscopy data were not part of the Danish health care monitoring system from which we extracted data. Therefore, we have followed your suggestion and removed the description of the bowel program. And added the text:

Setting, page 4, line 98-101:

“Lastly, biennially screening for bowel cancer. As the this program is based on faecal immunological test kits sent to people’s home, the program was not part of the Danish health care monitoring system from which we extracted data.”

3. In the results, I would suggest strictly reporting the outcomes of the analyses and avoiding making explanatory comments, as you could better expand on those in the discussion. For example, on page 9, lines 199-202, you state: "The activity decline was short-termed; however, it did not reach the previous levels in the first months after the first lockdown. Pre-lockdown levels were reached only after the issue of additional extra reminders for examination." The number of mammographies did recover but it is not obvious to me looking at your data that the decline is short-termed, considering that there was another big drop in 2021 and that the numbers seem to still be going down (although not statistically significant). Also, it is not immediately obvious where the extra reminders fit in your results since you cannot directly attribute the catch-up in the number of tests to the reminders based on the time series analysis only. It would be appropriate to suggest an effect in the discussion, however, as you did, and maybe encourage researchers to look further into that through a survey, for example.

Answer 3. We have removed the sentence as suggested from the results on page 9, and added the following text in the discussion on page 19, line 294-298:

“For mammography screening, our data showed activity decline in relation to the first lockdown and that the pre-lockdown levels were reached after several month. This could be due to the issue of extra reminders for screening, but further research would be needed to analyse the association, e.g. through a survey. Further, we saw a steep decrease in the screening activity during the second half of 2021, limited to two of the five Danish regions.”

4. Finally, the discussion would benefit from more insights and details regarding the regional differences in the number of screening tests and the downward trend in cervical cancer screening.

During the last ten years, Denmark has experienced a slight general decline in participation in cervical screening, which is also reported by the Danish Quality Database for Cervical Cancer Screening.

We have added the following text to the Discussion section:

Main finding, page 19, line 283-287:

“During the last ten years, Denmark has experienced a slight general decline in participation in cervical screening. The health authorities are aware of this tendency, and self-sampling kits for HPV-testing are currently being rolled out as an alternative to clinician collected samples (Danish Quality Database for Cervical Cancer Screening. DKLS Report 2021; 2022).”

Line 288-289:

“Further in-depth analyses are required to explain this phenomenon.”

On page 19, lines 295-300, you note that "A more detailed analysis showed that this was limited to two of the five Danish regions. We do not know the reason for this phenomenon. Furthermore, there were temporal drop-in activities in four out of five regions. It coincided with a national nurse strike, which is unlikely to be directly linked to the drop, as the strike did not involve nurses employed in GP/gynaecologist practices. It could have affected the minor part of the screening activity that the hospitals undertake." That definitely encourages further research but would you have any suggestions as to why? Could it be exacerbated by the lingering pandemic? Is this trend also seen for other health services within the same area?

After the nurse strike, there have been a notable increase in nurses leaving the Danish Health Care system, with the theoretical consequence of affecting several other activities. And this could potentially be exacerbated by the lingering pandemic, as the toll on the remaining nurses is substantial. However, it has not been possible to find any research regarding this for the time being.

Also, the large increase in number of mammography in Region South observed in 2020 and 2021 shown in Table 1 is not explained in the discussion. Was there a migration within Denmark? People moving to other regions considering the pandemic, wanting to be closer to family, now being able to work from home? These are questions that come to my mind as I look at your results.

To the best of our knowledge, there has not been a notable migration within Denmark under COVID-19. However, your comments made us aware of a mistake, we had made in Table 1. We had unfortunately reported the numbers for Region South for year 2017 to 2019 in the row for Region Zealand and vice versa. This has now been corrected and we deeply apologise for this mistake. We have afterwards looked through all tables and checked the numbers again to make sure that a similar mistake was not made elsewhere, and some minor mistakes in numbers have been adjusted.

There is therefore no longer a large increase in the number of mammographies in Region South.

One final comment, I would avoid a definite statement about COVID-19 not affecting cancer screening in Denmark. I would rephrase the statement between pages 22-23, lines 369-371 to say that the pandemic did have an effect even if temporarily, and I would limit that observation to breast and cervical cancer screening since you did not include an analysis of bowel cancer screening. And in your conclusion, on page 23, lines 396-398, I would not agree that your study shows that screening activity rapidly went back to pre-pandemic level, but that it did recover after a few months from a sudden and steep drop when the pandemic was declared.

We have thought-out the manuscript changed the focus to breast and cervical screening, as suggested. Moreover, taking your comments on the effect on screening in Denmark into account, we have changed the wording on pages 23-24, line 378-381. We suggest the following changes:

“Our analysis showed a steep, temporal drop in breast and cervical screening activity following the WHO declaration of the pandemic and the simultaneous lockdown of the Danish society, indicating that COVID-19 had a limited and temporary effect on breast and cervical screening in Denmark.”

We have changed the last part of the conclusion on page 24, line 406-408 to the following:

“Our study showed that screening activity dropped suddenly at the time of the first lockdown of the society but recovered in the following months and went back to the pre-pandemic level.”

According to your suggestion, we made the following chances to the conclusion in the abstract to fit with the conclusion in the manuscript:

“Denmark continued screening during the pandemic, but following the first lockdown a temporary drop was seen in breast and cervical screening activity.”

Reviewer #2 (Recommendations for the authors):

This paper estimates the effect of the Covid19 lockdown on the number of screens in the breast cancer and cervical cancer screening program in Denmark, where the program was not stopped after the start of the pandemic. The authors give several reasons for a decline in the number of screens in the discussion of their paper, including concern about becoming infected, concern about unnecessary burdening on the health system, etc.

The main strengths of the paper are the high quality of the registry data and the advanced statistical approach for estimating the Covid19-related drop in the screening coverage. A statistical time series model was applied to estimate the change in coverage after the start of the pandemic, adjusted for seasonal variation. The statistical analyses are advanced, but the appropriateness of the statistical model has not been checked. Nevertheless, the model seems to provide strong evidence of a short, steep decline in coverage after the start of the Covid19 pandemic. The authors are very cautious in not overinterpreting the data, for instance, by repeatedly stating that it is difficult to draw causal conclusions based on time series data. A causal interpretation of the results can be strengthened by better connecting the breast cancer and cervical cancer screening coverage data. The two cancers were analyzed and discussed independently but a causal association seems plausible since for both cancers a sharp temporary decline in the number of screens was observed after the start of the pandemic. Moreover, separate time series models fitted to the different regions in Denmark showed a similar decline in coverage right after the start of the pandemic, which again strengthens the interpretation of the presented results. A limitation of the paper is that it does not provide information on the effect of Covid19 on follow-up procedures for screen-positive women.

Thank you for these reflections. Please see our responses to the points below.

The data are interesting and the statistical analyses are advanced. The following, mainly methodological comments can be raised.

Thank you for your positive comments.

1. The appropriateness of the GARCH model is not discussed. Why has the GARCH model been chosen and it is appropriate when adjusting the data for seasonal variation? The model does not do a very good job of fitting breast cancer screening data in region North (Figure S1): The decline in the time series data after the start of the pandemic was missed by the model. Therefore, it would be interesting to compare the fit of the GARCH model to other time series models.

In order to check the validity of the model, a number of tests were performed:

– Before the model fitting, the autocorrelations and partial autocorrelations of the seasonal-adjusted time series were analysed, in order to select the most appropriate model (i.e. previous observations and conditional means to be regressed on).

– The fitted model was evaluated through cumulative periodogram of Pearson residuals, non-randomized PIT histogram and marginal calibration plot. These tests were all considered passed by visual inspection.

– The model’s residuals were checked for normality through normal QQ plot and Shapiro-Wilk test. By both the above tests, all residuals appeared normally distributed

The above was not reported in the manuscript to reduce technicalities.

Regarding the North Region, in our interpretation the failure to detect the decline after the start of the pandemic is mostly due to the wide oscillations in the time series before the pandemic, which somehow prevented the tests for outliers to detect the drop in the early pandemic period.

Nonetheless, we agree with the Reviewer that it would be interesting to compare the GARCH model with other time series models, which in principle might fit the data better. However, we believe that such comparison would go far beyond the aims of the present study. We added to the limitations of the study the following, page 21 line 337-341:

“Finally, even though a number of tests have been carried out to check the validity of our analyses (i.e. analysis of (partial) autocorrelation functions, periodogram of Pearson residuals, non-randomized PIT histogram, marginal calibration plot, normality of residuals), our results should be further checked in different settings, such as different time series models and/or using data with higher time resolution (weekly, daily).”

2. How are the estimated reductions in the number of screens and their 95% confidence intervals measured?

Outliers in the seasonal-adjusted time series, once detected, are manually inserted into the regression model as covariates. Estimated reductions are given by 1-exp(β_i) (as the models have been chosen with a logarithmic link function), where β_i is the coefficient of the i-th covariate. The 95% confidence intervals are then obtained from the β’s standard errors using normal approximation. The following has been added to the text on page 8, line 180-182:

“Estimated reductions in the number of screens were obtained from the coefficients of the relevant “intervention” covariate of the model. The corresponding 95% confidence intervals are calculated from the coefficients’ standard error using normal approximation.”

3. The y-axes of some figures do not include the value 0. This gives an exaggerated picture of the reduction in the number of screens and makes it difficult to compare different figures.

Figures have been changed.

4. Figure 2 is a nice decomposition of the breast cancer screening data. A similar figure might be included for cervical cancer.

A new figure was added, kindly see page 16.

We have furthermore deleted the previous Figure 3, as this information was also included in Figure 4, which is now instead Figure 3. Seasonal-adjusted time series and fitted model of the number of mammography screens, Denmark 2017-2021. There is equal numbers of figures related to cervical and mammography screenings, respectively.

5. The interpretation of the cervical cancer plot is incomplete. The strong increase in screens in 2017 is not mentioned in the Results section. Besides, the interpretation of the region-specific cervical cancer plots (S3) is incomplete. The reduction in all regions except region North in July 2021 is mentioned, but the large increase in region North in April 2021 is not mentioned.

The increase in cervical screening activity in 2017 was due to a special initiative for screening of elderly women. The following text has been added on page 14, line 244-245:

“The special initiative for screening elderly women was issued in 2017, which the model illustrates in Figure 4-6 and Table 1”

Even though Region North had a large increase in April 2021, the model detected an annual reduction from 9 September 2019 until 2 January 2022. We do not have an explanation for the large increase in April 2021. However, there is the possibility that it is an artifact in the model.

6. The discussion about registries in Table 4 seems a bit technical. Maybe it should be moved to the Supplementary Appendix.

Table 4 has been moved to Supplementary File.

7. Did the Covid19 influence follow-up procedures in Denmark? If the number of screens remained the same but follow-up medical procedures (diagnostics + treatment) were postponed, will this not lead to long waiting times that could have been avoided by adjusting the number of screens downwards?

We understand the interest in the possible effect of COVID-19 on the follow-up procedures, however the analysis was not possible based on the Danish health care monitoring data. However, we have added the following text, page 20 line 307-313:

“In our analysis, we focused on the potential effect of the COVID-19 epidemic on breast and cervical screening activity. One can speculate that the pandemic might have affected waiting times for follow-up and treatment of detected lesions. To answer this question further in-depth analysis of health care data would be needed. However, it should be stressed that reminder systems for follow-up of abnormal findings were in place also during the pandemic.”

Reviewer #3 (Recommendations for the authors):

In this study, the authors aim to show how mammography screening and cervical cancer screening in Denmark were affected by the covid-19 pandemic. By using data from national registries and analyzing the data using a time-series approach, they find that a drop in attendance during the first lock-down period, although screening was not suspended. They also found that the screening attendance rapidly returned to the pre-pandemic level in 2020.

Strengths:

The use of data from national register data, i.e. data that covers the whole Danish population and is not suffering from selection bias is a major strength of this paper. The data analysis, using a seasonally adjusted time series, identifying seasonal trends and patterns are sound and appropriate methodology for this type of data. The authors have also handled problems with outliers by both analytical and visual inspection.

Weaknesses:

As the authors clearly state, it is not possible to draw causal conclusions from trend analysis. However, with the pandemic having so severe consequences, it is highly likely that the change in the observed data is a consequence of the pandemic. Although the study analyses data from national registries there are some missing data, but not to an extent that would change the conclusions in the paper.

There are two major weaknesses of the paper. Firstly, the trend analysis is very data-driven and the actual dates of the first lock-down are not explicitly taken into the model. The first lockdown was effectuated on March 13th, 2020, while the results presented found a decrease of 22.6% from February 24 to March 22nd, 2020, and a decrease of 43.3% from March 23rd to April 19th, 2020 compared to pre-pandemic levels. The 22.6% drop is then estimated for both pre and during the lockdown, and one would not expect a decrease in February due to the lockdown in March. A more "mix" approach forcing the dates of the actual lock-down in the data analysis would probably give a better estimate of the effect. The second weakness is that subsequent covid measures during 2020 and 2021 such as enforcing masks/face shields in the autumn of 2020, and the subsequent lockdown at the end of 2021 are not mentioned in the paper.

Thank you for these reflections. Please see our responses to the points below.

The abstracts lack information on the data analysis and the conclusion should rather focus on empirical findings rather than the Danish policy of continuing screening.

Thank you for making us aware of this. We adjusted the abstract to better fit the scope of the manuscript as follows:

Method:

A time series analysis was carried out to discover possible trends and outliers in the screening activities in the period 2017-2021.

Conclusions

“Denmark continued cancer screening during the pandemic, but following the first lockdown a temporary drop was seen in breast and cervical screening activity”.

The authors should consider using the exact date of the lockdown in the data analysis rather than being fully data-driven, or at least trying to explain a reduction in screening activity before the lockdown. The comparisons in yearly screening, i.e. comparing 2019 with 2020, should rather be broken down more in line with important dates of the epidemic.

We agree with the Reviewer that it would be much more informative to use exact dates; however, two limitations prevented us to do so. (1) the data have been provided to us on a weekly base. In principle, we could have used the weekly date to have a better time resolution. However, (2) when using weekly data, the seasonal decomposition procedure, used to isolate the normal seasonal variations (mainly due to holidays), failed to identify moving holidays (e.g. Easter) or holidays falling on different weeks in different municipalities (e.g. Winter vacation). These problems were solved by aggregating the data on a monthly base. We added the following to the limitations of the study (this include changes to account for question 1 of Reviewer #2):

https://doi.org/10.7554/eLife.81605.sa2

Article and author information

Author details

Mette Hartmann Nonboe

Center for Epidemiological Research, Nykøbing Falster Hospital, Nykøbing Falster, Denmark

Contribution
Data curation, Investigation, Writing – original draft, Project administration

For correspondence
menon@regionsjaelland.dk

Competing interests
received HPV-test-kits free of charge for a method study from Roche. The author has no other competing interests to declare

"This ORCID iD identifies the author of this article:" 0000-0003-0085-9109
George Napolitano

Department of Public Health, University of Copenhagen, Copenhagen, Denmark

Contribution
Software, Formal analysis, Investigation, Visualization, Methodology, Writing – review and editing

Competing interests
No competing interests declared
Jeppe Bennekou Schroll

Department of Gynecology and Obstetrics, Odense University Hospital, Odense, Denmark

Contribution
Validation, Writing – review and editing

Competing interests
No competing interests declared
Ilse Vejborg

Department of Breast Examinations, Copenhagen University Hospital Herlev/Gentofte, Copenhagen, Denmark

Contribution
Validation, Writing – review and editing

Competing interests
No competing interests declared
Marianne Waldstrøm

Department of Pathology, Aarhus University Hospital, Aarhus N, Denmark

Contribution
Validation, Writing – review and editing

Competing interests
No competing interests declared
Elsebeth Lynge

Center for Epidemiological Research, Nykøbing Falster Hospital, Nykøbing Falster, Denmark

Contribution
Conceptualization, Resources, Supervision, Funding acquisition, Writing – original draft, Writing – review and editing

Competing interests
received HPV-test kits from Roche for a method study. The remaining authors have no conflicts of interest to declare

Funding

Region Zealand (R22-A597)

Elsebeth Lynge

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Acknowledgements

The Danish Health Data Authority provided data for the study. Region Zealand financially supported the study (grant number: R22-A597). The funder had no impact on the study design and interpretation of data.

Senior Editor

Eduardo L Franco, McGill University, Canada

Reviewing Editor

Nicolas Schlecht, Roswell Park Comprehensive Cancer Center, United States

Reviewers

Douglas M Puricelli Perin, Leidos Biomedical Research Inc., United States
Johannes Berkhof, Amsterdam UMC Location VUmc, Netherlands

Publication history

Received: July 4, 2022
Preprint posted: January 11, 2023 (view preprint)
Accepted: February 14, 2023
Version of Record published: March 21, 2023 (version 1)

Copyright

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

195

Page views
29

Downloads
0

Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Mendeley

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

Mette Hartmann Nonboe
George Napolitano
Jeppe Bennekou Schroll
Ilse Vejborg
Marianne Waldstrøm
Elsebeth Lynge

(2023)

Impact of COVID-19 pandemic on breast and cervical cancer screening in Denmark: A register-based study

eLife 12:e81605.

https://doi.org/10.7554/eLife.81605

Categories and tags

Research organism

Human

Share this article

Cite this article

Number of mammography and cervical screens by year and region, Denmark 2017–2021.

Weekly number of mammography screens, Denmark 2017–2021.

Decomposition of time series in number of mammography screens into seasonal, trend, and irregular component, Denmark 2017–2021.

Seasonal-adjusted time series and fitted model of the number of mammography screens, Denmark 2017–2021.

Estimated changes in number of seasonally adjusted mammography screens.

Time series of the weekly number of cervical, Denmark 2017–2021.

Decomposition of time series in number of cervical cancer screens into seasonal, trend, and irregular component, Denmark 2017–2021.

Seasonal-adjusted monthly time series and fitted model of the number of cervical cancer screens, Denmark 2017–2021.

Estimated changes in number of seasonally adjusted cervical cell samples.

Author details

Mette Hartmann Nonboe

Contribution

For correspondence

Competing interests

George Napolitano

Contribution

Competing interests

Jeppe Bennekou Schroll

Contribution

Competing interests

Ilse Vejborg

Contribution

Competing interests

Marianne Waldstrøm

Contribution

Competing interests

Elsebeth Lynge

Contribution

Competing interests

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

Categories and tags

Research organism

Further reading