1. Epidemiology and Global Health
Download icon

Estimates of the global burden of Japanese encephalitis and the impact of vaccination from 2000-2015

  1. Tran Minh Quan
  2. Tran Thi Nhu Thao
  3. Nguyen Manh Duy
  4. Tran Minh Nhat
  5. Hannah Clapham  Is a corresponding author
  1. Oxford University Clinical Research Unit, Wellcome Trust Asia Program, Viet Nam
  2. Biological Science Department, University of Notre Dame, United States
  3. Virology Department, Institute of Virology and Immunology, University of Bern, Switzerland
  4. Centre for Tropical Medicine and Global Health, Nuffield Department of Medicine, University of Oxford, United Kingdom
  5. Saw Swee Hock School of Public Health, National University of Singapore, Singapore
Research Article
  • Cited 0
  • Views 420
  • Annotations
Cite this article as: eLife 2020;9:e51027 doi: 10.7554/eLife.51027

Abstract

Japanese encephalitis (JE) is a mosquito-borne disease, known for its high mortality and disability rate among symptomatic cases. Many effective vaccines are available for JE, and the use of a recently developed and inexpensive vaccine, SA 14-14-2, has been increasing over the recent years particularly with Gavi support. Estimates of the local burden and the past impact of vaccination are therefore increasingly needed, but difficult due to the limitations of JE surveillance. In this study, we implemented a mathematical modelling method (catalytic model) combined with age-stratifed case data from our systematic review which can overcome some of these limitations. We estimate in 2015 JEV infections caused 100,308 JE cases (95% CI: 61,720–157,522) and 25,125 deaths (95% CI: 14,550–46,031) globally, and that between 2000 and 2015 307,774 JE cases (95% CI: 167,442–509,583) were averted due to vaccination globally. Our results highlight areas that could have the greatest benefit from starting vaccination or from scaling up existing programs and will be of use to support local and international policymakers in making vaccine allocation decisions.

Introduction

Japanese encephalitis (JE) is caused by Japanese encephalitis virus (JEV) – an arbovirus that belongs to the flavivirus genus, family flaviviridae. The main vectors are mosquitoes of the Culex genus, especially Culex tritaeniorhynchus. These mosquitoes thrive in rice-paddy fields (Buescher and Scherer, 1959; Self et al., 1973). JEV has a wide range of vertebrate hosts, noticeably the amplifying hosts are thought to be pigs and wading birds (SAGE Working Group on Japanese encephalitis vaccines, 2014). Humans are dead-end hosts as viremia is not believed to reach levels that are infectious to mosquitoes (SAGE Working Group on Japanese encephalitis vaccines, 2014). Only 1 in 25 to 1 in 1000 infections result in symptoms (Vaughn and Hoke, 1992; SAGE Working Group on Japanese encephalitis vaccines, 2014). However, the mortality rate of symptomatic cases is high - around 20–30% (Fischer et al., 2008), and around 30–50% of survivors experience significant neurological and psychiatric sequelae (Fischer et al., 2008).

The first JE case was documented in Japan in 1871 (WHO, 2015). In 1924, a first JE outbreak in Japan caused more than 6, 000 cases and 3, 000 deaths in 6 weeks (Solomon, 2006). Several outbreaks occurred subsequently in Asia (Hullinghorst, 1951; Erlanger et al., 2009; Barzaga, 1990). More recently, in 2005 large outbreaks occurred in northern India and Nepal, with 5000 cases and 1300 deaths (Solomon, 2006). Currently, 24 Asia-Pacific countries are thought to be endemic for JE, with 3 billion individuals at risk of infection (WHO, 2015).

The first vaccine was an inactivated mouse brain vaccine produced in Japan, used worldwide for 50 years. Although vaccine production halted in 2006, similar inactivated mouse brain vaccines are still produced locally in South Korea, Taiwan, Thailand and Vietnam (Yun and Lee, 2014). The use of the next vaccine, an inactivated a Vero cell vaccine (SAGE Working Group on Japanese encephalitis vaccines, 2014), has been gradually replaced (since 1988) by a live attenuated vaccine (SA 14-14-2) produced in China, with PATH support. SA 14-14-2 is now widely used in Asia and funded by Gavi which has led to a great increase in vaccination. This vaccine requires only a single dose, is cheap to produce, and is safer than the mouse brain vaccine (SAGE Working Group on Japanese encephalitis vaccines, 2014). In addition, a live attenuated chimeric vaccine was first licensed in Australia in 2012 (SAGE Working Group on Japanese encephalitis vaccines, 2014).

WHO recommends two JE surveillance systems that are important for monitoring burdens of JE and changes over time (WHO, 2019), (i) a subnational system with sentinel hospitals, or (ii) case-based nationwide surveillance. Each country implements one of these systems depending on available resources (Hills et al., 2009). WHO recommends diagnosis using JEV-specific IgM antibody-capture enzyme-linked immunosorbent assay (MAC-ELISA) in CSF at two time points (Donadeu et al., 2009; Burke and Leake, 1988). Serum samples can be used, but false positives may result from cross-reactivity with other flaviviruses or vaccination (Solomon et al., 1998; Hills et al., 2009). Other tests that can confirm JE are plaque reduction neutralizing (PRNT), haemagglutination inhibition (HI), immunohistochemistry or immunofluorescence assay, reverse transcription polymerase chain reaction (RT-PCR) or virus isolation (Hills et al., 2009), though these are not often used.

The previous estimate of annual global JE cases was 67,900 with 13,600–20,400 deaths (Campbell et al., 2011). For this estimate a systematic review in 2011 collated case incidence data from endemic JE countries. Countries were then stratified into 10 incidence groups (Group A, B, C1-2 and D-I) based on geographic, ecological and vaccine program similarities. The systematic review resulted in 12 key studies, which were then used to infer the incidence rate (IR) of the 10 incidence groups. However the estimation had some limitations; the surveillance quality of the 12 key studies varied, and as the case incidence rate combines both the infection rate and vaccination, (e.g. a low risk of infection with no vaccination could have a similar incidence as a high risk of infection but with high vaccination coverage), it is not possible to estimate the impact of vaccination.

The use of age-stratified case data to infer the FOI has been of use recently for dengue (Imai et al., 2016; Cattarino et al., 2020; Rodriguez-Barraquer et al., 2019). The advantage of this method is that the age distribution will be insensitive to differential reporting or tests used in different places, and the important information from the age distribution remains; the higher the rate of infection, the earlier in life individuals will acquire infection. By fitting models of the infection process (including acquisition of immunity) to this age-stratified data we can quantify this rate of acquiring infection, known as the force of infection (FOI) (Hens et al., 2010).

Poor clinical outcomes and lack of specific treatment makes JE prevention a priority. Vaccination is the most effective method of prevention, however it is difficult to decide where vaccination should be implemented or to estimate the quantitative impact of vaccination (Fischer et al., 2008). In Nepal, one study estimated 3,011 JE cases were prevented in vaccinated districts from 2006 to 2012 (Upreti et al., 2017). Another study in Sarawak Malaysia estimated a 61% reduction in JE cases after the vaccination program, where climate effects were not taken into account, and 45% when the effects of climate were included (Impoinvil et al., 2013). The methods used in both these papers require good surveillance data before and after vaccination, which, though data are improving, are currently not widely available. Hence, new approaches are needed to estimate burden and vaccine impact.

In this study, we provide updated global JE burden and vaccination impact estimates using a modelling method which helps overcome some of the limitations of sparse and variable surveillance data. In addition, by simulating the model with and without the undertaken vaccination programs we are able to estimate the impact of vaccination on the number of global JE cases to date and identify areas that would benefit most from future vaccination.

Results

There are two main stages to our analysis, summarized in flowcharts in Figure 1. In the fstage I, we conducted a systematic review to collate age-stratified case data and a literature review to obtain vaccination information. We then fit a model to this data to estimate the transmission intensity or force of infection (FOI) for each study. In stage II, we extrapolated the FOI from our previous estimates to all endemic areas. Using the processed population and vaccination data in all endemic areas, we used the model to generate burden quantities (cases) in two scenarios, with or without the JE vaccination programs that have been implemented.

Flowchart describes two main stages in our analysis: Estimating FOI (force of infection) and generating burden.

In Stage I we estimate FOI (force of infection) of all studies’ catchment area. In Stage II we then used the FOI estimates to generate global burden. Abbreviation: WPP: World Population Prospects.

Systematic review

A systematic review on October 11th 2017 yielded 2337 initial results (Figure 2). 407 relevant studies were obtained after eliminating 1931 irrelevant titles and abstracts that were about molecular biology, policy, entomology, hosts other than humans, or were review papers. The obtained studies mainly comprised of reports of JE surveillance or epidemiological studies in one specific location. We also included modelling, economic evaluation or vaccine program assessment studies for possible eligible data sources in the references. We retrieved and read 261 full-text papers. Most of the papers that we could not access were either old or not in English. In the systematic review process, a further four eligible studies were retrieved from references. 202 papers were then excluded as they did not contain age-stratified case data, and another 14 papers were also excluded because they had limited samples (less than 15 cases) or the study’s catchment area was not clear. Another four datasets from JE national reports were collated from Taiwan, Japan, and Sri Lanka. Finally, we had 53 studies that contained age-stratified case data (Figure 2). 42 of the 53 studies (79%) contained data from after 2000 only, 7 from before 2000 only and three from both time periods (Figure 2—source data 1). 34 studies (64%) had data from 1 to 4 year time periods, six studies had data for periods of between 5 and 9 years, and 11 studies had data for more than 10 years. The majority of the studies used the WHO JE case definition: JE IgM antibody in CSF or serum as confirmed by MAC-ELISA on patients with acute encephalitis syndrome. In the majority of studies patients were recruited from a sentinel hospital surveillance system, though these ranged in size from one to several hospitals. For studies with a consistent catchment area but for which data was collected in multiple years, we aggregated the age-stratified case data across years. Further details of the selected studies and data, including about catchment areas, sample collection methods, and vaccination programs are in Figure 2—source data 1.

Figure 2 with 1 supplement see all
Flowchart describing the systematic review procedure searching for Japanese encephalitis age- stratified case data.

We obtained the vaccination information from three main sources: literature review, WHO, and Gavi (Figure 3—source data 1). Campaign vaccination information was mainly from Gavi and routine vaccination was from WHO, while the literature contains both. When there were disagreements between the different vaccination information sources, we chose to use the information from the literature review. The total vaccinated population in each country from 2000 to 2015 using information obtained from this data is shown in Figure 3 and Figure 3—source data 1. This information was included as a prior in the model fitting (see Figure 4—source data 3).

Reported number of individuals vaccinated in each region from multiple data sources by region from 2000 to 2015.

If the country is not listed there is no vaccination reported. Abbreviations: AUS: Australia, CHN: China, IND: India, JPN: Japan, KHM: Cambodia, KOR: South Korea, LAO: Laos, LKA: Sri Lanka, MYS: Malaysia, NPL: Nepal, PRK: People’s Republic of Korea, THA: Thailand, TLS: Timor-Leste, TWN: Taiwan, VNM: Vietnam. The supplementary file: Figure 3—source data 1 lists the vaccination data and the sources for each country.

Force of infection (FOI) estimation from collated age-stratified data

From 53 studies, we made FOI estimates using the catalytic model from 53 unique catchment areas in 15 countries (Figure 4). Force of infection (FOI) is the per capita rate at which susceptible individuals are infected by an infectious disease and a catalytic model estimates the FOI from age- stratified case data (Hens et al., 2010). All the catalytic models converged well (see convergence plots in Figure 4—source data 4) and fit well to data in all but one study (Figure 4—source data 2 - 95% CIS of model output and case data shown). Our FOI estimates varied from 0.001 (95% CI: 0.000 - 0.002) in Japan to 0.507 (95% CI 0.419 - 0.582) in Guigang in China. Besides those extreme values, FOI were generally between 0.05 and 0.2, with a median of 0.09 (Figure 4). In this model-fitting the reporting rate is the proportion of all infections that are reported. The reporting rate includes both the proportion that are symptomatic and the proportion of those cases that present at each hospital or be counted in each surveillance system, so it is the proportion of infections reported. We therefore observed a wide variation in estimated reporting rates ρ between studies (Figure 4—figure supplement 1). This number is not used in the estimates of cases in the next section. For China, India, Japan, and Nepal, the posterior estimates of the proportion of the population in study k and age group i that remained susceptible after vaccination sk,i, for some age groups was slightly different to the prior population vaccination that was included in the model fitting (Figure 4—source data 3). When the posterior and prior did not agree, for most datasets this suggested missing vaccination data with the prior saying higher susceptibility than the posterior, however for some areas in India the reverse was estimated.

Figure 4 with 2 supplements see all
FOI distribution estimated from all studies’ catchment areas (on the left), each distribution represents FOI from one study, which were used to infer the FOI distribution in all endemic areas (on the right).

The colors are coded after the endemic areas as in the legend. Abbreviation: AUS: Australia, BGD: Bangladesh, CHN: China, IDN: Indonesia, IND: India, JPN: Japan, KHM: Cambodia, KOR: South Korea, LAO: Laos, LKA: Sri Lanka, MYS: Malaysia, NPL: Nepal, PHL: Philippines, RUS: Russia, SGP: Singapore, THA: Thailand, TWN: Taiwan, VNM: Vietnam. Countries have low, medium or high following the classification in Campbell et al., 2011.

Inference of force of infection for all endemic areas

Based on the rules in the methods, we are able to infer FOI from available data for 24 endemic areas (Figure 4—source data 1, and Figure 4) from across the Campbell et al. groupings. In the Campbell et al. grouping, FOI is assumed to be homogeneous across each country, except Indonesia, China and Nepal with low and high groups, and India with a low, medium and high transmission groups. We kept this grouping for our work except for Indonesia, as the collated data for Indonesia was combined across various provinces across both the low and high incidence areas, we assumed the FOI to be the same in both areas. There were no studies from countries in the Campbell et al., 2011 group B (Australia, Pakistan, North Korea, Russia, Singapore and the low incidence region in India). Since this group contains extremely low incidence areas, the FOI was assumed to have a lognormal distribution ln(X)N(0.01,1) (see Figure 4).

Burden and vaccine impact estimation

We estimate the burden from 2000 as the majority of studies used to estimate FOI were from after this time period. We estimate that from 2000 to 2015, there were 1,976,238 (95% CIs: 1,722,533–2,725,647) JE cases globally. By including known annual vaccination information in the catalytic model we estimate that in the same period had there been no vaccination there would have been 2,284,012 (95% CIs: 1,495,964–3,102,542) JE cases. Therefore we estimate that vaccination programs have prevented 307,774 JE cases globally (95% CI: 167,442–509,583) from 2000 to 2015 and vaccination programs similarly prevented 74,769 deaths from JE (95% CIs: 37,837–129,028). We estimate the greatest impact of vaccination from 2005 to 2010 due to large increase in vaccination in China in this time, and the impact of vaccination became more obvious over time (Figure 5). In 2015, we estimate vaccination reduced the number of cases globally by around 45,000 (from 145,542 (95% CI: 96,667–195,639) to 100,308 (95% CI: 61,720–157,522) (Figure 5).

Number of estimated cases with and without vaccination of the 30 endemic areas and of the world from 2000 to 2015.

The two scenarios, with or without vaccination, are also shown in blue and red respectively. In all areas, the boxplots represent the estimated cases with 95% credible intervals (also shown 1 st quartile, 3rd quartile) with the solid lines showing the mean value of each interval. Abbreviation: AUS: Australia, BGD: Bangladesh, BRN: Brunei, BTN: Bhutan, CHN: China, IDN: Indonesia, IND: India, JPN: Japan, KHM: Cambodia, KOR: South Korea, LAO: Laos, LKA: Sri Lanka, MMR: Myanmar, MYS: Malaysia, NPL: Nepal, PAK: Pakistan, PHL: Philippines, PNG: Papua New Guinea, PRK: North Korea, RUS: Russia, SGP: Singapore, THA: Thailand, TLS: Timor-Leste, TWN: Taiwan, VNM: Vietnam.

We estimated the highest number of cases in the high endemic area of China (around 40,000 annual cases in the no vaccination scenario and around 20,000 annual cases in vaccination scenario), and medium or high endemic areas in India (around 20,000 annual cases in no vaccination scenario and 15,000 annual cases in vaccination scenario for each area in recent years). On the contrary, areas like Australia, Brunei, Bhutan and Russia were estimated to have less than 100 annual cases with or without vaccination (Figure 5, Figure 6). All visualized burden estimates for every years and areas can be found in our interactive map (Duy, 2018).

Maps of estimated cases (in thousand) in 30 endemic areas for two scenarios in 2015.

Each endemic area is shaded in proportion to the area’s estimated cases in thousand as seen in the legend, with yellow shade is the lowest value and red shade is the highest value. The map on the left is the estimates from no vaccination scenario, and the right is from the vaccination scenario. The maps were made by leaflet package in R (Joe et al., 2017).

Vaccination impact can be observed in 19 areas where vaccination has been used (Figure 5). In areas like the low and high endemic area in China, medium and high endemic area in India, Cambodia, Laos, Nepal, North Korea, and Timor-Leste though vaccination started recently, we estimate that the programs have achieved significant cases averted. Indeed, in the high endemic area in China, the routine vaccination programs only started in 2008 but contributed the most to the global cases reduction, with around 20,000 cases averted in China in 2015. We also observed a clear difference in cases between vaccination and no vaccination scenario in areas with intensive vaccination programs such as South Korea, Sri Lanka, Thailand, Taiwan, and Vietnam. For Japan, Australia, and Malaysia, though vaccination began a long time ago, we estimated there has been minimal vaccine impact. From the data we collated, no vaccine programs had occurred in in Bangladesh, Brunei, Bhutan, the low and high endemic areas in Indonesia, Myanmar, Pakistan, Philippines, Papua New Guinea, Russia, or Singapore so we estimated of course vaccination has had no impact.

Sensitivity analysis

To assess the impact of uncertainties in our data and assumptions we performed extensive sensitivity analyses. Sensitivity analyses were conducted for endemic areas with uncertain vaccination coverage data, where both national and subnational data were available (China, India and Nepal), or where we did not have any studies. The majority of the results showed minimal changes compared to our original estimates (Figure 5—source data 1). Cases estimated from Taiwan subnational data were higher by about 200 to 400 cases before 2004 (Figure 5—source data 1). In some areas, we observed significant differences in the estimated cases when the vaccination coverage was changed: when the vaccination coverage reduced by 10% and 30% in Sri Lanka or by 30% in Thailand and Taiwan, the mean values of estimated cases increase by around 40, 100, 300, and 220 respectively (Figure 5—source data 1). However these changes account for a small fraction of our original global estimates. Sensitivity analysis varying the assumed 100% vaccine effectiveness to 90% and 70% showed global case estimates changed minimally with this assumption (Figure 5—source data 1). In addition due to concerns about possible changes in FOI over times, we also tested our assumption of constant FOI by fitting multiple-year data to a time-dependent catalytic model. Overall, the annual FOI estimates are comparable with the constant FOI (Figure 5—source data 1).

Discussion

In this paper, we updated the JE burden estimates with a mathematical modelling method using data we collated from a systematic review. We estimated that in 2015 there were around 100,000 JE cases globally. In addition, we estimate that vaccination programs averted around 45,000 JE cases in 2015.

For JE, since humans are dead-end hosts and therefore vaccination does not lead to herd immunity, the FOI we estimate represents the constant spread of the disease from the animal reservoirs to humans. This spread depends on epidemiological factors related to JE transmission such as climate, rural-urban, mosquito distribution (especially Culex tritaeniorhynchus), and pig and rice field distributions (Le Flohic et al., 2013). This explains why our estimated FOI varies widely. Looking crudely at the pig density (Nicolas and Gilbert, 2010) and a Culex tritaeniorhynchus probability maps of Miller et al., 2012 and Longbottom et al., 2017 there appears to be a broad correlation of these factors with our estimates. The high FOI estimated in the south of China, Vietnam, and Philippines is consistent with the high pig density and high probability of Culex in these areas (for Vietnam and Philippines with Miller et al., 2012 only). We also estimated high FOI in India and Indonesia; however these countries only have high probability of Culex (in Miller et al., 2012 but not Longbottom et al.) but low pig density in Longbottom et al., 2017. This suggests that other potential animal reservoirs may contribute to the transmission in these countries, likely the wading bird or even poultry, although current evidence is limited (Lord et al., 2015). In Taiwan and South Korea the current estimated FOI is lower compared to other areas, respectively 0.061 (95% CI 0.013–0.093), and 0.041 (95% CI 0.026–0.057) despite these areas having high probability of Culex mosquito and high pig density. These countries have had high JE burdens in the last 40 years, but we do not estimate so for 2000–2015. This could be due to lack of recent data, or perhaps suggests urbanization, which reduces the proximity of humans to pig farms and rice fields (where the mosquitoes thrive), may play an important role in lowering transmission. This could also be due to uncertainties in the long term vaccination information in these areas. Further work will use environmental covariates to gain estimates of FOI on a smaller spatial scale and over time. In addition, changes in these covariates into the future should be considered in estimates of the future vaccine impact.

A strength of our Bayesian approach was the possibility to include prior information on vaccination, but also assess whether this was consistent with the ages distribution of observed cases. For China and Japan we estimated lower susceptible proportions after vaccination in certain age groups compared to calculated proportions from the available data. This suggests that there are a large number of immunized people in certain age groups due to past vaccination, for which we did not have information. In Nepal and India, we also observed differences between the data and estimated susceptible proportion after vaccination, though the vaccination information for these countries was more readily available. There was still an impact of vaccination- but expected impact on the age distribution in the model fitting was not as extreme as the data we collated would have suggested. For India, this artefact as picked up by the model is consistent with data on vaccine efficacy and vaccination coverage data from India. From 2006 to 2011, SA 14-14-2 vaccine was used in India for campaigns. Though the vaccine reported nearly 100% efficacy in vaccine trials and case-control studies (Kumar et al., 2009; Bista et al., 2001), the efficacy in India was reported to be as low as 30% to 40% (Vashishtha and Ramachandran, 2015; Tandale et al., 2018) and lower seroconversion has also been reported in India (Singh et al., 2015). A previous evaluation of vaccination coverage also showed that the vaccination coverage data in India was lower than reported (Murhekar et al., 2017). Further studies are needed to explore whether there are different vaccine efficacies in different places, particularly India, and to explore possible explanations for this. One possible explanation could be cross-reactive immunity to other flaviviruses, or differences in circulating JE genotypes.

Using the FOI from 30 endemic areas, we projected the regional and global JE burdens as well as the vaccine impact. By region, our burdens estimates are highest in China and India, which aligns with previous literature (Heffelfinger et al., 2017). Our global estimate of around 100,000 cases annually is about 1.5 times higher than the previous estimate of around 70,000 cases (Campbell et al., 2011). Similar patterns are seen for the comparison area by area, in which our case number estimates are either higher than or comparable to the previous estimates (Table 1). It is not surprising that our estimates are higher, since our method more robustly takes into account under-reporting and different surveillance quality. In addition, the numbers we reported here are time-dependent and not static because our estimates include population changes and the progression of vaccination programs over time.

Table 1
Comparing annual case estimates from Campbell et al. to our estimates for the year 2015 (as this was the year of estimation of the previous estimates).

Group A: Taiwan, Japan, South Korea; Group B: Australia, low endemic area in India, Pakistan, Russia, Singapore; Group C1: high endemic area in China; Group C2: low endemic area in China; Group D: Cambodia, high endemic area in Indonesia, Laos, Sabah and Labuan in Malaysia, Myanmar, Philippines, Timor-Leste; Group E: low endemic area in Indonesia, Peninsular Malaysia, Papua New Guinea; Group F: high endemic area in India, high endemic area in Nepal; Group G: Bangladesh, Bhutan, Brunei, low endemic area in Nepal; Group H: Medium endemic area in India, Sarawak in Malaysia, Sri Lanka, Thailand, Vietnam; Group I: North Korea.

Incidence GroupCase numbers:
Previous estimates
Case numbers:
Our no vaccination scenario
Mean estimates (and 95% Cis)
Case numbers:
Our vaccination scenario
Mean estimates (and 95% Cis)
A62,307 (1,175–3,497)863 (453–1,469)
B22595 (388-6,243)2540 (381-6,071)
C133,84938,789 (26,128–51,482)*22,013 (3,778–42,375)*
C22810,752 (7,297–14,152)7,094 (4,230–10,579)
D791713,710 (9,333–18,135)13,700 (9,325–18,125)
E364512,932 (8,804–17,059)12,932 (8,804–17,059)
F12,35022,514 (1,503–36,423)*17,304 (846-27,930)*
G13589,538 (6,322–12,881)9,277 (6,133–12,548)
H807229,942 (17,431–40,933)23,201 (13,647–31,542)
I670465 (77–1,022)*433 (74–912)*
Total67,897143,545 (94,469 – 194,940)109,358 (65,968–156,669)
  1. *Our estimates are comparable to the previous estimates.

Though our methods are more robust, collating 53 studies (an additional 41 from the studies used in the previous burden estimate) (Campbell et al., 2011), and using age-stratified data to circumvent issues with reporting variation, there are still some limitations. As in the previous estimates of JE burden (Campbell et al., 2011), we made inferences for the whole country based on data from a few studies. However in our method we sampled from the FOI estimates from all studies to account for some of this uncertainty and variation. In addition, as in previous studies, a limitation is that we inferred the incidence metric (in our case, FOI) for areas without data, from FOI from other areas, based on previous classification of transmission in these countries. However, our sensitivity analysis shows that this does not alter the global burden estimates greatly, though it may affect the country-specific burden estimates. (Campbell et al., 2011). Our future work incorporating the epidemiological factors into machine learning algorithms to extrapolate the FOI on smaller spatial scales will help in refining these estimates in the future. Similarly, we assume transmission is constant over time. Further work fitting models with time varying forces of infection as well as looking at covariates of infection that are changing over time, will be necessary for future refinements of these estimates. An additional limitation of our analysis is the uncertainty in the proportion of infections that lead to disease, we sample from this range, and this uncertainty is included in our uncertainty analysis. Further studies, for example in cohorts may enable better estimates of this proportion. In addition, we assumed the FOI is constant across age, only susceptibility changes due to acquistion of immunity, further assessment of seroprevalence studies may be able to assess this further. In addition, though our method accounts for reporting rates within these studies, future work should assess the impact of cross-reactivity and further issues with diagnosis on the estimates, such as including AES data in the fitting. In addition, we did not include papers not in English in our literature review. The papers we could not include were all from China and so further work including these papers should enable better estimation of the FOI in China.

We estimated only the impact of vaccination on cases from 2000 to 2015. Because the impact of vaccination will continue into the future as vaccinated individuals remain protected, our estimate will be an underestimate of the total impact of vaccination. In addition, our estimates will be an under-estimate of total vaccine impact as in some places vaccination programs have been running before 2000, and so vaccination had a large impact before 2000. However there is limited information in order to estimate transmission intensity before this time, so we focused our work on 2000–2015. In this paper, we focused on cases (and to some extent deaths) from JE. However because a large number of cases have long-term sequelae after JE infection, focus just on case numbers does not describe fully the total burden of JE. Future work will refine the estimates of the proportion of individuals that die and that experience different long-term sequelae, to generate update our model to estimate JE Disability-Adjusted Life Year (DALY), particularly relevant for use in cost-effectiveness analyses for introduction of vaccination into new locations.

Since JE vaccination does not produce herd immunity, the transmission intensity can only be reduced by influencing the animal transmission cycle. Previous attempts to break the transmission cycle have been vector control and vaccination in pigs and wading birds, and this has been considered in modelling work (Khan et al., 2014). However they were either ineffective or up to now have been deemed economically and logistically intensive (Fischer et al., 2008). Further work considering pig vaccination in the context of these updated estimates of the burden of JE should be considered. We estimate that despite not interrupting transmission, human vaccination can be an effective strategy to reduce JE case numbers. This can be seen from the estimate that the majority of the reduction in global burden is due to the routine vaccination program in China from 2008. We estimate that India, Timor-Leste, and Vietnam also have high transmission intensity, and residual cases despite vaccination, and therefore could further benefit from scaling-up the existing vaccination program (Figure 3—source data 1). We estimated high transmission intensity in Indonesia, Papua New Guinea, and Philippines where there are no current vaccination programs, suggesting that vaccination in these areas should be a future priority. Future smaller scale estimates will support decisions on where within these countries could be best targeted for vaccination. For areas with a long history of JE vaccination (see Figure 3—source data 1). such as South Korea, Sri Lanka, Thailand and Taiwan, (Figure 4), we estimate a substantial vaccine impact (Figure 5), though with cases still occurring. In other countries with a long vaccination history however, we estimate a minimal impact of vaccination (Figures 4 and 5), due to low estimated transmission intensity in Japan, low vaccination coverage in Malaysia, or both in Australia (though age-stratified data were not available in Australia). Our estimate of transmission intensity for Japan also has great uncertainty, as half the studies included data pre-2000 and we were able to find limited information on the long-running vaccination program there. In addition, there are limitations to how well our method will work, given the high rate of vaccination there. This may mean we are under-estimating the impact of vaccination in Japan. Further work with serological data both from humans and animals and further exploration of the drivers of JE transmission will help refine this estimate.

Assessing JE disease burden and vaccination program performance is important though difficult due of the lack solid surveillance programs worldwide. In our paper, we are able to estimate the disease burden and vaccine impact using a modelling method that is able to overcome some of the limitations of current surveillance. We estimate annually there are still 100,000 cases of JE in Asia, making a 2/3 of all cases of this severe but vaccine preventable still not being averted. The majority of remaining cases are focussed in countries with still developing healthcare systems. Given there is a cheap vaccination now available, our results will help with the rational assessment of JE vaccination cost and benefit for each country and will help guide Gavi and other international and national public health agencies in making decision on their future investment into JE vaccination.

Materials and methods

Systematic review

Request a detailed protocol

We performed a systematic review to find all available age-stratified case data for Japanese encephalitis in PubMed. We used the search terms ‘epidemiology’ or ‘incidence’ or ‘prevalence’ or ‘public health’ or ‘surveillance’ or ‘distribution’ in all fields with ‘Japanese encephalitis’ in the title or abstract. All titles and abstracts were screened and we selected those in which the study contained age-stratified case data. We retrieved the full-texts for these selected abstracts and the abstracts were read by two independent reviewers to extract the age-stratified case data. From each study we also collected other information about the catchment areas, sample collection methods, diagnosis tests, and regional vaccination programs from the papers. A final consensus was reached for the final list of eligible full-texts. If abstracts were not available, the two independent individuals also tried to access and examine the full-texts. We also searched online for age-stratified case data from national JE surveillance reports.

We obtained vaccination information either from the study itself or from the literature review. Based on the review of JE vaccination programs reported from the World Health Organization (WHO) (Heffelfinger et al., 2017), we found that previous vaccination programs had occurred in 13 countries. We then undertook a literature search to find all vaccination information (target age group, vaccination coverage, types of vaccine used, years of vaccination) for these countries. We also collated historical routine vaccination program from country reported administrative doses data time series (from 2000 to 2015) compiled from WHO-UNICEF Joint Reporting (World Heath Organization, 2018) and additional data from Gavi.

Force of infection estimation

Request a detailed protocol

Force of infection (FOI) is the per capita rate at which susceptible individuals are infected by an infectious disease. In this study, we used a basic Muench’s catalytic model (Muench, 1958) to estimate the constant age and time independent FOI using the case data we extracted during the systematic review process. A similar approach has been used to estimate the global dengue transmission intensity (Imai et al., 2016; Rodriguez-Barraquer et al., 2019). As humans are dead-end hosts for JE, the FOI represents the FOI from the animal reservoir, and therefore is not impacted by human vaccination. This means vaccination can be included in the model simply as a removal of susceptible individuals by vaccination (or a reduction in risk of infection in this vaccinated group depending on vaccine efficacy) and will not alter the FOI. Therefore in this model, individuals can become immune to infection either by natural infection (depending on the force of infection) or vaccination.

To estimate the FOI (notated as λk), for each study k, taking into account vaccination and reporting rate for each study k, the modelled number of cases in a specific age group i is:

Ek,i=Pk,ipopk,isk,iρk, where

(1) Pk,i=(e||λkak,ileλk(ak,iu+1))

Where Pk,i estimates the incident rate of infection in each age group i (with lower and upper ak,il and ak,iu respectively), accounting for force of infection and susceptibility in that age group due to natural infection before this age. popk,i is the population size in each age group i of each study k, calculated from World Population Prospects 2017 data (United Nations-Department of Economic and Social Affairs-Population Division, 2017). sk,i is the estimated susceptible proportion in each age group i after vaccination for population in study k. The prior distribution of λk was an uninformative non-negative, normal distribution, λkNormal(0,1000). To include the uncertainty in the vaccination information, we used an informative prior: sk,iBeta(Φ(1sk,i),Φsk,i), with sk,i' is the proportion of the population that remain susceptible after vaccination in age group i of study k, calculated from the vaccination information and the population demographics in the study’s catchment area. Φ represents the uncertainty of the vaccination information (we set Φ=5 to account for the possibility that this information was incomplete or did not reflect the actual vaccinations delivered. The chosen uncertainty value represents moderate trust in the vaccination information. The value of 7 or 10 gave a very strong belief, hence not chosen here (these analyses are not shown)) (see Figure 4—source data 2 for priors). ρk is the reporting rate for each study, which is comprised of symptomatic rate and the reporting rate of the surveillance system and accounts for the different surveillance qualities of the different studies. Since ρk contains the symptomatic rate which reported to be less than 1% (SAGE Working Group on Japanese encephalitis vaccines, 2014; Vaughn and Hoke, 1992), we used an informative prior: ρkBeta(0.1,9.9).

The log-likelihood function for each study k is the sum of the multinomial log-likelihood and Poisson log-likelihood of total cases across all age groups.

(2) LkMN+P=log(tk!)ilog(C||k,i!)+iCk,ilog(Ek,iiEk,i)+tklog(iEk,i)iEk,ilog(tk!)

Where tk is the total number of cases and Ck,i is the number of age-stratified cases in age group i in each study k. Ek,i is the modelled number of cases in a specific age group i.

For each dataset, we fit the model in a Bayesian framework in RStan (Stan Development Team, 2016), estimating parameters λk,ρk,sk,i. RStan uses a No-U-Turn sampler (NUTS) (Hoffman and Gelman, 2014), a variant of Hamiltonian Monte Carlo to obtain posterior simulation (Stan Development Team, 2016). The parameters sk,i,ρk were all estimated on a logit scale. We started 4 random chains, each with 16000 iterations and 50% burn-in period. Smaller step size of the Hamiltonian transition was manually set by increasing the adapt delta parameter in RStan to be 0.99. Model convergence was assessed visually.

We assumed that the JE vaccine has 100% effectiveness, which is reasonable given the reported high effectiveness of the vaccine (World Health Organization, 2012a; WHO, 2014; World Health Organization, 2012b) and that the protection acquired from natural infection or vaccination was life-long.

For our estimate, the endemic areas were defined to be the same as in the previous JE burden estimate (Campbell et al., 2011). For China, India, Nepal and Indonesia, where transmission intensity is diverse these countries were broken down to low, medium, or high endemic areas. In total, there are 30 endemic areas, spanning 24 countries. We inferred the FOI for each endemic area based on the FOI estimated from collated studies. The inference was based on two rules: (1) For each area, the FOI was obtained by sampling from the estimated FOI of all the studies that had catchment areas within that endemic area (if any). (2) For endemic areas in which no studies were conducted, the FOI was inferred to be equal to the FOI of the area in the same incidence group defined by Campbell et al., 2011.

Burden and vaccine impact estimation

Request a detailed protocol

Once the distributions of inferred FOIs for each endemic area were obtained, we generated the distributions of the estimates of the number of cases in each year t (from 2000 to 2015) in endemic area d for each age group a from 0 to 99 years old and scenario m (described below) using the function (similar to the model used to estimate FOI (Equation 1)):

(3) casesm,d(a,t)=(1eλd)eλdaρsympopm,d(a,t)

λd is the FOI of that area (assumed constant over time and age independent) which is sampled from the posterior estimates from the previous model fitting. The term e-λda is the decrease in proportion of susceptible population due to natural infection. ρsym is symptomatic rate, sampled from Uniform(1500,1250) (SAGE Working Group on Japanese encephalitis vaccines, 2014). The symptomatic rate is the proportion of infections that are estimated to show symptoms, this is different to the reporting rate in the FOI estimation section, which includes differential reporting and testing by study. popm,d(a,t) is the susceptible population of age a in endemic area d in year t under scenarios m and was interpolated from World Population Prospects 2017 data (United Nations-Department of Economic and Social Affairs-Population Division, 2017). To assess the impact of previous vaccination programs, the population popm,d(a,t) was different for each vaccination scenario m: with or without vaccination. The vaccination scenario used the collated information about past vaccination programs and assumed that the number of vaccinations given each year to each age meant that this number of the relevant age groups in the population were not susceptible to infection from this year onwards. This takes into account aging of the vaccinated population and any changes in the vaccination programs over time.

Although the mortality rate of JE varies, the reported ranges are from 20-30% (Fischer et al., 2008). We sampled the mortality rate from Uniform(0.2,0.3) and multiplied it by the estimated number of casesm,d(a,t) to generate age-specific JE-induced deaths.

All code and data are available at: https://github.com/tranquanc123/JE_burden_estimates (Quan, 2020; copy archived at https://github.com/elifesciences-publications/JE_burden_estimates).

References

  1. 1
    A review of Japanese Encephalitis cases in the Philippines (1972-1985)
    1. NG Barzaga
    (1990)
    The Southeast Asian Journal of Tropical Medicine and Public Health 20:587–592.
  2. 2
  3. 3
  4. 4
    Japanese encephalitis
    1. DS Burke
    2. CJ Leake
    (1988)
    WHO-Recommended Standards for Surveillance of Selected Vaccine-Preventable Diseases.
  5. 5
  6. 6
  7. 7
  8. 8
    Estimating Japanese Encephalitis Global Burden 2018
    1. NM Duy
    (2018)
    Estimating Japanese Encephalitis Global Burden 2018.
  9. 9
  10. 10
    Japanese Encephalitis Prevention and Control: Advances, Challenges, and New Initiatives
    1. M Fischer
    2. S Hills
    3. E Staples
    4. B Johnson
    5. M Yaich
    6. T Solomon
    (2008)
    In: W Scheld, S Hammer, J Hughes, editors. Emerging Infections. American Society of Microbiology. pp. 93–124.
    https://doi.org/10.1128/9781555815592.ch6
  11. 11
  12. 12
  13. 13
  14. 14
    The No-U-Turn sampler: adaptively setting path lengths in hamiltonian monte carlo
    1. MD Hoffman
    2. A Gelman
    (2014)
    Journal of Machine Learning Research 15:1593–1623.
  15. 15
    Japanese B Encephalitis in Korea
    1. RL Hullinghorst
    (1951)
    Japanese B Encephalitis in Korea.
  16. 16
  17. 17
  18. 18
    Leaflet: Create Interactive Web Maps with the JavaScript ‘Leaflet’ Library
    1. C Joe
    2. K Bhaskar
    3. X Yihui
    (2017)
    Leaflet: Create Interactive Web Maps with the JavaScript ‘Leaflet’ Library.
  19. 19
  20. 20
  21. 21
  22. 22
  23. 23
  24. 24
  25. 25
    Catalytic Models in Epidemiology
    1. H Muench
    (1958)
    Harvard University Press.
  26. 26
  27. 27
  28. 28
  29. 29
  30. 30
    Background Paper on Japanese Encephalitis Vacccines
    1. SAGE Working Group on Japanese encephalitis vaccines
    (2014)
    Background Paper on Japanese Encephalitis Vacccines.
  31. 31
    Ecological studies on culex tritaeniorhynchus as a vector of japanese encephalitis
    1. LS Self
    2. HK Shin
    3. KH Kim
    4. KW Lee
    5. CY Chow
    6. HK Hong
    (1973)
    Bulletin of the World Health Organization 49:41–47.
  32. 32
  33. 33
  34. 34
  35. 35
    RStan: The R Interface to Stan
    1. Stan Development Team
    (2016)
    R Package Version 2.14.1.
  36. 36
  37. 37
    World Population Prospects: The 2017 Revision, DVD Edition
    1. United Nations-Department of Economic and Social Affairs-Population Division
    (2017)
    World Population Prospects: The 2017 Revision, DVD Edition.
  38. 38
  39. 39
  40. 40
  41. 41
    GRADE Table 2 . What Is the Effectiveness of Live Attenuated JE Vaccine in PreventingJE Disease in Vaccinees Living in JE- ‐ Endemic Areas ?
    1. WHO
    (2014)
    GRADE Table 2 . What Is the Effectiveness of Live Attenuated JE Vaccine in PreventingJE Disease in Vaccinees Living in JE- ‐ Endemic Areas ?.
  42. 42
    Japanese Encephalitis
    1. WHO
    (2015)
    Japanese Encephalitis.
  43. 43
    WHO World Health Organization: Immunization, Vaccines And Biologicals. VaccinePreventable Diseases Vaccines Monitoring System 2019 Global Summary Reference Time Series:JAPANESE ENCEPHALITIS
    1. WHO
    (2019)
    WHO World Health Organization: Immunization, Vaccines And Biologicals. VaccinePreventable Diseases Vaccines Monitoring System 2019 Global Summary Reference Time Series:JAPANESE ENCEPHALITIS.
  44. 44
    GRADE Table 1 . What Is the Effectiveness of Inactivated Vero Cell- ‐ Derived JE Vaccine in Preventing JE Disease in Vaccinees Living in JE- ‐ EndemicAreas ?
    1. World Health Organization
    (2012a)
    GRADE Table 1 . What Is the Effectiveness of Inactivated Vero Cell- ‐ Derived JE Vaccine in Preventing JE Disease in Vaccinees Living in JE- ‐ EndemicAreas ?.
  45. 45
    Grading of Scientific Evidence - Table 3. What Is the Effectiveness of LiveRecombinant JE Vaccine in Preventing JE Disease in Vaccinees Living in JE-‐endemic Areas?
    1. World Health Organization
    (2012b)
    Grading of Scientific Evidence - Table 3. What Is the Effectiveness of LiveRecombinant JE Vaccine in Preventing JE Disease in Vaccinees Living in JE-‐endemic Areas?.
  46. 46
    Reported Administrative Target Population, Number of Doses Administrered and Administrative Coverage
    1. World Heath Organization
    (2018)
    Reported Administrative Target Population, Number of Doses Administrered and Administrative Coverage.
  47. 47

Decision letter

  1. Eduardo Franco
    Senior and Reviewing Editor; McGill University, Canada

In the interests of transparency, eLife publishes the most substantive revision requests and the accompanying author responses.

Acceptance summary:

Significance statement: Because of its high morbidity and mortality, Japanese Encephalitis (JE) is an important public health concern in Asia. Vaccination is a realistic preventive strategy but requires reliable epidemiologic surveillance across endemic countries. To overcome this problem, the authors used age-stratified data combined with mathematical modelling to estimate the global transmission intensity and burden of JE, as well as the impact of vaccination, thus expanding on earlier work. The annual number of cases is about 50% higher than originally estimated, correcting for under-reporting and variations in countries' surveillance systems.

Decision letter after peer review:

Thank you for submitting your article "Estimates of the global burden of Japanese Encephalitis and the impact of vaccination from 2000-2015" for consideration by eLife. Your article has been reviewed by three peer reviewers, and the evaluation has been overseen by Mark Jit as Reviewing Editor and Eduardo Franco as the Senior Editor. The reviewers have opted to remain anonymous.

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

This review and modelling study updates global estimates of the burden of disease due to Japanese Encephalitis (JE). Based on a systematic review of the current literature, the paper makes revised models of the disease burden of JE and of the effects of vaccination. The revised estimate of the global burden of JE is 100,000 cases per year. This is an entirely plausible number, and, importantly, the paper estimates that only about 1/3 of cases are prevented by vaccination.

This is an important paper in the field, and should be published, with some modifications. The estimates are important given the lack of robust surveillance systems. The goal of this study – to improve understanding of the global burden of JE and the impact of existing vaccination efforts – is critical for informing future public health strategies. The manuscript is clear and well-written.

General comments

1) Could you add more detail about the Bayesian model used to estimate the force of infection and other key parameters as currently there is not sufficient information to review it? It would be useful to see both prior and posterior (including joint posterior) distributions for all the parameters, as well as trace plots, autocorrelation plots and any other diagnostic plots that you think are relevant. We may ask a Bayesian statistician to review this once we have received it.

2) How were asymptomatic infection incorporated in the estimations? Were individuals with asymptomatic infection considered susceptible or immune after infection? Particularly since the rate of asymptomatic to symptomatic JE is wide ranging (25:1 to 1000:1), this may have implications on estimations.

3) The model makes a number of assumptions which may have modified the accuracy of their conclusions. These assumptions include (i) vaccine efficacy is 100%, (ii) force of infection remains constant over time, (iii) the relative incidence of JE across different countries has remained the same since the Campbell et all review, (iv) population distribution is homogenous and (v) the force of infection is the same for all age groups.

Assumptions (i) and (ii) have been directly addressed by sensitivity analyses, and do not seem to affect the model outcomes that much, despite the fact that neither is true. The argument made that the force of infection need not vary over time to make the model valid is a little difficult to follow from Figure 5—figure supplement 1M for the non-modeller reader. See below for more on vaccine efficacy in India. Assumption (iii) appears not to make very much difference as most of the areas for which there are little data are areas of low incidence. However it is worth noting that the assumption itself otherwise may be incorrect – future workers would be wrong to take this as a precedent. Assumptions (iv) and (v) have not been addressed by the authors. They too may not affect the conclusions that much but they should at least be discussed.

Specific comments

1) Results section. Reliability of JE case data. While MAC-ELISA is the WHO recommended procedure for JEV diagnosis, concerns have been raised regarding the sensitivity (50-70%) and specificity of this test (E.g., Robinson et al., 2010 Am J Trop Med Hyg, Dubot-Peres et al., 2015, Lancet Infect Dis). Further to this, parallel testing for dengue-specific antibodies is often not done. It is not clear from Figure 2—figure supplement 1 whether they reported testing samples against other endemic antigenically related flaviviruses (e.g. dengue). Given that the diagnostic test employed by most studies included in your analysis has a low confirmatory level for JEV infection, could this uncertainty somehow be incorporated in your study (also the possibility that diagnostic trends have changed over the study time period)? Additionally, could any adjustments be made where diagnostic methods less confirmatory than MAC-ELISA were used? At the very least you should acknowledge this issue and discuss how it may have impacted their results and conclusions.

2) Subsection “Force of infection estimation from collated age-stratified data”: FOI estimates are very low for Japan. Could this just be because there's successful vaccination making the data uncertain due to low numbers? E.g. there are some papers that suggest the FOI in parts of Japan can be much higher, e.g. 10%, or 100 fold higher. What would the effect of this be on the model? See for example Konishi et al. Vaccine 2002; 21:98-107 and Vaccine 2006; 24:3054-6.

3) “We estimated that the proportion of the population in study 𝑘 and age group 𝑖 that remained susceptible after vaccination 𝑠𝑘,𝑖 , was different to the prior collated vaccination information in areas such as China, India, Japan, and Nepal (Figure 2—figure supplement 4).” -This sentence is hard to understand without the figure – what/where is Figure 2—figure supplement 4?

4) Subsection “Inference of force of infection for all endemic areas” – Presumably incidence group B refers to the same incidence group as per Campbell et al., 2011? This isn't actually explained until several pages later – it would be much easier for a reader who is not so well versed in the JE literature to understand if this were to be defined earlier.

5) Subsection “Burden and Vaccine Impact estimation” – only approx. 15% of JE cases have been prevented! This figure is worthy of more discussion, as although it has approximately doubled in recent years, still only about 1/3 of cases are prevented. Clearly there is still much to do.

6) Subsection “Burden and Vaccine Impact estimation”/Figure 5 – Are you suggesting that there has been no effect of JE vaccination in India? See also Discussion paragraph three.

7) Discussion paragraph three – Efficacy was high in case control studies too, not only in trials. See for example Ohrr et al. Lancet 2005; 366: 1375-78 and Kumar et al.,2009.

8) This efficacy estimate is now formally published and would make a better reference (Tandale et al., 2018). Vaccine efficacy does appear to be lower in India, for reasons which are not entirely clear. A lower rate of sero-conversion to the SA14-14-2 vaccine has also been described in India (Singh et al., 2015). Have you provided further support for these observations? Can you suggest why the efficacy of SA14-14-2 vaccine should be lower in India?

9) Table 1 – State more clearly that these are case numbers, and what the numbers in brackets are.

10) Discussion paragraph six – The paper acknowledges that its estimates will be underestimates for various reasons. However one important reason has been left out – that is that JE can be difficult to diagnose and there is a miss rate whereby even genuine cases will test negative because the test has been done too early in the illness (it takes a week for CSF IgM to be positive in close to 100% of cases and lumbar punctures are often not repeated this late). The authors should they acknowledge that these reported figures will be minimum estimates. Eg see Susan Hills' work in Nepal showing all AES went down after JE vaccination, not only JE (Upreti et al. Am. J. Trop. Med. Hyg., 88(3), 2013, pp. 464-468).

11) Could there be a table of countries with JE vaccination programs?

12) Subsection “Force of infection estimation”: Limited JE surveillance systems could lead to inaccurate reporting of disease. The authors took this into consideration in their estimation of number of cases in a specific age group by including pk (reporting rate) into consideration. It's not clear how the reporting rate accounts for different surveillance qualities. For example, in countries where JE virus IgM antibody was not confirmed with CSF, how was surveillance quality (or potential heterogeneity) addressed?

13) The paper notes that convergence was assessed visually. Were you also able to assess statistical measures of goodness of fit, such as R2 or residual plots to confirm the visual assessments?

14) Figure 3 plots the "estimated number of vaccinated individuals by region". Please make it clear in the figure caption that these are not model-based estimates, but calculated from vaccination data. Even referring to these values as "Reported number of individuals vaccinated in each region, summed [?] from multiple data sources". If any adjustments were made in making these calculations, please specify.

15) Figure 3 – Please provide further information on the data from Gavi and WHO. Does the Gavi data represent the number of doses delivered or health records of vaccinated individuals or follow-up survey data? Is the WHO joint reporting data publicly available? If so, please include the link. What uncertainties exist in these various different data sources and why are these not incorporated in Figure 3? The point estimates imply a high level of confidence in these values. Please consider acknowledging uncertainty in Figure 3.

16) Figure 3 – The paper later acknowledges uncertainty in these values and include it in their analysis through the selection of priors. Why are many of the priors so constrained for some countries e.g. Laos, Bangladesh, Philippines (Figure 4—figure supplement 3)? Does this not suggest a high level of confidence? How much data at each location informed the posterior?

[Editors' note: further revisions were suggested prior to acceptance, as described below.]

Thank you for resubmitting your work entitled "Estimates of the global burden of Japanese Encephalitis and the impact of vaccination from 2000-2015" for further consideration by eLife. Your revised article has been evaluated by Eduardo Franco as Senior Editor and Mark Jit as Reviewing Editor.

A second review round for your submission is complete. Four reviewers spent a considerable amount of time examining your work. We are happy to see the effort you made at amending the paper to accommodate the concerns and suggestions from the reviewers of the original version. Once again, we are unable to accept it in its present form for publication. However, we are willing to consider a new revised version if you can address the additional items in the reviewers' critiques. These comments were edited to eliminate redundancy and to help you focus on the revisions. In general, the required revisions have to do with clarity of the presentation and making the model reproducible, which requires providing the data and code in a user-friendly format.

1) Clarity of presentation:

Please revise Figure 3 to use integers, not scientific numbers for ease of interpretation.

Subsection “Force of infection (FOI) estimation from collated age-stratified data”: The authors indicate that the reporting rate = total rate of symptomatic JE (I assume that means with or without presentation to facility/surveillance system), plus % of people presenting to a facility or surveillance system. As written, it seems like this would lead to double counting and result in overinflated estimates? Please explain or clarify the text.

What was the magnitude of variation between prior and posterior estimates of susceptible post vaccination?

Table 1: What is the time frame for these estimates? (annual or for the 2000-2015 timeframe?)

Subsection “Force of infection estimation”: Can you give brief rationale for why you set your uncertainty of vaccination information to 5? The text references Figure 4—figure supplement 3 but this graph is difficult to read and does not include sources for selection of prior distributions.

Supplemental Appendix: In general, this information is important but I find the figures in Figure 4—figure supplement 3 to be a bit impenetrable. I think this could be addressed fairly simply by renaming the graphs (for example, from s_prop[2]), legends, and adding footnotes.

There are a few formatting issues – in particular some versions of some of the figures in the PDF file are of very poor resolution.

This may be a function of the publisher's format for review, but I found the figures, legends and nomenclature very hard to follow. I don't fully understand the hierarchy of "Figure X- Supp Y" etc. As long as the authors proofread the typeset manuscript very carefully this will be fine and it is not a reason for re-review – rather a word of caution that in the current form it's difficult to read and this needs to be improved on publication. Labelling each figure on the page on which it is presented would help a lot. My reading of the current figures is that they are appropriate and in some cases improved from the first version. eg Figure 3—source data – I am not sure what/where this is? What/where is S2 Fig?)

Figure 4—figure supplement 4-6 – I agree these should remain supplementary as they are indeed confusing for the non-statistician reader.

For those not familiar with FOI rates, it might help to have an explanation of what the number actually means – eg does FOI 0.05 mean 5% of the population infected per year? This would make it more accessible.

I agree with the authors choice of wording concerning the efficacy in India – I do not have any better explanation than this – except possibly a nutritional effect – but this is speculative and it is not clear why this should be different to other countries.

2) Issues related to the Bayesian model:

A) The overall model written in stan seems good. It is a shame that the data were not provided alongside the R code, and it would seem easy to remedy this (I would recommend packaging them together as an RMarkdown file along with the curated dataset). eLife is committed to reproducible research and having easy to re-run code and data is quite important for a study whose conclusions are entirely dependent on the underlying statistical model and available data. Asking people to extract data from a Word document is prone to errors etc.

B) I couldn't work out whether or not there was propagation of uncertainty from model 1 (the FOI model) to model 2 (the burden and vaccine impact model). i.e. is the model 2 using a point estimate from model 1, or is it integrating over the full posterior? If there is no propagation of uncertainty I would recommend this is changed (otherwise it will underestimate the uncertainty in the final output); if there is propagation of uncertainty, then please make it clearer.

C) The notation confused me in parts. Is λd the FOI or is it FOI(λd)? If so what is the function "FOI"? This is probably obvious to people used to dealing with these catalytic models, but as I've never seen one before I had trouble understanding.

D) A prior of N(0,1000) is probably not the best choice. If λd is the FOI, then Figure 4 shows that all the values are between 0 and ~0.65. So values of 2000 are completely implausible. Your chains will get better mixing if you use weakly informative priors. See https://github.com/stan-dev/stan/wiki/Prior-Choice-Recommendations for a nice overview of recommendations from the stan team (your choice is under the "not usually recommended" category!). A guess of a better prior might be an exponential distribution (values are always positive), with mean value close to what you expect in most cases.

Please provide an easy to use (e.g. RMarkdown) implementation, this would have the data as a csv file, for example, included.

https://doi.org/10.7554/eLife.51027.sa1

Author response

General comments

1) Could you add more detail about the Bayesian model used to estimate the force of infection and other key parameters as currently there is not sufficient information to review it? It would be useful to see both prior and posterior (including joint posterior) distributions for all the parameters, as well as trace plots, autocorrelation plots and any other diagnostic plots that you think are relevant. We may ask a Bayesian statistician to review this once we have received it.

We have added details on the model fitting methods and results throughout the paper and uploaded additional supplementary files as detailed below:

1) In the Materials and methods we added more detail on RStan algorithm: “RStan uses a No-U-Turn sampler (NUTS) (Hoffman and Gelman, 2014), a variant of Hamiltonian Monte Carlo to obtain posterior simulation (Stan Development Team, 2016).”

2) Convergence plots have been uploaded as Figure 4—figure supplement 4 and autocorrelation plots as Figure 4—figure supplement 5.

3) In the Results we added details on the model fit and point the reader towards the convergence plots: “Force of infection (FOI) is the per capita rate at which susceptible individuals are infected by an infectious disease. All the catalytic models converged well (see convergence plots in Figure 4—figure supplement 4 and autocorrelation plots Figure 4—figure supplement 5) and fit well to data in all but one study (Figure 4—figure supplement 1- 95% CIS of model output and case data shown).”

4) The prior and posteriors for the susceptible proportion after vaccination are shown in Figure 4—figure supplement 3. The priors for the reporting rate and λ have been plotted in Figure 4—figure supplement 6. I thought about putting on the results figures, but thought that would be confusing for the non-statistician reader. I can add if you prefer.

5) In our results, we estimated that the posterior estimate of the proportion of the population in study k and age group i that remained susceptible after vaccination sk,i, was different to the prior collated population vaccination that was included in the model fitting vaccination information in areas for areas such as China, India, Japan, and Nepal (Figure 4—figure supplement 3).

2) How were asymptomatic infection incorporated in the estimations? Were individuals with asymptomatic infection considered susceptible or immune after infection? Particularly since the rate of asymptomatic to symptomatic JE is wide ranging (25:1 to 1000:1), this may have implications on estimations.

Yes this range is indeed wide. This is a large part of the uncertainty in our estimates as we sample from across this range. We have added emphasis to this in the Discussion: “An additional limitation of our analysis is the uncertainty in the proportion of infections that lead to disease, we sample from this range, and this uncertainty is included in our uncertainty analysis. Further studies, for example in cohorts may enable better estimates of this proportion.”

3) The model makes a number of assumptions which may have modified the accuracy of their conclusions. These assumptions include (i) vaccine efficacy is 100%, (ii) force of infection remains constant over time, (iii) the relative incidence of JE across different countries has remained the same since the Campbell et all review, (iv) population distribution is homogenous and (v) the force of infection is the same for all age groups.

Assumptions (i) and (ii) have been directly addressed by sensitivity analyses, and do not seem to affect the model outcomes that much, despite the fact that neither is true. The argument made that the force of infection need not vary over time to make the model valid is a little difficult to follow from Figure 5—figure supplement 1M for the non-modeller reader. See below for more on vaccine efficacy in India. Assumption (iii) appears not to make very much difference as most of the areas for which there are little data are areas of low incidence. However it is worth noting that the assumption itself otherwise may be incorrect – future workers would be wrong to take this as a precedent. Assumptions (iv) and (v) have not been addressed by the authors. They too may not affect the conclusions that much but they should at least be discussed.

We thank the reviewers for their careful thoughts about our assumptions. We have added more discussion of these assumptions into the Discussion section.

For assumption (iii) we have added additional statement about this in the Discussion: “Similarly, we assume transmission is constant over time. Further work fitting models with time varying forces of infection as well as looking at covariates of infection that are changing over time, will be necessary for future refinements of these estimates.”

For assumption (iv) population age distribution is assumed homogenous within a country, but not across age- the age distribution is as per population data for each country. I think the way this is currently stated is not clear and unnecessary here, so have deleted this sentence. The source of the population age distribution for each country is given: “popk,i is the population size in each age group i of each study k, calculated from World Population Prospects 2017 data (United Nations-Department of Economic and Social Affairs-Population Division, 2017).”

For assumption (v), we have added the following to the Discussion: “In addition, we assumed the FOI is constant across age, only susceptibility changes, further assessment of seroprevalence studies may be able to assess this further.” The vaccine in India comments are addressed below.

Specific comments

1) Results section. Reliability of JE case data. While MAC-ELISA is the WHO recommended procedure for JEV diagnosis, concerns have been raised regarding the sensitivity (50-70%) and specificity of this test (E.g., Robinson et al., 2010 Am J Trop Med Hyg, Dubot-Peres et al., 2015, Lancet Infect Dis). Further to this, parallel testing for dengue-specific antibodies is often not done. It is not clear from Figure 2—figure supplement 1 whether they reported testing samples against other endemic antigenically related flaviviruses (e.g. dengue). Given that the diagnostic test employed by most studies included in your analysis has a low confirmatory level for JEV infection, could this uncertainty somehow be incorporated in your study (also the possibility that diagnostic trends have changed over the study time period)? Additionally, could any adjustments be made where diagnostic methods less confirmatory than MAC-ELISA were used? At the very least you should acknowledge this issue and discuss how it may have impacted their results and conclusions.

This is a really important comment, the sensitivity of the test and how this varies between studies in different places at different times, will be taken into account by the reporting rate- missing cases across all ages will be estimated as a lower reporting rate for this study, but will not alter the FOI estimates (see comment above). These are not the reporting rates used in the burden and vaccine estimates- so it will not alter our case estimates as well. We have added the following to the Discussion: “In addition, though our method accounts for reporting rates within these studies, future work should assess the impact of cross-reactivity and further issues with diagnosis on the estimates, such as including AES data in the fitting.”

2) Subsection “Force of infection estimation from collated age-stratified data”: FOI estimates are very low for Japan. Could this just be because there's successful vaccination making the data uncertain due to low numbers? Eg there are some papers that suggest the FOI in parts of Japan can be much higher, eg 10%, or 100 fold higher. What would the effect of this be on the model? See for example Konishi et al. Vaccine 2002; 21:98-107 and Vaccine 2006; 24:3054-6.

We thank the reviewer for this comment. We have added to the Discussion so the section on the estimates for Japan now reads as follows: “Our estimate of transmission intensity for Japan also has great uncertainty, as half the studies included data pre-2000 and we were able to find limited information on the long-running vaccination program there. In addition, there are limitations to how well our method will work, given the high rate of vaccination there. This may mean we are under-estimating the impact of vaccination in Japan. Further work with serological data both from humans and animals and further exploration of the drivers of JE transmission will help refine this estimate.”

We have removed discussion of Japan from elsewhere in the Discussion.

3) “ We estimated that the proportion of the population in study κ and age group i that remained susceptible after vaccination sk,i, was different to the prior collated vaccination information in areas such as China, India, Japan, and Nepal (Figure 2—figure supplement 4).” -This sentence is hard to understand without the figure – what/where is Figure 2—figure supplement 4?

Rephrased as follows: “In our results, we estimated that the proportion of the population in study and age group that remained susceptible after vaccination , was different to the prior population vaccination that was included in the model fitting for areas such as China, India, Japan, and Nepal (Figure 4—figure supplement 3)”. Apologies we gave the wrong figure reference previously- it has been updated to Figure 4—figure supplement 3.

4) Subsection “Inference of force of infection for all endemic areas” – Presumably incidence group B refers to the same incidence group as per Campbell et al., 2011? This isn't actually explained until several pages later – it would be much easier for a reader who is not so well versed in the JE literature to understand if this were to be defined earlier.

Thanks for pointing this out, we have added more detail on the grouping names in the Introduction and then added the reference and reference to the Campbell groupings in the text and Figure 4 legend.

5) Subsection “Burden and Vaccine Impact estimation” – only approx. 15% of JE cases have been prevented! This figure is worthy of more discussion, as although it has approximately doubled in recent years, still only about 1/3 of cases are prevented. Clearly there is still much to do.

We thank the reviewer for focussing on this. We have tried to add emphasis to this by rewording the last section of the manuscript. The final paragraph now reads as: “We estimate annually there are still 100,000 cases of JE in Asia, making a 1/3 of all cases of this severe but vaccine preventable still not being averted. The majority of remaining cases are focussed in countries with still developing healthcare systems. Given there is a cheap vaccination now available, our results will help with the rational assessment of JE vaccination cost and benefit for each country and will help guide Gavi and other international and national public health agencies in making decision on their future investment into JE vaccination.”

6) Subsection “Burden and Vaccine Impact estimation”/Figure 5 – Are you suggesting that there has been no effect of JE vaccination in India? See also Discussion paragraph three.

We apologise this was unclear- there has been little impact in the low incidence areas because the incidence rate is estimated to be very low even without vaccination.

We have clarified that this is only the low incidence sub-area of India. In we state that “We estimated the highest number of cases …….and medium or high endemic areas in India (around 20,000 annual cases in no vaccination scenario and 15,000 annual cases in vaccination scenario for each area in recent years).”

We have added clarification: “There was still an impact of vaccination- but expected impact on the age distribution in the model fitting was not as extreme as the data we collated would have suggested.”

7) Discussion paragraph three – Efficacy was high in case control studies too, not only in trials. See for example Ohrr et al. Lancet 2005; 366: 1375-78 and Kumar et al., 2009.

Thanks- we added the following references: (Kumar, Tripathi, and Rizvi, 2009); Bista et al., 2001),

8) This efficacy estimate is now formally published and would make a better reference (Tandale et al., 2018). Vaccine efficacy does appear to be lower in India, for reasons which are not entirely clear. A lower rate of sero-conversion to the SA14-14-2 vaccine has also been described in India (Singh et al., 2015). Have you provided further support for these observations? Can you suggest why the efficacy of SA14-14-2 vaccine should be lower in India?

Thanks very much for the additional references. This is a really interesting point for further study. We have added the Tandale et al. reference and added Singh reference with the statement: “lower seroconversion has also been reported in India (Singh et al., 2015).”

This section now reads: “Further studies are needed to explore whether there are different vaccine efficacies in different places, particularly India and to explore possible explanations for this. One possible explanation could be cross-reactive immunity to other flaviviruses or differences in circulating JE genotypes.”

Does the reviewer have any thoughts on why this might be?

9) Table 1 – State more clearly that these are case numbers, and what the numbers in brackets are.

Added to table column headers.

10) Discussion paragraph six – The paper acknowledges that its estimates will be underestimates for various reasons. However one important reason has been left out – that is that JE can be difficult to diagnose and there is a miss rate whereby even genuine cases will test negative because the test has been done too early in the illness (it takes a week for CSF IgM to be positive in close to 100% of cases and lumbar punctures are often not repeated this late). The authors should they acknowledge that these reported figures will be minimum estimates. Eg see Susan Hills' work in Nepal showing all AES went down after JE vaccination, not only JE (Upreti et al. Am. J. Trop. Med. Hyg., 88(3), 2013, pp. 464-468).

(Response as for comment 1) This is a really important comment, the sensitivity of the test and how this varies between studies in different places at different times, will be taken into account somewhat by the reporting rate- missing cases across all ages will be estimated as a lower reporting rate for this study, but will not alter the FOI estimates (see comment above). These are not the reporting rates used in the burden and vaccine estimates- so it will not alter our case estimates as well. We have added the following to the Discussion: “In addition, though our method accounts for reporting rates within these studies, future work should assess the impact of cross-reactivity and further issues with diagnosis on the estimates, such as including AES data in the fitting.”

We have also added the following for clarification on the reporting vs symptomatic rate: “The symptomatic rate is the proportion of infections that are estimated to show symptoms, this is different to the reporting rate in the FOI estimation section, which includes differential reporting and testing by study.”

11) Could there be a table of countries with JE vaccination programs?

This is in Figure 3—source data 1which was not properly referenced in the previous version. In order to guide the reader to this we have added reference throughout this section to Figure 3—source data 1.

12) Subsection “Force of infection estimation”: Limited JE surveillance systems could lead to inaccurate reporting of disease. The authors took this into consideration in their estimation of number of cases in a specific age group by including pk (reporting rate) into consideration. It's not clear how the reporting rate accounts for different surveillance qualities. For example, in countries where JE virus IgM antibody was not confirmed with CSF, how was surveillance quality (or potential heterogeneity) addressed?

(Response as for comment 1 and 10). This is a really important comment, the sensitivity of the test and how this varies between studies in different places at different times, will be taken into account somewhat by the reporting rate- missing cases across all ages will be estimated as a lower reporting rate for this study, but will not alter the FOI estimates (see comment above). These are not the reporting rates used in the burden and vaccine estimates- so it will not alter our case estimates as well. We have added the following to the Discussion: “In addition, though our method accounts for reporting rates within these studies, future work should assess the impact of cross-reactivity and further issues with diagnosis on the estimates, such as including AES data in the fitting.”

13) The paper notes that convergence was assessed visually. Were you also able to assess statistical measures of goodness of fit, such as R2 or residual plots to confirm the visual assessments?

We further describe the model fit and point the reader towards the correct file showing the models fits in the Results: “and fit well to data in all but one study (Figure 4—figure supplement 1- 95% CIS of model output and case shown).” and added into the supplementary materials the trace plots for the model fits.

14) Figure 3 plots the "estimated number of vaccinated individuals by region". Please make it clear in the figure caption that these are not model-based estimates, but calculated from vaccination data. Even referring to these values as "Reported number of individuals vaccinated in each region, summed [?] from multiple data sources". If any adjustments were made in making these calculations, please specify.

The legend title has been updated to: “Reported number of individuals vaccinated in each region from multiple data sources by region from 2000-2015. If the country is not listed there is no vaccination reported. ”. We also added a signpost to the supplementary file that contains all the source information: The supplementary file: Figure 3—source data 1 lists the data and the sources for each country.

15) Figure 3 – Please provide further information on the data from Gavi and WHO. Does the Gavi data represent the number of doses delivered or health records of vaccinated individuals or follow-up survey data? Is the WHO joint reporting data publicly available? If so, please include the link. What uncertainties exist in these various different data sources and why are these not incorporated in Figure 3? The point estimates imply a high level of confidence in these values. Please consider acknowledging uncertainty in Figure 3.

All vaccine data from the two sources, interpretation and what was chosen if there are discrepancies is shown in Figure 3—source data 1 and we have added the reference to this in the Figure 3 legend and in the manuscript. Figure 3 legend has also been altered to read “Reported number of individuals vaccinated in each region from multiple data sources. If the country is not listed there is no vaccination reported. ” rather than estimated to make it clear that this is simply the reported numbers.

The data is available in Figure 3—source data 1 and we have added reference to this data throughout this section.

16) Figure 3 – The paper later acknowledges uncertainty in these values and include it in their analysis through the selection of priors. Why are many of the priors so constrained for some countries e.g. Laos, Bangladesh, Philippines (Figure 4—figure supplement 3)? Does this not suggest a high level of confidence? How much data at each location informed the posterior?

We have altered the legend to Figure 3: “Reported number of individuals vaccinated in each region from multiple data sources by region from 2000-2015. If the country is not listed there is no vaccination reported.”

Thanks for looking at this figure so closely. This figure is susceptible proportion after vaccination and so 1 means no vaccination. In places like Laos, Bangladesh, Philippines where there has been no reported vaccination, we assumed in the prior vaccination would have been reported had it happened, thus the tight intervals. We have added clarification to the legend of Figure 4—figure supplement 3 as follows “This is the susceptible proportion so (1- vaccinated proportion) therefore estimates of 1 here means no vaccination.”

In addition we have clarified the section on discussing when the prior and posterior do not match. “In our results, We estimated that the proportion of the population in study k and age group i that remained susceptible after vaccination sk,i, was different to the prior collated population vaccination that was included in the model fitting vaccination information in areas such as China, India, Japan, and Nepal (Figure 4—figure supplement 3).”

[Editors' note: further revisions were suggested prior to acceptance, as described below.]

In general, the required revisions have to do with clarity of the presentation and making the model reproducible, which requires providing the data and code in a user-friendly format.

1) Clarity of presentation:

Please revise Figure 3 to use integers, not scientific numbers for ease of interpretation.

Done

Subsection “Force of infection (FOI) estimation from collated age-stratified data”: The authors indicate that the reporting rate = total rate of symptomatic JE (I assume that means with or without presentation to facility/surveillance system), plus % of people presenting to a facility or surveillance system. As written, it seems like this would lead to double counting and result in overinflated estimates? Please explain or clarify the text.

We apologise this was misleading. We have updated the text to read: “In this model-fitting the reporting rate is the proportion of all infections that are reported, this is both the proportion that are symptomatic, and the proportion of those cases that present at each hospital or be counted in each surveillance system, we do not estimate these separately”

What was the magnitude of variation between prior and posterior estimates of susceptible post vaccination?

We re-ordered this to be clear this was only for some countries and described the variation more: “For China, India, Japan, and Nepal in our results, the posterior estimates of the proportion of the population in study k and age group i that remained susceptible after vaccination sk,i, for some age groups was slightly different to the prior population vaccination that was included in the model fitting for areas such as China, India, Japan, and Nepal (Figure 4—figure supplement 3). When the posterior and prior did not agree, for most datasets this suggested missing vaccination data, with the prior saying higher susceptibility than the posterior, however for some areas in India the reverse was estimated.”

Table 1: What is the time frame for these estimates? (annual or for the 2000-2015 timeframe?)

We have updated the figure legend to: Table1: Comparing annual case estimates from Campbell et al. to our estimates for the year 2015 (as this was the year of estimation of the previous estimates).

Subsection “Force of infection estimation”: Can you give brief rationale for why you set your uncertainty of vaccination information to 5?

Added: “to account for the possibility that this information was incomplete or did not reflect the actual vaccinations delivered. The chosen uncertainty value represents moderate trust in the vaccination information. The value of 7 or 10 gave a very strong belief, hence not chosen here (these analyses are not shown))”.

The text references Figure 4—figure supplement 3 but this graph is difficult to read and does not include sources for selection of prior distributions.

Supplemental Appendix: In general, this information is important but I find the figures in Figure 4—figure supplement 3 to be a bit impenetrable. I think this could be addressed fairly simply by renaming the graphs (for example, from s_prop[2]), legends, and adding footnotes.

We have redone the Figure 4—figure supplement 3 with the figures being a much larger size and with a description for each study and relabelled the y-axis label.

We have uploaded a better quality figure of Figure 4—figure supplement 1 too also with larger figures.

There are a few formatting issues – in particular some versions of some of the figures in the PDF file are of very poor resolution.

We apologise for this. We have improved the format and re-uploaded. Resolution problem: Figure 3 and 4; Format problem: Figure 4—figure supplement 1 and Figure 4—figure supplement 3 which we have updated with better quality figures

This may be a function of the publisher's format for review, but I found the figures, legends and nomenclature very hard to follow. I don't fully understand the hierarchy of "Figure X- Supp Y" etc. As long as the authors proof read the typeset manuscript very carefully this will be fine and it is not a reason for re-review – rather a word of caution that in the current form it's difficult to read and this needs to be improved on publication. Labelling each figure on the page on which it is presented would help a lot. My reading of the current figures is that they are appropriate and in some cases improved from the first version. eg Figure 3—source data 1 – I am not sure what/where this is? What/where is S2 Fig?)

We agree this was not intuitive, unfortunately eLife would not allow uploading of one supplementary information and so supplementary files had to be uploaded in this way. We have checked all references, and where necessary corrected and added more details in the supplementary files to help the reader understand what they are.

Figure 4—figure supplement 4-6 – I agree these should remain supplementary as they are indeed confusing for the non-statistician reader.

Great, we will leave in the supplementary information.

For those not familiar with FOI rates, it might help to have an explanation of what the number actually means – eg does FOI 0.05 mean 5% of the population infected per year? This would make it more accessible.

We are nervous to do this, because the relationship is not linear and we would have to give a number of examples. We hope that the explanation is enough for people to get an intuited understanding of what the FOI means.

I agree with the authors choice of wording concerning the efficacy in India – I do not have any better explanation than this – except possibly a nutritional effect – but this is speculative and it is not clear why this should be different to other countries.

We agree.

2) Issues related to the Bayesian model:

A) The overall model written in stan seems good. It is a shame that the data were not provided alongside the R code, and it would seem easy to remedy this (I would recommend packaging them together as an RMarkdown file along with the curated dataset). eLife is committed to reproducible research and having easy to re-run code and data is quite important for a study whose conclusions are entirely dependent on the underlying statistical model and available data. Asking people to extract data from a Word document is prone to errors etc.

Thank you for your suggestion. We apologise this was not done previously. The data and codes are uploaded here https://github.com/tranquanc123/JE_burden_estimates, and we have added the link to the github at the end of the Materials and methods.

B) I couldn't work out whether or not there was propagation of uncertainty from model 1 (the FOI model) to model 2 (the burden and vaccine impact model). I.e. is the model 2 using a point estimate from model 1, or is it integrating over the full posterior? If there is no propagation of uncertainty I would recommend this is changed (otherwise it will underestimate the uncertainty in the final output); if there is propagation of uncertainty, then please make it clearer.

Yes there was propagation of uncertainty of the FOI from model 1 to model 2. We have added the following to the burden and vaccine impact estimation methods section to make this clear: “Once the distributions of inferred FOIs for each endemic area were obtained, we generated the distributions of the estimates of the number of cases “ and “ FOI… which is sampled from the posterior estimates from the previous model fitting.”

C) The notation confused me in parts. Is λd the FOI or is it FOI(λd)? If so what is the function "FOI"? This is probably obvious to people used to dealing with these catalytic models, but as I've never seen one before I had trouble understanding.

We apologise this was unclear, we have added the following sentence: “FOI (notated as λd)”

D) A prior of N(0,1000) is probably not the best choice. If λd is the FOI, then Figure 4 shows that all the values are between 0 and ~0.65. So values of 2000 are completely implausible. Your chains will get better mixing if you use weakly informative priors. See https://github.com/stan-dev/stan/wiki/Prior-Choice-Recommendations for a nice overview of recommendations from the stan team (your choice is under the "not usually recommended" category!). A guess of a better prior might be an exponential distribution (values are always positive), with mean value close to what you expect in most cases.

Yes you are right. In hindsight we probably could have run for a shorter time with different prior distribution. The parameter values were on the log scale so the range is not quite as wide (and positive). We used the prior as in a previous paper (10.1093/infdis/jiv470) Though this may speed up our model convergence, and would involve a re-running of the model system, we do not think it would have altered our conclusions, as convergence was reached, so would request to not re-run this now.Next time we will probably use a β distribution or exponential as the reviewer suggests.

Please provide an easy to use (e.g. RMarkdown) implementation, this would have the data as a csv file, for example, included.

All csv data and Rmarkdown are now uploaded at: https://github.com/tranquanc123/JE_burden_estimates. Apologies we did not do this before. This is also referenced at the end of the Materials and methods section.

https://doi.org/10.7554/eLife.51027.sa2

Article and author information

Author details

  1. Tran Minh Quan

    1. Oxford University Clinical Research Unit, Wellcome Trust Asia Program, Ho Chi Minh City, Viet Nam
    2. Biological Science Department, University of Notre Dame, Notre Dame, United States
    Contribution
    Resources, Data curation, Software, Formal analysis, Investigation, Visualization, Methodology, Writing - original draft
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-3337-161X
  2. Tran Thi Nhu Thao

    1. Oxford University Clinical Research Unit, Wellcome Trust Asia Program, Ho Chi Minh City, Viet Nam
    2. Virology Department, Institute of Virology and Immunology, University of Bern, Bern, Switzerland
    Contribution
    Resources, Data curation, Writing - review and editing
    Competing interests
    No competing interests declared
  3. Nguyen Manh Duy

    Oxford University Clinical Research Unit, Wellcome Trust Asia Program, Ho Chi Minh City, Viet Nam
    Contribution
    Data curation, Software, Formal analysis, Writing - review and editing
    Competing interests
    No competing interests declared
  4. Tran Minh Nhat

    Oxford University Clinical Research Unit, Wellcome Trust Asia Program, Ho Chi Minh City, Viet Nam
    Contribution
    Resources, Data curation, Undertook the literature review, Reviewed and approved the final manuscript for submission
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-9500-8341
  5. Hannah Clapham

    1. Oxford University Clinical Research Unit, Wellcome Trust Asia Program, Ho Chi Minh City, Viet Nam
    2. Centre for Tropical Medicine and Global Health, Nuffield Department of Medicine, University of Oxford, Oxford, United Kingdom
    3. Saw Swee Hock School of Public Health, National University of Singapore, Singapore, Singapore
    Present address
    Saw Swee Hock School of Public Health, National University of Singapore, Singapore, Singapore
    Contribution
    Conceptualization, Supervision, Methodology, Writing - original draft, Writing - review and editing
    For correspondence
    hannah.clapham@nus.edu.sg
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-2531-161X

Funding

Bill and Melinda Gates Foundation and Gavi (Vaccine Impact Modelling Consortium)

  • Tran Minh Quan
  • Tran Thi Nhu Thao
  • Nguyen Manh Duy
  • Tran Minh Nhat

Wellcome Trust (089276/B/09/7)

  • Hannah Clapham

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Senior and Reviewing Editor

  1. Eduardo Franco, McGill University, Canada

Publication history

  1. Received: August 12, 2019
  2. Accepted: May 17, 2020
  3. Accepted Manuscript published: May 26, 2020 (version 1)
  4. Version of Record published: June 9, 2020 (version 2)

Copyright

© 2020, Quan et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 420
    Page views
  • 84
    Downloads
  • 0
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Download citations (links to download the citations from this article in formats compatible with various reference manager tools)

Open citations (links to open the citations from this article in various online reference manager services)

Further reading

    1. Epidemiology and Global Health
    Huan Song et al.
    Research Article
    1. Epidemiology and Global Health
    2. Genetics and Genomics
    Linda Grigoraki et al.
    Research Article Updated