Coverage and system efficiencies of insecticidetreated nets in Africa from 2000 to 2017
Abstract
Insecticidetreated nets (ITNs) for malaria control are widespread but coverage remains inadequate. We developed a Bayesian model using data from 102 national surveys, triangulated against delivery data and distribution reports, to generate yearbyyear estimates of four ITN coverage indicators. We explored the impact of two potential 'inefficiencies': uneven net distribution among households and rapid rates of net loss from households. We estimated that, in 2013, 21% (17%–26%) of ITNs were overallocated and this has worsened over time as overall net provision has increased. We estimated that rates of ITN loss from households are more rapid than previously thought, with 50% lost after 23 (20–28) months. We predict that the current estimate of 920 million additional ITNs required to achieve universal coverage would in reality yield a lower level of coverage (77% population access). By improving efficiency, however, the 920 million ITNs could yield population access as high as 95%.
https://doi.org/10.7554/eLife.09672.001eLife digest
Malaria is a major cause of death in many parts of the world, especially in subSaharan Africa. Recently, there has been a renewed emphasis on using preventive measures to reduce the deaths and illnesses caused by malaria. Insecticidetreated nets are the most prominent preventive measure used in areas where malaria is particularly common. However, despite huge international efforts to send enough nets to the regions that need them, the processes of delivering and distributing the nets are inefficient. This problem is compounded by the fact that little information is available on how many nets people actually own and use within each country.`
Bhatt et al. have now created a mathematical model that describes the use and distribution of nets across Africa since 2000. This is based on data collected from national surveys and reports on the delivery and distribution of the nets. The model estimates that in 2013, only 43% of people at risk of malaria slept under a net. Furthermore, 21% of new nets were allocated to households that already had enough nets, an inefficiency that has worsened over the years. Nets are also lost from households much more rapidly than previously thought.
It’s currently estimated that 920 million additional nets are required to ensure that everyone at risk from malaria in Africa is adequately protected. However, Bhatt et al.’s model suggests that given the current inefficiencies in net distribution, the extra nets would in reality protect a much smaller proportion of the population. Taking measures to more effectively target the nets to the households that need them could improve this coverage level to 95% of the population. The next challenge is to devise distribution strategies to send nets to where they are most needed.
https://doi.org/10.7554/eLife.09672.002Introduction
Insecticidetreated nets (ITNs), which comprise conventional (cITNs) and longlasting insecticidal nets (LLINs), are the single most widely used intervention for malaria control in Africa, proven to significantly reduce morbidity and mortality via direct protection and communitywide reductions in transmission (Lim et al., 2011; Lengeler and Lengeler, 2004; Eisele et al., 2010; Killeen et al., 2007). The World Health Organization (WHO) promotes a target of universal coverage for all populations at risk with either ITNs or indoor residual spraying (IRS), with the former representing the primary vector control tool in nearly all endemic African countries (WHO, 2013a). The international community has invested billions of dollars in the provision of at least 700 million LLINs since 2004 (WHO, 2013a). While these investments have led to enormous scale up in population access to ITNs (Noor et al., 2009; Monasch et al., 2004), the target of universal coverage remains distant and millions of African households at risk remain unprotected (WHO, 2013a).
Bridging this gap is a key component of future strategies to reduce further the burden of malaria in Africa (WHO, 2014), and will require sustained commitment from donors, policy makers and national programmes. Central to these efforts is the capacity to monitor reliably current levels of ITN coverage in populations at risk and evaluate the systems that give rise to this coverage. This, in turn, enables progress towards international goals to be tracked and opportunities for efficiency gains to be identified. Such information is essential for evaluating the existing commodity and financing shortfalls and assessing future requirements if the target of universal coverage is to be achieved.
Modelling coverage
To facilitate standardised and comparable monitoring of ITN coverage through time, WHO and the Roll Back Malaria Monitoring and Evaluation Reference Group (RBMMERG) has over the past decade defined a series of indicators to capture two different aspects of ITN coverage: access and use. Gold standard measurements of these indicators are provided by nationally representative household surveys such as Demographic and Health Surveys (DHS) (Measure, 2014), Multiple Indicator Cluster Surveys (MICS) (UNICEF, 2012), and Malaria Indicator Surveys (MIS) (RBM, 2014a). These surveys are carried out relatively infrequently, however, meaning they cannot be used directly for evaluating yearonyear coverage trends or for generating timely estimates of continentwide coverage levels. In contrast, programmatic data such as the number of ITNs delivered and distributed within countries, while not describing coverage directly, are available for most countries and years (WHO, 2013a ). In a 2009 study, Flaxman and colleagues (Flaxman et al., 2010) used a compartmental modelling approach to link these programmatic and survey data, generating annual estimates of the two ITN indicators recommended at that time on access (% households with at least one ITN) and use (% children < 5 years old who slept under an ITN the previous night).
Since that study, there has been increasing recognition that a richer set of indictors is required to identify the complex nature of ITN coverage (Kilian et al., 2013). An intrahousehold 'ownership gap' may exist whereby many households with some nets may not have enough for one net between two occupants (the recommended minimum level of protection (WHO, 2013b). Similarly, a 'usage gap' may exist whereby individuals with access to a net do not sleep under it. In response, the measurement of two additional indicators was recommended: % households with at least one ITN for every two people and % population with access to an ITN within their household (assuming each net was used by two people) (RBM, UNICEF, WHO, 2013; RBM, 2011). In addition, the indicator on usage was extended to include the entire population rather than only children under 5 years old. This updated set of four indicators, used individually and in combination, has the potential to provide a nuanced picture of ITN access and use patterns that can directly guide operational decision making (Kilian et al., 2013). To achieve this, there is a need to develop modelling frameworks to allow all four to be tracked through time.
Evaluating efficiency
Countries have an ongoing struggle to maintain high LLIN coverage in the face of continuous loss of nets from households due to damage, repurposing, or movement away from target areas. In response, systems need to be responsive to emerging coverage gaps by ensuring nets are distributed to households that need them and avoiding overallocation (i.e. distribution of nets to those that already have them). Together, the rate of net loss and the degree of overallocation of new nets play a key role in determining how efficiently delivery to countries will translate into household coverage levels. These factors are not currently well understood but triangulation of survey and programmatic data allows new insights into both.
Estimating future needs
The WHO define universal access to ITNs on the basis that two people can share one net. Using the working assumptions of a 3year ITN lifespan and a 1.8 personpernet ratio (onebetweentwo but allowing for oddnumbered households), a simple calculation yields an indicative estimate of 150 million new nets required each year to provide universal coverage to an African population at risk of around 810 million (WHO, 2013a). To support country planning and donor application processes (RBMHWG, 2014), a more elaborate needs assessment approach has been developed by the RBM Harmonization Working Group (RBMHWG) and implemented by 41 of the 47 endemic African countries (RBM, 2014; Paintain et al., 2013). The tool takes into account the size and structure of national target populations, a 1.8 personpernet ratio for mass campaigns, additional routine distribution mechanisms employed by countries, and volumes of previously distributed nets and their likely rates of loss through time. Countries have used these inputs to calculate requirements for new nets to achieve national coverage targets, leading to an estimated continentwide need for 920 million ITNs over the 2014–2017 period (approximately 230 million per year) (RBM, 2014). This tool provides a transparent, intuitive and standardised mechanism for comparing forecasted needs against current financing levels and identifying likely shortfalls. However, calculated needs are sensitive to assumptions about how a given volume of new nets will translate into population coverage, and inefficiencies in the system such as such as overallocation and rate of net loss are not accounted for explicitly in the current needs assessment exercise.
The purpose of this study is to define a new dynamic modelling approach, triangulating all available data on ITN delivery, distribution and coverage in subSaharan Africa in order to (i) provide validated and datadriven timeseries estimates for all four internationally recommended ITN indicators; (ii) explore and quantify different aspects of system efficiency and how these contribute to reduced coverage levels; and (iii) estimate future LLIN needs to achieve universal access by 2017 under different efficiency scenarios and how these compare to existing needs assessment estimates.
Results
Net stock estimates
Figure 1A summarises the main inputs to and outputs from the stockandflow model for LLINs when aggregated at the continental level. Some 718 million LLINs have been delivered across the 40 endemic countries since their introduction in 2004. As is well documented (WHO, 2013a), annual LLIN deliveries increased yearonyear from 2004 to 2010, reaching 145 million in that year, but then declined dramatically in 2011 and 2012 to less than half that amount before rising again to 143 million in 2013 (green line). Taking into account rates of loss in households, these LLIN deliveries led to a continental net crop shown by the red line. We estimate that there were 252 million LLINs in subSaharan households by the end of 2013, with that net crop growing approximately linearly from 2004, with the exception of a slowdown resulting from the reduced supply of nets in 2011–2012. Figure 1B shows equivalent distribution and resulting net crop estimates for cITNs, which constituted nearly all ITNs prior to 2005 but diminished rapidly in importance following the introduction of LLINs thereafter.
Coverage estimates
Figure 2 shows continentlevel timeseries estimates of the four internationally recommended ITN indicators, along with the 'access gap' indicator. All four indicators show a similar temporal trend: very low coverage levels and modest yearonyear increases for the first 5 years from 2000, with a marked inflexion point in 2005 and much more rapid gains thereafter. Importantly, however, all four indicators show that the pace of increase has, overall, slowed since 2005. By the end of 2013, we estimate that around twothirds (66%, 95% CI 62%–71%) of households at risk owned at least one ITN. However, less than onethird (31%, 29%–34%) owned enough for one ITN between two people. This much lower level of adequate ownership is reflected in the levels of access and use, with 48% (45%–51%) of people at risk having access to an ITN within their household (on a onebetweentwo basis) and 43% (39%–46%) sleeping under an ITN the previous night. Comparison of Figure 2A,B demonstrates that many households that own some ITNs do not own enough for onebetweentwo, and this is captured in the timeseries for the 'ownership gap' (Figure 2E). Encouragingly, this gap has been narrowed from 77% (76%–78%) of netowning households having insufficient nets in 2000 to 56% (54%–57%) in 2013. Analysis of the 'use gap' suggested a large majority (89%, 84%–93%) of those with access to an ITN in the household slept under it the previous night, and we found no evidence of significant change in this proportion through time.
The relatively smooth temporal trends seen at continental level obscure a great deal of complexity in the patterns of ITN scaleup occurring at national level (Figure 3). Nearly all countries began with very low coverage levels in 2000 and display a marked inflection point towards the middle of the decade, although there was considerable variation in the timing of onset of concerted scaleup activities. Importantly, the monotonic increases in coverage seen at the aggregated continental level are often replaced at national level with pronounced periods of rise and fall, and in many cases, 2013 does not represent the peak year. Variation in contemporary levels of coverage remains stark. The population with access to ITNs within the household, for example, was at or below 15% in seven countries in 2013, while above 70% for the top four.
Overallocation
Over the 14year period since 2000, on average 15% (12%–18%) of all ITNs distributed to households were overallocated (owned by households already owning sufficient nets for onebetweentwo). Figure 4 illustrates how these overallocation rates have changed through time. Around 7% (6%–9%) of ITNs were overallocated in 2000, and this has risen steadily to 27% (22%–32%) in 2013. The yearonyear increase in overallocation is to some extent an expected consequence of the overall growth in ITN provision: we found that overallocation increased approximately 15 percentage points for each oneITNpercapita increase in net crop. Overallocation also varied substantially between countries, for example ranging in 2013 from 50% (36%–65%) in the Republic of the Congo to 11% (9%–15%) in Côte D’Ivoire.
Net loss
Averaged over all years and all countries, we found the median retention time for LLINs in households was 23 (20–28) months. We found no statistically significant evidence of continentwide temporal trends in retention times, but substantial betweencountry variation. Figure 5 plots the LLIN loss function representing the most recent three years (2011–2013) for each country individually (blue lines), along with the aggregated continentallevel curve (red line). For reference, we also overlay on Figure 5 some alternative loss functions that have been proposed. Flaxman et al. (orange line) fitted very small annual loss rates (5%) for years 1, 2 and 3  with all LLINs then assumed lost after 3 years (Flaxman et al., 2010). The RBMHWG proposed rate of loss (green line) is 8, 20 and 50% of LLINs to remain after 1, 2 and 3 years, respectively, with all nets being lost thereafter (Networks, 2014). As can be seen, we found rates of loss for the first 3 years to be greater than both these alternatives for all countries. Both alternatives impose a threeyear maximum retention time and our decision not to do so meant that we modelled a small proportion of LLINs lasting some years beyond that point.
ITN requirements to achieve universal coverage
Figure 6 shows the projected levels of coverage that we estimate would be achieved by the end of 2017 with LLIN deliveries for the 2014–2017 period varying from zero to 2.5 billion and under a range of different efficiency scenarios. The most important characteristic of our results is the pronounced shallowing of the deliverycoverage curves: proportionately smaller gains are made in coverage as more LLINs are delivered in an archetypal 'law of diminishing marginal returns'. This means that under a businessasusual scenario, where current levels of overallocation and LLIN loss persist, very large increases in LLIN delivery would be required to achieve high coverage. Under this scenario, we estimate that 1 billion LLINs (i.e. an average of 250 million per year) would be required to achieve 80% of the population with access to an LLIN in the household by the end of 2017, although this would only translate into 70% population use.
The extent to which coverage gains diminish as deliveries increase is mitigated substantially when overallocation and ITN loss rate are reduced. In a scenario with minimised overallocation (where over allocation is set to zero), 80% population access in 2017 would be achievable with just 700 million nets (175 million per year). Reducing ITN loss rate to a 3year median retention time would have a broadly similar impact, acting in isolation, to minimising overallocation. If these two hypothetical efficiency gains were combined, however, 80% access could be reached in 2017 with around 560 million nets (140 million per year). We found that the relative importance of the overallocation and LLIN loss rates changed as more LLINs were introduced. Increasing LLIN retention times was the most important factor at low levels of net delivery, but as more and more nets were provided, overallocation became progressively more important. This is intuitive since it becomes increasingly difficult to avoid overallocation as more households obtain adequate numbers of nets.
For reference, we also plot on Figure 6 the 920 million additional LLINs calculated by countries as required for universal coverage of targeted populations by 2017 under the RBMHWG needs assessment exercise. Under current levels of overallocation and net loss, we estimate that by the end of 2017 this quantity of new LLINs would translate into 77% access (among those populations targeted by countries for ITN coverage) and, assuming current behaviour patterns continue, 68% sleeping under an ITN. Under the combined efficiency scenario with minimised overallocation and 3yr median ITN retention time, however, the 920 million nets would approach universal access (slightly over 95%).
Discussion
By linking manufacturer, programme and national survey data using a conceptually simple model framework, the intention has been to provide a transparent and intuitive mechanism for tracking net crops and resulting household coverage that reflects the input data while simultaneously providing a range of insights about the system itself. In doing so we have been able to (i) provide a new approach for estimating past trends and contemporary levels of ITN coverage; (ii) explore the effects of uneven net distribution between households and the rates of net loss once in households; and (iii) use these insights to estimate how many LLINs are likely to be required to achieve different coverage targets in subSaharan Africa.
We have, for the first time, extended dynamic modelderived estimation of ITN coverage to all four internationally recognized indicators, along with the two 'gap' metrics. Our results reinforce a simple message: while gains in ITN coverage have been impressive, there remains an enormous challenge if the goal of universal access is to be achieved and sustained. The importance of the new expanded suite of indicators is also exemplified: while an encouraging twothirds of households now own at least one ITN, less than half of these have enough to protect everyone who lives there. This ownership gap is narrowing but the disparity remains evident across nearly all countries. Conversely, there is little evidence that nonuse of available nets contributes substantially to low coverage levels. We therefore reinforce earlier studies that suggest the overwhelming barrier to not sleeping under an ITN is lack of access rather than lack of use WHO, 2013a; Eisele et al., 2009; 2011; Koenker and Kilian, 2014; Koenker et al., 2014). Of course, nonuse may be important in certain local contexts, and finerscale analysis can support identification of areas where behaviour change communication interventions may be appropriate to reduce it (Kilian et al., 2013).
We found substantial overallocation of nets to households already owning a sufficient quantity, and that this became more pronounced as overall ownership levels increased through time. Mass distribution campaigns can, in principle, be designed to minimise overallocation and maximise evenness of nets allocated to households strictly on the basis of households members and preexisting nets. As other studies have highlighted, however, any possible commodity savings achieved by such strategies must be compared against the operational cost of these more complex distribution mechanisms (Yukich et al., 2013). What is certain is that overallocation becomes a major barrier to achieving universal coverage when levels of ITN provision are high because most new incoming nets are simply leading to surpluses in many households, while elsewhere there remains a shortfall. This may have a disproportionately high public health impact if those surplus nets are concentrated in households at lowest risk. Wealthier, better educated and more urban households may be better placed to obtain available nets but are often located in regions of lower transmission (Steketee and Eisele, 2009; Webster et al., 2005). While beyond the scope of the present study, the approaches we have developed here could be extended to consider these issues of equity in coverage versus risk in more detail.
One of the most important observations in our study is that LLINs may be lost from households at a substantially faster rate than is currently assumed. Importantly, we assess loss by comparing total inputs to countries (from deliveries) to total numbers in households (net crop), and so we measure real losses rather than, for example, reallocation of nets between relatives (Koenker et al., 2014). Longer retention times of the sort observed in some local studies are not supported by the body of evidence we have provided by triangulating largescale net distributions and household survey data. This more rapid loss rate has potentially important implications for existing guidelines. Current RBM guidance is for mass ITN campaigns to be conducted every 3 years, complemented by continuous distribution of nets via routine channels in order to maintain coverage levels between those campaigns. However, whatever levels of coverage are achieved by a given campaign, we estimate that onehalf of the campaign nets distributed, on average, will not be present in households just 2 years later. Our coverage timeseries for many countries suggest that routine distribution channels are not yet compensating fully for this rate of loss, often displaying pronounced dips in coverage levels between mass campaigns. Maintaining higher continuous coverage therefore clearly requires some combination of more frequent campaigns, greater ongoing distribution between campaigns, or more durable nets and improved care behaviour by users that lead to longer overall retention times.
We considered nets in households as simply present or absent, with no allowance for their condition. In reality, of course, nets may be retained by households (and thus 'present' in our calculations) even when they are badly torn, or have diminished insecticidal properties. As such, our estimates of 'coverage' would be revised downwards if additional measures of net efficacy were included. Our model is able to provide an estimate for every country and every year of the ageprofile of ITNs in households. This raises the possibility of extending the predictions to incorporate modelled or observational data on average rates of net degradation in different contexts (Briët et al., 2012) to explore measures of entomologically effective coverage.
Tools developed to assist countries to calculate LLIN requirements, have tended to define need using a simple ratio to populations at risk (such as 1.8 people per net), and have made allowances for net loss from households using predefined rates of loss. We have been able to show that true LLIN requirements are likely to be considerably larger when the more rapid rates of loss are taken into account, along with the additional effect of likely overallocation patterns. This more realistic framework not only provides the basis for more accurate needs assessments but identifies the relative importance of these different factors in determining the coverage that can be achieved for a given delivery level. Our analysis of future LLIN needs from the present time to 2017 demonstrates how these factors lead to a pronounced law of diminishing returns: as more nets are introduced to a population, proportional increases in coverage diminish, with overallocation a particular problem at high net provision levels.
Under businessasusual, the number of nets required to approach full coverage is prohibitively large. Clearly, however, reducing current system inefficiencies and increasing net retention are not straightforward and already the subject of much attention by countries and international partners. Overallocation is the complex result of different distribution strategies and varying levels of population access to services, and any solution comes with its own cost. Net retention can doubtless be increased by improved LLIN technology coupled with behaviourchange communication efforts, although it is also feasible that retention times may reduce when overall net provision increases (with new nets displacing older ones). Additionally, we look only at the RBM definition of use and ignore the effectiveness of nets in repelling mosquitoes once they are being used. This is potentially an important confounder when considering retention times. While not aiming to provide solutions to these complex challenges, the results we present here provide an analytical framework in which the impact of theoretical efficiency gains can be assessed and this could be extended to include formal cost–effectiveness analysis.
In conclusion, our results provide evidence that LLIN requirements to achieve universal coverage have been underestimated. If obtaining higher coverage remains an accepted goal of the international community, then larger LLIN volumes must be considered and planned for at national and international levels. We emphasise, however, that this would be best achieved in parallel with a renewed focus on maximising the efficiency of coverage achieved for each new net financed. Given that the pattern of diminishing coverage returns for each dollar spent is likely to be unavoidable, the cost–effectiveness of pursuing universal coverage rather than a lower operational target must ultimately be weighed against alternative malaria control investments.
Materials and methods
Overview
Two important preceding studies have sought to model nationallevel ITN delivery, distribution, and coverage: the Flaxman et al. study (Flaxman et al., 2010) and the work of Albert Killian culminating in the NetCALC tool (Networks, 2014) and a series of related publications (Paintain et al., 2013; Yukich et al., 2013). Although very different in implementation, both approached the problem in a similar twostage process. First, a mechanism was defined for estimating net crop — the total number of ITNs in households in a country at a given point in time—taking into account inputs to the system (e.g. deliveries of ITNs to a country) and outputs (e.g. the discard of worn ITNs from households). Second, empirical modelling was used to translate estimated net crops into resulting levels of coverage (e.g. access within households). We have adopted a similar analytical outline, but the models we have developed for each stage differ structurally and conceptually from these earlier efforts. Our underlying principle has been to represent the ITN system in a simple and intuitive way and to parameterise that system using a datadriven approach that minimises the reliance on assumptions or small external datasets. In this Methods section, we describe: (i) the main data sources used; (ii) a new compartmental model for estimating net crop that also offers insights into rates of ITN loss from households; (iii) a new coverage model linking net crop to household net access and use that also assesses the efficiency of betweenhousehold distribution (i.e. the extent of overallocation); and (iv) the use of our models to predict future ITN requirements to meet the goal of universal access. A schematic overview of our analytical framework is provided in Figure 7, and additional methodological detail is provided in the Supplementary Information.
Data
We used three principal sources of data to fit our models. These are described briefly below and in more detail in Supplementary Information.
LLINs delivered to countries: data provided to WHO by Milliner Global Associates on the number of LLINs delivered by approved manufacturers to each country each year (WHO, 2013a; AMP, 2014). These were complete for each country from 2000 to 2013 inclusive.
ITNs distributed within countries: data provided to WHO by National Malaria Control Programmes (NMCPs) on the number of cITN and LLINs distributed annually within each country (WHO, 2013a). Data were available for 365 of the 560 countryyears addressed in the study. We treated these data as only partial records of distribution activities because the extent to which NMCP reporting captures distribution by nongovernment agencies is not known for all countries.
Nationally representative household surveys. We assembled 99 national surveys from 39 subSaharan African countries from 2001–2013, covering 18% of all possible countryyears since 2000 (Figure 8). More recent surveys provided householdlevel data on the number of cITNs, LLINs, people within each dwelling, and people sleeping under nets the previous night. RBMMERG guidelines detail the conversion of these data into the standardised ITN indicators (RBM, UNICEF, WHO, 2013) and, in combination with national population data (UNPD, 2012), they can yield an estimate of national net crop (see Supplementary Information). Older surveys had less information: providing data on use but not ownership, for example, or for cITNs but not LLINs (see Supplementary Information). For most surveys (95/99), we were able to access the underlying data, while for the remaining four we used only the survey report.
Countries and populations at risk
Request a detailed protocolOur main analysis covered 40 of the 47 (WHO, 2013a) malaria endemic countries of subSaharan Africa. We excluded six endemic countries on the basis that ITNs do not form an important part of their vector control programme, as reported by the respective NMCPs to the African Leaders Malaria Alliance, ALMA (M. Renshaw, pers. comm. 3rd August 2014). These were Botswana, Cape Verde, Namibia, São Tomé and Príncipe, South Africa and Swaziland. We also excluded the small island nation of Mayotte, for which no ITN delivery or distribution data were available. We limited all analyses to those populations categorized as being at risk by NMCPs (WHO, 2013a). When interpreting NMCP distribution and household ownership data, we made the simplifying assumption that all reported ITNs were distributed among, and owned within, households situated in malaria endemic regions (Burgert et al., 2012). Additionally, we used data from African Leaders Malaria Alliance (ALMA) on the proportion of populations at risk targeted for ITNs versus IRS, and downscaled targeted populations at risk accordingly. It should be noted that restricting the distribution of ITNs to populations at risk makes the assumption that no ITNs are distributed to populations not at risk.
Estimating national net crops through time
Request a detailed protocolLike Flaxman et al.(Flaxman et al., 2010), we represented national ITN systems using a discrete time stockandflow model. In this structure, a series of compartments were defined that contained a given number of nets at each timestep, with possible movement of nets from one compartment to another between timesteps (see Supplementary Information). Nets delivered to a country by manufacturers were modelled as first entering a 'country stock' compartment (stored incountry but not yet distributed to households). Nets were then available from this stock for distribution to households by the NMCP or other distribution channels. Years where NMCP distributions were smaller than available country stock represented potential ‘underdistribution’, with nets left to stockpile rather than reaching households. However, because of the uncertainty associated with NMCP distribution data, these discrepancies could simply reflect underreporting of distribution levels. To accommodate this uncertainty, we specified the number of nets distributed in a given year as a range, with all available country stock as one extreme (the maximum nets that could be delivered) and the NMCPreported value (the assumed minimum distribution level) as the other.
New nets reaching households joined older nets remaining from earlier timesteps to constitute the total household net crop, with the duration of net retention by households described by a loss function. In this representation, the net crop simply reflected the differences over time between inputs to and outputs from households. This meant that distribution, net crop, and the loss function together formed a closed system: the three must triangulate exactly and knowledge of any two components allowed the third to be calculated directly. Flaxman et al. (Flaxman et al., 2010) assembled data from six studies on ITN durability and rates of loss. Using a loss function fitted to these data, however, they found that the three components tended not to triangulate: net crops observed in surveys were too small, given the data on nets distributed to households and their modelled rate of loss. Their interpretation was that the number of ITNs distributed each year may be systematically overreported by NMCPs, and a 'bias parameter' was included in the model, adjusting downward the volume of nets entering households in each country compared with reported levels. As described above, we took a different approach: with no a priori expectation that NMCP distribution reports exaggerate distribution levels. Rather than fitting the loss function to a small external dataset, we fitted this function directly to the distribution and net crop data within the stockandflow model itself. Conceptually, this reflected the view that the 560 countryyears of distribution data triangulated against the 102 surveyderived national net crop values represented a more impartial and datadriven way of inferring rates of loss than using limited data from local ITN retention studies. Loss functions were fitted on a countrybycountry basis, allowed to vary through time, and defined separately for cITNs and LLINs. We compared these fitted loss functions to existing assumptions about rates of net loss from households. The stockandflow model was fitted using Bayesian inference and Markov chain Monte Carlo (MCMC), providing timeseries estimates of national household net crop for cITNs and LLINs in each country along with evaluation of underdistribution, all with posterior credible intervals. A complete technical description is provided in the Supplementary Information.
Estimating national ITN access and use indicators from net crop
Request a detailed protocolLevels of ITN access within households depend not only on the total number of ITNs in a country (i.e. net crop), but on how those nets are distributed between households. In simple terms, a more even distribution yields a greater proportion of households owning nets than if those same nets are concentrated in fewer households. Many recent national surveys report the number of ITNs observed in each surveyed household. This allows, a histogram to be generated that summarises the net ownership pattern (i.e. the proportion of households with zero nets, one net, two nets and so on). By analysing such data from multiple surveys, previous studies have demonstrated that histograms for different countries vary in a broadly predictable way according to national net crop (Flaxman et al., 2010; Yukich et al., 2013). By representing these histograms using a formal statistical distribution (such as the negative binomial), and linking its parameters to net crop, predicted histograms can be generated for any countryyear for which a net crop estimate is available (Flaxman et al., 2010; Yukich et al., 2013). These histograms, in turn, allow direct calculation of the first access coverage indicator (% households owning one or more ITN). We took the view that this approach—linking net crop to a statistical distribution, and using the distribution to calculate access indicators—is preferable to the alternative of regressing the access indicators against net crop directly. The latter approach, used in the NetCalc tool (Networks, 2014), is simpler but provides less direct insight into the patterns of betweenhousehold ITN distribution that ultimately link net crops to access levels.
One aspect that is known to strongly influence the relationship between net crop and household ownership distribution is the size of households found in different countries (Networks, 2014; Yukich et al., 2013), which varies greatly across subSaharan Africa (Swaziland, for example has an average household size of around three members, while in Senegal the average is nearly ten). Household size also, of course, determines whether a given number of owned nets will be sufficient to provide access to all residents. We extended earlier analyses (Flaxman et al., 2010; Yukich et al., 2013) to explicitly account for household size: using a bivariate (i.e. two rather than onedimensional) histogram model to link net crop to ownership distributions for each household size stratum (see Supplementary Information). We replaced the negative binomial distribution with a 2d zerotruncated Poisson distribution and, for each household size stratum, fitted the distribution using two parameters: (i) the proportion of households with zero ITNs and (ii) the mean number of ITNs per ITNowning household. Using the householdlevel data from 83 national surveys, we found that both parameters were strongly related to national net crop, allowing bivariate histograms to be generated for every countryyear that were closely representative of the true ITN ownership distribution.
Stratifying our analysis by household size had three important advantages over earlier approaches. First, the distribution of net ownership tended to vary substantially between households of different sizes within a given country and this variation would be missed if all households were considered together. Accounting for this enabled better fits to the data. This makes sense: all else being equal, larger households would be expected to own more nets than smaller ones and so distribution patterns would differ systematically. Second, the bivariate ownership histograms predicted for each countryyear could be used to directly calculate all three indicators of household access. While a simple univariate histogram allows calculation of % households with at least one ITN, a bivariate histogram means the number of both ITNs and people in every household can be triangulated which, in turn, allows direct calculation of the two additional indicators: % households with at least one ITN for every two people and % population with access to an ITN within their household, along with the 'ownership gap' (see Supplementary Information). Linking these bivariate histograms to our annual net crop estimates for each country meant we could predict timeseries of the access indicators at the national level from 2000–2013, with all parameters fitted in a Bayesian framework providing posterior credible intervals around each timeseries. We also combined the countrylevel results to generate a set of continentlevel indicator timeseries, representing overall coverage levels among populations at risk in the 40 endemic countries. Third, the bivariate histograms allowed analysis of overallocation: certain cells of the histogram represented households owning more ITNs than were required to achieve access on a onebetweentwo basis, and the proportion of the total net crop falling in this category was examined through time for every country.
We took a different approach for the final indicator, % population who slept under an ITN the previous night. ITN use is less directly linked to national net crop and is primarily determined by the availability of nets within households (Eisele et al., 2009). A total of 83 of the 102 national surveys contained data allowing the relationship to be explored between ITN use and each of the three access indicators with, perhaps unsurprisingly, % population with access to an ITN within their household displaying the largest correlation (adjusted R^{2}= 0.96). We fitted this relationship across the 83 surveys using a simple Bayesian regression model (see Supplementary Information) and used it to predict timeseries of the ITN use indicator for every country. The ratio of population use to access revealed the 'usage gap'—the fraction of the population with access to ITNs not using them—and betweencountry variation in this ratio was also explored.
Estimating ITN requirements to achieve universal access
Request a detailed protocolOur twostage modelling framework represented the pathway from ITN delivery into countries through to resulting levels of net access and use in households. It also accounted for two potential factors that act to reduce access levels, and allowed these to be quantified through time for each country. Using this architecture, it was possible to simulate delivery of any hypothetical volume of ITNs to a given country over a given future time period, to predict the levels of access and use that would result, and to examine the impact of different amounts of overallocation and net loss. The current needs assessment exercise that countries are undertaking (RBMHWG, 2014; RBM, 2014) is designed to identify the number of LLINs required to achieve coverage targets by 2017. We used our model to estimate the levels of access likely to be achieved if these forecast LLIN commodity needs were met across the 2014–2017 period under a 'business as usual' scenario, that is, with current levels of overallocation and net loss, and compared these predicted levels with the objective of universal access among target populations. We then generalized this experiment to predict the likely level of coverage (for all four indicators) achievable by 2017 under a broad spectrum of LLIN delivery levels, equivalent to a total for subSaharan Africa (two of the 40 endemic countries in our study did not participate in the RBMHWG needs assessment exercise [Djibouti and Equatorial Guinea], and so our scenario analysis is based on the set of 38 remaining countries; to maintain comparability through time, we combined needs assessment data for mainland Tanzania and Zanzibar, and for Sudan and South Sudan) of between zero and 2.5 billion nets across the 4year period. Further, we ran these simulations under four scenarios: (i) 'businessasusual' (where current levels of overallocation and net loss were maintained); (ii) with no overallocation (new LLINs are distributed preferentially to those households with zero LLINs, then to those with less than onebetweentwo); (iii) with reduced LLIN net loss by households (using a modelled 3year median retention time); and (iv) with both no overallocation and a 3year median retention time.
Appendix 1. Supplementary information on data and methods
1.0 Outline of document
In this Supplementary Information document, we augment the main manuscript by providing additional details on the data and modelling architecture developed in this study to allow prediction of insecticide treated bednet coverage, system efficiency and future needs. In Section S2, we describe the acquisition and processing protocols for data from national household surveys, NMCPs and ITN manufacturers. In Section 3, we describe the Bayesian compartment model which estimates, at quarterly intervals, the total number of ITNs in households in each country (the net crop). In Section 4, we describe a second modelling stage that predicts nationallevel ITN coverage indicators as a function of predicted net crop. Finally, in Section 5, we describe the use of the modelling framework to predict future ITN coverage levels under a range of hypothetical ITN delivery volumes and efficiency scenarios.
2.0 Data collection
The various modelling stages were fitted to data from three principal sources.
2.1 ITN manufacturer reports
Manufacturer reports provided information on the number of LLINs delivered to each country each year by international manufacturers. These data were provided to the WHO by Milliner Global Associates and were complete for each country from 2000 to 2013 inclusive.
2.2 National malaria control program reports
NMCP reports provided information about the number of LLINs and cITNs distributed in a county within a given year. These data were provided to WHO by NMCPs and were available for 365 of the 560 countryyears addressed in the study. We treated these data as only partial records of distribution activities because the extent to which NMCP reporting captures distribution by nongovernment agencies is not known for all countries.
2.3 Household survey reports
We identified and obtained data for the ownership and use of ITNs from household surveys conducted in subSaharan countries since 2000, including DHS, MIS, MICS, AIDS indicator surveys (AIS) and a malaria and anaemia prevalence survey (EA & P) (Supplementary file 1). Data at the household level were acquired from 95 national surveys from 39 countries from 2001 to 2014. In addition, we acquired national level data from four household survey reports for which we were unable to obtain the household level dataset. The range and number of the household survey data collected is depicted in Appendix Figure 7. For those surveys where householdlevel data were available on the type of nets owned (see Supplementary file 1), the number of ITNs owned was determined by the sum of each ITN in the household. A net was considered an ITN if it was an LLIN, or a pretreated net obtained within the past 12 months, or a net that has been soaked with insecticide within the past 12 months. ITNs were then subdivided into the two classes, LLINs and cITNs.
For the surveys where data on the type of net was only available for one net in each household (see Supplementary file 1), the overall surveylevel proportion of total nets for each net classification (nonITN, LLIN, cITN) was determined and multiplied by the number of nets in the surveyed households to estimate the number of LLINs and conventional ITNs owned by each surveyed households.
3.0 Compartment model
3.1 Introduction
This section describes our implementation of a Bayesian hierarchical model to impute, at quarterannual intervals, the total number of insecticidetreated bed nets (which can be LLINs or cITNs) in a given country.
To achieve this goal, we built a model incorporating the three distinct sources of information on ITNs described in the preceding section: manufacturer reports on ITNs delivered to countries, NMCP reports on ITNs distributed to households within countries, and household survey data providing direct crosssectional estimates of net crop in households at a given time point.
The key challenge in our model was linking the delivery and distribution data sources to the net crop measurements. Figure 2 shows our schematic representation of the system and the evidence synthesis of these three data sources. In a given year, a given volume of nets are delivered to a country from manufacturers (Appendix figure 8, green arrows), giving rise to a (as yet undistributed) country stock (Appendix figure 8, orange arrow). Nets from this stock then may be distributed to households in that or subsequent years, as captured by NMCP distribution data (Appendix figure 8, blue arrows). To link the NMCP distributed nets to the direct estimates of the net crop existing within households, we needed to account for the rate of loss of nets from households. That is, once nets are distributed, they remain within households for a given length of time until they are discarded. By tracking the net loss from NMCP deliveries (Appendix figure 8, brown arrows), we were able to estimate the total number of nets in a country at a given time point (e.g. in 2008, when a household survey was conducted [ Appendix figure 8, red arrow] by summing the nets of all ages [Appendix figure 8, purple line]). The compartment model was therefore calibrated by parameterising the loss function in such a way that the net crop observed in each national survey was consistent with the known influx of nets to households (following manufacturer delivery and NMCP distribution) and an estimated rate of loss from households. Our loss function was modelled to be temporally varying for a given country and parameterises the proportion of nets discarded through time.
3.2 Model specification
3.2.1 Observed data
3.2.1.1 Observations of net crop from household surveys
Conceptual description
Household survey reports provided a direct measure of how many ITNs, of any age, existed in households in a country over a sampling period. For use in fitting the compartment model, householdlevel survey data on ITN ownership needed to be translated into a representative estimate for the total number of nets in a given country. To accomplish this, we summarised each household survey into two variables: the average number of LLINs and cITNs per household and the average household size (i.e. number of residents). The total number of ITNs was then calculated as the product of the population of the country at the time of the survey and the estimated ITNs percapita as observed in the survey data, taking into account the survey weighting to ensure the arithmetic mean was nationally representative. The calculation of the total number of nets was completed by specifying probability distributions that allowed propagation of uncertainty from the household surveys.
Formal description
To estimate national net crop (total ITNs in households) from each national survey, three summary statistics were required:
a) The average household size, $h{h}_{\mu ,}$, with an associated standard error, $h{h}_{\sigma}^{2}$.
b) The average number of LLINs per household, $AvgLLI{N}_{\mu \text{'}}$, with an associated standard error, $AvgLLI{N}_{\sigma}^{2}$ .
c) The average number of cITNs per household, $AvgcIT{N}_{\mu \text{'}}$, with an associated standard error, $AvgcIT{N}_{\sigma}^{2}$.
It should be noted here that, throughout this Supplementary document, we use the term standard error when referring to the standard deviation of the sample mean or the variability of an estimator.
Survey data came from four different sources—DHS Program, MICS4, MICS3 and other surveys for which only the published report (and not underlying data) were available, mainly MIS undertaken unilaterally by NMCPs or supporting partners. Different protocols were required to obtain summary statistics (a), (b), and (c) above for each survey type.
DHS and MICS4 data: These household survey reports provided direct data about the number of LLINs and cITNs per household and the household size. Therefore, means across all households were calculated using a weighted average that incorporated the published survey sample weights and the standard errors obtained through Taylor linearization (Kish and sampling, 1965).
MICS3 data: These household surveys provided direct data about the household size and number of bed nets per household, but did not provide information on the type of each net observed (e.g. LLINs, cITNs or untreated bet nets). The surveys did, however, provide a full description of a single net within each household. Therefore, across all such single nets in each survey, we determined the mean proportion of nets that were LLINs, cITNs or untreated nets along with the standard errors. We then multiplied these proportions with the computed average number of nets (of any kind) to determine the average number of LLINs and cITNs per household. Propagation of uncertainty was achieved using MCMC sampling (Plummer, 2003; Gelman et al., 2013).
Other reports: These surveys contained no disaggregated household data and only reported the averages of household size and numbers of LLIN and cITN nets with no standard errors. We therefore assumed a small 1% error on these estimates, consistent with the magnitude of sampling errors seen in other surveys.
Using metrics (a), (b) and (c) obtained from the 95 processed surveys, the total number of LLINs and associated standard error in a country reported by a survey were defined probabilistically as:
And similarly the total number of cITNs:
Where $\text{\Epsilon}[\bullet ]$ is the expected value, $\sqrt{\text{VAR}[\bullet ]}$ is the standard error, c is a given country and $\overline{t}$ is the mean sampling time of the survey. Equations 1–4 ${\mu}_{SURVE{Y}_{LLIN}{}_{\overline{t},c}},{\sigma}_{SURVE{Y}_{LLIN}{}_{\overline{t},c}}$,${\mu}_{SURVE{Y}_{cITN}{}_{\overline{t},c}}$ and ${\sigma}_{SURVE{Y}_{cITN}{}_{\overline{t},c}}$ were therefore observed data inputs into the compartment model.
3.2.1.2 Manufacturer reports of LLIN deliveries
As shown in Appendix figure 8, manufacturer reports provided data on the number of LLINs delivered to a country in a given year. There were no corresponding reports for cITNs.
Conceptual description
The main purpose of the manufacturer data was to determine stock levels and ‘cap’ estimated NMCP distributions in each year (i.e. more nets could not be distributed than were potentially available in country stock). The manufacturer reports were complete (no missing values) and assumed to be of high fidelity. We therefore modelled the likelihood of the manufacturer data in a given countryyear as a normally distributed random variable. The manufacturer reports did not report standard errors in the delivery numbers and therefore we assumed a uniformly distributed prior probability on the error. Because the manufacturer data served only to inform stock and cap NMCP distributions, no considerations of subannual timing were required.
Formal description
We define $Manufacture{r}_{c,t}$ (observed data) as the number of manufacturer LLIN nets delivered to a country c at year t. The number of manufacturer LLIN nets sent (${\mu}_{c,t}$) was modelled as:
With error ${\sigma}_{m,t}=Uniform(0,0.075)$0.3
3.2.1.3 NMCP reports of ITN distributions
As shown in Appendix figure 8, NMCP reports provided information on the number of both LLIN and cITNs nets distributed in a given country c at year t.
Conceptual description
As with the manufacturer data, the NMCP reports were assumed to be of reasonably high fidelity. Unfortunately, NMCP reports were not complete and contained missing values where countries failed to report the number of nets distributed in a year. To impute these missing values, we defined an informative prior probability distribution on the NMCP distributions for both LLINs and cITNs. We tested multiple different parameterisations of NMCP prior distributions, and evaluated performance of parameterisations using outofsample cross validation to choose the best model.
Our final choice of NMCP prior distributions was datadriven, using combined reports across all African countryyears that had NMCP data. We scaled the reports to percapita NMCP distributions to remove countryspecific differences. We observed that this combined percapita distribution approximately followed an exponential distribution with a zero inflation hurdle (to account for no deliveries). Additionally, when looking at the combined reports across time, it was clear that combined distribution varied temporally. We therefore modelled NMCP LLINs and cITNs separately for each year by disaggregating the continent wide percapita distribution temporally into separate exponentials with zero inflation. Finally, to account for variability due to sample size, we fitted splines through these time series.
Formal description
We defined $\text{NMCP}\_{\text{LLIN}}_{c,t}$ (observed data), $\text{NMCP}\_{\text{ITNc}}_{c,t}$ (observed data) and $\text{NMCP}\_{\text{TOTAL}}_{c,t}$ (observed data) as the number of NMCP LLIN nets, NMCP cITN nets and the sum of LLIN and ITN NMCP nets reported distributed in a country c at time t.
As described above, the NMCP reports did not have standard errors and contained missing values. We therefore defined informative prior distributions on $\text{NMCP}\_{\text{LLIN}}_{c,t}$ and $\text{NMCP}\_{\text{cITN}}_{c,t}$. First, we defined two sets ${\mathcal{L}}_{t}$ and ${\mathcal{I}}_{t}$ as the combined set of percapita rates of NMCP LLIN and cITN across all African countries at times t respectively. The sets ${\mathcal{L}}_{T}\text{and}{\mathcal{I}}_{T}$ with $T\in [2000,2001,\dots ,2012]$ therefore contained all NMCP LLIN and cITN distributions across all countryyears.
To characterise the hurdle exponential distribution, we used two parameters: the zero inflated components (${p}_{LLIN}{0}_{t}$,${p}_{cITN}{0}_{t}$), which defined the probability that in a given countryyear no nets were distributed, and the exponential component (${p}_{LLIN}{1}_{t},{p}_{cITN}{1}_{t}$) which, given some distribution of nets took place, gave the percapita rate of distribution (i.e. how many nets percapita were distributed).
We therefore define the proportion of zero NMCP deliveries for LLINs and cITNs as:
And the mean delivery rate at time t as:
Using these two parameters we then defined the prior distributions on NMCP LLIN and NMCP cITN as:
Where:
The terms $rat{e}_{LLIN,t}$ and $rat{e}_{cITN,t}$ therefore characterised the prior distribution on the percapita rates of NMCP LLIN and cITN distributions, respectively. The values of terms $rat{e}_{LLIN,t}$ and $rat{e}_{cITN,t}$ also contained missing values for some years and showed variability due to different sample sizes. Therefore, we fitted penalised regression splines through the time series for each parameter to create smooth prior parameters. The spline fitting was done using restricted maximum likelihood with rigorous selection done to find the optimal number of basis functions for the spline (see Appendix figure 1 for the spline fits).
To form the likelihoods, these prior rates needed to be scaled by the population in a given countryyear. Therefore, we defined terms ${\delta}_{c,t}=Populatio{n}_{c,t}rat{e}_{LLIN,t}$ and ${\pi}_{c,t}=Populatio{n}_{c,t}rat{e}_{cITN,t}$ as the final prior distribution on the NMCP distributions (not percapita).
Using these prior distributions and allowing for some uncertainty, we model the likelihoods for the observed $\text{NMCP}\_{\text{LLIN}}_{c,t}$, $\text{NMCP}\_{\text{ITNc}}_{c,t}$ and $\text{NMCP}\_{\text{TOTAL}}_{c,t}$ as random normal variables with added error:
Where ${\sigma}_{d1,t}={\sigma}_{d2,t}={\sigma}_{d3,t}=Uniform(0,0.01)$. Additionally, all distributions (equations 14 and 16) were zero truncated to prevent negative numbers of nets.
It should be noted that NMCP distribution numbers are not always accurate and potentially under or overestimate the number of nets distributed. This may result, for example, from nets being distributed through other sources and therefore not contributing to the total NMCP distribution sum. To account for this uncertainty, we allow the number of nets distributed to take a uniformly distributed value with a lower bound of the NMCP distribution and an upper bound of the number of nets able to be distributed (found from the net stock—see Equation 22).
3.2.2 Compartment model structure
Following the methods outlined above, we arrived at a set of observed data, with standard errors, of the number of nets delivered for all countryyears (manufacturer), the number of nets distributed withincountry for all countryyear (NMCP) and sparse estimates of the number of nets of any age in households (i.e. net crop) in 95 countryyears with available surveys
Our compartment model linked all these processes together by modelling four different processes. (i) Using delivery and distribution information to allow for LLIN net stock to accumulate, thereby allowing for more LLIN nets to be distributed than were delivered in a countryyear. (ii) Disaggregating the total nets delivered in a countryyear into quarteryearly intervals to allow for a more realistic modelling of the temporal dynamics. (iii) Linking yearbyyear distributions through a loss function that accounted for the rate of nets being lost from households after distribution as a function of the time from distribution. (iv) Calibrating the compartment model quarteryearly estimates on the observed survey reports. These four modelled processes are now described in turn.
3.2.2.1 Accumulation of LLIN net stock
Conceptual description
Following the methodology described in Flaxman et al (2010) (Flaxman et al., 2010), we define a net stock variable $Stoc{k}_{c,t}$ for a given countryyear. $Stoc{k}_{c,t}$ links together LLIN manufacturer deliveries and NMCP LLIN distributions by allowing for a surplus of nets to be built up in a country. Essentially, $Stoc{k}_{c,t}$ added a ‘cap’ on the number of nets NMCPs report to be delivered, and therefore created an upper bound on erroneously reported deliveries.
We also allowed for NMCP data to represent an underestimate of true distribution levels. This could occur, for example, if the NMCP reporting system did not capture those nets being distributed by nongovernmental agencies. To accommodate this uncertainty, we specified the number of nets distributed in a given year as a range, with all available country stock as one extreme (the maximum nets that could be delivered) and the NMCPreported value (the assumed minimum distribution level) as the other. It should be noted that due to the lack of manufacturer data for cITNs, this uncertainty was only incorporated for LLINs.
Formal description
First we define ${\widehat{\delta}}_{c,t}$, the adjusted ${\delta}_{c,t}$, as the modelled parameter for the number of NMCP LLIN nets delivered in a countryyear. We define ${\widehat{\delta}}_{c,{t}_{0}}$ and $Stoc{k}_{c,{t}_{0}}$ at the first time point ${t}_{0}=2000$ as:
And then define ${\widehat{\delta}}_{c,t}$ and $Stoc{k}_{c,t}$ at subsequent time points recursively as:
As shown in the above equations, if a country did not distribute as many nets as were delivered, stock levels can increase, but with a limit that a country cannot deliver more nets than stock permits. ${\widehat{\delta}}_{c,t}$ has a probabilistic interpretation reflecting our uncertainty about whether the NMCP values reported the total number of LLINs that were able to be distributed or if the calibration of the stock and flow model on the survey data required more nets to be delivered. For cITNs, it was not possible to include a stock component in the compartment model as there were no manufacturer reports for cITNs.
3.2.2.2 Temporal disaggregation of NMCP LLIN and cITN deliveries
Conceptual description
Modelled variables ${\widehat{\delta}}_{c,t}$ and ${\pi}_{c,t}$ defined the number of LLINs and cITNs distributed within a countryyear. However, these variables were modelled on data that did not provide any information about when in the given year nets were distributed. This led to the potential for temporal inconsistency when calibrating survey estimates at an average time in a countryyear against NMCP distribution information with nosub annual temporal resolution. Therefore, we needed to specify a prior on when NMCP distributions occurred within a given countryyear. In striving to keep the model as parsimonious as possible, we first modelled a scenario where all NMCP nets were distributed at either the start or end of the year. This, however, did not represent reality adequately, and led to poor calibrations with survey estimates in some instances. We then relaxed this assumption to allow all nets to be delivered at a random point in the year, but again this led to poor calibrations. Finally, we opted for a more realistic distribution scenario where we disaggregated distributions to a quarter yearly temporal resolution. We then assigned priors on NMCP quarterly net distributions to allow any proportion of nets to be delivered at the start, first quarter, second quarter, third quarter or end of the year. This scheme allowed for maximum flexibility in the model with minimal subjective prior assumptions, and yielded excellent calibrations with survey estimates.
Formal description
We disaggregated modelled variables ${\widehat{\delta}}_{c,t}$ and ${\pi}_{c,t}$ (the number of LLINs and cITNs distributed within a countryyear) into intervals $Q\in [0.25,0.5,0.75,1]$, representing the number of nets distributed by the first, second, and third quarter or end of a given year. We defined the proportions of ${\hat{\delta}}_{c,t}$ and ${\pi}_{c,t}$ in each interval as:
Where $i\in Q$ and ${\sum}_{i\in Q}{q}_{i}=1$ and of course ${\sum}_{i\in Q}{\hat{\delta}}_{c,t}{q}_{i}={\hat{\delta}}_{c,t}$ and ${\sum}_{i\in Q}{\pi}_{c,t}{q}_{i}={\pi}_{c,t}$ i.e. the sum across the year is preserved.
3.2.2.3 Rate of net loss
Conceptual description
Our compartment model estimates NMCP distributions at quarterly intervals through time from manufacturer delivery data and the estimated stock accumulation. The final link to calibrate these quarterly distributions with survey observations of net crop is the rate of net loss. We model the net loss function as a smooth compactly supported function defined previously as part of the NetCALC tool (Koenker et al., 2013). We also model the loss function as non stationary in time, and represent this change through time using a moving average. By using a moving average as opposed to individual loss functions for each year, or quarter, we were able to learn temporal changes given the sparse data and not over represent the prior.
Formal description
We tried several different functional forms for net loss (Weibull, exponential, hill) and decided on specifying the form using a smoothcompact function defined previously (Koenker et al., 2013):
Where k and L are loss function parameters with $k,L>0$. The smoothcompact loss function produced models with a the lowest information criteria (DIC) from the other forms and has been validated in previous studies (Koenker et al., 2013; Yukich et al., 2013).
For both LLINs and ITNs we use the same functional form in Equation 25 but restrict the bounds on parameter k as uniform priors on this parameter produced strongly nonuniform functions (Bornkamp, 2012). Therefore, to achieve a diffuse uniform prior on the loss functions, we allowed L to vary within large bounds, thereby producing priors that allowed candidate loss functions with halflives from 0.7 to 5 years. These priors were necessarily vague to allow for adequate flexibility in fitting countryspecific loss functions.
To model the loss function through time, we define a moving average on parameters k and L. Therefore the moving average on the loss function for both LLINs and cITNs is defined as:
Where n is the moving average lag and $t\in \left[2000,2001,\dots ,2013\right]$, and any terms with $t<2000$ are ignored. From outofsample cross validation we found the optimal lag to be $n=5$, i.e. a balance between over and undersmoothing. It should be noted that t is restricted to the range $2000\le t\le 2013$, which is the range for which we have real data on NMCP reports, manufacturer reports and household surveys. For the future scenario predictions (described later), we assume any future net loss behaviour is the same as that occurring in 2013, that is:
3.2.2.4 Calibrating the compartment model against survey information
Conceptual description
Given the net distributions defined at quarterly intervals and the temporally varying loss function, a continuous prediction of the number of nets of any age in a country could be found at quarterly intervals simply by summing across nets of all ages for a given quarter. These quarterly predictions of the number of nets of any age were then calibrated in a likelihood of the observed survey estimates. In the presence of observed survey estimates, this likelihood helped ‘learn’ values for all the prior probability parameters outlined in the compartment model. In the absence of survey information, the model defaulted to the prior probabilities for all parameters and relied on the NMCP and Manufacturer reports.
Formal description
Consider the stock and flow model evaluated over a period 2000:2017, which yielded 73 quarterly intervals. Now consider two 73× 73 matrices labelled ${M}_{LLIN}$ and ${M}_{cITN}$. The rows and columns of these matrices represent the entire time period in quarterly intervals.
Consider the stock and flow model progressing column wise through these matrices, at year t and quarter $\widehat{t}\in Q$, the column and row index $ind=4t+\hat{t}$ (e.g 2004.5 or year 5 quarter 3 would be index 23) stores ${\hat{\delta}}_{c,t}{q}_{\hat{t}}$ and ${\pi}_{c,t}{q}_{\widehat{t}}$ (LLINs and cITNs in year t and quarter $\widehat{t}\text{}\in Q$).
Then for each quarter after the distributions ${\delta}_{c,t}^{adj}{q}_{\widehat{t}}$ and ${\pi}_{c,t}{q}_{\widehat{t}}$, the remaining nets in subsequent quarters were filled row wise according to the loss function defined in Equation 25.
By summarising the stock and flow process in this manner, the total number of LLIN and cITN nets of all ages in a given quarterly time period was simply the column sums ${\sum}_{j=1}^{73}{M}_{LLIN,ind}$, ${\sum}_{j=1}^{73}{M}_{cITN,ind}$.
Finally, we calibrated estimates of the total number of nets of all ages, against those reported from the household survey reports:
Where the normal standard deviation is given by those found from the survey reports (Equations 2 and 4). Additionally, ${P}_{LLIN}$ and ${P}_{cITN}$ was defined as a linear interpolation between the two closest quarters and the average survey time $\overline{t}$.
It should be noted that, when calculating the nets percapita, two scalings were used: (1) Countries with the proportion of population at risk being less than 1 were scaled according to the WHOdefined populations at risk proportion. (2) Countries partially dependant on IRS as a means of vector control were scaled as the proportion of the population at risk targeted with ITNs.
4.0 Indicators model
4.1 Introduction
Section 2 provides details on how, using yearly data on manufacturer deliveries and NMCP distributions calibrated using household survey reports, we estimated the number of LLINs and cITNs in households in each countryyear at quarterly intervals. From these net crop estimates, we used population information to derive the nets percapita for LLINs and cITNs. Standardized ITN coverage indicators (Kilian et al., 2013) were then estimated from net crop by leveraging the household survey information with the estimates of nets per capita to derive a set of indicators on net ownership and usage. These were:
Indicator 1: % households with at least one ITN
Indicator 2: % households with at least one ITN for every two people
Indicator 3: % population with access to an ITN within their household
Indicator 4: % population who slept under an ITN the previous night
Indicator 5: The ratio of population use to access or the ‘ownership gap’
4.2 Indicator model structure
Previous models attempting to evaluate total nets and nets percapita have utilised negative binomial models unstratified by household size to estimate Indicator 1 (Flaxman et al., 2010). However, using these previous approaches, it was impossible to estimate Indicators 2–5. Here we introduce a new zerotruncated Poisson model stratified by household size, which has the ability to estimate all Indicators 1–5 with excellent precision.
To begin the model derivation, consider a household survey $H$. Contained within $H$ we were able to calculate a density/histogram of the number of households with a given number of ITNs (both LLINs and cITNs). Appendix figure 2 summarises this density plot, and it is clear that Indicator 1 is trivially calculated (sum of the red bars divided by the total) from this histogram, but Indicators 2–5 are not. Previous modelling approaches (Flaxman et al., 2010) used this unstratified density and assigned a probability distribution (e.g negative binomial or Poisson) parameterised such that the observed density could be recreated using a small number of model parameters (for the negative binomial 2 and Poisson 1).
There are two key problems with this approach, first as highlighted above Indicators 2–5, which provide additional richness of information for decision makers, cannot be directly calculated from this onedimensional histogram. Second, after experimenting with a large suite of probability distributions, we found that fitting twodimensional summary histograms to household survey data often provided very poor fits. The key to these poor fits is the lack of stratification of the number of ITNs by household size, the absence of which ignores an important determinant on the number of ITNs per household. A more useful summary of $H$ is the inclusions of a second dimension for household size (Appendix figure 3). From this twodimensional model, it becomes possible to estimate Indicators 1–3 (Appendix figure 4). However, the problem remains: how do we recreate this twodimensional density when we have no household survey information?
To accomplish this, we developed a model which, given a household size distribution, translates an estimate of nets percapita (derived from the compartment model) into an accurate realisation of the threedimensional histogram.
4.2.1 Zero truncated poisson model
The most logical model to recreate the threedimensional densities for a given household strata is the Poisson distribution (or a negative binomial distribution for added over dispersion). However, we found that these models did not recreate the observed pattern accurately. We tried more complicated zeroinflated versions but these did not improve fits.
After looking across all 83 surveys for which we had all the relevant information to recreate the threedimensional histograms, we realised the process to create the histograms had to be separated into two processes: (a) a process which, for a given household strata, gave the density of households with no nets $({P}_{0})$, and (b) a process which, for a given household strata owning nets, gave the density of a given number of nets (1,2,3…) $({P}_{1})$. To think about this intuitively, consider a process that first fills the zero category of ITNs per household in Appendix figure 3, and then fills the categories 1,2,3 etc.
Consider a household strata $h$ (e.g. households of size three persons) from $H$, it is easy to calculate the proportion of households with no nets. This is the ${P}_{0,h}$ parameter. Of the remaining households owning one or more nets, we calculated ${P}_{1,h}$ as the average number of nets in a household strata. The most logical probability distribution to fill the densities given ${P}_{1,h}$ is again the Poisson distribution as ${P}_{1,h}$ is simply the mean of the Poisson.
However, because we have already filled the household with no nets density, the correct distribution is a zero truncated Poisson distribution. Unfortunately, the mean of the zero truncated Poisson distribution is now no longer just ${P}_{1,h}$ but $\mu =\lambda /(1{e}^{\lambda})$, which does not have the same useful interpretation. Therefore, we solved (using a simple root finding) for the value of $\lambda $ that gave a zero truncated Poisson with the same mean as a standard Poisson with mean ${P}_{1,h}$, but excluded a zero category.
Using this model, parameterised by just two parameters per household strata ${P}_{0,h}$ and ${P}_{1,h}$, we evaluated Indicators 1–3 across the 83 relevant surveys with correlation values of more than 0.98 showing that the model reproduced the complex density pattern in the twodimensional histograms with excellent accuracy.
4.2.2 Zero truncated poisson model in the absence of survey information
The zero truncated Poisson is a probabilistic distribution that translates $\mathscr{H}$ into parameters ${P}_{0,h}$ and ${P}_{1,h}$ that were able to recreate the twodimensional densities in Appendix figure 4 from which Indicators 1–3 could be calculated. However, we still needed to estimate ${P}_{0,h}$ and ${P}_{1,h}$ for country years without survey information.
Given the logical dependence of ${P}_{0,h}$ and ${P}_{1,h}$ on the underlying nets percapita, we created two functions which, given a household strata, h, translated nets percapita, npc, into ${P}_{0,h}$ and ${P}_{1,h}$. i.e.:
After experimenting with non parametric spline models, we found that simple polynomial surfaces worked remarkably well and had the added benefits of computational efficiency and compatibility with the compartment model.
We divided household sizes into 10 sizes $\left(1,2,3,4,5,6,7,8,9,\ge 10\right)$ and then modelled ${f}_{0}\left(npc,h\right)$ as:
and ${f}_{1}\left(npc,h\right)$ as:
The model for ${f}_{0}\left(npc,h\right)$ is therefore a twodimensional surface that varies according to household size but the model for ${f}_{1}\left(npc,h\right)$ is a separate linear straight line function for each household category.
Equations 34 and 35 were fitted using Bayesian linear regression with the uncertainty in the coefficients being propagated through the compartment model.
When performing 10fold outof sample cross validation (leaving out entire surveys), we found that ${f}_{0}\left(npc,h\right)$ predicted ${P}_{0,h}$ with a correlation of 0.98 and ${f}_{1}\left(npc,h\right)$ predicted ${P}_{1,h}$ with a correlation of 0.97, indicating extremely good fits.
Given these two functions, which parameterise the zero truncated Poisson model, we can calculate Indicators 1–3 from an estimate of nets percapita for a countryyear (whether we have a survey or not) from the resulting twodimensional density.
4.2.3 Estimating % population who slept under an ITN the previous night and the 'ownership gap'
The proportion of people who slept under an ITN (Indicator 4) was highly correlated with the proportion of people with access to an ITN (the ‘use gap’, Indicator 5). Therefore, to evaluate Indicator 4, we used a simple linear relationship between access and use evaluated across all 83 surveys with the relevant information (see Appendix figure 5). Therefore, all that was required to evaluate Indicator 4 was to take Indicator 3 (which contained all the rich information about household size strata) and translate it through a linear relationship with noise:
Finally, Indicator 5 (the ownership gap) was calculated as 1–(Indicator 4/Indicator3).
4.2.4 Additional note on household sizes
It should be noted that one missing piece in this analysis is the distribution of household sizes for every countryyear. This information does not exist and is very difficult to model. Therefore, we make two assumptions. First, while the populations are known to change over time, we assume that in the 13year window of our analysis the distribution of household sizes stays constant within each country. This assumption is, to some degree, warranted as countries with serially sampled surveys showed extremely similar household size distribution patterns, and the resulting indicators do not change significantly if a different time point household size distribution is used. Second, for countries with no household size information (due to no relevant surveys), we use an average across all surveys.
5.0 Future predictions
Using the methods described in sections 2 and 3, we were also able to simulate the delivery of any volume of ITNs to a given country over a given future time period to predict the nets percapita and full suite of indicators. Additionally, we were able to change the dynamics of this simulated future period to allow nets to be retained for a longer period (by varying the net loss function prior) and account for overallocation of nets (where there is a skewed distribution of net distributions in households with some households having too many nets and some too few).
When simulating forwards in time from 2013, we made several assumptions:
No cITNs were distributed or delivered. This is a justifiable as, with the exception of Gabon in 2013, none of the 40 countries in our analysis delivered or distributed any ITNs in 2012 or 2013. Therefore, it is reasonable to assume that for future years, these countries continued using LLINs exclusively. This assumption also follows country recommendation by the WHO to distribute LLINs and not cITNs (Measure, 2014).
Because only projected information on manufacturer data is known, we assume that all nets delivered to a county are distributed. It should be noted that the longterm consequences of this assumption are insignificant as data from 2000–2013 show that nets that are delivered are ultimately distributed, however, shortterm particulars of how many nets are distributed and how many are retained as stock are not captured.
Given a number of nets delivered to all of Africa during the period 2014–2017 (e.g. 500 million LLINs), we use the yearly proportions defined by the harmonisation working group (RBMHWG) to determine how many nets a country gets in a given year. The working group estimates represent the most reliable projection estimates available.
If the loss function was chosen to be that fitted within the model, we assume the loss function for future years was fixed to that learnt for 2013. We justify this assumption by looking at the changes in loss function on fitted data from 2000–2013, which suggests that the loss function does not vary dramatically and is stable temporally.
5.1 Future scenarios
For the future scenarios the number of nets delivered to all of Africa during the period 2014–2017 was varied from 0 nets to 2.5 billion nets with increments of 100 million nets.
Business as usual
In the businessasusual scenario, the loss function was fixed to that learned for 2013 and no accounting for overallocation was performed.
Minimised overallocation
In the minimised overallocation scenario (where over allocation is set to zero), the loss function was fixed to that learned for 2013. To account for overallocation, we ran the compartmental model to estimate net crop, as we would have under the business as usual scenario. However, when evaluating the coverage indicators, we implemented a 'redistribution' algorithm that operated on the modelled twodimensional histogram, summarising the proportion of households of a given size having a given number of nets for each countryyear. From this histogram, we removed those 'surplus' nets from households that that would normally occur due to overdistribution under businessasusual. These were then redistributed to households with two few nets. This reallocation started with the biggest households with the largest shortfalls and, while surplus nets remained, continued into progressively smaller households until, potentially, every single household had sufficient nets for onebetweentwo.
Longer net retention
In the longer net retention scenario, we did not use the loss function fitted in the model, but rather defined a fixed loss function with a mean halflife of 3 years. The prior uncertainty was taken as the standard deviation across all actual fitted loss functions calculated retrospectively.
References

Assessment of insecticidetreated bednet use among children and pregnant women across 15 countries using standardized national surveysThe American Journal of Tropical Medicine and Hygiene 80:209–214.

Protective efficacy of interventions for preventing malaria mortality in children in plasmodium falciparum endemic areasInternational Journal of Epidemiology 39 Suppl 1:i88–i101.https://doi.org/10.1093/ije/dyq026

Insecticidetreated bed nets and curtains for preventing malariaThe Cochrane Database of Systematic Reviews CD000363.https://doi.org/10.1002/14651858.CD000363.pub2

Child coverage with mosquito nets and malaria treatment from populationbased surveys in african countries: a baseline for monitoring progress in roll back malariaThe American Journal of Tropical Medicine and Hygiene 71:232–238.

JAGS: a program for analysis of bayesian graphical models using gibbs sampling.Proceedings of the 3rd International Workshop on Distributed Statistical Computing (DSC 2003) pp. 20–22.

ConferenceChanges to guidance for vector vontrol indicatorsSeventeenth Meeting of the RBM Partnership Monitoring and Evaluation Reference Group (MERG).

Websitehttp://www.childinfo.org/ecd_indicators_mics.htmlECD Indicators in Multiple Indicator Cluster Surveys (MICS).

BookWorld Population Prospects: The 2012 Revision Population DatabaseNew York: United Nations Population Division.

BookMethods for Achieving Universal Coverage with LongLasting Insecticidal Nets in Malaria Control Vector Control Technical Expert Group Report to MPACGeneva: World Health Organization.

BookGlobal Technical Strategy for Malaria 2016–2030Geneva: World Health Organization.
Decision letter

Catherine KyobutungiReviewing Editor; Africa Population Health Research Center, Kenya
eLife posts the editorial decision letter and author response on a selection of the published articles (subject to the approval of the authors). An edited version of the letter sent to the authors after peer review is shown, indicating the substantive concerns or comments; minor concerns are not usually shown. Reviewers have the opportunity to discuss the decision before the letter is sent (see review process). Similarly, the author response typically shows only responses to the major concerns raised by the reviewers.
Thank you for submitting your work entitled "Coverage and system efficiencies of insecticidetreated nets in Africa from 2000 to 2017" for peer review at eLife. Your submission has been favorably evaluated by Prabhat Jha (Senior editor), Catherine Kyobutungi (Reviewing editor) and two reviewers, one of whom has agreed to reveal her identity: Eline Korenromp.
The reviewers have discussed the reviews with one another and the Reviewing editor has drafted this decision to help you prepare a revised submission.
Summary:
The authors, using data from 102 national surveys, triangulated against delivery data and distribution reports, developed a Bayesian model to generate yearbyyear estimates of four ITN coverage indicators: i)% households with at least one ITN; ii)% of children under five who slept under an ITN the previous night; iii)% household with at least one ITN for every two people; and iv)% of the population with access to an ITN within their household.
The authors also explored the impact of two potential 'inefficiencies': uneven net distribution among households and rapid rates of net loss from households. The findings show worsening of overallocation (a measure of inefficiency) over time as overall net provision has increased, and more rapid ITN loss from households than previously thought. This has implications on the universal coverage (only 77%) that can be expected from the current estimate of 920 million additional ITNs that have been computed as the need through 2017. By improving efficiency, however, the 920 million ITNs could yield population access as high as 95%.
The study presents an important situation analysis, of progress in distribution and resulting protective coverage with ITNs, the primary malaria prevention strategy in most of the highmalariaburden countries in Africa. The dynamic model developed for this purpose is elegant, building on and refining earlier analysis, and adds value by producing more credible and considerably different estimates of the extent that ITNs are misallocated to some households, and the high rates of loss of ITNs from households – with the resulting inefficiencies varying among countries. The uptodate estimates of ITN use will be useful to other researchers. Building on similar previous work, the authors have developed a model which allows them to break down coverage into estimates of multiple indicators that take account of the variation in household size, and this is an important advance on previous work. The paper also has projections of the coverage achieved if distribution can be done more efficiently or if people retain their nets for longer – this is also novel.
Essential revisions:
The areas of concern include the following:
1) The absence of concrete suggestions on how national malaria control programs could aim to reduce the identified inefficiencies, specifically overallocation of ITNs. The authors should give examples of the operational strategies for better targeting and/or prescreening of households' current ITN situation, and indicate which of these has proven to work, in what settings.
2) Different choices could have been made in the modelling, and this makes some of the conclusions more uncertain than what is currently presented. The authors should discuss these points and consider which outputs are most sensitive to the modelling choices. The estimates of the various indicators up to the present might not be much affected, but the projections forwards would be.
3) The net loss function (the rate at which people discard nets) is estimated by combining the numbers of nets entering a country, the number reported as distributed by control programmes, and the numbers owned by households. Flaxman et al. found that nets were retained were much longer than would be expected from matching up the distribution data and coverage data from crosssectional surveys and as a result they introduced bias parameters to account for the discrepancy between data sources. It is not clear how this was handled in the modelling. The authors should describe this aspect of the methodology and the implications on the findings.
Minor points:
The estimated sharp fluctuations over time in ITN coverage indicators in some countries: are these real (if periodic campaigns dominate distributions?), or reflecting missing or incomplete data, e.g. in NMCPreported distributions? In particular, comment on the odd time patterns in DRC and Ghana and/or a few other countries with irregular patterns.
[Editors' note: further revisions were requested prior to acceptance, as described below.]
Thank you for resubmitting your work entitled "Coverage and system efficiencies of insecticidetreated nets in Africa from 2000 to 2017" for further consideration at eLife. Your revised article has been favorably evaluated by Prabhat Jha (Senior editor), a Reviewing editor, and two reviewers. The manuscript has been improved but there are some remaining issues that need to be addressed before acceptance, as outlined below:
The reviewers still have concerns about the conclusions on net retention. The model in this paper only estimates ITN retention indirectly while the six studies that Flaxman depended on were directly measuring it. The bias parameters of Flaxman et al. were introduced because of a discrepancy between the different sources of data, if a loss function of longer duration is used. This discrepancy could be due to systematic differences between the data sources, and so the reviewers feel that the conclusion of the present paper that nets are retained for a shorter time than was previously thought is less certain than the authors suggest. This also has implications for the point about sharp fluctuations in the predicted ITN coverage over time, which presumably would be smoother with a loss function of longer duration.
We recommend that the authors conduct a sensitivity analysis that assumes longer retention of nets, showing how any projections change. In any case this uncertainty should be acknowledged and the implications for the projections discussed.
In addition, net loss and retention consider the physical presence of the ITNs in households, but not their effectiveness in repelling mosquitoes. According to manufacturer and WHOPES evidence and guidance on ITN's effective durability, is it justified to assume a median 36month retention (in the optimized scenario), or does this imply a proportion of ITNs modelled as effective for longer than is realistically assumed? The authors should expound on this point in the discussion – for LLINs specifically
For the countrylevel estimates: Are the 4 international indicators (ownership, access, usage) calculated and shown for each country's overall national population (including subpopulations at no risk, just as the DHS, MICS and MIS surveys) or for the subpopulation at risk of malaria? What assumptions, if any, are made about (mis)allocation of ITNs to populations not at risk of malaria? Please clarify this in the appropriate section of the manuscript
Minor suggested edits:
If the world limit allows, please define 'Population access' in the Abstract.
Paragraph three, subsection “Net Loss”, 'minimised overallocation': From the annex (and subsection “Estimating ITN Requirements to Achieve Universal Access”) it seems that this is ZERO overallocation; please qualify or rephrase accordingly.
a) Parts of the diagram are confusing or not clear – please correct as needed:b) An arrow (or, equality sign) is missing from HHlevel ITN ownership to the box below (National total ITNs in HH?). c) An arrow missing from National net crop to, Observed bivariate histogramsd) National net crop equals National total ITNs in HH 5 boxes below, but the equality is not indicated.
Appendix 1: The scaling across countries to get to continenttotals is not very clear.
https://doi.org/10.7554/eLife.09672.021Author response
Essential revisions:
The areas of concern include the following:
1) The absence of concrete suggestions on how national malaria control programs could aim to reduce the identified inefficiencies, specifically overallocation of ITNs. The authors should give examples of the operational strategies for better targeting and/or prescreening of households' current ITN situation, and indicate which of these has proven to work, in what settings.
In this work we sought simply to identify and measure the magnitude of overallocation and underretention that exists and the changes in these factors over time. We felt this was a vital initial step and, of course, agree that it now paves the way for subsequent work to explore strategies for better targeting. We would propose, though, that this substantial body of new work would form one (or likely many) new studies in their own right, and would not be appropriately placed within the current submission which already covers a lot of new ground.
Solving the over allocation problem from an operational standpoint is something currently being discussed in policy fora. In the early years of integrated campaigns nets were allocated to a specific target group (e.g. children under five), but since the shift of policy to universal coverage this has become more difficult. Currently the distribution of nets requires a registration process within a county in order to identify beneficiaries thereby leaving large scope for under and overallocation. Work has been done at the Global Fund to strengthen the registration process but they have yet to identify concrete solutions that work at scale. Mozambique, Burundi and Madagascar have implemented systems based on village hierarchy that have been partially effective at small scales, but these do not lend to implementations at the scale of our analysis (country or continent level). Increasingly it is thought that solutions to the problem of over allocation need to be addressed in the planning stages rather than post hoc.
Given these complications, it is clear that defining strategies to address overallocation in the field will remain a challenge – one to which modelling may be able to make a contribution alongside those with expertise of the reality of the logistics on the ground. For now, however, we maintain that by identifying that the phenomenon of overallocation exists, is widespread, and contributing substantially to lower coverage yields, our study makes an important first step in motivating future work to improve distribution systems incountry.
2) Different choices could have been made in the modelling, and this makes some of the conclusions more uncertain than what is currently presented. The authors should discuss these points and consider which outputs are most sensitive to the modelling choices. The estimates of the various indicators up to the present might not be much affected, but the projections forwards would be.
We thank the reviewers for this comment. We felt that, because of the policy nature of this paper, most modelling assumptions were better described in full in the Appendix. For every component of the model we provide a conceptual and formal distribution of the model that outlines all the assumptions made and justifications for why we made them. As the reviewers expressed concern for the projection assumptions, we have added additional text to the assumptions section for future projections in the Appendix (Future predictions).
3) The net loss function (the rate at which people discard nets) is estimated by combining the numbers of nets entering a country, the number reported as distributed by control programmes, and the numbers owned by households. Flaxman et al. found that nets were retained were much longer than would be expected from matching up the distribution data and coverage data from crosssectional surveys and as a result they introduced bias parameters to account for the discrepancy between data sources. It is not clear how this was handled in the modelling. The authors should describe this aspect of the methodology and the implications on the findings.
The Flaxman et al. paper was the inspiration for this work. However, there were several fundamental problems with their work. Their loss function was constant over a given year with very minor reductions for subsequent years until all nets are discarded at once after three years (see brown line Figure 7). This function is almost piecewise linear and does not resonate with available evidence on net retention from field settings (e.g. see Killian et al.’s Net Calc). This limitation of their loss function was compensated for by introducing a ‘bias’ parameter that helped triangulate the entire modelling chain. As a result of this statistical compensation we do not believe their loss function represented reality particularly well – it is inconceivable that everyone in a given country discards all their nets at once at the three year mark. Additionally the Flaxman et al. net loss was fitted from 6 studies outside the main compartment model and fixed, whereas we learn a data driven, time varying loss function directly from the survey, manufacturer and NMCP data. We have discussed these limitations in the Methods section.
Minor points:
The estimated sharp fluctuations over time in ITN coverage indicators in some countries: are these real (if periodic campaigns dominate distributions?), or reflecting missing or incomplete data, e.g. in NMCPreported distributions? In particular, comment on the odd time patterns in DRC and Ghana and/or a few other countries with irregular patterns.
We believe these fluctuations to be real. The priors on distributions are set as uniform and therefore in the absence of concrete data driving these sharp fluctuations the model will prefer smoother distributions. Sharp fluctuations only arise when triangulation between the NMCP, manufacturer and survey data demand it. In the absence of data the model will choose a more parsimonious and even distribution pattern. It should also be noted that these sharp distributions are accompanied by full Bayesian credible intervals capturing our uncertainty about the distribution pattern (if that uncertainty is there).
[Editors' note: further revisions were requested prior to acceptance, as described below.]
The reviewers still have concerns about the conclusions on net retention. The model in this paper only estimates ITN retention indirectly while the six studies that Flaxman depended on were directly measuring it. The bias parameters of Flaxman et al. were introduced because of a discrepancy between the different sources of data, if a loss function of longer duration is used. This discrepancy could be due to systematic differences between the data sources, and so the reviewers feel that the conclusion of the present paper that nets are retained for a shorter time than was previously thought is less certain than the authors suggest.
The key point of contention here revolves around how the NMCP distribution data triangulates with the loss function. We disagree that the measurement of the loss function, as modelled in Flaxman et al., is suitable and disagree that the bias parameter allows a loss function of longer duration. Below we explain the shortcomings of the Flaxman et al. approach and the true purpose of the bias parameter.
There are two modelling choices that can be made here:
1) The approach of Flaxman et al. – use a loss function based on 6 in field studies, and adjust the NMCP distribution data for a hypothesised systematic bias.
2) Our approach – learn a highly flexible loss function directly from the data and only adjust the NMCP distribution data for random noise.
To convince you that our approach is inherently a less subjective modelling choice consider the two following issues with the Flaxman et al. approach:
a) Net loss function:
Flaxman et al. paramaterised their loss function based on 6 studies with differing time points under no standardised conditions between studies. In these highly controlled studies key factors of net loss predominantly driven by living conditions, household behaviour and attitudes (Killian et al., 2015) are unaccounted for by the very nature of the study. Additionally some of the studies used in Flaxman et al. are retrospective studies where recall bias was a significant and unaccounted for confounder (Killian et al., 2015). We also must stress again just how implausible the loss function resulting from these studies is – the function is essentially constant (reductions with an average of 5% per year) for 3 years and then a complete stepwise reduction to zero at the 3year mark. Under this loss function, using the upper 95% confidence interval, at most only 30% of nets can be lost after 3 years; immediately after which all nets are discarded. We believe that the implausibility of this loss function highlights the inconsistencies in paramaterising a function based on a small sample of unstandardized studies.
In contrast to this approach, we learn the loss function directly from the data, imposing an agnostic uninformative prior on the underlying halflife and duration. In our approach nets are lost smoothly over time (with no sudden discards) and the model has the choice of a long or short halflife depending the country and the year. From our fitted function (Figure 7) we find large variability between countries that would not be able to be represented using the Flaxman et al. loss function. Additionally, we actually say the duration of nets is longer than what Flaxman et al. say, with nets lasting up to 5 years on average (Figure 7 red curve). The key disagreement between our fitted loss function and that of Flaxman et al. is the yeartoyear rate of reduction, and that all nets are lost exactly on the three year mark.
To further assure the reviewers of the implausibility of Flaxman et al. loss function, consider looking at the raw household survey data and extracting information on net age directly from each survey. When averaged across all surveys used in our study the average proportion of the total number of nets that were:
Less than one year old was 57%,
Between one and two years old was 20%
Between two and three years old was 5%
Greater than three years old was 18%
Obviously there are a lot more complexities to these simple summaries, including when the survey was sampled, the distribution pattern, recall bias etc. But one thing is clear, the yearto year loss rate is very nonlinear and probably much greater than 5% (which Flaxman et al. claim), and a large number of nets are older than 3 years (which the Flaxman et al. does not allow). This once again calls into question the plausibility of the Flaxman et al. loss function, but is consistent with our loss function.
b) The bias parameter:
The bias parameter in Flaxman et al. is used to systematically adjust the NMCP distribution data to match the household survey data. This is essential in their approach as given their constrained loss function (described above) the survey data would not match the distribution data without some form of adjustment. There are several conceptual and implementation problems with this bias parameter. Firstly this bias parameter is a very poor linear fit (see Figure 5, Supplementary information Flaxman et al.), and does not account for the sampling time of the survey or the recall bias inherent in the survey questions. The poor linear fit suggests substantial country specific variation. Second the prior does not model country specific variation making the assumption that all NMCP distribution data is biased in the same way.
Conceptually the problem with the bias parameter is a reliance on the assumption: the NMCP data is systematically biased (constant) across all countries and years. Flaxman et al. provide no justification for this assumption, and we believe it to be very unlikely that all NMCP data, reported from different countries, in different years are systematically biased in the same way. This is the equivalent to saying that for every country the number of nets reported to be distributed should be scaled by the same constant.
To further prove to the reviewers that using a bias parameter is not a sound modelling choice to explain the data, consider a like for like rerun of the Flaxman et al. bias parameter analysis. We take the weighted survey data totals for the survey questions:
Is your net less than one year old?
Is your net between 1 to 2 years old?
Is your net between 2 to 3 years old?
And regress these (on a logarithmic scale) against the corresponding NMCP distribution results in that year (with linear interpolation to match the average year of the survey). If the Flaxman et al. hypothesis of the NMCP data being systematically biased holds the intercepts of the three linear models (in Author response image 1 where the intercepts are in the brackets, the regression bias line is in green and the onetoone line is in red) should be the same. However, these intercepts are orders of magnitude different (note the logarithmic scale) suggesting steep losses of nets and no systematic bias. A far more parsimonious explanation is that the differences in the number of LLINs from the Survey and NMCP are explained by the loss of nets occurring through time. This is supported by the increasing bias parameter, as nets get older. Indeed this analysis suggests that the sole purpose of Flaxman et al. bias parameter is to compensate for the deficiencies in their poorly characterised loss function.
In contrast we do not assume any systematic bias on the NMCP data, we only assume that the NMCP data contains random noise and let the variation be described by a flexible loss function.
Therefore in summary we hope we have made clear that opting for modelling approach (2) over (1) is a considerably more parsimonious choice making far fewer assumptions and letting the data speak. We therefore have high confidence in the fits of our modelled loss functions. We stress that we have been completely transparent with the modelling assumptions in the Appendix, and have made very few subjective decisions and focused on a data driven approach.
This also has implications for the point about sharp fluctuations in the predicted ITN coverage over time, which presumably would be smoother with a loss function of longer duration.
We believe the reviewers are mistaken that the cause of sharp fluctuations in ITN coverage is due to the loss function. The predominant and overwhelming cause of these fluctuations is the delivery and distribution data, not the loss function. Sharp fluctuations arise due to real, large distributions of nets captured in both the distribution and survey data. See a typical example of this below for Togo.
In Author response image 2, green dots are the manufacturer deliveries, purple triangles are the NMCP distributions, red triangles are the household surveys, the red line is the number of LLINS, the yellow line is the number of cITNs and the black line is the residual stock. The fluctuations are caused by real large deliveries and distributions causing a saw tooth pattern in the number of nets. The reason the number of nets does not increase exponentially with the deliveries is due to loss, but this loss is smooth over time. Using the Flaxman et al. loss function would cause far less smoothness due to restriction of all nets being discarded at the threeyear mark. To highlight this consider what would happen in the above plot if all nets were discarded after 3 years. As highlighted in the previous rebuttal the distribution of NMCP ITNs is uniform intra years, as nothing in the data suggests a nonuniform pattern, thereby implicitly choosing a smooth distribution pattern unless the data would suggest otherwise.
We recommend that the authors conduct a sensitivity analysis that assumes longer retention of nets, showing how any projections change. In any case this uncertainty should be acknowledged and the implications for the projections discussed.
Given the above discussion about the suitability of our loss function we feel the reviewers will agree that a sensitivity analysis is unnecessary given our framework. If we used a loss function such as that in Flaxman et al., this sensitivity analysis would be warranted to test the validity of using such a constrained loss function. However, given the data driven, assumption free nature, of our analysis this is unnecessary as the sensitivity analysis is implicitly contained within the Bayesian framework. That is, during every fit, we explore a full range of candidate functions from differing retention times to sharp and smooth transitions and average over these to find the best function that fits the data. Therefore, during every fit, we perform a statistically robust sensitivity analysis. To state this differently our model allows for a loss function similar to that of Flaxman et al., but this loss function is not supported by the data at all.
That said, one of the important results of this paper is exploring the effects of a longer retention net halflife (3 year halflife). These results are presented and discussed in the main paper (Figure 8 red line) and in Appendix 1.
In addition, net loss and retention consider the physical presence of the ITNs in households, but not their effectiveness in repelling mosquitoes. According to manufacturer and WHOPES evidence and guidance on ITN's effective durability, is it justified to assume a median 36month retention (in the optimized scenario), or does this imply a proportion of ITNs modelled as effective for longer than is realistically assumed? The authors should expound on this point in the discussion – for LLINs specifically
The scope of this paper was restricted to ownership and usage indicators and not on the effectiveness of nets once they are being used. Of course the insecticidal and barrier effectiveness of a nets diminish in a complex multifaceted way through time but due to these complexities, we did not expand our analysis to effectiveness and focus on the standard RBM definitions of ownership and use. We have expanded on this point in the Discussion.
For the countrylevel estimates: Are the 4 international indicators (ownership, access, usage) calculated and shown for each country's overall national population (including subpopulations at no risk, just as the DHS, MICS and MIS surveys) or for the subpopulation at risk of malaria? What assumptions, if any, are made about (mis)allocation of ITNs to populations not at risk of malaria? Please clarify this in the appropriate section of the manuscript
We use population at risk as the denominator in all the analysis in this paper and in doing so we avoid the problem of biased estimates by including populations not at risk. This however does make the assumption that there is no allocation of nets outside populations at risk. We have added a line in the main manuscript explicitly stating this assumption and thank the reviewers for pointing out this omission. However it is also vital to recognise that this adjustment by populations of risk only applies to a handful of countries in subSaharan Africa (e.g. Ethiopia, Kenya etc.) as all others are universally at risk.
Minor suggested edits:
If the world limit allows, please define 'Population access' in the Abstract.
These terms are fully described in the Introduction, subsection “Modelling coverage”.
Paragraph three, subsection “Net Loss”, 'minimised overallocation': From the annex (and subsection “Estimating ITN Requirements to Achieve Universal Access”) it seems that this is ZERO overallocation; please qualify or rephrase accordingly.
We thank the reviewers for this suggestion and have clarified this statement in the Methods and Appendix.
a) Parts of the diagram are confusing or not clear – please correct as needed:b) An arrow (or, equality sign) is missing from HHlevel ITN ownership to the box below (National total ITNs in HH?). c) An arrow missing from National net crop to, Observed bivariate histogramsd) National net crop equals National total ITNs in HH 5 boxes below, but the equality is not indicated.
Appendix 1: The scaling across countries to get to continenttotals is not very clear.
Figure 2 describes the Bayesian compartment model only, we choose not to tie together the Poisson model for the indicators as felt this would make the figure too complex.
https://doi.org/10.7554/eLife.09672.022Article and author information
Author details
Funding
Medical Research Council (K00669X)
 Samir Bhatt
 Peter W Gething
Bill and Melinda Gates Foundation (OPP1068048)
 Daniel J Weiss
 Bonnie Mappin
 Ursula Dalrymple
 Ewan Cameron
 Donal Bisanzio
 Peter W Gething
Foundation for the National Institutes of Health (U19AI089674)
 David L Smith
Bill and Melinda Gates Foundation (OPP1110495)
 David L Smith
Fogarty International Center
 Simon I Hay
 David L Smith
Wellcome Trust (091835)
 Catherine L Moyes
World Health Organization
 Michael Lynch
 Cristin A Fergus
 Richard E Cibulskis
Global Fund to Fight AIDS, Tuberculosis and Malaria
 Jan Kolaczinski
Bill and Melinda Gates Foundation (OPP1106023)
 Simon I Hay
 Peter W Gething
Bill and Melinda Gates Foundation (OPP1119467)
 Simon I Hay
Wellcome Trust (095066)
 Simon I Hay
Bill and Melinda Gates Foundation (OPP1093011)
 Simon I Hay
Department for International Development
 Peter W Gething
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Acknowledgements
The authors acknowledge the valuable assistance from Melanie Renshaw in providing information from the Roll Back Malaria Harmonization Working Group Programmatic Gap Analysis and other guidance in the interpretation of our results. We thank members of the Roll Back Malaria Monitoring and Evaluation Reference Group (RBMMERG) and the World Health Organization Surveillance Monitoring and Evaluation Technical expert Group (SMETEG) for their feedback and suggestions. We thank Clara Burgert of the DHS Program for her assistance with DHS Survey access and interpretation.
Reviewing Editor
 Catherine Kyobutungi, Africa Population Health Research Center, Kenya
Publication history
 Received: June 25, 2015
 Accepted: November 26, 2015
 Accepted Manuscript published: December 29, 2015 (version 1)
 Version of Record published: February 8, 2016 (version 2)
Copyright
This is an openaccess article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Metrics

 3,122
 Page views

 646
 Downloads

 102
 Citations
Article citation count generated by polling the highest count across the following sources: Scopus, Crossref, PubMed Central.
Download links
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Further reading

 Epidemiology and Global Health
Background:
Shortterm forecasts of infectious disease burden can contribute to situational awareness and aid capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise the predictive performance of such forecasts if multiple models are combined into an ensemble. Here, we report on the performance of ensembles in predicting COVID19 cases and deaths across Europe between 08 March 2021 and 07 March 2022.
Methods:
We used opensource tools to develop a public European COVID19 Forecast Hub. We invited groups globally to contribute weekly forecasts for COVID19 cases and deaths reported by a standardised source for 32 countries over the next 1–4 weeks. Teams submitted forecasts from March 2021 using standardised quantiles of the predictive distribution. Each week we created an ensemble forecast, where each predictive quantile was calculated as the equallyweighted average (initially the mean and then from 26th July the median) of all individual models’ predictive quantiles. We measured the performance of each model using the relative Weighted Interval Score (WIS), comparing models’ forecast accuracy relative to all other models. We retrospectively explored alternative methods for ensemble forecasts, including weighted averages based on models’ past predictive performance.
Results:
Over 52 weeks, we collected forecasts from 48 unique models. We evaluated 29 models’ forecast scores in comparison to the ensemble model. We found a weekly ensemble had a consistently strong performance across countries over time. Across all horizons and locations, the ensemble performed better on relative WIS than 83% of participating models’ forecasts of incident cases (with a total N=886 predictions from 23 unique models), and 91% of participating models’ forecasts of deaths (N=763 predictions from 20 models). Across a 1–4 week time horizon, ensemble performance declined with longer forecast periods when forecasting cases, but remained stable over 4 weeks for incident death forecasts. In every forecast across 32 countries, the ensemble outperformed most contributing models when forecasting either cases or deaths, frequently outperforming all of its individual component models. Among several choices of ensemble methods we found that the most influential and best choice was to use a median average of models instead of using the mean, regardless of methods of weighting component forecast models.
Conclusions:
Our results support the use of combining forecasts from individual models into an ensemble in order to improve predictive performance across epidemiological targets and populations during infectious disease epidemics. Our findings further suggest that median ensemble methods yield better predictive performance more than ones based on means. Our findings also highlight that forecast consumers should place more weight on incident death forecasts than incident case forecasts at forecast horizons greater than 2 weeks.
Funding:
AA, BH, BL, LWa, MMa, PP, SV funded by National Institutes of Health (NIH) Grant 1R01GM109718, NSF BIG DATA Grant IIS1633028, NSF Grant No.: OAC1916805, NSF Expeditions in Computing Grant CCF1918656, CCF1917819, NSF RAPID CNS2028004, NSF RAPID OAC2027541, US Centers for Disease Control and Prevention 75D30119C05935, a grant from Google, University of Virginia Strategic Investment Fund award number SIF160, Defense Threat Reduction Agency (DTRA) under Contract No. HDTRA119D0007, and respectively Virginia Dept of Health Grant VDH215010141, VDH215010143, VDH215010147, VDH215010145, VDH215010146, VDH215010142, VDH215010148. AF, AMa, GL funded by SMIGE  Modelli statistici inferenziali per governare l'epidemia, FISR 2020Covid19 I Fase, FISR2020IP00156, Codice Progetto: PRJ0695. AM, BK, FD, FR, JK, JN, JZ, KN, MG, MR, MS, RB funded by Ministry of Science and Higher Education of Poland with grant 28/WFSN/2021 to the University of Warsaw. BRe, CPe, JLAz funded by Ministerio de Sanidad/ISCIII. BT, PG funded by PERISCOPE European H2020 project, contract number 101016233. CP, DL, EA, MC, SA funded by European Commission  DirectorateGeneral for Communications Networks, Content and Technology through the contract LC01485746, and Ministerio de Ciencia, Innovacion y Universidades and FEDER, with the project PGC2018095456BI00. DE., MGu funded by Spanish Ministry of Health / REACTUE (FEDER). DO, GF, IMi, LC funded by Laboratory Directed Research and Development program of Los Alamos National Laboratory (LANL) under project number 20200700ER. DS, ELR, GG, NGR, NW, YW funded by National Institutes of General Medical Sciences (R35GM119582; the content is solely the responsibility of the authors and does not necessarily represent the official views of NIGMS or the National Institutes of Health). FB, FP funded by InPresa, Lombardy Region, Italy. HG, KS funded by European Centre for Disease Prevention and Control. IV funded by Agencia de Qualitat i Avaluacio Sanitaries de Catalunya (AQuAS) through contract 2021021OE. JDe, SMo, VP funded by Netzwerk Universitatsmedizin (NUM) project egePan (01KX2021). JPB, SH, TH funded by Federal Ministry of Education and Research (BMBF; grant 05M18SIA). KH, MSc, YKh funded by Project SaxoCOV, funded by the German Free State of Saxony. Presentation of data, model results and simulations also funded by the NFDI4Health Task Force COVID19 (https://www.nfdi4health.de/taskforcecovid192) within the framework of a DFGproject (LO342/171). LP, VE funded by Mathematical and Statistical modelling project (MUNI/A/1615/2020), Online platform for realtime monitoring, analysis and management of epidemic situations (MUNI/11/02202001/2020); VE also supported by RECETOX research infrastructure (Ministry of Education, Youth and Sports of the Czech Republic: LM2018121), the CETOCOEN EXCELLENCE (CZ.02.1.01/0.0/0.0/17043/0009632), RECETOX RI project (CZ.02.1.01/0.0/0.0/16013/0001761). NIB funded by Health Protection Research Unit (grant code NIHR200908). SAb, SF funded by Wellcome Trust (210758/Z/18/Z).

 Epidemiology and Global Health
Background:
Affectionate touch, which is vital for mental and physical health, was restricted during the Covid19 pandemic. This study investigated the association between momentary affectionate touch and subjective wellbeing, as well as salivary oxytocin and cortisol in everyday life during the pandemic.
Methods:
In the first step, we measured anxiety and depression symptoms, loneliness and attitudes toward social touch in a large crosssectional online survey (N = 1050). From this sample, N = 247 participants completed ecological momentary assessments over 2 days with six daily assessments by answering smartphonebased questions on affectionate touch and momentary mental state, and providing concomitant saliva samples for cortisol and oxytocin assessment.
Results:
Multilevel models showed that on a withinperson level, affectionate touch was associated with decreased selfreported anxiety, general burden, stress, and increased oxytocin levels. On a betweenperson level, affectionate touch was associated with decreased cortisol levels and higher happiness. Moreover, individuals with a positive attitude toward social touch experiencing loneliness reported more mental health problems.
Conclusions:
Our results suggest that affectionate touch is linked to higher endogenous oxytocin in times of pandemic and lockdown and might buffer stress on a subjective and hormonal level. These findings might have implications for preventing mental burden during social contact restrictions.
Funding:
The study was funded by the German Research Foundation, the German Psychological Society, and German Academic Exchange Service.