Methodology

The Flu Tool relies on a baseline set of multipliers, derived from the peer-reviewed literature, that extrapolate from influenza-associated hospitalization and/or death data (depending on the data that the user enters into the tool) to estimate other levels of the burden of disease (BoD) pyramid. Data to inform multipliers and other assumptions were gathered through a series of systematic literature reviews that included data extracted from studies published in all six WHO regions. Data were included from studies published from January 2000 to August 2024.

The tool uses data from systematic reviews of the peer-reviewed literature in three ways:

1. To determine multipliers (and 95% confidence intervals [95% CIs]) for estimating deaths or hospitalizations (depending on the user-supplied data), critically ill (e.g., intensive care unit [ICU]) hospitalizations, medically attended mild/moderate cases, and medically unattended mild/moderate cases.
2. To determine expansion factors that account for under-detection and under-reporting of influenza-associated hospitalizations and deaths. These expansion factors function similarly to the multipliers described above, but rather than using reported hospitalization or death data to estimate other influenza severity levels (e.g., critical hospitalizations, mild/moderate cases), expansion factors use reported hospitalization or death data to estimate the true number of hospitalized cases or influenza deaths, respectively (i.e., the same level of severity).
3. To determine proportions of influenza-associated hospitalizations by syndrome (i.e., clinical presentation). While influenza typically presents as a respiratory disease, it can also present with other primary clinical manifestations, including circulatory disease or symptoms that are neither respiratory nor circulatory. Influenza surveillance typically focuses on respiratory symptoms, so influenza cases that present with non-respiratory symptoms are more likely to be excluded from reported surveillance data.

The default assumptions for influenza-associated hospitalizations and deaths (depending on user-entered data), critically ill (ICU-level) hospitalizations and medically attended and unattended mild/moderate case multipliers, and hospitalization and death expansion factors were derived from relevant peer-reviewed studies. Each included study was required to provide data for at least two BoD pyramid levels (e.g., hospitalizations and deaths) and break down data by age group. Because studies included many different age groupings, depending on the population and study design, the data needed be stratified into standardized age groups before multipliers could be calculated. Specifically, the default multipliers and expansion factors were based on WHO-recommended age groups: 0 to 4 years, 5 to 14 years, 15 to 49 years, 50 to 64 years, and 65 years and older. To utilize WHO-recommended age groups, the extracted data from the literature review was ungrouped into single-year age groups (e.g., age 0 [less than 1 year], age 1, age 2) using the ungroup package in the R coding language. The ungrouping statistical approach--described by M. Pascariu, et al, in their paper, “Ungroup: An R package for efficient estimation of smooth distributions from coarsely binned data”--utilizes a penalized composite link model (PCLM) to estimate the latent distribution of the assumed Poisson-distributed count data into single-year age group bins. This approach then allows for the regrouping of data into new, custom age ranges. It also enables estimation of open-ended age groups, such as 65 years and older. It is common practice for the oldest age bin in studies to not define a maximum age; however, the Flu Tool imposes an artificial maximum age of 100 years for all data sources to simplify the model. For each BoD pyramid level, the ungroup package was executed separately for each multiplier in each study, based on the single-year numerator and denominator, prior to being regrouped into the WHO-recommended standard age groups. The regrouped values were then incorporated into a Poisson regression model with mixed effects in order to account for correlation within studies and generate related multipliers and 95% confidence intervals (95% CIs) by the WHO age groups. This process was repeated for each included study within the respective BoD pyramid level to calculate the multipliers included as default assumptions for the Flu Tool. In addition, the updated tool uses R script to execute the Poisson regression of the numerator and denominator data in accordance with the user’s provided age groups to allow the user to customize their BoD estimates into age ranges most relevant to their context.

From these data, multipliers were calculated from hospitalizations by dividing the average (mean) number of cases or deaths by the average number of hospitalizations per age group, across all studies.

The Flu Tool allows users to select the year/influenza season for which to estimate burden, as well as to compare results between seasons. The format of the season—i.e., single-year season for southern hemisphere countries vs two-year season for northern hemisphere countries (e.g., 2020-2021) vs hybrid season for tropical countries (near the Equator)—is formatted based on the WHO Global Influenza Programme FluNET database.

The Flu Tool incorporates data from a literature review focused on understanding the proportions of influenza-associated hospitalizations that have a respiratory, circulatory, or non-respiratory/non-circulatory primary clinical presentation (or syndrome). The purpose of this analysis is to provide data on influenza-associated hospitalizations without the traditional respiratory presentation in order to improve understanding of the whole spectrum of the influenza burden of disease. Data from the peer-reviewed literature were limited and could not be broken down by age group due to small sample sizes, so we used data from all ages from the combined literature to calculate estimates for the proportions described above.

From the literature review, it was determined that, on average, 81.94% of all influenza-associated hospitalizations exhibit respiratory disease, 4.61% a primary circulatory presentation, and 13.45% have other non-respiratory/non-circulatory presentations (e.g., gastrointestinal). To determine the expected number of respiratory, circulatory, and non-respiratory/non-circulatory cases, the tool divides the number of influenza-associated hospitalizations calculated in the tool (including the expansion factor, if selected) by 81.94%, in order to estimate the total number of influenza-associated hospitalizations. The tool then multiplies that total by 4.61% and 13.45% to calculate the number of circulatory and non-respiratory/non-circulatory hospitalizations, respectively.

These estimates are intended to be approximate and should be considered notional due to the limited size and number of studies from which the underlying data were extracted.

The Flu Tool represents uncertainty through the display of 95% confidence intervals (95% CIs), which are applied to the multipliers and expansion factors calculated using the methods detailed above. The tool also allows (but does not require) users to enter 95% CIs with their hospitalization or death data. Users who elect to enter custom multipliers must also enter accompanying 95% CIs.

Uncertainty is propagated through the tool primarily using the normal Delta Method. For example, to compute the expected number of deaths given a starting number of user-entered hospitalizations (with an accompanying 95% CI) and a tool-generated multiplier (with 95% CI), the following Delta Method using a normal approximation is applied:

Step 1: Compute SEs from CIs

Step 2: Point estimate of deaths

Step 3: Variance of deaths (delta method)

Step 4: Standard error

Step 5: 95% CI (Normal approximation)