Effects of side-effect risk framing strategies on COVID-19 vaccine intentions: a randomized controlled trial
- Assistant Professorship of Behavioral Science for Disease Prevention and Health Care,Technical University of Munich, Germany
- Heidelberg Institute of Global Health, Heidelberg University, Germany
Figures
Figure 1
CONSORT flow diagram.
Of the 9002 enrolled participants, 8998 individuals completed the survey (response rate = 99.96%).
Figure 2
Effect of the vaccine side-effect framing strategies on participants’ vaccine intentions and perceived safety.
N=8998 for strategies 1 and 2; strategy 3 is estimated only among those that received a comparison risk (N=5998). We estimated the main effects of each treatment strategy on participants’ vaccine …
Figure 3
Differences in the framing effects by country, age, and sex.
Coefficients are the interaction effect of the main heterogeneity characteristic with the indicator for each treatment strategy and are presented as coefficients on the log-odds scale from logistic …
Appendix 1—figure 1
Screenshots of the online, Gorilla-based, experiment content for 2 of the 10 possible experimental combinations (A) no risk label, no control group; (B) risk label, comparison to motor-vehicle mortality, absolute comparison.
Appendix 1—figure 2
Detailed randomization diagram across the main study factors.
We have shown the randomization here with sample sizes referring to both countries combined; in practice, we conducted this procedure stratified by country such that each cell in the diagram has …
Appendix 1—figure 3
Screenshots of each experimental arm for US participants.
Note that we have labeled the arms for the figure, but participants were not shown this label. Reference mortality information for motor-vehicle and COVID-19 fatalities were taken from the Centers …
Appendix 1—figure 4
Screenshots of each experimental arm for UK participants.
Note that we have labeled the arms for the figure, but participants were not shown this label. Reference mortality information for motor-vehicle fatalities were taken from the Department of …
Tables
Table 1
Demographic composition of the study samples.
| United States(N=4502) | United Kingdom(N=4496) | |
|---|---|---|
| Gender | ||
| Male | 2,216 (49.2%) | 2,236 (49.7%) |
| Female | 2,205 (49.0%) | 2,216 (49.3%) |
| Other | 81 (1.8%) | 44 (1.0%) |
| Age | ||
| 18–30 | 3,090 (68.6%) | 1,859 (41.4%) |
| 30–45 | 1,077 (23.9%) | 1,531 (34.0%) |
| 45–60 | 268 (6.0%) | 776 (17.3%) |
| 60+ | 67 (1.5%) | 330 (7.3%) |
| Education | ||
| Less than secondary education | 142 (3.1%) | 498 (11.1%) |
| Completed secondary education | 987 (21.9%) | 1217 (27.1%) |
| Some college | 1786 (39.7%) | 1014 (22.5%) |
| Bachelor’s degree | 1061 (23.6%) | 1182 (26.3%) |
| More than bachelor’s degree | 526 (11.7) | 585 (13.0%) |
| Race* | ||
| White | 3607 (80.1%) | 3813 (84.8%) |
| Black/African | 357 (7.9%) | 139 (3.1%) |
| Asian | 441 (9.8%) | 316 (7.0%) |
| Other | 329 (7.3%) | 190 (4.2%) |
| Hispanic | 538 (11.9%) | – |
-
Notes: *Participants were allowed to choose multiple options if they identified as a member of multiple races. We used the race/ethnic categories used by the US Census Bureau and the UK Office for National Statistics when asking individuals to report their race and/or ethnicity.
Table 2
Balance table of baseline participant characteristics between the no risk label and risk label treatment arms.
| No risk label (N=4496) | Risk label(N=4502) | p-Value | |||
|---|---|---|---|---|---|
| Female (n, %) | 2224 (49.5%) | 2197 (48.8%) | 0.542 | ||
| Age (mean years, SD) | 31.7 (12.6) | 31.6 (12.7) | 0.623 | ||
| Education (n, %) | 0.836 | ||||
| Less than secondary education | 320 (7.1%) | 319 (7.1%) | |||
| Completed secondary education | 1097 (24.4%) | 1107 (24.6%) | |||
| Some college | 1392 (31.0%) | 1409 (31.3%) | |||
| Bachelor’s degree | 1143 (25.4%) | 1100 (24.4%) | |||
| More than bachelor’s degree | 544 (12.1%) | 567 (12.6%) | |||
-
Notes: p-Values for gender and education correspond to a chi-squared test and a two-sided t-test for age.
Table 3
Balance table of baseline participant characteristics between the no comparison, comparison with motor-vehicle mortality, and comparison with COVID-19 mortality treatment arms.
| No comparison (N=3000) | Comparison with motor-vehicle mortality(N=3002) | Comparison with COVID-19 mortality(N=2996) | p-Value | |
|---|---|---|---|---|
| Female (n, %) | 1478 (49.3%) | 1452 (48.4%) | 1491 (49.8%) | 0.547 |
| Age (mean years, SD) | 31.8 (12.8) | 31.4 (12.5) | 31.6 (12.7) | 0.516 |
| Education (n, %) | 0.873 | |||
| Less than secondary education | 210 (7.0%) | 202 (6.7%) | 227 (7.6%) | |
| Completed secondary education | 717 (24.0%) | 744 (24.8%) | 743 (24.8%) | |
| Some college | 931 (31.0%) | 933 (31.1%) | 937 (31.2%) | |
| Bachelor’s degree | 769 (25.6%) | 753 (25.1%) | 721 (24.1%) | |
| More than bachelor’s degree | 373 (12.4%) | 370 (12.3%) | 368 (12.3%) |
-
Notes: p-Values for gender and education correspond to a chi-square test and an ANOVA-test for age.
Table 4
Balance table of baseline participant characteristics between the absolute risk and relative risk treatment arms.
| Absolute risk (N=2997) | Relative risk(N=3001) | p-Value | |
|---|---|---|---|
| Female (n, %) | 1473 (49.1%) | 1470 (49.0%) | 0.918 |
| Age (mean years, SD) | 31.4 (12.4) | 31.6 (12.8) | 0.543 |
| Education (n, %) | 0.897 | ||
| Less than secondary education | 222 (7.4%) | 207 (6.9%) | |
| Completed secondary education | 737 (24.6%) | 750 (25.0%) | |
| Some college | 934 (31.2%) | 936 (31.2%) | |
| Bachelor’s degree | 743 (24.8%) | 731 (24.3%) | |
| More than bachelor’s degree | 361 (12.0%) | 377 (12.6%) |
-
Notes: p-Values for gender and education correspond to a chi-square test and a two-sided t-test for age.
Table 5
Independent and combined effects of risk labeling and motor-vehicle mortality comparisons on willingness to take a hypothetical COVID-19 vaccine.
| Effect size(percentage points) | p-Value | |
|---|---|---|
| Effect of risk labeling (ref: no risk label) N=8998 | 3.0 | 0.003 |
| Effect of motor-vehicle comparison (ref: no comparison) N=8998 | 2.4 | 0.049 |
| Effect of both risk labeling and a motor-vehicle comparison (ref: no risk label nor comparison) N=3002 | 6.1 | <0.001 |
-
Notes: Outcome: ‘Would you take this vaccine?’ (yes = 1, others = 0). All the results are from logistic regression models with the results presented as average marginal effects. As per our pre-analysis plan, all regressions include covariates for age, sex, education, and country. Sample sizes for the relative to absolute comparison and effect of both labeling and a motor-vehicle mortality comparison are smaller since they are only estimated among subset of the total sample. p-Values are from two-tailed t-tests.
Appendix 1—table 1
Robustness of main paper results to alternative regression types and the inclusion/exclusion of covariates.
| Logistic regression models with results presented as average marginal effects | Linear probability models | |||
|---|---|---|---|---|
| With covariates (main paper results) | Without covariates | With covariates | Without covariates | |
| Effect of labeled risk compared to unlabeled risk | 3.0 pp (p=0.003) | 3.0 pp (p=0.003) | 3.0 pp (p=0.003) | 3.0 pp (p=0.003) |
| Effect of comparison to motor-vehicle mortality compared to no comparison | 2.4 pp (p=0.049) | 2.4 pp (p=0.051) | 2.4 pp (p=0.0498) | 2.4 pp (p=0.052) |
| Effect of comparison to COVID-19 mortality compared to no comparison | 0.8 pp (p=0.496) | 0.7 pp (p=0.568) | 0.8 pp (p=0.497) | 0.7 pp (p=0.571) |
| Effect of relative comparison compared to absolute comparison | 1.3 pp (p=0.285) | 1.3 pp (p=0.277) | 1.3 pp (p=0.286) | 1.3 pp (p=0.277) |
| Effect of both risk labeling and a motor-vehicle comparison compared to no labeling or comparison | 6.1 pp (p<0.001) | 6.1 pp (p<0.001) | 6.1 pp (p<0.001) | 6.1 pp (p<0.001) |
-
Notes: Outcome: ‘Would you take this vaccine?’ (yes = 1, others = 0). As per our pre-analysis plan, covariates include age, sex, education, and country. Sample sizes for the relative to absolute comparison (N=5998) and effect of both labeling and a motor-vehicle mortality comparison (N=3002) are smaller since they are only estimated among subset of the total sample.
Appendix 1—table 2
Main paper results estimated using ordinal logistic regression models with a four-category outcome specification rather than logistic regression models with a binary outcome.
| Odds ratio (p-value) | |
|---|---|
| Effect of labeled risk compared to unlabeled risk | 1.15 (p=0.002) |
| Effect of comparison to motor-vehicle mortality compared to no comparison | 1.11 (p=0.050) |
| Effect of comparison to COVID-19 mortality compared to no comparison | 1.03 (p=0.541) |
| Effect of relative comparison compared to absolute comparison | 1.05 (p=0.327) |
| Effect of both risk labeling and a motor-vehicle comparison compared to no labeling or comparison | 1.31 (p<0.001) |
-
Note. Outcome: ‘Would you take this vaccine?’ (No, Unsure – leaning towards no, Unsure – leaning towards yes, Yes). As per the main paper results and pre-analysis plan, all regressions include controls for age, sex, country, and education. Sample sizes for the relative to absolute comparison (N=5998) and effect of both labeling and a motor-vehicle mortality comparison (N=3002) are smaller since they are only estimated among subset of the total sample.
