Meta-Research: Reader engagement with medical content on Wikipedia
- Department of Medicine, Uniformed Services University of the Health Sciences, United States
- Lane Medical Library, Stanford University School of Medicine, United States
- École Polytechnique Fédérale de Lausanne, Switzerland
- Graduate School of Education, Stanford University, United States
Figures
Figure 1
Page length of WikiProject Medicine (WPM) and other Wikipedia (W) pages.
Distribution of page lengths in characters for WikiProject Medicine (WPM) pages and the rest of Wikipedia (W) on April 20th, 2019. The difference between the two distributions is statistically significant according to Mann–Whitney U test (p<0.001, two-tailed).
Figure 2
Open access reference displayed in Wikipedia.
A research study cited on Wikipedia’s “diabetes” page displayed on desktop device with “freely accessible” icon for PubMed Central (PMC) link and on a mobile device without the open access icon. There is an “up click”( ^ ) in front of the citation on the desktop device, which returns reader back to footnote 46 in the text.
Figure 3
Example of information displayed when hovering over footnotes in Wikipedia.
(A) An external link on the Hepatitis page, revealed by “hovering” over the footnote number, indicating that the source is a systematic review conducted by the Cochrane Collaboration (with a green icon PMC link signaling open access to the source). (B) Hovering over a different footnote number on the same page reveals a different external link for the 19th and relatively recent edition of a medical textbook from a leading publisher.
Tables
Table 1
Types of engagement with external links
Different events captured by the CitationUsage Schema that reflect how readers of Wikipedia pages engage with external links.
| Engagement type | General description |
|---|---|
| External click | A click on a link located on a Wikipedia page leading to a web page outside of Wikipedia. |
| Hover | An event that occurs when a reader hovers over a link for at least 1000 milliseconds on a Wikipedia page. |
| Footnote click | A click of an internal footnote link – [1] – that takes the reader to the reference section at the bottom of the Wikipedia page. |
| Up click | A click of an arrow – ^ – that takes the reader from the reference at the bottom of the Wikipedia page back to the citation in the main text. |
Table 2
Factors that distinguish WPM pages from the rest of Wikipedia.
Differences between WPM pages and pages in the rest of Wikipedia (W) based on data collected on April 20th, 2019.
| WPM | W | |
|---|---|---|
| Wikipedia pages | 34,324 | 5,839,083 |
| Pages with external links | 32,609 | 5,210,746 |
| External links | 945,645 | 60,851,396 |
| Links per page (with links) | 29.0 | 11.7 |
| Page length (characters) | 13,084.9 | 7,676.4 |
| Characters per link (pages with links) | 450.3 | 657.3 |
Table 3
Comparing pageviews between WikiProject Medicine (WPM) and other Wikipedia (W) pages.
The average number of pageviews WikiProject Medicine (WPM) and the rest of Wikipedia (W) received per day for different types of devices between March 22nd and April 22nd, 2019.
| WPM (%) | W (%) | |
|---|---|---|
| Pageviews on desktop device | 1,957,821.6 (33.3) | 97,956,273.1 (42.9) |
| Pageviews on mobile device | 3,917,648.8 (66.7) | 130,488,855.3 (57.1) |
| Total number of pageviews | 5,875,470.4 (100) | 228,445,128.4 (100) |
Table 4
Frequency of different types of link engagement per day.
The average number of times per day that readers of WPM and W pages engaged with external links using one of the event types captured from March 22nd to April 22nd, 2019.
| Event type | WPM | W | ||||
|---|---|---|---|---|---|---|
| Total (%) | Desktop (%) | Mobile (%) | Total (%) | Desktop (%) | Mobile (%) | |
| Hover over link | 48,748.9 (46.9) | 45,814.8 (60.3) | 2,934.1 (10.5) | 1,122,704.0 (32.7) | 1,057,982.0 (47.2) | 64,722.0 (5.4) |
| Footnote click | 27,739.4 (26.7) | 10,948.8 (14.4) | 16,790.6 (60.3) | 722,131.0 (21.0) | 235,245.0 (10.5) | 486,886.7 (40.7) |
| External click | 25,811.9 (24.9) | 17,792.3 (23.4) | 8,019.7 (28.8) | 1,557,125.0 (45.3) | 915,445.1 (40.9) | 641,676.4 (53.6) |
| Up click | 1,539.5 (1.5) | 1,422.8 (1.9) | 116.4 (0.4) | 34,738.0 (1.0) | 31,230.1 (1.4) | 3,508.1 (0.3) |
| All events | 103,839.7 (100) | 75,978.7 (100) | 27,860.8 (100) | 3,436,698.0 (100) | 2,239,902.2 (100) | 1,196,793.2 (100) |
Table 5
Number of pageviews per engagement event.
The frequency of each event per day was divided by the average number of daily pageviews for WPM and W pages from March 22nd to April 22nd, 2019. The lower the number of pageviews per event the greater the event frequency. Difference between each pair of WPM and W distributions is statistically significant as derived from Fisher’s exact test (p<0.001, two-tailed).
| Pageviews/event | WPM | W | ||||
|---|---|---|---|---|---|---|
| Total | Desktop | Mobile | Total | Desktop | Mobile | |
| Hover over link | 120.5 | 42.7 | 1,335.2 | 203.5 | 92.6 | 2,016.1 |
| Footnote click | 211.8 | 178.8 | 233.3 | 316.3 | 416.4 | 268.0 |
| External click | 227.6 | 110.0 | 488.5 | 146.7 | 107.0 | 203.4 |
| Up click | 3,816.6 | 1,376.1 | 33,655.0 | 6,576.2 | 3,136.6 | 37,196.5 |
| All events | 56.6 | 25.8 | 140.6 | 66.5 | 43.7 | 109.0 |
Table 6
Summary of data collected.
Summary statistics selected from the tables above, comparing WikiProject Medicine (WPM) pages and readers to the rest of Wikipedia (W) pages and readers. (a) The “4.4 more pageviews” reflects the ratio of WPM to W pages (0.6%) compared to the ratio of WPM to W pageviews/day (2.6%). (b)“WPM readers” and “W readers” refer to the behaviors of those reading a WPM and/or W page during the data collection period.
| WPM pages | W pages | WPM pages, compared to other W pages,... | |
|---|---|---|---|
| Page length (characters) | 13,085 | 7676 | …are 70.5% longer by character count. |
| Characters/external link (on pages with links) | 450.3 | 657.3 | …possess a 31.5% greater external link density. |
| Pageviews/day | 5,875,470.40 | 228,445,128.40 | …receive 4.4 more pageviews per day.a |
| WPM readers | W readers | WPM readers, compared to rest of W readers,...b | |
| Time before engagement (sec) | 47.5 | 32.8 | …take 44.8% longer before engaging in a link activity. |
| Pageviews/link engagement | 56.60 | 66.47 | …engage 17.5% more often with links per pageview. |
| Pageviews/hover | 120.5 | 203.5 | …hover over links 68.9% more often per pageview. |
| Pageviews/footnote click | 211.8 | 316.3 | …click footnote numbers 49.3% more often per pageview. |
| Pageviews/external click | 227.6 | 146.7 | …click external links 35.5% less often per pageview. |
Additional files
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Meta-Research: Reader engagement with medical content on Wikipedia
eLife 9:e52426.
https://doi.org/10.7554/eLife.52426
