Tutorial: How to draw the line in biomedical research

Biomedical research relies on statistical analyses of data sets comprised of multiple variables and, in particular, on analyses of the relationships between pairs of variables within those data sets. In a typical analysis, data representing two variables are displayed in a two-dimensional scatter plot and the method of ordinary least squares is used to fit a regression line to the data. Here we examine an under-appreciated aspect of this approach: the slope of the regression line depends on which of the two variables we select as the independent variable. This means that the method of ordinary least squares can be used to calculate two different regression lines for the same scatter plot. While this issue has long been appreciated in the statistics community, it is not as widely known among biomedical researchers (Cornbleet and Gochman, 1979). The ubiquity of scatter plots and regression lines in biomedical research suggests that a brief discussion of this issue would be useful.

Consider a data set composed of pairs of variables, with individual data points represented by (x₁, y₁), (x₂, y₂), and so on. In some studies, there may be a symmetric relationship between x_i and y_i: for example, they might represent blood pressure measurements from pairs of siblings in a cohort study. Alternatively, there may be an asymmetry in the relationship between the variables: for example, x_i might represent the dose of an antihypertensive drug, and y_i might represent the change in blood pressure in a group of subjects treated with various doses of the drug. In this example, drug dose is the independent variable and change in blood pressure is the dependent variable.

Typically, one is interested in determining the most likely value of the dependent variable given the value of the independent one. Thus, in the example described above one might be interested in predicting the change in blood pressure in response to different doses of the drug. However, there are many instances in which it is not clear that one variable depends on the other (independent) variable. For example, individuals with metabolic syndrome have elevated levels of both serum triglyceride and elevated fasting glucose levels: therefore, in a cohort that contains both metabolic syndrome patients and control subjects, one would expect these two variables to be correlated, with a clustering of metabolic syndrome patients at the high end of both distributions (Ford et al., 2002). Although it would be inappropriate to consider one of these variables independent and the other dependent in a mechanistic sense, one might still be interested in calculating the expected level of serum triglycerides given the level of fasting glucose, or the expected level of fasting glucose given the level of serum triglycerides. As described below, estimations in either direction begin with the calculation of a best-fitting regression line.

The method of least squares

The straight line that constitutes the best fit to a set of data points in the x-y plane is typically calculated by minimizing the sum of the squares of the distances from the points to the line—a method that was introduced by Legendre and Gauss more than two hundred years ago. If one variable, conventionally represented by the y-axis, is known to depend on the other variable, conventionally represented by the x-axis, then one generally calculates a best-fitting line that represents the expected value of the dependent variable (y) as a function of the independent variable (x): this is known as the regression of y on x. In this case, the distance from a data point to the regression line (also known as the residual) is taken as the vertical distance from the point to the line (Figure 1A; Bulmer, 1965). This approach, referred to as ordinary least squares regression, is the default mode for line fitting in several commonly used software packages; for example, it is the algorithm represented by the ‘Trend line' and ‘Linest' functions in Microsoft Excel.

Figure 1

Download asset Open asset

If, on the other hand, the y-axis represents the independent variable and the x-axis represents the dependent variable, the best-fitting line can be calculated by taking the residuals as the horizontal distances from the points to the line, the regression of x on y (Figure 1B). Except in the limiting case in which all of the data points lie precisely on a straight line, these two best-fitting regression lines will not coincide.

A third type of best-fitting line can be calculated by squaring the perpendicular distances from the points to the line (Figure 1C). This method is referred to as an orthogonal or Deming regression. The latter name refers to the statistician W Edward Deming who described the method in the 1940s (Deming, 1943). The Deming regression method is symmetric with respect to the two variables and therefore makes no assumptions regarding dependence and independence (Cornbleet and Gochman, 1979; Linnet, 1993, 1998; Glaister, 2001).

For a scatter plot in which the data points do not fall on a straight line, the best-fitting line calculated with vertical residuals will have a relatively shallow slope (Figure 1A), whereas the best-fitting line calculated with horizontal residuals will have a steeper slope (Figure 1B). The best-fitting line calculated with perpendicular residuals will have an intermediate slope (Figure 1C). The data can also be described by a correlation coefficient (R) that is agnostic with respect to the dependence or independence of the variables (Bulmer, 1965). It is conventional to calculate the square of the correlation coefficient (R²), which is equal to the slope of the regression line for y on x (that is, calculated using vertical residuals) divided by the slope of the regression line for x on y (horizontal residuals). Thus, if the data points fall on a straight line (R² = 1), the slopes of these two best-fitting lines will be equal. With increasing scatter in the data (R² < 1), the slopes of the two best-fitting lines will diverge. If the data points are completely uncorrelated (R² = 0), then the best-fitting lines calculated with vertical and horizontal residuals will have slopes of 0 and infinity, respectively.

Examples with real data: three lines are better than one

These considerations are illustrated by three scatter plots in a paper on post-traumatic stress disorder by Kerry Ressler and co-workers (Ressler et al., 2011). The best-fitting lines for these three plots were calculated with vertical residuals (green lines in Figure 2). In each case, these differ substantially from the best-fitting lines calculated with horizontal residuals (blue lines), as expected from the relatively low values of R² for the three data sets; the best-fitting lines calculated with perpendicular residuals (red lines) occupy intermediate positions.

Figure 2

Download asset Open asset

As noted above, the choice of algorithm to use in calculating the best-fitting line reflects a decision regarding which variable is independent and which is dependent. In Ressler et al. the variables in the three scatter plots are: severity of post-traumatic stress disorder (PTSD) symptoms vs. serum concentration of pituitary adenylate cyclase activating polypeptide (PACAP, also known as ADCYAP1; Figure 2A); severity of PTSD symptoms vs. PACAP receptor gene methylation (also known as ADCYAP1R; Figure 2B); and abundance of PACAP receptor mRNA in the cerebral cortex vs. abundance of PACAP mRNA in the cerebral cortex (Figure 2C). In this example, as in many biological systems, the cause and effect relationships between the variables are likely to be complex. For example, while it is possible that PACAP secretion may alter the severity of PTSD symptoms, it is also possible that stress associated with PTSD may alter PACAP secretion. Furthermore, it is possible that these two variables may have no direct cause-and-effect relationship, and that changes in neural circuitry alter stress level and PACAP secretion by distinct mechanisms. A similar argument can be applied to the other pairs of variables.

Visual inspection of Figure 2 shows that the individual regression lines for y on x (green) or x on y (blue) do not fully capture the trend in the data points within any of the scatter plots. Plotting both regression lines gives a fuller picture of the data, and comparing their slopes provides a simple graphical assessment of the correlation coefficient. Plotting the orthogonal regression line (red) provides additional information because it makes no assumptions about the dependence or independence of the variables; as such, it appears to more accurately describe the trend in the data compared to either of the ordinary least squares regression lines.

1. Book

   1. Bulmer MG

   (1965)

   Principles of statistics

   New York: Dover Publications.

   • Google Scholar
2. 1. Cornbleet PJ
   2. Gochman N

   (1979)

   Incorrect least-squares regression coefficients in method-comparison analysis

   Clin Chem 25:432–438.

   • Google Scholar
3. 1. Deming WE

   (1943)

   Statistical adjustment of data

   Statistical adjustment of data, New York, Wiley, (Dover Publications edition, 1985; Dover Publications, New York).

   • Google Scholar
4. 1. Ford ES
   2. Giles WH
   3. Dietz WH

   (2002) Prevalence of metabolic syndrome among US adults: findings from the third National Health and Nutrition Examination Survey

   JAMA 287:356–359.

   https://doi.org/10.1001/jama.287.3.356

   • Google Scholar
5. 1. Glaister P

   (2001) Least squares revisited

   Mathematical Gazette 85:104–107.

   https://doi.org/10.2307/3620485

   • Google Scholar
6. 1. Linnet K

   (1993)

   Evaluation of regression procedures for methods comparison studies

   Clin Chem 39:424–432.

   • Google Scholar
7. 1. Linnet K

   (1998)

   Performance of Deming regression analysis in case of misspecified analytical error ratio in method comparison studies

   Clin Chem 44:1024–1031.

   • Google Scholar
8. 1. Ressler KJ
   2. Mercer KB
   3. Bradley B
   4. Jovanovic T
   5. Mahan A
   6. Kerley K

   et al. (2011) Post-traumatic stress disorder is associated with PACAP and the PAC1 receptor

   Nature 470:492–497.

   https://doi.org/10.1038/nature09856

   • Google Scholar

Author details

1. Lisa Huang

   Department of Molecular Biology and Genetics, Johns Hopkins University School of Medicine, Baltimore, United States

   For correspondence

   chuang36@jhmi.edu

   Competing interests

   The authors declare that no competing interests exist.

2. Amir Rattner

   Department of Molecular Biology and Genetics, Johns Hopkins University School of Medicine, Baltimore, United States

   For correspondence

   arattner@jhmi.edu

   Competing interests

   The authors declare that no competing interests exist.

3. Han Liu

   Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Baltimore, United States

   For correspondence

   hanliu@princeton.edu

   Competing interests

   The authors declare that no competing interests exist.

4. Jeremy Nathans, Reviewing Editor

   Department of Molecular Biology and Genetics, the Department of Neuroscience, the Department of Ophthalmology, Howard Hughes Medical Institute, Johns Hopkins University School of Medicine, Baltimore, United States

   For correspondence

   jnathans@jhmi.edu

   Competing interests

   The authors declare that no competing interests exist.

• 1,475

  views

• 118

  downloads

• 5

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.