Regression Analysis

A scatterplot displays paired quantitative measurements. The explanatory variable is plotted horizontally and the response vertically. A fitted line predicts responses; it does not establish causation.

A scatterplot places a quantitative explanatory variable on the horizontal axis and a quantitative response on the vertical axis. Describe direction, form, strength, and unusual points. A strong curved relationship can have a weak linear correlation, so inspect shape before summarizing with one number.

Correlation r measures direction and strength of linear association. It is unitless and lies between −1 and 1. Switching explanatory and response variables leaves r unchanged, but changing the sign of one variable reverses it. Correlation is sensitive to unusual points and does not establish causation.

The line ŷ = a + bx predicts y from x by minimizing the sum of squared vertical residuals. The slope is predicted change in y per one-unit increase in x, with units of y per x. The intercept predicts y at x = 0, which may have no practical meaning if zero lies far outside the observed domain.

The least-squares line passes through (x̄,ȳ). Its slope has the same sign as r and can be calculated as r(sy/sx). Reversing the roles of x and y does not generally give the algebraic inverse of the original least-squares line because the residuals being minimized change.

A residual is observed y minus predicted y. A positive residual means the observation lies above the fitted line. Plot residuals against x or predicted values to look for patterns. Random-looking scatter around zero supports the linear-form description; a curve suggests the line misses systematic structure.

Changing spread across x can indicate nonconstant variability. A residual standard deviation summarizes typical prediction error around the line in response-variable units. It is not the standard deviation of x, and it should be interpreted within the observed context and range.

For simple least-squares regression with an intercept, r² is the proportion of observed variation in y accounted for by the linear model using x. It does not mean that r² of individuals are predicted perfectly or that x causes that fraction of y.

A high r² does not guarantee that the linear form is appropriate, predictions are safe outside the data range, or a causal interpretation is valid. Inspect residuals, unusual points, and the collection process. Report the variables and model when interpreting the percentage.

A point with unusual x has high leverage. It may strongly change a fitted line, but leverage alone does not guarantee large influence. A large vertical residual is an unusual y given x. Investigate whether a point is an error, a different process, or a valid extreme observation before deciding how to handle it.

Extrapolation predicts outside the observed x range and assumes the relationship continues there. Physical or behavioral constraints can make that assumption unreasonable. A model fitted to a narrow range of ages, doses, or temperatures should not be treated as a universal law.

A lurking variable can influence both measured variables, reverse causation can be possible, and chance can contribute to observed patterns. Randomized experiments can strengthen causal interpretation, while observational regression alone cannot remove every confounding explanation.

State a prediction as a model-based estimate for the studied context. A residual describes one observation’s deviation, whereas unexplained variation describes the broader scatter. Keep statistical language specific rather than saying a model is simply “accurate” because its correlation is large.

Look for direction, form, strength, and unusual points in the scatterplot. Correlation summarizes the direction and strength of a linear relationship; a strong curved pattern can have a small correlation. A correlation has no units, and switching which variable is called explanatory does not change it. The regression line, however, does depend on which variable is predicted.

Interpret slope as the predicted response change for a one-unit increase in the explanatory variable, with both units stated. The intercept predicts the response at x=0, which may be outside the meaningful range. Avoid attaching a substantive interpretation to an impossible or distant zero value.

A residual equals observed response minus predicted response. A positive residual means the observation is above the fitted line; a negative one is below. A residual plot with a curved pattern suggests the line is missing systematic structure. A roughly patternless cloud around zero is consistent with a linear model, though it does not prove every modeling assumption.

The coefficient of determination, r², gives the fraction of observed response variation explained by the linear model using the explanatory variable. It does not mean that this fraction of observations lies exactly on the line, nor that x causes that percentage of y. Retain context: specify what response variation and what predictor the statement concerns.

A point far from the mean x value has high leverage and can strongly affect a fitted line, but influence depends on its position relative to the overall pattern. A point can have a large residual without large leverage. Compare fits with and without a suspicious point only with an honest explanation; never remove inconvenient observations solely to improve fit.

Extrapolation uses the model beyond the observed predictor range, where the relationship may change. Confounding offers a different explanation for an association: a third variable may relate to both x and y. Even a high r² cannot transform an observational association into evidence of causation.

Identify the explanatory and response variables and their units. Describe direction, form, strength, and unusual observations. Correlation measures linear association, so a strong curved relationship can have a small correlation. A scatterplot is necessary to see what a single coefficient hides.

The least-squares line minimizes the sum of squared vertical residuals. A residual is observed response minus predicted response; a positive residual lies above the line. The line predicts y from x, and reversing the roles generally produces a different regression, not simply the algebraic inverse.

Correlation has no units and lies from −1 to 1. Changing measurement units by positive scale factors does not change correlation, but it changes slope units. Neither correlation nor a fitted line establishes that changing x causes y to change.

A slope gives the predicted change in response for a one-unit increase in the explanatory variable. Include units and use predictive language. The intercept is the prediction at x=0; it may have no useful practical interpretation when zero lies outside the observed range or is impossible in context.

The coefficient of determination r² is the fraction of observed variation in the response accounted for by its linear relationship with the explanatory variable in this fit. It is not the fraction of predictions that are correct, nor the fraction of response caused by x.

A residual plot should not show a systematic pattern if a straight-line description is adequate. Curvature suggests missed structure; changing vertical spread suggests different prediction variability across x. A small average residual is not proof of a good model because least-squares residuals with an intercept balance around zero.

A point far from the others in x has high leverage. It is influential if removing it materially changes the fitted relationship. A large residual is a different property. A point can have high leverage and a small residual while still strongly shaping the line.

Extrapolation predicts beyond the observed x range and assumes a relationship continues where it has not been checked. Even an excellent fit within the sample can fail beyond that range. Contextual limits such as saturation or impossible negative predictions can reveal the problem.

A lurking variable may explain an association, and reverse causation may be possible. To support a causal claim, examine design and mechanisms rather than relying on a large r². This unit focuses on regression description and prediction within the revised assessed scope, not inference for a population slope.