Unit 10.00: Why a straight line fails on a 0/1 outcome

A straight line has no way of knowing that a probability cannot be negative.

Why linear regression fails on a 0/1 outcome

A binary outcome takes two values, and what you want to model is the probability of one of them. Probabilities live in [0, 1]; a straight line does not, so a linear model will happily predict −0.5.

That is not merely inelegant. Predictions outside the range are meaningless, and the model's error structure is wrong throughout, so the standard errors cannot be trusted either.

Logistic regression models the log-odds instead — a quantity that runs from minus infinity to plus infinity and can therefore be a linear function of the predictors — and converts back to a probability at the end.

This block fits a linear model to a churn indicator and looks at what it predicts.

set.seed(701)
n <- 400
tenure <- runif(n, 0, 48)
# True log-odds of churn falls with tenure.
p_true <- 1 / (1 + exp(-(1.2 - 0.06 * tenure)))
churn  <- rbinom(n, 1, p_true)
d <- data.frame(churn, tenure)

cat("Outcome values:", paste(sort(unique(d$churn)), collapse = ", "), "\n")
cat("Overall churn rate:", round(mean(d$churn), 4), "\n\n")

# Why a straight line is the wrong model for a 0/1 outcome.
linear <- lm(churn ~ tenure, data = d)
preds <- predict(linear, data.frame(tenure = c(0, 24, 60, 90)))
cat("Linear model predictions at tenure 0, 24, 60, 90:\n")
cat(" ", round(preds, 3), "\n")
cat("Predictions outside [0, 1]:", sum(preds < 0 | preds > 1), "of 4\n\n")

cat("A probability cannot be negative or exceed 1, and a straight line has no\n")
cat("way of knowing that. Logistic regression models the LOG-ODDS instead,\n")
cat("which is unbounded, then converts back to a probability.\n")

The overall churn rate is 0.425. The linear model predicts 0.746 at tenure 0, 0.422 at 24, and then −0.065 and −0.471 at 60 and 90 months. 2 of 4 predictions are impossible, and nothing in the fit objected.

The mistake this prevents

The mistake is running lm() on a 0/1 outcome because it is convenient. It produces a number for every case, some of which cannot be probabilities, and confidence intervals that do not mean what they say.

Takeaway

Use glm(..., family = binomial) for binary outcomes. Check that any predicted probability lies in [0, 1] — if it does not, the model is the wrong kind.

Mark complete