Unit 10.01: From bounded to unbounded

Three ways of saying the same thing, and logistic regression works on the one nobody finds intuitive.

From bounded to unbounded

Probability runs from 0 to 1. Odds is probability divided by its complement and runs from 0 to infinity. Log-odds is the logarithm of the odds and runs from minus infinity to plus infinity.

That last property is why logistic regression uses it: an unbounded quantity can be modelled by a straight line without ever producing an impossible prediction. The conversion back is p = 1 / (1 + exp(-logit)).

The scale is symmetric around p = 0.5, which corresponds to log-odds 0. And equal steps in log-odds are emphatically not equal steps in probability — the same coefficient moves the probability a lot near 0.5 and almost not at all near the extremes.

This block tabulates all three scales and then measures the non-linearity.

probs <- c(0.01, 0.10, 0.25, 0.50, 0.75, 0.90, 0.99)
odds  <- probs / (1 - probs)
logit <- log(odds)

cat(sprintf("%12s %12s %12s\n", "probability", "odds", "log-odds"))
for (i in seq_along(probs)) {
  cat(sprintf("%12.2f %12.3f %12.3f\n", probs[i], odds[i], logit[i]))
}

cat("\nProbability is bounded in [0, 1]; odds in [0, Inf); log-odds is\n")
cat("unbounded in both directions, which is what makes it modellable by a\n")
cat("straight line.\n\n")

cat("The scale is symmetric around 0.5:\n")
cat("  p = 0.25 -> log-odds", round(log(0.25/0.75), 3), "\n")
cat("  p = 0.75 -> log-odds", round(log(0.75/0.25), 3), "\n\n")

cat("Converting back:  p = 1 / (1 + exp(-logit))\n")
cat("  logit 0.847 ->", round(1 / (1 + exp(-0.847)), 3), "\n")
cat("Equal steps in log-odds are NOT equal steps in probability: from 0 to 1\n")
cat("moves p by", round(1/(1+exp(-1)) - 0.5, 3), ", from 3 to 4 by only",
    round(1/(1+exp(-4)) - 1/(1+exp(-3)), 3), ".\n")

At p = 0.5 the odds are 1 and the log-odds 0. The symmetry is exact: p = 0.25 gives −1.099 and p = 0.75 gives +1.099. The final lines are the ones to remember — moving log-odds from 0 to 1 raises the probability by 0.231, while moving from 3 to 4 raises it by only 0.029. The same coefficient means very different things depending on where you start.

The mistake this prevents

The mistake is reading a logistic coefficient as a change in probability. It is a change in log-odds, and the probability effect depends entirely on the baseline.

Takeaway

Learn the two conversions and use them. When reporting to non-specialists, convert to predicted probabilities at concrete values rather than quoting coefficients or odds ratios.

Mark complete