Unit 10.01: From bounded to unbounded

Three ways of saying the same thing, and logistic regression works on the one nobody finds intuitive.

From bounded to unbounded

Probability runs from 0 to 1. Odds is probability divided by its complement and runs from 0 to infinity. Log-odds is the logarithm of the odds and runs from minus infinity to plus infinity.

That last property is why logistic regression uses it: an unbounded quantity can be modelled by a straight line without ever producing an impossible prediction. The conversion back is p = 1 / (1 + np.exp(-logit)).

The scale is symmetric around p = 0.5, which is log-odds 0. And equal steps in log-odds are emphatically not equal steps in probability — the same coefficient moves the probability a lot near 0.5 and almost not at all near the extremes.

This block tabulates all three scales and then measures the non-linearity.

import numpy as np

probs = np.array([0.01, 0.10, 0.25, 0.50, 0.75, 0.90, 0.99])
odds = probs / (1 - probs)
logit = np.log(odds)

print(f"{'probability':>12s}{'odds':>12s}{'log-odds':>12s}")
for pr, o, lg in zip(probs, odds, logit):
    print(f"{pr:12.2f}{o:12.3f}{lg:12.3f}")

print("\nProbability is bounded in [0, 1]; odds in [0, inf); log-odds is")
print("unbounded both ways, which is what makes it modellable by a line.\n")
print("The scale is symmetric around 0.5:")
print(f"  p = 0.25 -> log-odds {np.log(0.25 / 0.75):.3f}")
print(f"  p = 0.75 -> log-odds {np.log(0.75 / 0.25):.3f}\n")

inv = lambda z: 1 / (1 + np.exp(-z))
print("Converting back:  p = 1 / (1 + exp(-logit))")
print(f"  logit 0.847 -> {inv(0.847):.3f}\n")
print("Equal steps in log-odds are NOT equal steps in probability:")
print(f"  0 -> 1 moves p by {inv(1) - inv(0):.3f}")
print(f"  3 -> 4 moves p by {inv(4) - inv(3):.3f}")

At p = 0.5 the odds are 1 and the log-odds 0. The symmetry is exact: p = 0.25 gives −1.099 and p = 0.75 gives +1.099. The final lines are the ones to remember — moving log-odds from 0 to 1 raises the probability by 0.231, while moving from 3 to 4 raises it by only 0.029, an eighth as much.

The mistake this prevents

The mistake is reading a logistic coefficient as a change in probability. It is a change in log-odds, and the probability effect depends entirely on the baseline.

Takeaway

Learn the two conversions and use them. When reporting to non-specialists, convert to predicted probabilities at concrete values rather than quoting coefficients or odds ratios.

Mark complete