Unit 07.00: Scatter plots and what a cloud actually shows

A correlation coefficient summarises a cloud into one number, and several very different clouds produce the same one.

Four patterns, four different meanings

Linear, curved, unrelated and two-cluster data, each with its correlation.

The code reports what a fitted line would claim about each.

import numpy as np

rng = np.random.default_rng(5)
x = rng.uniform(0, 100, 200)
patterns = {
    "linear":      x * 0.8 + rng.normal(0, 8, 200),
    "curved":      (x ** 1.6) / 12 + rng.normal(0, 8, 200),
    "no relation": rng.normal(50, 20, 200),
    "two groups":  np.where(x < 50, 20, 80) + rng.normal(0, 6, 200),
}
print(f"{'pattern':14} {'correlation':>12}  what a line through it would claim")
for name, y in patterns.items():
    r = np.corrcoef(x, y)[0, 1]
    claim = {"linear": "correct", "curved": "understates the high end",
             "no relation": "a relationship that is not there",
             "two groups": "a gradient where there are two clusters"}[name]
    print(f"{name:14} {r:>12.2f}  {claim}")

print("\nTwo of these have a respectable correlation and no linear relationship.")

The curved pattern has a respectable correlation and no linear relationship - a straight line through it understates the high end badly. The two-cluster pattern has a strong correlation and no relationship at all within either cluster.

Both would be reported as "correlated" by anyone reading the coefficient without the plot.

The mistake this prevents

The mistake is computing the correlation and not looking at the scatter. The coefficient is a summary, and summarising is exactly what loses the shape that determines whether the summary means anything.

Takeaway

Always plot the scatter before reporting a correlation. Curves and clusters both produce respectable coefficients and neither is a linear relationship.

Mark complete