What Deep Learning Adds

Deep learning is not "machine learning but bigger". It adds one specific capability, and you can see exactly where a classical model stops and a network keeps going.

The problem a straight line cannot solve

XOR is the smallest example of a pattern no linear boundary can separate. Two inputs, four cases, and the answer flips diagonally. No matter where you draw a single straight line, at least one point ends up on the wrong side.

A network with one hidden layer builds an intermediate representation first, then separates *that*. Watch the accuracy difference:

import torch
from torch import nn

# What a linear model cannot do: XOR. No straight line separates these.
X = torch.tensor([[0., 0.], [0., 1.], [1., 0.], [1., 1.]])
y = torch.tensor([[0.], [1.], [1.], [0.]])

def train(model, steps=2000):
    opt = torch.optim.Adam(model.parameters(), lr=0.05)
    for _ in range(steps):
        opt.zero_grad()
        nn.BCEWithLogitsLoss()(model(X), y).backward()
        opt.step()
    return ((torch.sigmoid(model(X)) > 0.5).float() == y).float().mean().item()

torch.manual_seed(0)
linear = nn.Linear(2, 1)
hidden = nn.Sequential(nn.Linear(2, 8), nn.ReLU(), nn.Linear(8, 1))

print("linear model accuracy   :", train(linear))    # 0.5 -- no better than guessing
print("one hidden layer        :", train(hidden))    # 1.0

# That hidden layer is the whole idea: learned intermediate representations
# solve problems a single linear boundary cannot.

The linear model sits at 0.5 — it is guessing. Add one hidden layer of eight units and accuracy reaches 1.0. Nothing changed about the data or the optimiser; the only difference is that the network was allowed to learn a representation before classifying.

The mistake this prevents

Assuming a network is always the stronger choice. On data that *is* linearly separable, logistic regression matches a network, trains in milliseconds, and is far easier to explain. Reach for depth when the relationship is non-linear, not by default.

Takeaway

Depth buys you learned intermediate representations. That is the whole advantage — and it only pays when the problem needs it.

Mark complete