Unit 02.03: Datasets and DataLoaders

A DataLoader turns a dataset into batches. Two of its defaults will change your results, so it is worth knowing what they do.

Batching, shuffling, and the ragged last batch

batch_size decides how many examples the model sees before each update. shuffle reorders the data every epoch, which matters because a model trained on sorted data learns the sort order.

With 20 items and a batch size of 6 you get four batches — and the last one is not full:

import torch
from torch.utils.data import DataLoader, TensorDataset

X = torch.arange(20).float().unsqueeze(1)
y = X * 2
dataset = TensorDataset(X, y)

loader = DataLoader(dataset, batch_size=6, shuffle=False)
print("dataset items:", len(dataset))
print("batches      :", len(loader))            # 4: 6+6+6+2

for i, (xb, yb) in enumerate(loader):
    print(f"  batch {i}: {tuple(xb.shape)}")
# The last batch is smaller. Code that assumes a fixed batch size breaks here.

dropped = DataLoader(dataset, batch_size=6, drop_last=True)
print("with drop_last:", len(dropped), "batches -- the final 2 rows are discarded")

# shuffle=True changes order every epoch, which is what you want for training
# and what you must turn off for validation if you want comparable batches.
shuffled = DataLoader(dataset, batch_size=20, shuffle=True)
print("shuffled first values:", next(iter(shuffled))[0][:5].flatten().tolist())

That final batch of 2 is where fixed-size assumptions break. drop_last=True discards it, which keeps shapes uniform at the cost of throwing away real data every epoch.

Use shuffle=True for training. Leave it off for validation, so the batches are comparable between runs.

The mistake this prevents

Shuffling the validation set, then wondering why the per-batch metrics jump around between epochs. The model did not change; the batches did.

Takeaway

Shuffle training data, never validation, and always check what your code does with a partial final batch.

Mark complete