Data Size, Feature Type, Noise, Missingness, and Dimensionality

Unit ID: AMLA-M01-U01 Estimated active time: 30-40 minutes

Classroom explanation

Algorithms respond differently to the shape of the data. Before fitting anything, inspect the data shape.

Start with five questions.

1. How many rows do we have?

Small data makes complex models easier to overfit. With very few rows, a simple baseline, regularised linear model, or carefully limited tree may be more honest than a complex ensemble.

Large data can support more flexible models, but large data also makes runtime and maintenance important.

2. What feature types do we have?

Numeric, categorical, text-like, sparse, ordered, and count features may need different preprocessing. Some algorithms are sensitive to scale. Some work naturally with sparse features. Some need careful encoding before they make sense.

3. How noisy is the signal?

If the label is noisy, a more complex model may learn noise confidently. A slightly lower score from a simpler model can be safer if the result is more stable.

4. What is missing?

Missing values are not just blanks. They may reveal process behaviour. Some algorithms need imputation. Some tree methods can handle missing values in specific ways, depending on the implementation. You must know what the runtime supports before relying on it.

5. How many features do we have?

High-dimensional data can make distance less meaningful, increase overfitting risk, and make inspection harder. Feature selection or dimensionality reduction may help, but those choices must happen inside the modelling pipeline.

Practice

For a dataset you already know, write one line for each question:

• Rows:
• Feature types:
• Noise:
• Missingness:
• Dimensionality:

Then name one algorithm family you would try early and one you would avoid at first.

Takeaway

The data shape narrows the candidate list before modelling begins.