Data Is a Record of a Process

Unit ID: ML-M02-U00 Estimated active time: 20-25 minutes

Why this matters

A dataset does not appear by magic. It is created by a process.

People click buttons. Systems log events. Rules decide what is saved. Some values are missing. Some values are recorded late. Some useful facts are never recorded at all.

Before modelling, ask:

What process created this table?

If we do not understand the process, we can easily trust the wrong column.

The running example

In this module, we use learner_progress_audit_v1.csv.

It is fictional synthetic data. One row represents one learner's first attempt at Module 1.

The practical question is:

At the end of day 3, can we understand which data might help predict whether a learner completes Module 1 by day 10?

We are not training a model yet. We are inspecting the evidence.

A table has a story

A row in this dataset exists because a fictional learner started Module 1. Some columns come from activity logs by the end of day 3. Some columns come from later outcomes.

That timing matters.

If we predict at the end of day 3, we cannot use a final quiz score. The score did not exist yet.

Practice

Open the data dictionary and answer:

• What does one row represent?
• What is the target?
• Which columns are recorded after the prediction time?
• Which columns have missing values?

Takeaway

Data is a record of a process. Before modelling, understand how the table was created and when each value became available.