pandas Summaries and Plots

Unit ID: ML-M02-U05 Estimated active time: 30-40 minutes

Inspect before you model

pandas gives you quick ways to inspect a dataset.

Start with:

df.shape
df.head()
df.info()
df.describe()

Do not stop there. These commands tell you what is stored, not whether it is safe to use.

Count categories

Use:

df["python_preparation"].value_counts()
df["completed_module1_by_day10"].value_counts()

Counts help you see imbalance and coverage.

Inspect missing values

Use:

df.isna().sum().sort_values(ascending=False)

Then explain the missing values in words. A count without interpretation is not a data audit.

Plot carefully

A simple bar chart can help inspect target balance.

df["completed_module1_by_day10"].value_counts().sort_index().plot(kind="bar")

Every plot must have a text explanation. Do not rely on colour alone.

Practice

Run summary code and answer:

• Which columns have missing values?
• How many learners completed Module 1 by day 10?
• Which columns would you exclude before modelling?
• Which column needs proxy-risk review?

Takeaway

pandas summaries help you inspect the table, but the audit comes from your explanation of what the summaries mean.