Module 05 Activity

Scenario

Clean a messy return and produce the log that lets a reviewer trust the result.

What you build

A cleaning script with one rule per issue and a log table of every removal.

Steps

1. Count the missing values before doing anything, then find out why each one is missing. Record whether the reasons differ.
2. Define the key that identifies one row, count duplicates on that key, and state which copy you keep and why.
3. Write each validation rule as a check column rather than a filter, so the failures can be counted and looked at.
4. Convert types deliberately: strip what you understand, convert, and flag what still fails rather than letting it become NA.
5. Normalise every category column with a whitespace squish and a case rule, then confirm the number of categories is what you expected.
6. Convert ordered categories to factors with declared levels, parse dates with the format stated, and build the log as you go.

Evidence to hand in

• The missing-value count and the reason for each gap.
• The key, the duplicate count, and the rule for which copy survives.
• The check columns and the count of failures.
• The conversion, with the still-unconvertible values visible.
• The category count before and after normalising.
• The cleaning log.

Review checklist

• No missing value was filled with zero without a stated reason.
• Duplicates were found on the key, not with duplicated() on whole rows.
• Validation failures are counted and reported, not silently filtered.
• Unconvertible values are flagged rather than becoming NA.
• Every row removed appears in the log with a named reason, and the removals sum to the total change.

Mark complete