File Validation and Failure Modes

Unit ID: M07-U05 Estimated active time: 25-35 minutes

Validate before transformation

Check:

• path exists;
• path is a file;
• suffix matches the expected format;
• file can be decoded as UTF-8;
• required CSV headers or JSON shape exist; and
• output paths are different from the source path.

Raise specific errors

if not source_path.exists():
    raise FileNotFoundError(f"source file not found: {source_path.name}")

if not source_path.is_file():
    raise ValueError("source path must identify a file")

if source_path.suffix.lower() != ".csv":
    raise ValueError("source file must use the .csv extension")

Do not include private absolute paths in learner-facing error reports when a safe file name is enough.

Catch at the user-facing boundary

Low-level reading functions can raise useful exceptions. The orchestration layer may catch expected file errors to show a clear message or record a failed run.

Do not continue to write partial outputs after required input validation fails.

Temporary output strategy

For a larger production workflow, write to a temporary file and replace the final output only after validation. The beginner checkpoint writes small outputs to a clean temporary test folder and verifies them after creation.

Unexpected extra fields

Decide explicitly whether extra CSV headers are allowed. The checkpoint allows extra fields but writes only the approved output schema. Missing required fields stop the conversion.

Practice

Describe expected exception types for a missing source, directory passed as source, wrong suffix, invalid UTF-8, missing CSV header, malformed JSON, and unwritable output folder.

Takeaway

File validation checks location, role, format, decoding, and structure before transformation. Catch expected failures at a clear boundary and do not publish partial results as success. Next, we will add provenance and protect source evidence.