Files, CSV and errors

Outcome: parse a small CSV without hiding bad rows. Prerequisite: Module 4. Profile: Python basics. Time: 2–3 hours.

Text is not yet validated data

A CSV is text representing rows and fields. A header names the fields, and later rows carry values. Those values arrive as text even if a column is called minutes. Parsing gives structure; validation determines whether the structure and values satisfy your task's rules. These are separate responsibilities.

Do not parse general CSV with line.split(","): a quoted field may contain a comma or a newline. Python's csv module understands CSV quoting rules. csv.DictReader uses the header to create a dictionary for each row, which makes row["minutes"] more readable than remembering a column number.

For browser practice we use StringIO, a file-like object backed by text in memory. It lets you practice a file-reading interface without relying on browser uploads or filesystem persistence. The same CSV reader can later consume a real opened file. StringIO is not a permanent saved file.

Errors belong in the design

A SyntaxError prevents Python from understanding the code. Runtime exceptions such as ValueError, KeyError and FileNotFoundError happen when an operation cannot complete during execution. A logic error may produce plausible output with no exception at all.

Use try around the operation whose expected failure you can actually handle. except ValueError here means an invalid numeric field should be recorded and the next row processed. It does not mean every problem should be ignored. A bare except: or an except Exception: pass would hide spelling errors, missing columns and other defects that deserve attention.

Worked example

# Profile: Python basics. StringIO gives the CSV reader file-like text.
import csv
from io import StringIO

csv_text = "topic,minutes\npython,20\ndata,not recorded\ncharts,30\n"
accepted = []
rejected = []
with StringIO(csv_text) as source:
    for line_number, row in enumerate(csv.DictReader(source), start=2):
        try:
            minutes = int(row["minutes"])
            if minutes < 0:
                raise ValueError("minutes must be non-negative")
        except ValueError:
            rejected.append({"line": line_number, "raw": row["minutes"]})
        else:
            accepted.append({"topic": row["topic"], "minutes": minutes})
print("Accepted:", accepted)
print("Rejected:", rejected)
print("Accepted minutes:", sum(row["minutes"] for row in accepted))

Download this worked script

Expected accepted rows are python/20 and charts/30. The rejected record preserves the text not recorded and line 3, including the header as line 1. The accepted total is 50. We have not decided that the rejected session lasted zero minutes; its duration remains unknown.

The with block closes its file-like object when the block exits. enumerate(..., start=2) supplies row labels beginning after the header. These line numbers work for this simple single-line-per-record fixture; for arbitrary CSV with quoted embedded newlines, call them record positions or use the reader's physical line tracking instead.

The final sum contains a generator expression: row["minutes"] for row in accepted supplies one value per accepted row to sum. It is a compact version of the accumulator loop you already know. If it is unfamiliar, rewrite it as total = 0 followed by a loop adding each row's minutes; the result must match.

else after try/except runs only when the try block completed without the handled exception. That makes the accepted path clear. If the conversion succeeds but the value is negative, we raise ValueError ourselves because the input breaks our rule. A negative number is valid Python text but an invalid duration under this task's contract.

A safe local file pattern

The optional examples/05_local_files.py reads datasets/study_sessions.csv using Path and with source_path.open(encoding="utf-8", newline=""). UTF-8 makes the text encoding explicit; newline="" lets the CSV module handle newlines. The file is located relative to the script's directory rather than wherever a terminal happened to start.

The script writes only outputs/session_summary.txt. Writing with replacement mode overwrites that destination, so generated outputs belong in their own folder and raw input should be preserved. The included sample has 180 minutes. Run it twice and inspect both the raw CSV and output: the raw source should be unchanged, and the summary should be reproducible.

Worked export: a small supplies list

Here is a separate export example before you build the session exporter. fieldnames specifies the column names and their order. writeheader() writes those names as the first CSV row. writerows(rows) writes each dictionary using that order.

# Profile: Python basics. An unrelated export example for Module 5C.
import csv
from io import StringIO

rows = [{"item": "pencils, blue", "count": 2}]
buffer = StringIO()
writer = csv.DictWriter(buffer, fieldnames=["item", "count"])
writer.writeheader()
writer.writerows(rows)
exported_text = buffer.getvalue()
buffer.close()
print(exported_text)

Download this export example

StringIO stores text in memory. Its getvalue() method returns all text written so far as an ordinary string; call it before closing the buffer. Closing the buffer does not destroy the string already returned. The output is:

item,count
"pencils, blue",2

The quotes around pencils, blue keep the embedded comma inside one field. DictWriter handles that quoting for you; do not concatenate fields manually. Exercise 5C uses the same sequence with different column names and the accepted session rows. A JSON file is another useful format for structured records, but it is not needed for this project.

Practice

• 5A: write parse_minutes(raw) using integer conversion and a non-negative check. Accept "0" and " 20 "; reject "unknown", "2.5" and "-1".
• 5B: complete read_sessions in the starter. Return accepted dictionaries and a separate rejection list retaining original text and record position. The fixture has two accepted and two rejected rows, totaling 20 accepted minutes.
• 5C: write accepted rows back to a StringIO CSV using DictWriter. Read that text again and verify its fields. Test a topic containing a comma to check that quoting survives the round trip.

Exit check

Why is “readable as an integer” different from “allowed by the task”? Why would replacing missing minutes with zero misrepresent the data? Which error should remain visible if a required header is missing? How does a context manager help when an exception occurs halfway through reading?

Further reading

• Python: CSV readers and writers
• Python: errors and exceptions