Module 2 practice · a generator's decisions

Attempt these before reading the solutions. Use methods.py decoding to inspect a trace, but calculate before running.

A. A changed training table. Train conceptually on “desk opens weekdays” three times and “desk closes holidays” once. What are the next-word probabilities after desk at temperature 1? What does greedy decoding select? At temperature 0.5, compute the two normalized weights. Does any result establish the actual opening policy?

B. Replay and stopping. In a Python session, use train_bigram and decode_bigram on the four sentences above. Generate greedily from desk, then from the unseen history laptop. Construct {"again": {"again": 1}} as a continuation table and decode from again with max_steps=4. Explain each stopping reason. Do not remove the bound to see how long a loop runs.

C. Attention arithmetic. For query 1, keys 0,0 and values 4,12, calculate the normalized weights and weighted value. What happens to the weights if both scores receive the same added constant? Explain why neither calculation establishes which value is factually correct.

D. A plain-language distinction. A colleague says putting a revised handbook in a prompt “retrained the model.” Write a correction that separates source retrieval, inference context and parameter updates. Explain why our word-token count cannot establish a future model's context usage.

Self-review: a strong response gives correct denominators, reports all three stopping cases, computes the average and names the limits of the toy model. This is practice feedback, not platform credit.