Answers and explanations · Understand what a generator does

A. Counts after desk are opens 3 and closes 1. At temperature 1, probabilities are 3/4 and 1/4; greedy chooses opens. At temperature 0.5, raise counts to power 2: weights 9 and 1 normalize to 0.9 and 0.1. These are continuation estimates from authored examples, not verified library facts. A confident-looking distribution can come from repeated mistaken examples.

B. A runnable check is:

from methods import train_bigram, decode_bigram
table = train_bigram(["desk opens weekdays"] * 3 + ["desk closes holidays"])
print(decode_bigram(table, "desk"))
print(decode_bigram(table, "laptop"))
print(decode_bigram({"again": {"again": 1}}, "again", max_steps=4))

The first emits opens weekdays and stops at end_token. The second emits nothing and stops at unseen_history: the implementation has no smoothing or fallback. The third emits four copies of again and stops at step_budget. A finite bound makes this cycle observable without leaving a process running.

C. Both scores are 0, so exponentiated weights are 1 and 1, normalized to 0.5,0.5. The weighted result is 0.5×4+0.5×12=8. Adding the same finite constant to both scores multiplies both exponentials by the same factor, which cancels under normalization. The averaging mechanism says how inputs contribute, not whether the inputs represent true statements. Numerical weights are not citations.

D. Supplying a new handbook changes the information presented during inference. Retraining would change learned parameters using a training procedure. Retrieval selects the supplied material; a model may then condition its next-token distribution on that context. Our toy model uses lowercase English words and one-word history, while another model can have a different tokenizer and context accounting. We must measure its actual serialized tokens before claiming anything about fit.