Deep Learning · Module 7
Answers and explanations · When fitting better means predicting worse
Compare these references after your own attempt. You can read them at any time; they do not record a live model or vendor run.
Deep Learning · Module 7
Compare these references after your own attempt. You can read them at any time; they do not record a live model or vendor run.
7A. Output is 0 with probability 0.4 or 6/(1-0.4)=10 with probability 0.6. Expected output is 0×0.4+10×0.6=6. Evaluation returns 6. This preserves expectation, not each individual training value.
7B. Epoch 20 has the smallest validation loss, 0.31. Later training improvements may fit sample-specific noise. The declared rule restores epoch 20; it does not let a test score override the choice.
7C. The wide candidate uses 24 deliberately noisy rows, width 64, no dropout/decay and learning rate .01; the smaller candidate uses width 16, dropout .15, decay .01 and rate .005. In the recorded run, wide best validation MSE was about .5413 at epoch 22, while its final validation MSE rose to 3.1869 despite training MSE about .00979. The regularized candidate's best validation MSE was about .8293 at epoch 296. This illustrates overfit and the value of checkpointing; it does not show regularization winning here or isolate one regularizer's causal effect.