Reference capstone · measured results

This is a report of the executed reference configuration, not a required score for every learner. Values are from the bundled output JSON.

Protocol

Predict the fictional nonlinear sensor response from two features. Use independently generated regression rows and fixed random 360/120/120 partitions. Fit mean/population standard deviation on training only. Compare a linear model and a two-hidden-layer width-16 MLP, each with 100 epochs. Select checkpoint and model by validation MSE. The visible bundled test is a teaching fixture, not a hidden assessment.

Selection and final assessment

• linear: best epoch 57; validation MSE 1.26921833, RMSE 1.126596

• mlp: best epoch 99; validation MSE 0.02235237, RMSE 0.149507

Selected mlp. On 120 test examples, MSE 0.02597234, RMSE 0.161159, MAE 0.125866. The constant train-mean baseline predicts 1.180640 and has test RMSE 2.285362. Only the already-selected learned model is evaluated on test; the naive baseline is fixed from train.

Interpretation and limitations

The nonlinear candidate better represents this generator’s curved relationship. The comparison is consistent with improved fit, not proof of a causal feature effect or real sensor utility. The fixed synthetic process has low noise and no deployment drift. A real application would require availability-aware features, realistic group/time holdouts, error-cost decisions and external validation. Repeated reference runs verify execution, not additional independent test evidence.

Classification extensions

rings: selected mlp by validation loss. Test accuracy 1.0000 on 120 examples; confusion counts {'tn': 57, 'fp': 0, 'fn': 0, 'tp': 63}. Fixed train-majority baseline test accuracy 0.4750.

bars: selected cnn by validation loss. Test accuracy 1.0000 on 120 examples; confusion counts {'tn': 57, 'fp': 0, 'fn': 0, 'tp': 63}. Fixed train-majority baseline test accuracy 0.4750.

Both classification fixtures are deliberately easy, balanced class-conditional samples. The bars linear baseline also achieved perfect validation accuracy; the CNN was selected by lower validation loss, not a demonstrated accuracy advantage. Zero test errors on these fixtures do not establish real-world performance, robustness or calibrated confidence.

Inspect the underlying files

See outputs/reference_results.json for full candidate histories, split IDs, settings and metrics. See outputs/reference_capstone/selection.json and final.json for the two-stage capstone record. These are reference outputs, not replacements for learner implementation or explanation.