# Primary sources and claim boundaries

Checked on **17 October 2026**. Teaching cases, policy numbers, code, and SVG are original to this local draft. No source text is reproduced at length. These references support the principles indicated, not the lab's thresholds or claims of production readiness.

| Primary source | What it supports here | Boundary |
|---|---|---|
| [OpenAI: Evaluation best practices](https://developers.openai.com/api/docs/guides/evaluation-best-practices) | Task-specific evaluation, defined objectives and metrics, case diversity, human calibration, and repeating evaluations after changes. | Provider-neutral principles only. This course neither calls nor teaches the hosted Evals service. Its current availability is time-sensitive; recheck official documentation before authoring any future integration. |
| [NIST: Generative Artificial Intelligence Profile, AI 600-1](https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence) | Trustworthiness considerations belong across design, development, use, and evaluation; the profile is a voluntary AI RMF companion. | The exercise is not certification, legal guidance, or proof of compliance. No numerical gate is attributed to NIST. |
| [Google SRE: Monitoring Distributed Systems](https://sre.google/sre-book/monitoring-distributed-systems/) | Operational signals, tail behavior, and separating successful-request and failed-request latency. | Invented latency records are not real operational monitoring. The fixture has no actual saturation measurements. |
| [scikit-learn: Common pitfalls, data leakage](https://scikit-learn.org/stable/common_pitfalls.html#data-leakage) | Why test information must not influence choices that are later judged on that test. | The course applies the principle to prompts/examples; no scikit-learn package or fitted model is used. |
| [Google SRE: Service Level Objectives](https://sre.google/sre-book/service-level-objectives/) | Measurement conditions, targets, and error-budget reasoning. | The 98% per-count-window objective and budget multiple are original toy choices, not an Aabha SLO or time-to-exhaustion forecast. |
| [Google SRE Workbook: Canarying Releases](https://sre.google/workbook/canarying-releases/) | Staged comparison, control attribution, and absolute as well as relative conditions. | The pure simulator is not a real canary, traffic test, deployment, or learner pilot. |
| [Google SRE: Postmortem Culture](https://sre.google/sre-book/postmortem-culture/) | Constructive incident review and checkable follow-up. | The dated timeline is an invented scenario, not real incident evidence or causal proof. |
| [NIST AI RMF Playbook](https://airc.nist.gov/airmf-resources/playbook/) | A voluntary set of suggested actions to adapt to the use case. | It is not a universal checklist or certification for this course. |
| [Python 3.11: unittest](https://docs.python.org/3.11/library/unittest.html) | Standard-library test discovery and assertions used in the lab. | The tested interpreter on this Mac is recorded separately; compatibility is targeted to Python 3.11+ rather than claimed tested across every version. |
| [Python 3.11: json](https://docs.python.org/3.11/library/json.html) | Parsing JSON and customizing object-pair and constant handling. | This evaluator deliberately rejects duplicate object keys and non-finite constants instead of relying on permissive defaults. |

## Interpretation rather than source prescription

The lab chooses route equality, three slices, a 90% overall floor, 75% slice floors, zero critical or new failures, a 700 ms nearest-rank p95 ceiling, and a mean budget of 2,000 micro-USD per case. Every number is an illustrative exercise decision. No reference establishes them as a universal standard. A real application needs domain evidence, representative data, an agreed failure cost, and explicit release authority.

The nearest-rank calculation is defined in the project itself so its convention can be inspected. Other percentile conventions can produce different values on small samples. The project uses no statistical library, confidence interval, model judge, or estimate of real service quality.