Qualification and oracles

Start with a claim you can test

Qualification establishes that declared checks passed against current evidence for a named scope. It needs a question, an independently established expectation and a retained observation. A successful notebook or convincing chart alone supplies none of those three in full.

For a Sales example, first agree what Sales Amount means. State the fact grain, period, currency, exclusions and required groups. A total across order headers can disagree with a total across order lines even when both queries execute successfully. Human grain ratification prevents the compiler from quietly deciding that business meaning for you.

Keep the expectation independent

An oracle is the source expectation used to judge the downstream result. Independence matters because comparing Gold with a DAX measure derived from the same mistaken transformation can reproduce the same error twice. The agreement would be real; the assurance would be weak.

Author the source queries and review their meaning separately from the implementation under test. Capture expectations under the approved source-read boundary. Source profiling, raw extraction and oracle capture are distinct operations. A value-free source index describes structure and coverage; it does not contain the independently captured answer key.

Freeze the reviewed expectations

Before acceptance, freeze the catalogue, definition and captured values bindings. This makes the intended comparison explicit. The lock binds those artifacts; it is not permission to expose their private contents or replace them when the implementation disagrees.

An expectation change needs deliberate authoring and an explanation. If a business definition changes, reconcile that decision and retain its history. Do not silently refreeze after a failure to turn the actual into the new expected value. History qualification adds explicit source, run, coverage and expectation epochs, batch identity and receipt selections.

Recompute the comparison from evidence

Producers emit structured step results. Only the ledger records receipts. For numeric checks, the gate recomputes the difference between actual and expected using the declared tolerance. It does not accept a producer's pass flag as a substitute for the numbers.

Grouped checks also require the exact expected key set. A matching grand total does not excuse a missing region, duplicated group or omitted exception. Tolerances, decimal scale and displayed rounding need explicit treatment; they answer different questions and should not be blurred into a visual approximation.

A gate needs current evidence. Changed inputs, missing retained files, contradictory observations or an unsuccessful newest retry cannot be rescued by pointing to an older green result. The receipt chain explains the dependencies that make this enforceable.

Keep business acceptance with people

A qualified result applies to the declared checks and captured observations. It does not decide whether the chosen measure supports the intended business decision, whether the page is usable or whether untested security contexts behave correctly.

Write a review statement that names the scope, expectation version, observation time and exclusions. Distinguish the machine result from the human's acceptance. That discipline makes a narrow proof useful without expanding it into a claim about every number, future refresh or interpretation. Use the status vocabulary when reporting a result to another reader.