Analytics qualification fundamentalsLesson 1 of 6
A correct-looking report can be wrong
Follow an order-header versus order-line mistake across the pipeline.
Begin with the business question
A report can look finished before its numbers have been qualified. Tables load, relationships resolve, and a polished card displays a sales total. Those are useful authoring observations. They do not establish that the total answers the question a reader intended to ask. This lesson follows one common ambiguity: an order header and its order lines describe different grains.
Grain means the business meaning of one fact row. An order header describes an order. An order line describes an item on that order. Before choosing a measure, ask which of those events the report needs to represent. A label such as “Sales” leaves that decision unstated. A reviewer needs the definition, the scope and the expected comparison as well as the label.
Follow the mistake downstream
Imagine a draft joins order headers to their lines, then treats an order-level amount as though it belongs to every line. The join can be syntactically valid while the resulting business interpretation is wrong. A successful query proves that the query ran. It does not prove that repeating an order amount is appropriate for the chosen fact grain.
Now carry that draft through the pipeline. Bronze captures a declared source batch. Silver prepares typed data and records accounting or quarantine observations. Gold implements the reviewed business structures. A semantic model provides relationships and measures. A report queries that model. If every layer inherits the same mistaken assumption, their outputs may agree with one another. Agreement among copies of an assumption does not make the assumption independent.
Write the grain in plain language before trying to repair a formula. “One row per order” and “one row per order line” imply different keys, relationships and valid aggregations. A human must ratify the intended definition. A domain seed can help draft that conversation, but cannot decide the client's meaning or supply business acceptance.
Compare with an independent expectation
An independent oracle is an expectation established separately from the implementation under test. Its source query and business interpretation must answer the agreed question. Copying the generated measure into a second query would carry the same mistake into the answer key. Comparing those two outputs could produce a reassuring agreement without detecting the defect.
Qualification therefore begins with declared checks and reviewed expectations. It compares observed actuals at the required boundaries with those expectations. The ledger records producer results and binds them to declared inputs, outputs and upstream evidence. A gate recomputes numeric comparisons rather than merely accepting a producer's pass flag. These mechanisms make a bounded claim reviewable; they cannot choose the right business question for you.
Separate the observations
Use this table to annotate the example. Each row answers a different question, so retain the distinction in your worksheet.
| Observation | What it can establish | What remains open |
|---|---|---|
| A model was generated | Definitions were produced | Deployment, refresh and business meaning |
| A query matched an independent expectation | The declared comparison matched | Other groups, contexts and future data |
| A report image was inspected | A particular rendered state was observed | Unobserved states and source reconciliation |
| A person accepted the report | A recorded human decision | Any machine checks outside that decision |
Completion check
Write a short explanation of how the header/line mistake could survive successful authoring. Name the person who must ratify grain, describe an independently established expectation, and identify a report state that still needs observation. Finish with a limitation: a passing comparison applies to its declared checks and current evidence. It does not establish universal report correctness or future freshness.