HumachLearnLearn · explore · apply
Start learning
Reporting & Reproducibility · Lesson 87

Data and Metric Definitions

Data and Metric Definitions is about connecting analysis to a decision.

ConceptWorked examplePracticeKnowledge check
Textbook walkthrough

What Data and Metric Definitions actually means

Data and Metric Definitions is about connecting analysis to a decision. Measures quantify something, dimensions describe how observations are grouped, and KPIs select a small set of measures that reflect progress toward an objective.

Data and Metric Definitions matters because analysis is only useful when another person can understand the evidence and another analyst can reproduce the calculation. Reporting therefore includes definitions, uncertainty, data lineage and execution details—not just final numbers.

Deeper walkthrough

Read Data and Metric Definitions as a mechanism, not a recipe

Treat this as a sequence of observable decisions rather than one opaque command. Stage 1: Name the stakeholder and decision. Stage 2: Define the unit of analysis (customer, order, day, device, etc.). Stage 3: Define each measure with numerator, denominator, time window and inclusion/exclusion rules where relevant. Final checkpoint: Check whether the metric can move because of data collection changes rather than real-world change.

Mechanism

Follow the transformation

Name the stakeholder and decision.

Define the unit of analysis (customer, order, day, device, etc.).

Define each measure with numerator, denominator, time window and inclusion/exclusion rules where relevant.

Evidence

Know what would convince you

  • Write the decision question, unit of analysis and metric formula in plain language before computing it.
  • Reconcile KPI numerators/denominators or grouped totals to source counts for a small slice.
Useful distinctionMeasure: A numeric quantity such as revenue or response time.
How it works

Trace the mechanism step by step

  1. Name the stakeholder and decision.
  2. Define the unit of analysis (customer, order, day, device, etc.).
  3. Define each measure with numerator, denominator, time window and inclusion/exclusion rules where relevant.
  4. Choose dimensions that support meaningful segmentation.
  5. Check whether the metric can move because of data collection changes rather than real-world change.
Worked demonstration

Make the concept concrete

Demonstration

Text example

Field: order_date = timestamp when payment is confirmed, UTC.
Metric: refund_rate = refunded_orders / completed_orders within the same 30-day cohort.
Exclusions: test orders and cancelled-before-payment orders.
Expected / illustrative result
Operational definitions prevent two analysts from calculating “the same” metric differently.
Interpret the result.

For Data and Metric Definitions, connect the displayed result to the specific input and mechanism above; independently verify one value/state change rather than treating successful execution as proof.

Distinctions & related ideas

Know what this is — and what it is not

MeasureA numeric quantity such as revenue or response time.
DimensionA grouping/category such as region, product or channel.
KPIA measure deliberately tied to an objective and decision cadence.
TargetA desired value or threshold against which a KPI is evaluated.
Use deliberately

When it is appropriate

Use Data and Metric Definitions when it connects a clearly framed stakeholder question to measurable evidence and an action or decision.

Boundary conditions

When to stop or reconsider

Reframe the analysis if the decision, unit of analysis, metric definition, comparison group or time window is still ambiguous; more computation will not repair an undefined question.

Common mistakes

Failure modes to recognise

  • Starting with a favourite chart/tool before defining the decision and unit of analysis.
  • Using an undefined KPI, denominator, cohort or time window and then comparing incomparable numbers.
  • Turning association or a descriptive pattern into a causal recommendation without supporting design/evidence.
Verification

How to check the result

  • Write the decision question, unit of analysis and metric formula in plain language before computing it.
  • Reconcile KPI numerators/denominators or grouped totals to source counts for a small slice.
  • Test whether the conclusion changes under one reasonable alternative definition or comparison window.
Hands-on practice

Demonstrate understanding

Try this:

Build a tiny, inspectable example of Data and Metric Definitions. First name the stakeholder and decision. Then define the unit of analysis (customer, order, day, device, etc.). Write the expected result before running it, and explain one condition that would make the result misleading or invalid.

Write the decision, unit and metric definition first. Use a tiny slice where you can recompute the KPI/table manually and explain what would change the recommendation.
Knowledge check

Check reasoning, not memorisation

Before trusting a result from Data and Metric Definitions, which check provides the strongest evidence that you understand and applied it correctly?

Quick reference

Keep the important distinctions visible

Step 1Name the stakeholder and decision.
Step 2Define the unit of analysis (customer, order, day, device, etc.).
Step 3Define each measure with numerator, denominator, time window and inclusion/exclusion rules where relevant.
Step 4Choose dimensions that support meaningful segmentation.
Lesson summary

What to remember

  • Data and Metric Definitions is about connecting analysis to a decision. Measures quantify something, dimensions describe how observations are grouped, and KPIs select a small set of measures that reflect progress toward an objective.
  • Name the stakeholder and decision.
  • Starting with a favourite chart/tool before defining the decision and unit of analysis.
  • Write the decision question, unit of analysis and metric formula in plain language before computing it.