The method, line by line

In short. Habitat holds every number it publishes to these lines. Where a result still falls short of one, its own page says so, and the open items are gathered on Limits, in one place.

1. The record is the subject: not the model, not the meaning, not the person. We look at what entered and in what order, and nothing else.
2. Every claim ships with its control and its limit on the same line; a claim without a control is not made. You never have to hunt for what a number was measured against.
3. The null is the record's own; anything declared is declared before the read and travels in the receipt. The baseline comes from the record itself, and the settings can't be changed after the fact.
4. The reader recomputes: bytes in, the same digest out, on their machine, without us; where our code is still needed the receipt says so and that is a build item. If you cannot check something without us, we say so.
5. Nothing is promised that does not run: past tense for what ran, present for what runs, future only for the ask. These pages describe what exists.
6. Nothing erased; corrections published; editions beside each other with a dated note; a withdrawn claim stays. When we get something wrong, you can still see what we said and when we changed it.
7. The register: rigorous and quiet, limits first, the punch is the recompute, the first move is the reader's. We would rather you ran the command than took our word.

What a result looks like

Every result we write up has four parts, always in this order:

1. What was declared before the run: the record, the rule, the seed, the number of draws, the threshold.
2. The numbers.
3. What they say, as facts. What they mean is left to the reader.
4. The limits: what the result cannot see, and what wasn't done.

How a change is graded

A change is real only if it moves the record further than the record moves on its own. That "own movement", the floor, is measured from the record: for example by re-sampling its own rows thousands of times with a fixed seed and taking a declared quantile. The floor, the seed, the number of draws and the quantile are all published with the verdict.

Two cautions, both stated on the receipt pages too:

• A floor can be read more than one way, for example at the median or at the 99th percentile. Each receipt page says which reading it uses, and a result too close to call is reported as "at the floor", not as a verdict.
• One older receipt, revision materiality, still grades by a fixed cut (3 standard deviations). That rule is under review, and the receipt's page says so.