What the scan measures, and what it cannot reach

Where each criterion falls

These four classes partition the 56. They do not overlap, and they add up.

• 36Graded automatically by a written ruleEach of these has a written rule that reads what was captured from the page and produces a finding carrying the observed value, the expected value, and where on the page it sits.
• 3Judged by a language modelThe language, terminology and tone criteria. They are issued at a reduced confidence ceiling, and a failing verdict is not accepted unless the model quotes text from the page to support it.
• 12Automatable, with no rule built yetThey appear in the report tagged “no rule built yet” and are counted neither for nor against the estimate. This is a work list, not a technical limit.
• 5Only a specialist can measure themClarity of visual hierarchy, intuitiveness of navigation, efficiency of task completion, predictability of behaviour, and consistency of interaction patterns. No external automated scan reaches these.

What the scan does not reach, and why

Each of these is a limit we declare, not a gap we are working around.

• 33 of 127Checkpoints that need interactionComponent states — hover, press, focus, disabled — appear only when the element is interacted with, and our scanning rules forbid interacting with an entity’s platform. This is an ethical constraint we chose, not a technical failure: every component finding says its states were not examined.
• Behind sign-inPages that require national single sign-onThe scan reads publicly published pages only. Most forms, file uploads and service steps sit behind a sign-in, so the tool never reaches them and issues no verdict on them.
• A sampleUp to 8 pagesThe whole platform is not scanned — a sample is, covering the mandatory templates and the page types. A component missing from the sample is not evidence that it is missing from the platform, which is why it is recorded as “not applicable” rather than “not applied”.
• ValuesNot all verified against the official source yetColour and spacing have been compared against the Platforms Code source. Type sizes, corner radii, shadows and gradients have not been verified yet, so no violation is issued from them; they are recorded and nothing more.

How this shows up in the report

• What is not measured leaves the denominator. An unmeasured criterion counts as neither a pass nor a failure. It leaves the estimate calculation entirely, so it neither raises nor lowers the result.
• The estimate is a range, not a number. The lower bound assumes the worst about what was not measured; the upper bound assumes the best. The width between them is the size of what we did not reach — declared, not hidden.
• Every finding carries its evidence. The observed value, the expected value and its location, so it can be verified or disputed without coming back to us.
• The detection method is stated. Where a component was detected from page structure rather than from semantic markup, the finding says so and its confidence is lowered.

The methodology behind all of this is set out on the methodology page, and the criteria themselves are indexed on the 56 criteria.