Skip to content

v0.5.0

Release Date: 2026-09-20

Highlights

  • Honest Conformance - Criteria that were not assessed are reported as "Not Evaluated" (never assumed "Supports"), with the evaluation method recorded per criterion
  • Proactive Criterion Evaluation - Judge criteria automation can't resolve, in two interchangeable modes: local (agent, no API key) or API — sharing one prompt and verdict schema
  • More Specialized Checks - Resize (1.4.10), hover (1.4.13), and flashing-content (2.3.1) checks now run in the specialized runner
  • Page Evidence - Screenshot, rendered HTML, and structure captured during audits and per journey state, for reuse across consumers
  • Richer Result API - Conformance, OpenACR, and proactive evaluation exposed directly on Result

Overview

v0.5.0 makes conformance reporting honest by construction. Earlier versions could imply support for criteria that were never actually assessed; now every criterion carries its evaluation method, and any criterion without a real result is reported as "Not Evaluated" rather than "Supports". To close that gap without a human in the loop, this release adds proactive criterion evaluation: a way to judge the criteria that automated rules cannot resolve (captions, use of color, consistent navigation, and the like), in either a local agent mode (no API key) or an API mode, both sharing a single prompt and verdict schema.

What's New

Honest conformance

report.EvaluateConformance assigns per-criterion conformance from real evidence. A criterion with no finding and a judgment method (LLM-Judge / Hybrid / Manual) is "Not Evaluated" — enabling an LLM does not flip it; only a real verdict does. report.ApplyVerdicts resolves those "Not Evaluated" criteria and never overrides a finding-backed result. The VPAT never marks an unevaluated criterion "Supports". A regression guard (report/vpat_honesty_test.go) locks this in.

Proactive criterion evaluation

llm/criteval.go provides two backends sharing one prompt and verdict schema:

  • API mode — APIEvaluator calls an LLM via omnillm.
  • Local mode — BuildBundle exports the unresolved-criteria worklist; an agent judges it; ParseVerdicts reads the verdicts back. No API key required.

See the Proactive Criterion Evaluation guide.

New specialized checks

The specialized runner now executes checks for:

  • Resize (1.4.10 Reflow / 1.4.4 Resize Text)
  • Hover (1.4.13 Content on Hover or Focus)
  • Flashing content (2.3.1 Three Flashes or Below Threshold)

Page evidence

The audit engine captures normalized page evidence — screenshot, rendered HTML, and structure — via pagecapture and stores it on PageResult.Evidence; the journey executor captures the same per state on StepExecutionResult.Evidence. Result.pageContext() builds the evaluator's context from it.

Richer Result API

Conformance and proactive evaluation are now first-class on Result: Conformance, OpenACR, EvaluationBundle, EvaluateCriteria, and ConformanceWithVerdicts.

Fixed

  • Corrected WCAG criterion evaluation-method classifications so only criteria the specialized runner actually executes are MethodSpecialized, and MethodAutomated requires real axe rules. Guarded by wcag/method_coverage_test.go.

Upgrading

go get github.com/plexusone/[email protected]

No breaking changes to existing audit flows. The new conformance honesty is additive: existing reports gain accurate "Not Evaluated" states and per-criterion methods, and proactive evaluation is opt-in.