v0.14.0

Release Date: 2026-08-15

Highlights

• Layered rubric classification: CriterionClass and EvaluationMethod taxonomies, plus Class/Blocking/Evaluation fields on categories and criteria, let a rubric separate advisory principle-based judgment from gating implementation checks instead of collapsing everything into one composite score
• RubricSet.JudgeInstructions: cross-category evidence-discipline rules for the LLM judge prompt, complementing any per-category EvaluationPrompt
• INV-3: RubricSet.Validate() rejects a leadership_principle class that is marked blocking — advisory judgment cannot gate implementation
• Fully additive: every new field is omitempty and defaults to its zero value, so a v0.13.0-shaped rubric parses and validates unchanged

Overview

A single composite score hides an important distinction. Advisory, principle-based judgment (e.g. "does this show enough long-term thinking?") is a different kind of thing from a mechanical implementation check (e.g. "is every requirement traceable to a stable ID?"). Collapsing both into one number lets a soft, debatable dimension silently sink — or worse, block — a decision that should hinge on hard, checkable ones.

v0.14.0 adds three optional fields on both Category and Criterion that keep these layers explicit, a RubricSet-level list of judge instructions, and a validation invariant that prevents the most dangerous misconfiguration: wiring advisory judgment as a hard gate.

New Features

CriterionClass and EvaluationMethod

Class names the kind of judgment a category or criterion represents; Evaluation names how it is checked.

type CriterionClass string

const (
    ClassLeadershipPrinciple     CriterionClass = "leadership_principle"
    ClassSpecificationQuality    CriterionClass = "specification_quality"
    ClassImplementationReadiness CriterionClass = "implementation_readiness"
    ClassDeterministicIntegrity  CriterionClass = "deterministic_integrity"
)

type EvaluationMethod string

const (
    EvalMethodDeterministic EvaluationMethod = "deterministic"
    EvalMethodSemantic      EvaluationMethod = "semantic"
    EvalMethodHuman         EvaluationMethod = "human"
)

CriterionClass	Meaning
leadership_principle	Principle-based decision lens (e.g. AWS Leadership Principles), not a completeness check. Never blocking.
specification_quality	Is the intended behavior complete and unambiguous?
implementation_readiness	Could an agent implement and verify the spec safely without guessing?
deterministic_integrity	Structural checks: headings, IDs, links, broken references, schema validity.

An empty Class means unclassified (legacy rubrics); consumers should treat it as specification_quality.

Class, Blocking, Evaluation on categories and criteria

cat := rubric.NewCategory("traceability", "Requirement Traceability",
    "Every requirement maps to a stable ID").
    WithPassPartialFail(pass, partial, fail)
cat.Class = rubric.ClassDeterministicIntegrity
cat.Evaluation = rubric.EvalMethodDeterministic
cat.Blocking = true // a broken reference is a hard stop

Blocking is an absolute veto — distinct from Required, which feeds minCategoriesPassing. A fail on a blocking category or criterion blocks approval regardless of other scores.

RubricSet.JudgeInstructions

Evidence-discipline rules that apply across all categories. Render them into the judge system prompt so the same rules govern every dimension.

rubricSet.JudgeInstructions = []string{
    "Cite the relevant section and requirement IDs for every score",
    "Do not reward length; reward completeness and precision",
    "Distinguish missing evidence from negative evidence",
}

INV-3: advisory judgment cannot gate implementation

RubricSet.Validate() enforces that a category or criterion whose Class is leadership_principle must not set Blocking:

issues := rubricSet.Validate()
// "category think_big: leadership_principle class must not be blocking
//  (advisory judgment cannot gate implementation)"

TypeScript / Zod

@plexusone/structured-evaluation@0.14.0 regenerates the Zod schemas and TS types for the new rubric layering fields (class, blocking, evaluation, judgeInstructions). This regeneration also catches up the v0.13.0 claims fields — sourceRole, minCorroboratingSources, corroborationCategories, and maxClaimAge — which shipped in the Go types and JSON Schema then but were not regenerated into the TS package at the time.

Backward Compatibility

Every new field is omitempty and defaults to its zero value. A rubric authored against v0.13.0 (no layering fields, no judge instructions) parses identically and validates cleanly — the only new validation error, INV-3, fires solely when a rubric explicitly sets both Class == ClassLeadershipPrinciple and Blocking == true. There are no breaking changes.

Commits

• 41e77fe feat(rubric): add layered classification and judge instructions
• 9ee8731 test(rubric): cover layered fields, legacy parsing, and INV-3
• 8b6b663 docs(rubric): document layered classification and judge instructions
• 572294c chore(ts): regenerate Zod schemas for rubric layering fields