Skip to the diagram
05 / 08Micro / research, contracts, evaluation and proof

From a claim to a release decision.

A useful evaluation names the task, criteria, scope and result. The Intent Eval Platform separates methodology, executable checks, shared evidence contracts and rollout decisions so the conclusion can be inspected.

INTENT SOLUTIONS / HOW A CLAIM BECOMES EVIDENCE

DEFINE
EXECUTE AND MEASURE
PRESERVE EVIDENCE
APPLY POLICY AND RECORD THE DECISION
From a claim to a release decision. — relationshipsThe six-repository Intent Eval Platform connects Lab methodology and Core contracts to task cases, deterministic checks and behavioral evaluation. Checks emit an Evidence Bundle; the rollout gate combines it with a policy and emits a decision record for the dashboard. Failure returns to the test or implementation. This shows the architecture, not universal deployment coverage. method contracts to cases criteria cases checks results schema evidence policy decision signed result failed → revisemethod criteria cases checks results schema evidence policy decision signed result failed → revise

METHODOLOGY

Intent Eval Lab

Define the question, test method, failure cases and interpretation limits.

SHARED CONTRACTS

Intent Eval Core

Versioned types, schemas, validators and state definitions make results comparable.

SCOPE THE CLAIM

Task cases & criteria

Choose representative work, expected behavior and cases that must fail.

DETERMINISTIC CHECKS

Audit Harness

Check declared source and test-policy requirements with reproducible gates.

BEHAVIORAL EVALUATION

J-Rig

Exercise an artifact against explicit criteria; preserve observations and judgments.

SHARED RESULT ENVELOPE

Evidence Bundle

Carry the result, scope and supporting evidence in the platform’s versioned contract.

  • Inspectable evidence · declared limits

DECLARED DECISION RULE

Rollout policy

State which results permit this release and which conditions must block it.

Published results are scoped observations, not blanket certification.

Labs investigates and explains. Evals defines result meanings. Harnesses perform checks. A rollout policy decides whether a specific result permits a release.
Work / artifact flowFeedbackControl / shared supportProposed boundarySelect a node to trace its connections.

Six repositories, one evidence language

Lab, Core, Audit Harness, J-Rig, Rollout Gate and Dashboard retain separate responsibilities and releases. Their shared contracts connect the work without pretending they are one monolithic service.

  • Methodology and contracts define what the result means.
  • Executable checks generate observations for the declared task.
  • A rollout rule turns evidence into a bounded release decision.

A failure is useful information

A failed evaluation can block a release and identify what must change. A missing or stale result establishes uncertainty; it must not silently become a pass or an invented failure.

  • The task, version and test conditions belong with the result.
  • A passing example does not establish performance for every workflow.
  • The same discipline applies when the model or provider changes.

What can be inspected today

The public Lab includes scoped skill evaluation examples and methodology. Its published boundary is explicit: skill checks run locally; agent loops and workflows need a custom test setup.

  • Demos show behavior and link evidence where available.
  • Evals preserves definitions for interpreting records.
  • The diagram explains the architecture; current integration coverage belongs to the source repositories.
Read the relationships as text

The six-repository Intent Eval Platform connects Lab methodology and Core contracts to task cases, deterministic checks and behavioral evaluation. Checks emit an Evidence Bundle; the rollout gate combines it with a policy and emits a decision record for the dashboard. Failure returns to the test or implementation. This shows the architecture, not universal deployment coverage.

  • Intent Eval Lab → Task cases & criteria: method (flow)
  • Intent Eval Core → Task cases & criteria (governance)
  • Task cases & criteria → Audit Harness: criteria (flow)
  • Task cases & criteria → J-Rig: cases (flow)
  • Audit Harness → Evidence Bundle: checks (flow)
  • J-Rig → Evidence Bundle: results (flow)
  • Intent Eval Core → Evidence Bundle: schema (governance)
  • Evidence Bundle → Intent Rollout Gate: evidence (flow)
  • Rollout policy → Intent Rollout Gate: policy (governance)
  • Intent Rollout Gate → Signed decision record: decision (flow)
  • Signed decision record → Intent Eval Dashboard: signed result (flow)
  • Intent Rollout Gate → Task cases & criteria: failed → revise (feedback)

Basis and scope

Reviewed 2026-10-11. Public links support the named properties and capabilities. The relationships explain the operating method and owner-directed vision; they do not imply that every initiative is deployed or commercially proven.

Mermaid sourceComplete source graphWorkshop PDFPrint view