Exact Version 1.0.0 walkthrough

Inspect the complete deterministic decision before buying.

The following contract, evidence, commands, seal, and evaluation result come from the reproducible agent-eval-kit-1.0.0.pyz release. The values are published so a buyer can review the product boundary without a sales call.

1. Versioned task contract

The contract pins the task, repository commit, exact test command, required checks, and fixture outcomes.

{
  "fixture_expectations": [
    {
      "expected_exit_code": 0,
      "name": "valid-submission",
      "path": "fixtures/valid.json"
    },
    {
      "expected_exit_code": 1,
      "name": "price-tampering",
      "path": "fixtures/price-tampering.json"
    }
  ],
  "immutable_invariants": [
    "Preserve every ID",
    "Do not change prices"
  ],
  "repository_commit": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
  "repository_url": "https://github.com/example/catalog",
  "required_checks": [
    "schema_exact",
    "business_facts_preserved"
  ],
  "schema_version": "1.0",
  "task_id": "catalog-normalization",
  "test_command": [
    "python3",
    "-m",
    "unittest",
    "discover"
  ]
}

2. Validate and seal

python3 agent-eval-kit-1.0.0.pyz validate contract.json

{
  "contract_sha256": "fac5f172f0f48e750505e208354a8a2d6f7d3c12882f40519189852e92050aca",
  "schema_version": "1.0",
  "task_id": "catalog-normalization",
  "valid": true
}

3. Recorded evidence

The caller runs its own tests and records only the pinned commit, command, exit code, and declared checks.

{
  "checks": {
    "business_facts_preserved": true,
    "schema_exact": true
  },
  "executed_command": [
    "python3",
    "-m",
    "unittest",
    "discover"
  ],
  "exit_code": 0,
  "repository_commit": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
  "schema_version": "1.0",
  "task_id": "catalog-normalization"
}

4. Evaluate offline

python3 agent-eval-kit-1.0.0.pyz evaluate contract.json passing-evidence.json

{
  "checks": [
    {
      "detail": "Evidence uses only the declared 1.0 fields.",
      "name": "evidence_schema_exact",
      "passed": true
    },
    {
      "detail": "Evidence schema version matches the product contract.",
      "name": "schema_version_exact",
      "passed": true
    },
    {
      "detail": "Evidence belongs to the sealed task.",
      "name": "task_id_exact",
      "passed": true
    },
    {
      "detail": "Evidence uses the pinned repository commit.",
      "name": "repository_commit_pinned",
      "passed": true
    },
    {
      "detail": "The recorded command exactly matches the contract.",
      "name": "command_exact",
      "passed": true
    },
    {
      "detail": "The recorded command completed successfully.",
      "name": "exit_code_zero",
      "passed": true
    },
    {
      "detail": "No required check is missing and no self-awarded check is present.",
      "name": "check_schema_exact",
      "passed": true
    },
    {
      "detail": "Every declared required check passed.",
      "name": "required_checks_passed",
      "passed": true
    }
  ],
  "contract_sha256": "fac5f172f0f48e750505e208354a8a2d6f7d3c12882f40519189852e92050aca",
  "passed": true,
  "reward": 1.0,
  "schema_version": "1.0",
  "task_id": "catalog-normalization"
}

5. Reject a changed commit

In a copy named tampered-evidence.json, only the recorded repository commit is changed:

{
  "repository_commit": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
}

python3 agent-eval-kit-1.0.0.pyz evaluate contract.json tampered-evidence.json

Process exit code: 1

{
  "checks": [
    {
      "detail": "Evidence uses only the declared 1.0 fields.",
      "name": "evidence_schema_exact",
      "passed": true
    },
    {
      "detail": "Evidence schema version matches the product contract.",
      "name": "schema_version_exact",
      "passed": true
    },
    {
      "detail": "Evidence belongs to the sealed task.",
      "name": "task_id_exact",
      "passed": true
    },
    {
      "detail": "Evidence uses the pinned repository commit.",
      "name": "repository_commit_pinned",
      "passed": false
    },
    {
      "detail": "The recorded command exactly matches the contract.",
      "name": "command_exact",
      "passed": true
    },
    {
      "detail": "The recorded command completed successfully.",
      "name": "exit_code_zero",
      "passed": true
    },
    {
      "detail": "No required check is missing and no self-awarded check is present.",
      "name": "check_schema_exact",
      "passed": true
    },
    {
      "detail": "Every declared required check passed.",
      "name": "required_checks_passed",
      "passed": true
    }
  ],
  "contract_sha256": "fac5f172f0f48e750505e208354a8a2d6f7d3c12882f40519189852e92050aca",
  "passed": false,
  "reward": 0.0,
  "schema_version": "1.0",
  "task_id": "catalog-normalization"
}