feat(test-preflight): add verify and prepare skill

2026-10-15 18:04:59 +08:00
parent ffa7cbd63e
commit c2eb207bd2
9 changed files with 1014 additions and 0 deletions
@@ -0,0 +1,85 @@
+# test-preflight matrix schema
+
+The input is YAML. `mode` is supplied by the CLI, so a file may be reused for
+`verify` and `prepare`.
+
+```yaml
+schema_version: 1
+project_root: /absolute/project/root
+host: https://test.example.invalid
+public_host: https://public-test.example.invalid # optional
+service_probes:
+  - id: api-health
+    url: "{HOST}/health"
+    expected_status: [200]
+fixtures:
+  required: true
+  references: [isolated-test-account]
+cleanup:
+  required: true
+  steps: [delete-run-data]
+matrix:
+  - platform: android
+    tester:
+      binding: tester.android
+      node: local
+      runner: {kind: local}
+    artifact:
+      path: /absolute/dist/writes.apk
+      sha256: 012345...
+      package: page.writes
+      source_commit: 40-character-commit-sha
+      identity_command: [apkanalyzer, manifest, application-id, "{artifact}"]
+```
+
+Required matrix fields are `platform` (`android` or `ios`), `tester.binding`,
+`tester.node`, and `artifact.path`. Artifact identity is required as
+`package` for Android or `bundle_id` for iOS. `sha256` and `source_commit` are
+optional but recommended; when present they are checked exactly.
+
+`service_probes` must contain at least one URL. `{HOST}` and `{PUBLIC_HOST}`
+are expanded from the top-level values. Each probe is checked from the actual
+tester node as DNS, TCP, and HTTP. A status outside 2xx–4xx fails transport
+readiness; `expected_status` can require an exact allow-list. HTTP 401/403 can
+therefore be a valid reachability result when explicitly expected.
+
+`fixtures` and `cleanup` are declarations, not commands. A required fixture
+needs non-empty `references`; a required cleanup needs non-empty `steps` or
+`run`. For a read-only run, use `required: false` with a non-empty `reason`.
+The skill never executes these values.
+
+## Tester-node contract
+
+`tester.runner.kind: local` executes platform and network probes on the current
+node. It is only valid for `node: local`/`localhost`. A non-local node must use
+`kind: contract` and a non-empty argv `command`. The command receives one JSON
+request on stdin and returns one JSON object on stdout; no shell string is
+accepted.
+
+Requests have `operation` (`platform_probe`, `network_probe`, `install`, or
+`start`), `platform`, `node`, and operation-specific fields. The response is:
+
+```json
+{
+  "status": "ready",
+  "observed": "short evidence",
+  "checks": {
+    "macos": {"status": "ready", "observed": "Darwin 24", "expected": "Darwin"}
+  }
+}
+```
+
+`checks` is optional; each key becomes a separate evidence row. A non-zero,
+invalid, or unavailable runner is `environment-fail`. The adapter is expected
+to execute on the resolved tester node, not on the coordinator. See
+[boundaries.md](boundaries.md).
+
+## Evidence
+
+YAML contains `mode`, overall `status`, timestamps, a summary, and `checks`.
+CSV uses these stable columns:
+
+`check_id,mode,platform,node,target,observed,expected,status,remediation,timestamp`
+
+Every check must use one of `ready`, `blocked`, or `environment-fail`; the
+report does not invent `fail` or `not-run` testcase results.