96 lines
3.4 KiB
Markdown
96 lines
3.4 KiB
Markdown
# test-preflight matrix schema
|
||
|
||
The input is YAML. `mode` is supplied by the CLI, so a file may be reused for
|
||
`verify` and `prepare`.
|
||
|
||
```yaml
|
||
schema_version: 1
|
||
project_root: /absolute/project/root
|
||
host: https://test.example.invalid
|
||
public_host: https://public-test.example.invalid # optional
|
||
service_probes:
|
||
- id: api-health
|
||
url: "{HOST}/health"
|
||
expected_status: [200]
|
||
fixtures:
|
||
required: true
|
||
references: [isolated-test-account]
|
||
cleanup:
|
||
required: true
|
||
steps: [delete-run-data]
|
||
# Optional project metadata. It is not a preflight gate.
|
||
capabilities:
|
||
assumed: [album_picker, photo_permission, ime_pinyin]
|
||
matrix:
|
||
- platform: android
|
||
tester:
|
||
binding: tester.android
|
||
node: local
|
||
runner: {kind: local}
|
||
artifact:
|
||
path: /absolute/dist/writes.apk
|
||
sha256: 012345...
|
||
package: page.writes
|
||
source_commit: 40-character-commit-sha
|
||
identity_command: [apkanalyzer, manifest, application-id, "{artifact}"]
|
||
```
|
||
|
||
Required matrix fields are `platform` (`android` or `ios`), `tester.binding`,
|
||
`tester.node`, and `artifact.path`. Artifact identity is required as
|
||
`package` for Android or `bundle_id` for iOS. `sha256` and `source_commit` are
|
||
optional but recommended; when present they are checked exactly.
|
||
|
||
`service_probes` must contain at least one URL. `{HOST}` and `{PUBLIC_HOST}`
|
||
are expanded from the top-level values. Each probe is checked from the actual
|
||
tester node as DNS, TCP, and HTTP. A status outside 2xx–4xx fails transport
|
||
readiness; `expected_status` can require an exact allow-list. HTTP 401/403 can
|
||
therefore be a valid reachability result when explicitly expected.
|
||
|
||
`fixtures` and `cleanup` are declarations, not commands. A required fixture
|
||
needs non-empty `references`; a required cleanup needs non-empty `steps` or
|
||
`run`. For a read-only run, use `required: false` with a non-empty `reason`.
|
||
The skill never executes these values.
|
||
|
||
## Client capabilities
|
||
|
||
The project assumes the mobile test nodes provide the album picker, photo permission,
|
||
test image, and Chinese IME. These are not preflight declarations or gates. Journey
|
||
cases exercise the real UI; failures are recorded as product failures. `AUTHORIZE_ALBUM`
|
||
and `IME_PINYIN` are not read.
|
||
|
||
## Tester-node contract
|
||
|
||
`tester.runner.kind: local` executes platform and network probes on the current
|
||
node. It is only valid for `node: local`/`localhost`. A non-local node must use
|
||
`kind: contract` and a non-empty argv `command`. The command receives one JSON
|
||
request on stdin and returns one JSON object on stdout; no shell string is
|
||
accepted.
|
||
|
||
Requests have `operation` (`platform_probe`, `network_probe`, `install`, or
|
||
`start`), `platform`, `node`, and operation-specific fields. The response is:
|
||
|
||
```json
|
||
{
|
||
"status": "ready",
|
||
"observed": "short evidence",
|
||
"checks": {
|
||
"macos": {"status": "ready", "observed": "Darwin 24", "expected": "Darwin"}
|
||
}
|
||
}
|
||
```
|
||
|
||
`checks` is optional; each key becomes a separate evidence row. A non-zero,
|
||
invalid, or unavailable runner is `environment-fail`. The adapter is expected
|
||
to execute on the resolved tester node, not on the coordinator. See
|
||
[boundaries.md](boundaries.md).
|
||
|
||
## Evidence
|
||
|
||
YAML contains `mode`, overall `status`, timestamps, a summary, and `checks`.
|
||
CSV uses these stable columns:
|
||
|
||
`check_id,mode,platform,node,target,observed,expected,status,remediation,timestamp`
|
||
|
||
Every check must use one of `ready`, `blocked`, or `environment-fail`; the
|
||
report does not invent `fail` or `not-run` testcase results.
|