diff --git a/skills/ack/references/delivery.md b/skills/ack/references/delivery.md index 5dc6590..a2729f4 100644 --- a/skills/ack/references/delivery.md +++ b/skills/ack/references/delivery.md @@ -114,12 +114,15 @@ intents: PR/MR。普通任务使用项目已确认的 Forge 流程；只有本次是版本发布生命周期且用户 明确要求时才调用独立的 `releaser`。没有对应能力或认证时标记 blocked，不用 带 token 的临时 curl 兜底。remote 与 base branch 必须来自该步骤，不能临时猜测。 -- `build`：调用 artifact 的 build entrypoint。DEB 必须记录包名、版本、架构和 - SHA-256；OCI image 必须记录完整引用、platform 和 digest。产物必须绑定当前 source - revision，不能在目标机器重新拉源码构建。 +- `build`：对当前 source revision 调用已安装的 `builder` `obtain.sh --sha `， + 不要假定本机 `make`。`BUILD_VENUE=ci` 时监听该 SHA 的流水线；没有流水线或 job 失败 + 则本步 `failed`。DEB 必须记录包名、版本、架构和 SHA-256；OCI image 必须记录完整 + 引用、platform 和 digest。产物必须绑定当前 source revision，不能在目标机器重新拉 + 源码构建。 - `publish`：验证 artifact/destination 类型兼容，上传精确产物。DEB 与 Docker 均可使用 已安装的 `builder`；Docker 轨道保持显式触发——只有用户明确点名（builder / 发布镜像） - 时才加载，否则必须走契约中已审查的 upload entrypoint。项目入口只接受 + 时才加载，否则必须走契约中已审查的 upload entrypoint。`BUILD_VENUE=ci` 且 obtain + 已带回 registry digest 时，本步只复核，不再本机 push。项目入口只接受 刚校验的精确 artifact。preview/staging 使用不可覆盖的 commit/PR 标识，不隐式使用 `latest`。既没有可用 skill 也没有 upload 入口时标记 blocked。 - `deploy`：把同一不可变 artifact 交给 environment 的 deploy entrypoint；获取目标 diff --git a/skills/bugfix-ship/SKILL.md b/skills/bugfix-ship/SKILL.md index 2f7aeb0..72b8acd 100644 --- a/skills/bugfix-ship/SKILL.md +++ b/skills/bugfix-ship/SKILL.md @@ -29,6 +29,9 @@ description: >- 4. 读取并执行共享 `delivery-loop` 的状态机。停止点由用户选择或项目 delivery profile 决定，使用 ACK canonical 状态：`verified`、`validation_ready`、`review_ready` 或 `released`。 5. 只有达到配置的停止点并拿到证据后才结束；提交、推送、合并、发布和部署不能静默跳过。 + 需要产物时只调用 builder `obtain.sh --sha `：CI 项目等该 + SHA 的流水线，失败（无流水线、job 失败、digest 不对）则停止，不要本机 `docker push`。 + 合进默认分支若项目不构建，把预览推到 `dev/*` 或等 releaser 打 tag。 ## 安全门 diff --git a/skills/builder/README.md b/skills/builder/README.md index d17d3ee..8583e93 100644 --- a/skills/builder/README.md +++ b/skills/builder/README.md @@ -15,6 +15,7 @@ Android、远程 Mac iOS 或未来其它平台都可以各自绑定 environment/ - "把 1.2.3 发布到包仓库 / 镜像仓库 / File Browser" - "检查这个项目的 makefile.builder 是否符合 builder 契约" - "看看项目现在的发布流程" +- "看这个 SHA 的 CI 构建 / obtain" 只构建不上传时明确说明即可；上传永远需要你显式授权。 @@ -65,6 +66,8 @@ COPY_AS=latest.apk # COPY_SCRIPT= # 仅外部/临时脚本覆盖；项目内脚本写 makefile.builder ``` +`BUILD_VENUE=ci` 写在 `makefile.builder`：产物由 CI 构建。`make -f makefile.builder obtain`（或 `push-docker`）等待该 commit 的流水线并 inspect registry，不在本机 `docker push`。 + 4. 日常发布就是两条命令： `make -f makefile.builder deb && make -f makefile.builder push-deb`、 `make -f makefile.builder push-docker`。 @@ -219,6 +222,8 @@ Docker 只有你点名镜像时才走。token 不会写进输出、提交或回 | `scripts/publish_docker.sh` | buildx 构建 + 推送镜像，远端 digest 验证 | | `scripts/verify_deb.sh` | 核对包元数据、内容与 SHA-256 | | `scripts/copy_artifact.sh` | 按 `COPY_SCRIPT` 分发产物，未配置时拷到 `COPY_DEST` | +| `scripts/obtain.sh` | 对精确 SHA 拿到产物；`BUILD_VENUE=ci` 时等流水线再 inspect | +| `scripts/wait_ci.sh` | 只等该 SHA 的 GitLab/GitHub 流水线，不触发 | | `templates/makefile.android.builder` | Android APK + copy 轨道骨架 | | `templates/makefile.ios.builder` | iOS IPA + copy 轨道骨架 | diff --git a/skills/builder/SKILL.md b/skills/builder/SKILL.md index bf8b893..5ac442b 100644 --- a/skills/builder/SKILL.md +++ b/skills/builder/SKILL.md @@ -2,8 +2,9 @@ name: builder description: >- 初始化或检查 makefile.builder 契约，再构建或发布 deb/镜像/Android APK/iOS IPA， - 或把产物拷到目录/容器。触发词：初始化 builder、检查 makefile.builder、构建/发布 - deb、推送 apt、构建/发布 Docker 镜像、打 APK/IPA、拷到 File Browser、make push。 + 或把产物拷到目录/容器。产物可在本机或 CI 上构建；拿到产物用 obtain。触发词：初始化 + builder、检查 makefile.builder、构建/发布 deb、推送 apt、构建/发布 Docker 镜像、 + 打 APK/IPA、拷到 File Browser、make push、看构建、监听 CI、obtain。 未授权不上传。Docker 仅用户点名镜像时才走。 --- @@ -34,6 +35,7 @@ worktree，也可以指向已配对的远程 Mac/其它构建机；不要把远 - 初始化、接入 builder，或还没有 `makefile.builder`：执行「初始化」。 - 检查契约或发布配置：执行「检查」。 - 构建、发布、上传：执行「工作流」。不要静默初始化。 +- 看构建、监听 CI、ACK/bugfix-ship 要产物：执行「拿到产物」。不要当成第五条轨道。 - 指定 `builder.`、远程构建机或跨 session：先读共享 `agent-runtime`。 - 不适用：本地安装/卸载 DEB；RPM/语言包；从零设计打包体系（先出方案）；普通编码与 Dockerfile 编辑。 - 改本 skill 自身：契约先改 `scripts/check.py`，再同步 [contract.md](references/contract.md) 与 templates；不在生产上传上试脚本。 @@ -79,5 +81,15 @@ COPY_AS=latest.apk 2. 仅当用户明确要求发布、上传或提交时才上传。不猜测生产端点；不把 token 写入输出、文件、提交或回复；不用 `set -x` 跑含凭据的命令。可能覆盖同版本时先问。脏工作树默认拒绝发布；用户明确接受时设 `ALLOW_UNCOMMITTED=1` 并注明未提交修改。 3. 构建：用户指定了架构才传 `ARCH=`；否则不要传，让 makefile 按 `ARCHES` 展开（`make -s -f makefile.builder arches`）。`make -f makefile.builder build` 始终单架构；DEB 用 `make -f makefile.builder deb`。版本按契约从 Git 祖先稳定 tag 推导，不要调用 releaser，不要用全仓库最新 tag。仅获构建授权时不要走会自动上传的目标。脚本含 `rm -rf` 时先确认为受限构建目录。若通过 `builder.` 执行，先精确解析 `environment + host + project + path` 对应的 worktree；远程 binding 不能 fallback 到本机。持久 session 只显式 attach/resume，不能按标题复用旧 terminal。 4. DEB 上传前：对每个包 `verify_deb.sh `。多个包时不凭文件时间猜测。镜像轨道由 `publish_docker.sh` 自带远端 inspect。 -5. 发布优先 `make -f makefile.builder push[-deb|-docker|-copy]`，或直接调 `upload_deb.sh` / `publish_docker.sh` / `copy_artifact.sh`。未指定 `ARCH=` 时 `push-deb` 上传清单里每一个 `.deb`。拷贝轨道至少配置 `COPY_DEST`（本地目录或 `container:/dir`）或 `COPY_SCRIPT`（项目内脚本优先在 `makefile.builder` 设置；同时配置时命令行/环境值优先）。自定义脚本按 `FILE [FILE ...]` 接收产物路径，自行完成重命名、上传、远端校验和通知；相对路径按项目根解析，`BUILDER_PROJECT_ROOT` 提供项目根。`COPY_AS` 只由内置 `COPY_DEST` 路径处理。脚本缺环境变量时加载 `.env.builder`（shell 显式值优先），不读 `.env`。不要把 token 当命令行参数；脚本可随项目放在 `.pouch/builder/` 下。多平台镜像只能走 `publish_docker.sh`，不能拆进 make；不要把项目架构清单写进 `.env.builder`。`build` 里禁止 `docker cp`。 +5. 发布优先 `make -f makefile.builder push[-deb|-docker|-copy]`，或直接调 `upload_deb.sh` / `publish_docker.sh` / `copy_artifact.sh` / `obtain.sh`。`BUILD_VENUE=ci` 时 `push-docker` 就是 obtain。未指定 `ARCH=` 时 `push-deb` 上传清单里每一个 `.deb`。拷贝轨道至少配置 `COPY_DEST`（本地目录或 `container:/dir`）或 `COPY_SCRIPT`（项目内脚本优先在 `makefile.builder` 设置；同时配置时命令行/环境值优先）。自定义脚本按 `FILE [FILE ...]` 接收产物路径，自行完成重命名、上传、远端校验和通知；相对路径按项目根解析，`BUILDER_PROJECT_ROOT` 提供项目根。`COPY_AS` 只由内置 `COPY_DEST` 路径处理。脚本缺环境变量时加载 `.env.builder`（shell 显式值优先），不读 `.env`。不要把 token 当命令行参数；脚本可随项目放在 `.pouch/builder/` 下。多平台镜像只能走 `publish_docker.sh`，不能拆进 make；不要把项目架构清单写进 `.env.builder`。`build` 里禁止 `docker cp`。 6. 发布成功不能只看「curl/push 已执行」。要有退出码、HTTP 成功或远端 digest；索引异步时报告「上传已接受，索引尚待更新」。最终回复给出包名/镜像引用、版本、架构、SHA-256 或 digest、源 commit、工作区状态和未完成项。 + +## 拿到产物 + +调用方是 bugfix-ship、ACK（delivery-loop 的 `artifact_built`），或用户要看 CI 构建。 +按 [obtain.md](references/obtain.md) 执行 `scripts/obtain.sh --sha `。 + +- `BUILD_VENUE=local`：当前 HEAD 必须是该 SHA，跑 makefile 的 docker/deb。 +- `BUILD_VENUE=ci`：等该 SHA 的 GitLab/GitHub 流水线，再 inspect `version.sh --docker` 的镜像 tag。不触发流水线，不本机 push。 +- 失败（无流水线、job 失败、digest 不对）必须原样返回给调用方，不得标 `artifact_built`。 +- 合进默认分支若项目不跑 CI，这是正确失败；预览用 `dev/*`，正式用 tag。 diff --git a/skills/builder/references/contract.md b/skills/builder/references/contract.md index ecd8f5f..7f590ad 100644 --- a/skills/builder/references/contract.md +++ b/skills/builder/references/contract.md @@ -52,6 +52,8 @@ make -f makefile.builder | `push-docker` | 同时有多种发布轨道的项目 | 仅调 builder 的 `publish_docker.sh` | | `push-copy` | 同时有多种发布轨道的项目 | 仅调 builder 的 `copy_artifact.sh`；由它按配置选择内置拷贝或自定义脚本 | | `push` | 单一产物类型时必备；多产物项目为聚合 | 依序调用对应 push-* 或直接调脚本；是发布的唯一 make 入口 | +| `obtain` | 有产物的项目 | 调用 `obtain.sh`，对精确 commit SHA 拿到已存在的产物。`BUILD_VENUE=ci` 时等待该 SHA 的 forge 流水线再 inspect，不在本机 `docker push` | +| `venue` | 可选（新模板必备） | 打印一行 `BUILD_VENUE`（`local` 或 `ci`） | 规则： @@ -80,6 +82,7 @@ make -f makefile.builder | 变量 | 默认 | 说明 | |------|------|------| +| `BUILD_VENUE` | `local` | 产物在哪构建：`local` 本机 make；`ci` 由 GitLab/GitHub 流水线构建。这不是新轨道。写在 `makefile.builder` 并提交。`ci` 时 `push-docker` 委托 `obtain.sh`（等流水线 + inspect），不跑 `publish_docker.sh` | | `ARCHES` | `amd64` | 本项目要发布的架构清单，空格分隔，子集只能是 `amd64` `arm64`。写在 `makefile.builder` 里并提交；不要放进 `.env.builder`。缺省或无 `arches` 目标时视为单架构 `amd64`。不要默认写成双架构 | | `ARCH` | `$(firstword $(ARCHES))` | 这一次构建的架构。必须落在 `ARCHES` 里；非法值必须 `$(error)` 并提示合法值。命令行/环境传入时只构建/上传这一项 | | `VERSION` | `` （空） | 为空时按「版本号」节从 Git 祖先稳定 tag 推导；禁止 `sort -V` 取全局最新 | @@ -186,6 +189,8 @@ include $(HOME)/.pouch/skills/builder/scripts/version.mk `PLATFORMS` 优先级（高 → 低）：用户这轮 `--platform` 或已 export 的 `PLATFORMS` → 命令行 `ARCH=` 映射的 `linux/$(ARCH)`（由 `push-docker` 传入）→ `.env.builder` 的 `PLATFORMS` → `makefile.builder` 的 `ARCHES` → `linux/amd64`。 +`BUILD_VENUE=ci` 时 Docker 发布路径是 `obtain.sh`：只认 **该 commit SHA** 的流水线。没有流水线、job 失败、超时、或 Harbor inspect 不到 `version.sh --docker` 算出的 tag，一律非零退出。不触发流水线。合进默认分支若项目声明不构建，这是正确失败；调用方应打 tag 或推送到项目会构建的 ref（例如 `dev/*`）。CI job 若已经 `docker push`，obtain 成功即视为产物已在 registry，不要再本机 push 一次。 + 配置来源优先级：shell 已显式设置的值 > 项目根 `.env.builder` > 失败并询问用户。 `.env.builder` 由 builder 脚本加载（不回显任何值）；当前 shell 已设置的值优先。 不要读取或改写用户 `.env`。空值视为未配置。不要提交 `.env.builder`。 @@ -201,6 +206,7 @@ push 目标与 `version.sh` 定位 builder 脚本时按以下顺序，命中即 1. `$BUILDER_SKILL_DIR/scripts/`（特殊安装位置） 2. `$HOME/.pouch/skills/builder/scripts/`（标准 clone 位） +3. `$HOME/.agents/skills/builder/scripts/`（本机 agent 安装位） 两个位置都不可用时必须失败并提示：设置 `BUILDER_SKILL_DIR`，或把 pouch 仓库 clone 到 `~/.pouch`。 diff --git a/skills/builder/references/obtain.md b/skills/builder/references/obtain.md new file mode 100644 index 0000000..ce061a8 --- /dev/null +++ b/skills/builder/references/obtain.md @@ -0,0 +1,54 @@ +# 拿到产物（obtain） + +Builder 的交付入口是 **按精确 Git commit 拿到产物**，不是「在哪台机器编译」。 +docker / deb / android / ios / copy 仍是轨道；`BUILD_VENUE` 只说明构建发生在本机还是 CI。 + +## 谁来调用 + +- `bugfix-ship` 和 ACK 走到 delivery-loop 的 `artifact_built` 时。 +- 用户说「看构建」「监听 CI」「拿到镜像」时。 +- `make -f makefile.builder obtain` 或 `push-docker`（`BUILD_VENUE=ci`）。 + +代码必须已经推到远端（`ci` 时流水线由 git ref 触发）。Builder 不代替 releaser 打 tag，也不创建分支。 + +## 命令 + +```bash +/scripts/obtain.sh --sha [--branch ] [--root ] [--timeout 1200] +``` + +成功打印： + +```text +ARTIFACT=harbor.example.com/ns/name:1.4.2 +DIGEST=sha256:… +SHA= +VENUE=ci +TAG=1.4.2 +``` + +镜像 tag 对该 SHA 跑 `version.sh --docker`（可 `--branch` 注入 `CI_COMMIT_BRANCH`），禁止 `git describe`。 + +## 退出码 + +| 码 | 含义 | 调用方 | +|---|---|---| +| 0 | 产物已核验 | `artifact_built` 完成 | +| 1 | 该 SHA 无流水线、job 失败、超时、inspect 失败 | 标 `failed`，不得 publish/deploy | +| 2 | 用法或配置错误 | 标 `blocked` | +| 3 | glab/gh 缺失或读不到该项目 | 停；可用**已打开且已登录**的同 host Orca tab 查流水线。禁止新开未登录页 | + +## `ci` 规则 + +- 只认该 SHA 的 pipeline，不认默认分支「最新一次绿」。 +- 不触发流水线。 +- job 变绿不够，docker 轨道必须 inspect 到 digest。 +- 合进 `main` 若项目不构建，`no_pipeline` 是正确失败。预览走项目会构建的 ref（如 `dev/*`），正式走 tag。 + +## Agent + +1. 固化 source revision 为完整 SHA。 +2. 跑 `obtain.sh --sha `。 +3. 退出 0：记录 ARTIFACT/DIGEST/SHA。`venue=ci` 且 job 已 push 时，`published` 视为已满足，不要本机再 `publish_docker.sh`。 +4. 退出 1/2：停止 delivery-loop，报告 pipeline URL 或 `no_pipeline`。 +5. 退出 3：先说明 CLI 身份不对；只有已有同 host 登录 tab 时才用来读 pipeline。 diff --git a/skills/builder/scripts/check.py b/skills/builder/scripts/check.py index 734754b..6a96eb5 100755 --- a/skills/builder/scripts/check.py +++ b/skills/builder/scripts/check.py @@ -41,7 +41,13 @@ SECRET_PATTERNS = ( ) FLOATING_TAGS = (":latest", ":stable") DEB_SHAPE = re.compile(r"^[^_\s]+_[^_\s]+_[^_\s]+\.deb$") -VALID_SCRIPT_NAMES = ("upload_deb.sh", "publish_docker.sh", "copy_artifact.sh") +VALID_SCRIPT_NAMES = ( + "upload_deb.sh", + "publish_docker.sh", + "copy_artifact.sh", + "obtain.sh", + "wait_ci.sh", +) BUILDER_MAKEFILE = "makefile.builder" BUILDER_ENV = ".env.builder" @@ -354,6 +360,9 @@ SCRIPT_RESOLVE_SNIPPETS = tuple( "$$HOME/.skills/skills/builder/scripts", "$HOME/.skills/skills/builder/scripts", "~/.skills/skills/builder/scripts", + "$$HOME/.agents/skills/builder/scripts", + "$HOME/.agents/skills/builder/scripts", + "~/.agents/skills/builder/scripts", ) for name in VALID_SCRIPT_NAMES ) @@ -379,7 +388,12 @@ def check_push_delegates( missing = [] inline = [] thin = [] - builder_names = ("upload_deb.sh", "publish_docker.sh", "copy_artifact.sh") + builder_names = ( + "upload_deb.sh", + "publish_docker.sh", + "copy_artifact.sh", + "obtain.sh", + ) for target in targets: dry = run_make(project, target) if has_no_rule(dry): @@ -576,12 +590,68 @@ def check_copy_script( ) +def read_venue(project: Path) -> str: + result = run_make(project, "venue") + if not has_no_rule(result): + echoed = parse_dry_echo(result) + if echoed in ("local", "ci"): + return echoed + cleaned = "\n".join(clean_make_output(result)).strip() + if cleaned in ("local", "ci"): + return cleaned + raw = makefile_values(project).get("BUILD_VENUE", "local") + raw = re.split(r"\s+#", raw, maxsplit=1)[0].strip() + return raw or "local" + + +def origin_forge(project: Path) -> str: + result = subprocess.run( + ["git", "-C", str(project), "remote", "get-url", "origin"], + capture_output=True, + text=True, + check=False, + ) + url = (result.stdout or "").strip() + if "github.com" in url: + return "github" + if url: + return "gitlab" + return "" + + def check_ready_toolchain( - report: Report, *, deb_project: bool, docker_project: bool, copy_project: bool = False + report: Report, + project: Path | None = None, + *, + deb_project: bool, + docker_project: bool, + copy_project: bool = False, ) -> None: - lines = [f"make: {'found' if shutil.which('make') else 'MISSING'}"] + venue = read_venue(project) if project is not None else "local" + lines = [ + f"make: {'found' if shutil.which('make') else 'MISSING'}", + f"BUILD_VENUE: {venue}", + ] problems = [] - if docker_project: + if venue not in ("local", "ci"): + problems.append("BUILD_VENUE must be local or ci") + if docker_project and venue == "ci": + docker_ok = shutil.which("docker") is not None + lines.append( + f"docker: {'found' if docker_ok else 'optional for inspect; obtain fails without it'}" + ) + forge = origin_forge(project) if project is not None else "" + if forge == "github": + gh_ok = shutil.which("gh") is not None + lines.append(f"gh: {'found' if gh_ok else 'MISSING (blocks ci venue)'}") + if not gh_ok: + problems.append("install gh and authenticate for this GitHub repo") + else: + glab_ok = shutil.which("glab") is not None + lines.append(f"glab: {'found' if glab_ok else 'MISSING (blocks ci venue)'}") + if not glab_ok: + problems.append("install glab and authenticate for this GitLab project") + elif docker_project: docker_ok = shutil.which("docker") is not None lines.append(f"docker: {'found' if docker_ok else 'MISSING (blocks docker track)'}") if not docker_ok: @@ -682,6 +752,7 @@ def check_script_paths(report: Report) -> None: home = Path(os.environ.get("HOME", "")) candidates.append(home / ".pouch" / "skills" / "builder" / "scripts") candidates.append(home / ".skills" / "skills" / "builder" / "scripts") + candidates.append(home / ".agents" / "skills" / "builder" / "scripts") found = next((c for c in candidates if c.is_dir() and any((c / n).is_file() for n in VALID_SCRIPT_NAMES)), None) if found: report.add(PASS, 9, "builder 脚本路径可达", str(found)) @@ -761,7 +832,12 @@ def main(argv: list[str] | None = None) -> int: report.add(SKIP, number, title, detail) docker_project = detect_docker_project(project) deb_project = any(project.glob("debian/*")) - check_ready_toolchain(report, deb_project=deb_project, docker_project=docker_project) + check_ready_toolchain( + report, + project, + deb_project=deb_project, + docker_project=docker_project, + ) check_ready_env_keys( report, project, deb_project=deb_project, docker_project=docker_project ) @@ -786,6 +862,8 @@ def main(argv: list[str] | None = None) -> int: "push-deb", "push-docker", "push-copy", + "obtain", + "venue", ): result = run_make(project, target) if result.returncode == 0: @@ -847,6 +925,7 @@ def main(argv: list[str] | None = None) -> int: if args.ready: check_ready_toolchain( report, + project, deb_project=deb_project, docker_project=docker_project, copy_project=copy_project, diff --git a/skills/builder/scripts/obtain.sh b/skills/builder/scripts/obtain.sh new file mode 100755 index 0000000..167b75b --- /dev/null +++ b/skills/builder/scripts/obtain.sh @@ -0,0 +1,197 @@ +#!/usr/bin/env bash +set -euo pipefail + +# Obtain a verified artifact for an exact git commit. +# Build may have happened locally or in CI; this script only reports identity +# after the artifact exists. Does not print tokens. +# +# Usage: +# obtain.sh [--sha SHA] [--branch NAME] [--root DIR] [--timeout SEC] +# +# BUILD_VENUE (makefile.builder, default local): +# local HEAD must be SHA; run `make -f makefile.builder docker` or `deb` +# ci wait for the forge pipeline of this SHA, then inspect the image +# +# Exit: +# 0 artifact verified; prints ARTIFACT= / DIGEST= / SHA= / VENUE= +# 1 no pipeline, job failed, inspect failed +# 2 usage / config +# 3 forge CLI cannot read the project (ci venue) + +usage() { + sed -n '2,20p' "$0" +} + +sha= +branch= +root= +timeout_sec=1200 + +while (($#)); do + case "$1" in + --sha) sha=$2; shift 2 ;; + --branch) branch=$2; shift 2 ;; + --root) root=$2; shift 2 ;; + --timeout) timeout_sec=$2; shift 2 ;; + -h|--help) usage; exit 0 ;; + *) echo "Error: unknown argument: $1" >&2; usage >&2; exit 2 ;; + esac +done + +if [[ -z "$root" ]]; then + root=$(git rev-parse --show-toplevel 2>/dev/null || true) +fi +if [[ -z "$root" ]]; then + echo "Error: --root must be a git project" >&2 + exit 2 +fi +cd "$root" + +if [[ -z "$sha" ]]; then + sha=$(git rev-parse HEAD) +else + sha=$(git rev-parse "$sha") +fi + +# Load .env.builder without printing values; shell values win. Do not read `.env`. +if [[ -f "$root/.env.builder" ]]; then + while IFS='=' read -r key value; do + key=${key%%[[:space:]]*} + [[ -z "$key" || "$key" == \#* ]] && continue + if [[ -n "${!key:-}" ]]; then + continue + fi + value=${value%\"}; value=${value#\"}; value=${value%\'}; value=${value#\'} + [[ -z "$value" ]] && continue + printf -v "$key" '%s' "$value" + export "$key" + done < <(grep -v '^[[:space:]]*$' "$root/.env.builder") +fi + +script_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) +version_sh=$script_dir/version.sh +wait_ci=$script_dir/wait_ci.sh + +venue=${BUILD_VENUE:-} +registry=${DOCKER_REGISTRY:-} +repository=${DOCKER_REPOSITORY:-} +if [[ -f "$root/makefile.builder" ]]; then + if [[ -z "$venue" ]]; then + venue=$(make -s -C "$root" -f makefile.builder venue 2>/dev/null || true) + fi + if [[ -z "$registry" ]]; then + registry=$(make -s -C "$root" -f makefile.builder print-DOCKER_REGISTRY 2>/dev/null || true) + fi + if [[ -z "$repository" ]]; then + repository=$(make -s -C "$root" -f makefile.builder print-DOCKER_REPOSITORY 2>/dev/null || true) + fi +fi +venue=${venue:-local} + +docker_tag_for_sha() { + local tmp tag + tmp=$(mktemp -d "${TMPDIR:-/tmp}/builder-obtain.XXXXXX") + git -C "$root" worktree add --detach "$tmp" "$sha" >/dev/null + if [[ -n "$branch" ]]; then + tag=$(CI_COMMIT_BRANCH="$branch" "$version_sh" -C "$tmp" --docker) + else + tag=$("$version_sh" -C "$tmp" --docker) + fi + git -C "$root" worktree remove --force "$tmp" >/dev/null + printf '%s' "$tag" +} + +inspect_image() { + local image=$1 out digest + if ! command -v docker >/dev/null 2>&1; then + echo "Error: docker is required to inspect $image" >&2 + exit 1 + fi + out=$(docker buildx imagetools inspect "$image" 2>/dev/null \ + || docker manifest inspect "$image" 2>/dev/null \ + || true) + if [[ -z "$out" ]]; then + echo "Error: cannot inspect $image" >&2 + exit 1 + fi + digest=$(printf '%s\n' "$out" | awk '/^Digest:/ {print $2; exit}') + if [[ -z "$digest" ]]; then + digest=$(python3 -c 'import json,sys +try: + d=json.load(sys.stdin) +except Exception: + sys.exit(0) +print(d.get("digest") or "")' <<<"$out") + fi + if [[ -z "$digest" ]]; then + echo "Error: inspect of $image had no digest" >&2 + exit 1 + fi + printf '%s' "$digest" +} + +case "$venue" in + ci) + "$wait_ci" --sha "$sha" --root "$root" --timeout "$timeout_sec" + if [[ ! -x "$version_sh" ]]; then + echo "Error: version.sh not found: $version_sh" >&2 + exit 2 + fi + if [[ -z "$registry" || -z "$repository" ]]; then + echo "Error: DOCKER_REGISTRY and DOCKER_REPOSITORY are required to inspect the CI image" >&2 + exit 2 + fi + tag=$(docker_tag_for_sha) + image="${registry}/${repository}:${tag}" + digest=$(inspect_image "$image") + echo "ARTIFACT=$image" + echo "DIGEST=$digest" + echo "SHA=$sha" + echo "VENUE=ci" + echo "TAG=$tag" + ;; + local) + head=$(git rev-parse HEAD) + if [[ "$head" != "$sha" ]]; then + echo "Error: local obtain requires HEAD=$sha (HEAD is $head)" >&2 + exit 2 + fi + if [[ ! -f "$root/makefile.builder" ]]; then + echo "Error: no makefile.builder" >&2 + exit 2 + fi + if make -s -C "$root" -f makefile.builder -n docker >/dev/null 2>&1; then + make -C "$root" -f makefile.builder docker + if [[ -n "${IMAGE_TAG:-}" ]]; then + tag=$IMAGE_TAG + elif [[ -x "$version_sh" ]]; then + tag=$("$version_sh" --docker) + else + echo "Error: cannot derive IMAGE_TAG" >&2 + exit 2 + fi + name=$(make -s -C "$root" -f makefile.builder print-PROJECT_NAME 2>/dev/null || basename "$root") + image="${name}:${tag}" + if [[ -n "$registry" && -n "$repository" ]]; then + image="${registry}/${repository}:${tag}" + fi + echo "ARTIFACT=$image" + echo "DIGEST=local" + echo "SHA=$sha" + echo "VENUE=local" + echo "TAG=$tag" + elif make -s -C "$root" -f makefile.builder -n deb >/dev/null 2>&1; then + make -C "$root" -f makefile.builder deb + echo "ARTIFACT=deb" + echo "SHA=$sha" + echo "VENUE=local" + else + echo "Error: local venue has no docker or deb target" >&2 + exit 2 + fi + ;; + *) + echo "Error: BUILD_VENUE must be local or ci (got ${venue})" >&2 + exit 2 + ;; +esac diff --git a/skills/builder/scripts/wait_ci.sh b/skills/builder/scripts/wait_ci.sh new file mode 100755 index 0000000..69736b6 --- /dev/null +++ b/skills/builder/scripts/wait_ci.sh @@ -0,0 +1,269 @@ +#!/usr/bin/env bash +set -euo pipefail + +# Wait until the forge pipeline for an exact commit SHA finishes. +# Does not trigger a pipeline. Does not print tokens. +# +# Usage: +# wait_ci.sh --sha SHA [--root DIR] [--timeout SEC] +# +# Exit: +# 0 pipeline succeeded; prints "OK pipeline= sha=" +# 1 no pipeline for this SHA, or a job failed/canceled +# 2 usage / bad arguments +# 3 forge CLI missing, or it cannot read this project + +usage() { + sed -n '2,16p' "$0" +} + +sha= +root= +timeout_sec=1200 +poll_sec=10 +appear_sec=90 + +while (($#)); do + case "$1" in + --sha) sha=$2; shift 2 ;; + --root) root=$2; shift 2 ;; + --timeout) timeout_sec=$2; shift 2 ;; + -h|--help) usage; exit 0 ;; + *) echo "Error: unknown argument: $1" >&2; usage >&2; exit 2 ;; + esac +done + +if [[ -z "$root" ]]; then + root=$(git rev-parse --show-toplevel 2>/dev/null || true) +fi +if [[ -z "$root" || ! -d "$root/.git" && ! -f "$root/.git" ]]; then + echo "Error: --root must be a git project" >&2 + exit 2 +fi +if [[ -z "$sha" ]]; then + echo "Error: --sha is required" >&2 + exit 2 +fi +sha=$(git -C "$root" rev-parse "$sha") + +origin=$(git -C "$root" remote get-url origin 2>/dev/null || true) +if [[ -z "$origin" ]]; then + echo "Error: no origin remote" >&2 + exit 2 +fi + +python_json() { + python3 -c "$1" +} + +parse_origin() { + python_json ' +import os, re, sys +url = os.environ["WAIT_CI_ORIGIN"] +url = url.strip() +url = re.sub(r"^git@", "ssh://git@", url) +url = re.sub(r"^ssh://git@([^/]+):", r"ssh://git@\1/", url) +url = re.sub(r"^https?://", "ssh://git@", url) +# ssh://git@host/group/repo.git +m = re.match(r"ssh://git@([^/]+)/(.+?)(?:\.git)?$", url) +if not m: + print("", "", "") + raise SystemExit(0) +host, path = m.group(1), m.group(2) +kind = "github" if host == "github.com" or host.endswith(".github.com") else "gitlab" +print(kind, host, path) +' +} + +export WAIT_CI_ORIGIN=$origin +read -r forge_kind forge_host forge_path <<<"$(parse_origin)" +if [[ -z "$forge_kind" || -z "$forge_path" ]]; then + echo "Error: cannot parse origin: $origin" >&2 + exit 2 +fi + +gitlab_pipelines() { + local encoded + encoded=$(python_json 'import os,urllib.parse; print(urllib.parse.quote(os.environ["WAIT_CI_PATH"], safe=""))') + glab api "projects/${encoded}/pipelines?sha=${sha}&per_page=20" +} + +gitlab_jobs() { + local encoded pid=$1 + encoded=$(python_json 'import os,urllib.parse; print(urllib.parse.quote(os.environ["WAIT_CI_PATH"], safe=""))') + glab api "projects/${encoded}/pipelines/${pid}/jobs" +} + +export WAIT_CI_PATH=$forge_path + +if [[ "$forge_kind" == github ]]; then + if ! command -v gh >/dev/null 2>&1; then + echo "Error: gh is required to watch GitHub Actions for $forge_host" >&2 + exit 3 + fi +else + if ! command -v glab >/dev/null 2>&1; then + echo "Error: glab is required to watch GitLab CI for $forge_host" >&2 + exit 3 + fi +fi + +started=$(date +%s) +first_seen=0 +empty_since=$started + +summarize_gitlab() { + python_json ' +import json, os, sys +raw = sys.stdin.read() +try: + data = json.loads(raw) +except json.JSONDecodeError: + print("error") + raise SystemExit(0) +if isinstance(data, dict) and data.get("message"): + print("auth", data.get("message", "")[:80].replace(" ", "_")) + raise SystemExit(0) +if not isinstance(data, list): + print("error") + raise SystemExit(0) +if not data: + print("empty") + raise SystemExit(0) +p = data[0] +print(p.get("id"), p.get("status"), p.get("web_url", "")) +' +} + +summarize_jobs() { + python_json ' +import json, sys +data = json.loads(sys.stdin.read()) +parts = [] +for j in data if isinstance(data, list) else []: + parts.append("%s=%s" % (j.get("name"), j.get("status"))) +print(",".join(parts)) +' +} + +github_runs() { + gh run list --repo "$forge_path" --commit "$sha" --limit 20 \ + --json databaseId,status,conclusion,url,headSha +} + +summarize_github() { + python_json ' +import json, sys +data = json.loads(sys.stdin.read()) +if not data: + print("empty") + raise SystemExit(0) +r = data[0] +status = r.get("status") +conc = r.get("conclusion") +if status == "completed" and conc == "success": + st = "success" +elif status == "completed": + st = conc or "failed" +else: + st = status or "running" +print(r.get("databaseId"), st, r.get("url", "")) +' +} + +probe_auth() { + if [[ "$forge_kind" == github ]]; then + local out rc + set +e + out=$(github_runs 2>&1) + rc=$? + set -e + if [[ $rc -ne 0 ]]; then + echo "Error: gh cannot list runs for $forge_path" >&2 + echo "$out" | sed '/token/Id; /Bearer/d' >&2 + exit 3 + fi + return + fi + local out rc + set +e + out=$(gitlab_pipelines 2>&1) + rc=$? + set -e + if [[ $rc -ne 0 ]]; then + echo "Error: glab cannot list pipelines for $forge_path on $forge_host" >&2 + echo "$out" | sed '/token/Id; /Bearer/d' >&2 + exit 3 + fi + local kind + kind=$(printf '%s' "$out" | summarize_gitlab | awk '{print $1}') + if [[ "$kind" == auth ]]; then + echo "Error: glab is authenticated but cannot read $forge_path" >&2 + exit 3 + fi +} + +probe_auth + +status_line() { + if [[ "$forge_kind" == github ]]; then + github_runs | summarize_github + else + gitlab_pipelines | summarize_gitlab + fi +} + +while true; do + now=$(date +%s) + if (( now - started > timeout_sec )); then + echo "FAIL reason=timeout sha=$sha" >&2 + exit 1 + fi + line=$(status_line) + set -- $line + kind=${1:-empty} + + if [[ "$kind" == empty ]]; then + if (( now - empty_since > appear_sec )); then + echo "FAIL reason=no_pipeline sha=$sha origin=$forge_host/$forge_path" >&2 + echo "This SHA has no CI pipeline. Tag or push a ref the project actually builds." >&2 + exit 1 + fi + sleep "$poll_sec" + continue + fi + if [[ "$kind" == error || "$kind" == auth ]]; then + echo "Error: unexpected forge response for $forge_path" >&2 + exit 3 + fi + + pid=$1 + st=$2 + url=$3 + empty_since=$now + if [[ "$first_seen" -eq 0 ]]; then + echo "Watching pipeline $pid status=$st $url" >&2 + first_seen=1 + fi + + case "$st" in + success) + echo "OK pipeline=$url sha=$sha id=$pid" + exit 0 + ;; + failed|canceled|cancelled|skipped) + jobs_s= + if [[ "$forge_kind" != github ]]; then + jobs_s=$(gitlab_jobs "$pid" | summarize_jobs || true) + fi + echo "FAIL reason=job_$st sha=$sha pipeline=$url jobs=$jobs_s" >&2 + exit 1 + ;; + created|pending|running|waiting_for_resource|preparing|waiting|queued|in_progress|waiting_for_resource) + sleep "$poll_sec" + ;; + *) + sleep "$poll_sec" + ;; + esac +done diff --git a/skills/builder/templates/makefile.builder b/skills/builder/templates/makefile.builder index 8e5a98a..f639f64 100644 --- a/skills/builder/templates/makefile.builder +++ b/skills/builder/templates/makefile.builder @@ -8,6 +8,9 @@ ARCHES ?= amd64 ARCH ?= $(firstword $(ARCHES)) +# local: make docker/deb on this machine. ci: obtain.sh waits for the +# forge pipeline of an exact SHA, then inspects the artifact. Not a track. +BUILD_VENUE ?= local ifeq ($(strip $(ARCHES)),) $(error ARCHES must not be empty) endif @@ -34,7 +37,7 @@ endif BUILDER_SCRIPTS := $(or $(BUILDER_SKILL_DIR),$(HOME)/.pouch/skills/builder)/scripts -.PHONY: help version arches clean build deb docker push push-deb push-docker +.PHONY: help version arches venue clean build deb docker obtain push push-deb push-docker help: @echo "$(PROJECT_NAME) $(VERSION)" @@ -45,6 +48,7 @@ help: @echo " make -f makefile.builder clean - 删除 $(DIST_DIR)" @echo " make -f makefile.builder deb - 打 DEB（按 ARCHES；只构建不上传）" @echo " make -f makefile.builder docker - 本地单平台镜像" + @echo " make -f makefile.builder obtain - 按 BUILD_VENUE 拿到产物" @echo " make -f makefile.builder push / push-deb / push-docker" version: @@ -53,6 +57,12 @@ version: arches: @echo $(ARCHES) +venue: + @echo $(BUILD_VENUE) + +print-%: + @echo $($*) + clean: rm -rf $(DIST_DIR) @@ -83,7 +93,13 @@ docker: push-deb: $(BUILDER_SCRIPTS)/upload_deb.sh $(foreach a,$(_PUSH_DEB_ARCHES),$(DIST_DIR)/$(PROJECT_NAME)_$(VERSION)_$(a).deb) -ifdef _ARCH_FROM_FILE +obtain: + $(BUILDER_SCRIPTS)/obtain.sh + +ifeq ($(BUILD_VENUE),ci) +push-docker: + $(BUILDER_SCRIPTS)/obtain.sh +else ifdef _ARCH_FROM_FILE push-docker: $(BUILDER_SCRIPTS)/publish_docker.sh else diff --git a/skills/delivery-loop/SKILL.md b/skills/delivery-loop/SKILL.md index c29aa19..357db17 100644 --- a/skills/delivery-loop/SKILL.md +++ b/skills/delivery-loop/SKILL.md @@ -18,7 +18,8 @@ description: >- ## 不可违反的规则 - 复用现有 `hunt`、`tester`、`check`、`builder`、`deployer`、`releaser` 和 - `orchestration` 能力；不要在本 skill 里复制它们的实现。 + `orchestration` 能力；不要在本 skill 里复制它们的实现。`artifact_built` 只调用 + builder `obtain.sh`，不要在本 skill 里 poll GitLab/GitHub。 - 先读取项目的 delivery profile、分支策略和当前工作树，再决定可执行的停止点。 - `stopAt` 未配置或与用户目标冲突时停止并报告缺口，不猜测是否应合并或部署。 - 提交、推送、合并、打 tag、发布产物和生产部署都必须同时满足项目配置与当前请求的授权。 diff --git a/skills/delivery-loop/references/state-machine.md b/skills/delivery-loop/references/state-machine.md index c415976..c404ab4 100644 --- a/skills/delivery-loop/references/state-machine.md +++ b/skills/delivery-loop/references/state-machine.md @@ -31,8 +31,8 @@ ACK 配置必须使用以下 canonical `stopAt`： - `pr_opened`：提交说明、测试证据、风险和回滚信息已写入 PR；不能把草稿 PR 当 `review_ready`。 - `merged`：目标分支的合并 commit 已确认，不能只确认源分支 commit。 - `regression_passed`：以合并 commit 执行项目规定的完整回归；失败则回到修复或停止。 -- `artifact_built`：产物由合并后的 commit 构建，记录版本、文件和 SHA256。 -- `published`：镜像/APK 等外部分发位置可复核，记录 digest 或文件校验和。 +- `artifact_built`：对**合并后的精确 commit SHA** 调用 builder `obtain.sh`。`BUILD_VENUE=ci` 时监听该 SHA 的流水线并 inspect 产物，不在本机重新编译。无流水线、job 失败或 digest 对不上则本阶段失败，不得进入 `published`。项目若声明默认分支不构建，合进 `main` 会 `no_pipeline`：预览推 `dev/*`，正式打 tag。 +- `published`：镜像/APK 等外部分发位置可复核，记录 digest 或文件校验和。`venue=ci` 且 CI 已经 push 到 registry 时，obtain 成功即同时满足本阶段，不要再本机 `publish_docker.sh`。 - `deployed`：目标服务运行在预期版本，健康检查和关键路径验证通过；生产部署还必须进入 `released`。 ## 配置和授权