Build matching signed Python payloads for ComfyUI candidates #90
No reviewers
Labels
No labels
app-migration
bug
ci-cd
cluster
Documentation
duplicate
enhancement
gpu
invalid
networking
question
security
Storage
wontfix
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
Homelab/workflows!90
Loading…
Reference in a new issue
No description provided.
Delete branch "kiyi/comfy-candidate-payload"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
ComfyUI update runs 229 and 234 resolve a new requirements.lock but still copy the fixed Python payload, so the application correctly fails its checksum. Add an opt-in ComfyUI Python payload build to the existing source-installed container workflow: verify the signed PyTorch/CUDA build parent, build the exact candidate lock with the reviewed recipe, scan the entire scratch payload, and publish signed evidence in ComfyUI’s existing namespace.
The source gate scans the complete original candidate checkout, all fetched Git refs/history and working-tree files, including files omitted from the minimal Docker context. Required history scans reject missing, empty, nested or shallow checkouts. The passing source-decision.json is retained and hash-bound in the signed image receipt. Payloads retain full-image scanning; application builds retain exact signed-payload verification and lock checksums.
Automatic selections remain restricted to the payload FROM digest and matching declaration, plus application evidence matching both manifests. Workflow names, job IDs, status-check names, default dispatch behavior and legacy signed releases are preserved. This branch incorporates released v1.34.0. The current preinstalled image embeds v1.33.0 and does not expose the new payload opt-in; its existing application contract stays unchanged.
Validation: the full 346-test suite passed with one local Trivy integration skip. Five new tests use native Git and pinned Gitleaks 8.30.1 to cover historical-only, other-ref and omitted working-tree secrets; missing/incomplete history; clean receipt binding and evidence tampering. YAML/Bash validation and 16 mocked scanner control-flow cases passed, as did Ruff, git diff --check and the staged native Gitleaks hook. Container commands and unrelated scanners are mocked in the regressions; hosted security checks and actual payload publication remain separate acceptance gates.
Release order: after fresh hosted checks and approval, merge this and any other ready changes, publish the next unused minor after v1.34.0 through the protected library release workflow, then update ComfyUI PR #38 to that exact release before adopting it. No manual build, release or deployment was dispatched. Follow-up is tracked in ComfyUI issue #39.
Addressed run 562 in 29e2af7564. Its sole blocker was the same public Terraform acceptance-object name encountered through all-ref history, not ComfyUI payload credentials.
Applied the shared exact rule/path/public-value review documented on PR #89, with immutable provenance and expiry 2026-12-01. History and working-tree scans remain complete; changed values and unrelated credentials still block. The originating fixture is clarified on #89, and #87 carries the same review so these PRs can rerun independently.
All 49 affected source-secret/policy tests passed on this branch, including native Gitleaks 8.30.1 synthetic history/tree checks and changed-value rejection. No workflow files, status-check names, job IDs or ComfyUI delivery behavior changed. Fresh hosted checks were triggered by the push; their results are pending.
Request changes
P1 — Preserve the original checkout/history source-security gate for Python payloads.
In
pipeline/python_payload.py:43–47,build_container(context, ...)substitutes the minimal temporary Docker context for the original fetched checkout. This context has no.gitand includes only the copied payload inputs.container.build()runs its sole source scan against that context (pipeline/container.py:29), so the mandatory history scan (pipeline/security.py:600–607) cannot inspect the fetched application history, and working-tree scans omit noncopied files. The precedingprotected()checks validate provenance/policy but do not scan the checkout.This is a silent fail-open, not merely a scanner failure: reproduced with the checksum-verified, pinned Gitleaks 8.30.1. Running
gitleaks git <non-repository-context> --log-opts=--all --redact=100 --report-format json --report-path <report>logsfatal: not a git repositoryand0 commits scanned, yet exits 0 and writes[]. A credential existing only in fetched history or an omitted working-tree file therefore no longer blocks this payload publication, while its source evidence can still appear passing. Full image scanning does not cover deleted historical files.Required change: Separate the minimal Docker build context from the complete original-checkout source gate, and retain that complete gate's passing decision in the published evidence. Add regression coverage demonstrating that both a historical secret and a secret in an omitted working-tree file block payload publication. Fail closed when a required history scan has no Git repository; initializing an empty repository around the reduced context would not restore the fetched-history guarantee.
Validation: reviewed exact head
29e2af7564df184a4f9d87378ad8bbf1b544ea51; all three hosted checks are successful; the full unittest suite passed (316 tests, 3 skipped); 12 source-secret tests passed with native pinned Gitleaks;tests/check_scaffold.pypassed YAML/Bash and 16 mocked cases;git diff --checkpassed. Existing payload unit tests mockcontainer.build, so those green checks do not exercise this source-scan boundary. No real payload build or publication was dispatched.The signed-parent verification, full-image payload scan, and narrow digest/evidence selection guards are otherwise a sound direction. This source-gate regression is the reason I cannot approve the current head.
Addressed the P1 in a6a4d5c220, in response to this review.
The payload builder now supplies the original candidate checkout separately from its minimal Docker context. container.build scans that complete checkout with its original policy and retains the resulting source-decision.json in the hash-bound evidence signed by the existing publisher. Image inputs remain minimal, and the scratch payload still receives the full-image gate.
Required history scans now verify the actual Git root, HEAD and non-shallow history before invoking Gitleaks. Missing/empty repositories, nested directories inheriting another checkout, unavailable Git and shallow clones fail closed. Linked worktrees remain supported; no temporary repository is initialized around the payload context.
Five new regression tests exercise the real preparation/container source-gate path with native Git and pinned Gitleaks 8.30.1. Historical-only secrets, secrets on another fetched ref and secrets in omitted working-tree files all block before the mocked build boundary, with no archive or receipt created. Clean-source coverage verifies full-image scan selection, exact source-decision retention/hash binding, and rejection of modified evidence.
Validation: full suite 346 tests, OK with one local native-Trivy integration skip; all five new regressions rerun successfully after lint cleanup; YAML/Bash and 16 mocked scanner cases, Ruff and git diff --check passed. The installed native Gitleaks 8.30.1 staged hook also passed. Container commands and unrelated scanners are mocked; no real build/publication was dispatched.
This commit also integrates released v1.34.0, preserves every existing scan/workflow/job name, and documents that the current preinstalled v1.33.0 image cannot expose this new opt-in. The PR description now specifies the next unused minor release after v1.34.0 and matching consumer pins. Hosted validation is currently blocked during runner setup, before any library step: repository security run 597 reports “No such image” for the configured main-runner digest, and image startup run 598 reports the same for the accepted security image digest. Restore runner/registry image availability, then rerun this exact head. The source fix is ready for re-review; this top-level review has no inline thread to resolve, and the reviewer’s approval remains pending.
Reviewed current head
a6a4d5c220against Dock's earlier source-gate finding. The payload builder now passes the original checkout separately from its curated Docker context; container.build scans that checkout before build, copies the passing source decision into hash-bound receipt evidence, and require_git_history rejects missing, empty, nested, or shallow checkout roots. The added native Git/Gitleaks regressions cover historical, other-ref, and omitted working-tree secrets, but I have not independently executed those tests or built/published a real payload.Not approving on this head: the exact-head repository-security run 7018 and security-image-compatibility run 7019 are failed. Available diagnostics for jobs 8732 and 8735 say the runner could not create its pinned job image (No such image), before checkout; this is an image-startup/infrastructure failure, not evidence of a scanner rejection of the PR source. The legacy/image sibling logs returned HTTP 500, so their failure causes remain unverified. The library-tests job 8737 in run 7020 is still running; acceptance run 7017 passed. Please verify availability of the exact pinned runner image and runner-local pull/startup, restore those gates and rerun this exact head; then re-review against the fresh completed checks. No approval, merge, or image publication is implied.
Approved at exact head
3bb6bab4d4ef3b8e03164189dd74f066ae069115. The earlier full-checkout blocker is resolved: the payload path scans the original checkout/history separately from the minimal Docker build context, and regressions cover historical/other-ref and omitted working-tree secrets before any build. This head is the clean merge of current main4202677614380f4e58151253792541bcc50c0f84with the reviewed PR source. All 7 exact-head checks pass; acceptance artifact 5343 binds this SHA and verifies clean/history/working-tree gate behavior, repository-security artifact 5344 reports PASS with 0 blocking and 0 scanner errors, and compatibility artifacts 5340/5341 confirm both legacy and authenticated-image startup modes. Local full suite: 352 passed, 20 skipped because pinned native external-tool fixtures were unavailable locally.Approved at exact head
3bb6bab4d4ef3b8e03164189dd74f066ae069115. The earlier full-checkout blocker is resolved: the payload path scans the original checkout/history separately from the minimal Docker build context, and regressions cover historical/other-ref and omitted working-tree secrets before any build. This head is the clean merge of current main4202677614380f4e58151253792541bcc50c0f84with the reviewed PR source. All 7 exact-head checks pass; acceptance artifact 5343 binds this SHA and verifies clean/history/working-tree gate behavior, repository-security artifact 5344 reports PASS with 0 blocking and 0 scanner errors, and compatibility artifacts 5340/5341 confirm both legacy and authenticated-image startup modes. Local full suite: 352 passed, 20 skipped because pinned native external-tool fixtures were unavailable locally.