fix(delivery): package images once and publish in the build job #49

Merged

gitchegumi merged 1 commit from kiyi/simple-bundle into main

2026-09-20 15:48:32 -05:00

kiyi commented

2026-09-20 09:32:30 -05:00

Member

Package the image inventory once in a shared image-only Zarf package, followed by two small manifest-only packages. Signed app metadata selects images first plus dev or prod; both deployments still use the same immutable archive. Older signed bundles retain their existing selection. The Zarf agent handles pod image rewrites to the populated registry.

Use temporary per-package image caches and UDS scratch directories; remove intermediate Zarf archives after successful assembly. Sign/publish directly from the build job, retaining only small Actions security artifacts. Protected owner dispatch, hash checks, security gates and final-step signing credentials remain; step scoping is not separate-job isolation. No runner storage/RAM increase or node restriction.

Validation: 167 Linux tests pass, workflow YAML/Bash and 16 scaffold cases pass. Real UDS 0.35.1 / Zarf 0.83.0 built a tiny BusyBox bundle with all three packages, verified its receipt, and removed intermediate archives/caches. Tests verify one image-bearing package, dev/prod selection order, identical signed archive bytes, legacy selection, and no inter-job archive transfer. Existing workflows-repo HTTP exceptions were rebound only to moved lines, without expanding scope.

Hard storage-limit test, 2026-09-20: ran the real published ComfyUI image (sha256:b822d4d8a0c328dca57beb890bcf5f38aedfe295796648dd8b2282e0a54aa987) through the old v1.15.2 and this commit's packaging code, each on a fresh 24 GiB ext4 loopback disk with 6 GiB preallocated as runner/tools/scanner overhead. UDS binary checksum matches tools.lock.json. HOME, caches, temporary files and output all stayed inside the capped filesystem; filesystem use was sampled every second.

Old: failed writing the prod Zarf archive with 'no space left on device'; sampled peak 23.36 GiB. Its registry transfer retried after an unexpected EOF, so elapsed time and exact disk difference are not a controlled performance comparison.
New: completed packaging and receipt/hash verification in 613 seconds, peak 22.33 GiB including the reserve, minimum usable free space 1.11 GiB. Final usage 10.05 GiB including reserve; final bundle 4,340,987,812 bytes. Temporary image caches, UDS scratch and intermediate archives were removed.
Scope: packaging-only test with synthetic scan receipts, not a repeat of security scanning, signing/upload or live deployment. The 6 GiB reserve is a test allowance, not a measurement of complete CI overhead. Ext4 usable capacity was 23.46 GiB; headroom is modest. Temporary test disk was unmounted and deleted afterward.

Release/adopt this together for bundle build and deployment, then build a new bundle using the existing image digest. Full CI and live deployment remain to be verified. Related: Homelab/Comfy-UI#1 and #5.

ComfyUI run 129 passed security but its worker was evicted during UDS assembly: docker-data exceeded its 24 GiB EmptyDir limit. The builder packaged the same image in both environment packages and retained their caches. A separate publishing job would then upload/download the complete bundle again. Package the image inventory once in a shared image-only Zarf package, followed by two small manifest-only packages. Signed app metadata selects images first plus dev or prod; both deployments still use the same immutable archive. Older signed bundles retain their existing selection. The Zarf agent handles pod image rewrites to the populated registry. Use temporary per-package image caches and UDS scratch directories; remove intermediate Zarf archives after successful assembly. Sign/publish directly from the build job, retaining only small Actions security artifacts. Protected owner dispatch, hash checks, security gates and final-step signing credentials remain; step scoping is not separate-job isolation. No runner storage/RAM increase or node restriction. Validation: 167 Linux tests pass, workflow YAML/Bash and 16 scaffold cases pass. Real UDS 0.35.1 / Zarf 0.83.0 built a tiny BusyBox bundle with all three packages, verified its receipt, and removed intermediate archives/caches. Tests verify one image-bearing package, dev/prod selection order, identical signed archive bytes, legacy selection, and no inter-job archive transfer. Existing workflows-repo HTTP exceptions were rebound only to moved lines, without expanding scope. Hard storage-limit test, 2026-09-20: ran the real published ComfyUI image (sha256:b822d4d8a0c328dca57beb890bcf5f38aedfe295796648dd8b2282e0a54aa987) through the old v1.15.2 and this commit's packaging code, each on a fresh 24 GiB ext4 loopback disk with 6 GiB preallocated as runner/tools/scanner overhead. UDS binary checksum matches tools.lock.json. HOME, caches, temporary files and output all stayed inside the capped filesystem; filesystem use was sampled every second. - Old: failed writing the prod Zarf archive with 'no space left on device'; sampled peak 23.36 GiB. Its registry transfer retried after an unexpected EOF, so elapsed time and exact disk difference are not a controlled performance comparison. - New: completed packaging and receipt/hash verification in 613 seconds, peak 22.33 GiB including the reserve, minimum usable free space 1.11 GiB. Final usage 10.05 GiB including reserve; final bundle 4,340,987,812 bytes. Temporary image caches, UDS scratch and intermediate archives were removed. - Scope: packaging-only test with synthetic scan receipts, not a repeat of security scanning, signing/upload or live deployment. The 6 GiB reserve is a test allowance, not a measurement of complete CI overhead. Ext4 usable capacity was 23.46 GiB; headroom is modest. Temporary test disk was unmounted and deleted afterward. Release/adopt this together for bundle build and deployment, then build a new bundle using the existing image digest. Full CI and live deployment remain to be verified. Related: Homelab/Comfy-UI#1 and #5.

kiyi added 1 commit

2026-09-20 09:32:30 -05:00

fix(delivery): package images once and publish in the build job

Workflow acceptance / acceptance (pull_request) Successful in 2m22s

Details

Library checks / tests (pull_request) Successful in 2m46s

Details

Repository security / security-1 (pull_request) Successful in 6m6s

Details

Repository security / security (pull_request) Successful in 0s

Details

c5006322ea