Share bundle source validation and complete package image checks #100
No reviewers
Labels
No labels
app-migration
bug
ci-cd
cluster
Documentation
duplicate
enhancement
gpu
invalid
networking
question
security
Storage
wontfix
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
Homelab/workflows!100
Loading…
Reference in a new issue
No description provided.
Delete branch "kiyi/reusable-bundle-ci"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Homelab/GitchAlerts#17 still implements manifest/startup validation and its exact-image scan as app-specific jobs. Add reusable entry points so its workflows can become thin callers while keeping those checks.
validate-source.ymlexecutes a nonempty declaration of caller-owned Python scripts using the existing hash-verified Python lock; its setup profile downloads no scanner binaries.package-image-security.ymlderives the dev/prod inventory through shared package preparation and uses the same package-context function as signed build/deploy. It requires complete full-image decisions, clean malware evidence and a valid CycloneDX SBOM for every image in a fresh evidence directory. No risk acceptance is added.Registry credentials are exposed only in the scan step after the same-repository PR guard and are paired with the exact registry host. Source validation receives no delivery credentials. Both entry points compile as frozen stable-v1 adapters and support the existing executable job display names. Caller wrapper names must remain distinct from the executable required checks; fresh hosted expansion and failure propagation must be verified before adoption. Release and promote the reviewed adapter before activating consumer paths.
Validation at
545600da7b682bc6af4143a047215ae6ed557d09: 46 focused source/contract tests passed, covering Forgejo schemas/credentials/runner contracts, all 23 frozen adapters and generated Bash, source-script failure handling, complete image decisions, reporting, package contexts and mocked packaging. The actual preparation/scanner fixture covers one and two distinct digest-pinned images, per-image SBOM contents and rejection when only the second SBOM is absent. The reporting inventory includes the new sanitized scan workflow. Native Gitleaks 8.30.1 and diff checks passed; no local container execution occurred. Forgejo reports workflow acceptance, repository security and Library checks successful on this head. The prepared GitchAlerts patch passed its six thin-caller contracts, 27 resource checks, 28 negative manifest cases and 10 reference cases; Linux startup skipped locally. Hosted consumer expansion, exact emitted required-check names, startup and full-image acceptance remain pending.This does not remediate the old image bytes: GitchAlerts run 83 correctly blocks 15 vulnerability occurrences plus Redis DS-0002/DS-0025, with zero scanner errors. Reviewed application/base-image publication and four-digest adoption remain required before deployment approval. Refs: #3 and GitchAlerts tracker #18.
Blocking:
pipeline/package_image_security.py::require_completecheckssbom-1.cdx.jsonthroughsbom-N.cdx.json, but the existingsecurity.scanenumerates images from zero and writessbom-0.cdx.jsonthroughsbom-(N-1).cdx.json. On any real successful scan, the final requested SBOM is absent, so this new image gate rejects the clean result; the mocked test createssbom-1and misses the production naming contract. Match the existing zero-based filenames, and exercise a real (or fixture-backed actual security.scan) one-image/two-image SBOM path in tests rather than only a mocked scanner. Also investigate the two exact-headpackage-image-security.ymlchecks failing before runner logs (run IDs 7186/7188; job log endpoints returned 404); their cause and hosted expansion remain unverified. Workflow acceptance has succeeded, while other required checks were pending at review time. No release, adoption, or merge until the corrected head and required gates are verified.Blocking at this head:
pipeline/package_image_security.py::require_completestill iteratesrange(1, len(images) + 1)and requiressbom-1.cdx.jsonetc. The existingsecurity.scanwrites zero-indexed image SBOM filenames (sbom-0.cdx.jsononward); a clean real one-image or two-image scan therefore lacks the final filename this gate requires and fails.tests/test_bundle_ci.pystill createssbom-1.cdx.jsoninside its mocked scan, masking that production mismatch. Align the gate with the actual scanner's SBOM naming and add a real or fixture-backed scanner-output regression for one and two images (including missing-SBOM rejection). At this exact head the workflow acceptance, repository security and library checks are still running/pending; those results, hosted reusable-workflow expansion, and actual image scan acceptance are unverified. Do not release/promote/adopt or merge on this review.Blocking on current head
e14a3daf:pipeline/package_image_security.py::require_completestill readssbom-1.cdx.jsonthroughsbom-N.cdx.json, andtests/test_bundle_ci.pystill fabricatessbom-1.cdx.json. The existing scanner's zero-based SBOM output (sbom-0.cdx.jsononward, documented in my review of00449ef) has not been reconciled. A real successful one-image scan will fail this new gate when it looks for nonexistentsbom-1. Align the consumer with the producer and exercise the actual scanner output (or a fixture produced from it) for one and two images; the current mocked scanner cannot validate this contract. Run 672 / API 7190 on00449efwas cancelled and is superseded, not evidence that current-head checks failed. Current-head Library, security, and acceptance checks remain pending; release/adoption/merge are not accepted.The zero-based SBOM consumer is now aligned with the producer (
range(len(images))), and the new fixture exercises the realsecurity.scanpath for one image. My earlier requested multi-image regression is still missing:test_actual_package_preparation_and_scanner_report_contractconstructs only one digest, asserts onlysbom-0.cdx.json, and the missing-SBOM negative test also covers only one image. This is the reusable full-inventory gate, so please extend the actual-scanner fixture to two distinct digest-pinned images and assert both zero-based SBOMs are required, including rejection when onlysbom-1.cdx.jsonis absent. That closes the previous review's one-/two-image producer/consumer contract rather than trusting a mocked scan for the multi-image case. At this head, acceptance, repository security, and library checks are still running; hosted reusable-workflow expansion, scanner acceptance and consumer startup remain unverified. No release, adoption, or merge on this review.Source review at
545600da7b: the previous SBOM index defect is corrected (require_completecheckssbom-0throughsbom-(N-1)), and the actualpromotion.prepare+security.scanfixture now covers one and two distinct digest-pinned images, checks both generated zero-based SBOMs and rejects a missing second SBOM while the first remains. The full-image gate still rejects incomplete decisions, malware warnings/errors, missing images and invalid SBOMs; image scan uses a fresh evidence directory and credentials remain in the scan step after the same-repository PR guard. Source validation executes nonempty, confined caller-owned scripts without delivery credentials. The shared package-context preparation preserves existing signed build/deploy image metadata; this change does not add a UDS bundle, gateway, route, controller or policy exemption and does not change runtime routing. Source approval only: exact-head Library, repository-security and workflow-acceptance checks were still running at review, and hosted reusable-workflow expansion, consumer startup and full-image scan acceptance remain unverified. Do not merge, release/promote adapters or adopt a consumer until required head-bound checks and hosted acceptance succeed; the separately blocked GitchAlerts image findings are not remediated by this PR.CI at submission: pending. This is a source-review approval, not a scan verdict; required checks still block merge.
Addressed the two-image request in commit 545600d. The fixture now runs both one- and two-image inventories through actual promotion preparation, image-package metadata,
security.scanandrequire_complete, stubbing only external Trivy/ClamAV operations. It verifies per-image scan/conversion calls, package context, malware coverage, and the contents of both zero-based SBOMs. Removing onlysbom-1.cdx.jsonwhile retainingsbom-0.cdx.jsonrejects the two-image result. The one-image missing-SBOM case also remains covered.The earlier requests (6361, 6364, 6366) are covered by the existing zero-based consumer fix in
321dd90and this expanded fixture. The original parser failure was the unsupportedworkflow_call.secretsmapping, removed ine14a3daand checked by the Forgejo contract suite.I inspected Library checks run 679: its sole failure among 391 tests was the reporting test’s expected upload-workflow inventory omitting the new
package-image-security.yml. This commit adds that entry while retaining all sanitized-report assertions. Acceptance run 677 and repository security run 678 passed on the preceding head.Validation: 46 focused source/contract tests passed, including bundle checks, reporting, frozen adapters/generated Bash, Forgejo schemas/credentials/runner contracts, package contexts and mocked packaging. Native Gitleaks 8.30.1 and diff checks passed; no containers ran locally. The new hosted head, reusable expansion, consumer startup and real image acceptance remain pending. I have not released, promoted, activated consumers or merged this PR.
New commits pushed, approval review dismissed automatically according to repository settings
Source review at
5926bea0ca: the previously requested production SBOM naming contract remains zero-based, and the actual preparation/security.scan regression covers one and two distinct images and missing second SBOM. The package-image gate still requires the entire sorted image inventory, full-image passing decisions, clean malware for each image and valid per-image CycloneDX files; its evidence directory is fresh and its credentials are confined to the guarded scan step. Source validation checks confined nonempty caller-owned scripts without delivery credentials. I also inspected the newly present Node adapters/planner/cache helper and the Terraform state-bootstrap profile: Node keeps the original credential-free Node 24 path, requires all five gates and a manifest-pinned pnpm version, checks the preinstalled Node 26 runtime before install, and exports writable job-local caches; Terraform restricts the new state-bootstrap selector to inspect/bootstrap, binds endpoint/bucket/role/credential in library policy, and requires protected dispatch and bucket encryption/versioning/conditional-write checks. Existing app plan/apply contract is not broadened by this selector. The shared UDS image-package context is metadata for prepared bundles, not a new runtime gateway, route, controller or admission exemption; pinned UDS packaging/integration acceptance remains a hosted check. No source-level blocker found in the inspected head. Source approval only: current-head acceptance, repository security, image compatibility/startup and Library checks were pending during review; no hosted consumer expansion, image startup/full-image acceptance, SeaweedFS backend acceptance, release/promotion or deploy is established here. Required exact-head checks and actual consumer acceptance remain separate gates; do not merge/release/promote/adopt on this approval alone.CI at submission: pending. This is a source-review approval, not a scan verdict; required checks still block merge.
CI follow-up for kiyi at
5926bea0ca: Library checks run 718 (API 7297), executingtestsjob 9173, is failed. The authenticated job-log diagnostics are truncated during the verbose unit/contract-test output; they do not expose the first FAIL/ERROR traceback or the final failing step. The manyfailed or interruptedClamAV/Trivy lines occur in test-progress output and do NOT establish a real malware/image-scan failure. Please inspect the complete job log, identify the first failing assertion or step, reproduce it against this exact head on the owning Linux runner, and provide the narrow correction if source/test behavior is wrong. The test-library workflow runs unit tests, UDS packaging integration, YAML/shell checks, Kubernetes schema validation, source-scan integration, then Node cache/setup probes: this excerpt does not establish which stage failed. Workflow acceptance run 715, Repository security run 716, and image-compatibility run 717 report success at this head, but the Library gate remains red. My existing APPROVED review #1205 already covers source at this SHA; it is not CI acceptance. Preserve all checks; no merge, release or promotion on this failed gate. The reviewed source change is not a new UDS gateway/route/admission policy boundary; live packaging and deployment acceptance remain distinct.New commits pushed, approval review dismissed automatically according to repository settings
Follow-through for the current Library checks review: the complete run 722 traceback identifies a selected-context check rejecting the existing generated Python payload, rather than a Trivy/ClamAV verdict. Shared correction
35f595fconfines caller-selected build folders at the CLI boundary while retaining full checkout/history scanning for the curated payload;dfa1c07includes it here.The earlier SBOM review remains addressed: actual scanner/preparation fixtures require both zero-based SBOMs for two distinct images and reject a missing second SBOM. Both package-image and RKE2 reporting inventories, the Node cache probe and isolated Seaweed bootstrap are retained in the combined release branch. Final combined validation completed 127 local tests with eight Windows symlink skips, plus staged Gitleaks/diff checks. Fresh exact-head hosted checks and consumer expansion/acceptance remain required. No security gate or required status name was removed or renamed.
Merge #103 → #101 → #99 → #100 → #102, then publish one release and promote stable-v1 once. The owner selected checkout-based consumers for this batch. GitchAlerts image remediation remains a separate blocker; this source fix does not make its current images acceptable.
Source review at
dfa1c07897: the earlier zero-based SBOM producer/consumer issue remains corrected, with actual preparation/security.scan fixture coverage of one and two distinct images and rejection when the second SBOM is removed. The complete-image gate still requires full-image passing decisions, clean malware evidence for each selected image and valid per-image CycloneDX files in a fresh output directory. Source checks remain caller-confined and credential-free; registry credentials are limited to the guarded image-scan step. I inspected the new selected-context correction in container.py: the CLI requires the chosen application build folder to be the checkout or a descendant before build, while build() still scans the full supplied checkout/history and curates the selected folder as build context; its negative unrelated-checkout test and selected-tool test cover this boundary. The Node cache helper and Library workflow now export and probe a writable job-local Node tool cache before real setup-node, while the separate preinstalled-image path checks exact runtime and manifest-pinned pnpm. Terraform state-bootstrap remains an inspect/bootstrap-only policy-bound operation with protected dispatch and bucket security/conditional-write checks. This is not a change to UDS runtime gateway/route ownership, admission exemptions, or controllers; pinned UDS packaging integration remains a hosted acceptance gate. No source-level blocker found on this head. Approval is source-only: exact-head Library, workflow acceptance, repository security and image-compatibility/startup checks are waiting/pending. The previous Library failure is not resolved until its new check executes and passes; hosted consumer expansion, full-image scan acceptance, backend acceptance and GitchAlerts' separate image findings remain unverified. Do not merge, release/promote or adopt consumers based on this approval alone. Preserve every required gate.CI at submission: pending. This is a source-review approval, not a scan verdict; required checks still block merge.
New commits pushed, approval review dismissed automatically according to repository settings
Source review of
0bacbfd83c: I rechecked the authenticated current head, changed workflow adapters and implementation, and prior reviews. The earlier SBOM indexing defect remains fixed: the complete-image decision requires the sorted entire image inventory, full-image passing status, clean malware records and zero-based regular CycloneDX SBOMs; the actual preparation/security.scan fixture tests one and two distinct digests and rejection when only the second SBOM is removed. The image workflow keeps registry credentials in the scan step after its same-repository PR guard and uses a fresh evidence directory. Source validation requires confined nonempty caller scripts without delivery credentials. The newly present Node and Terraform paths preserve manifest-pinned runtime/tooling and the inspect/bootstrap-only, reviewed-backend boundary respectively; I found no source-level blocker in the inspected head. UDS image-package context is package metadata, not a new gateway/route/controller or admission exemption; actual pinned UDS packaging remains a hosted gate. This approval is source-only. Workflow acceptance, repository security and Library checks at this SHA are running/pending, not passed; the preceding Library failure is not accepted as fixed until the new job passes. Hosted reusable expansion, consumer startup/full-image acceptance, state backend acceptance, release/promotion, and GitchAlerts' separate image remediation remain unverified. Keep required checks and security gates; no merge, release, promotion or consumer adoption based on this approval alone.CI at submission: pending. This is a source-review approval, not a scan verdict; required checks still block merge.
Run 697 batch recovery is included in
0bacbfdthrough the normal dependent-branch merge. See the diagnosis and fix in #101: the clean source fixture passed, then the next Semgrep rule download suffered a connection reset. Three bounded attempts recover temporary transport failures; TLS, full-content pinning and blocking failure reports remain intact.Combined branch
b955d25completed 177 local cases with eight Windows symlink skips, including scanner recovery, Node/bundle contracts, package image gates and native Gitleaks payload regressions. Staged Gitleaks, diff checks and the #99 → #100 merge simulation passed. Hosted current-head checks remain required. Preserve #101 → #99 → #100 → #102 and one combined release; no required status names changed.Reviewed current PR #100 source at
9ad31f80aeafter its merge from main and prior human reviews. The prior SBOM producer/consumer defect remains corrected: full-image decisions require the complete sorted digest inventory, clean malware records and regular zero-based CycloneDX files; the actual preparation/scanner fixture covers one and two images and fails if the second SBOM is removed. Source validation confines nonempty caller-owned Python checks without delivery credentials; image credentials remain limited to the guarded scan step. Node cache flows through writable RUNNER_TEMP and GITHUB_ENV before setup-node; the owning Library job retains a real Node setup probe. No source-level blocker found in inspected changed paths. UDS package-image context is packaging metadata, not a new runtime gateway, route, controller or admission exemption; pinned UDS integration remains a hosted gate. Source-only approval: acceptance run 767 (API 7353) initially failed before checkout because its runner could not create the pinned job image ('No such image'), not because a source/security test failed; at review it had been natively retried and was waiting. Repository security and Library checks were still running/pending, NOT passed. Require successful exact-head checks, including actual runner startup and error-free scans, before merge/release. Hosted consumer expansion, image scans/acceptance, GitchAlerts image remediation and deployment are separate; no merge or publication authorized.CI at submission: pending. This is a source-review approval, not a scan verdict; required checks still block merge.
Resolved the README merge conflict against current main in
cf054151e8. The merge retains both the bundle validation/image-check entry and the Node CI entry, including the owner’s prior branch updates.Validation: the complete resulting Git tree is identical to the previously passing
9ad31f8head; there are no unmerged files or conflict markers, diff checks passed, and the active native Gitleaks 8.30.1 hook passed. No workflow/job/status names or source behavior changed. Forgejo now shows no merge-conflict banner. Fresh head-bound acceptance, repository-security and Library checks are queued (775–777) and remain required before merge.Source review at
cf054151e8: I inspected the authenticated live changed workflows, source gate, image gate, shared delivery/setup changes, tests, documentation, prior reviews and the conflict-resolution comment. The previous zero-based SBOM defect remains corrected: the full-image decision requires the complete sorted image inventory, clean malware evidence, and regular CycloneDX SBOMs for each zero-based image index; the actual preparation/security.scan fixture covers one and two distinct digest-pinned images and rejects loss of only the second SBOM. Source checks require a confined, nonempty caller script declaration and receive no registry/delivery credentials; the image workflow guards same-repository PR access before its scan-only registry credentials. The README retains both bundle and Node entries after conflict resolution. The shared UDS image-package metadata does not introduce a new runtime gateway, route, admission exemption or controller; pinned UDS packaging remains a separate hosted gate. No source-level blocker found in the inspected head. This is source approval only: workflow acceptance, repository security and Library checks at this exact SHA are pending, not passed. Hosted reusable-workflow expansion, emitted required-check names, consumer startup/full-image acceptance, and the separately blocked GitchAlerts image remediation remain unverified. Preserve all gates; do not merge, release, promote or adopt consumers on this review alone.CI at submission: pending. This is a source-review approval, not a scan verdict; required checks still block merge.