fix(security): verify Semgrep rules across registry encodings #69

Merged

gitchegumi merged 1 commit from kiyi/canonical-semgrep-rules into main

2026-09-30 22:17:04 -05:00

kiyi commented

2026-09-30 20:50:02 -05:00

Member

Failure and evidence

Renovate run 28 failed source security with zero blockers, zero mitigations and one Semgrep checksum error. Its image job was correctly skipped by the new source prerequisite. Tracks #4 and Homelab/renovate#6; does not close the broader security issue.

The reviewed v1.25.1 response was JSON (433,040 bytes, raw SHA256 5509b33e...). Three current urllib downloads returned YAML (473,625 bytes, raw SHA256 b109a039...). Comparing every field of all 225 rules by ID finds no additions, removals or changes. JSON includes missed: 0; YAML omits that zero envelope count. The changing serialization explains why changing only the raw checksum is unreliable. CI did not retain its raw response, so its exact encoding is not independently established.

Change

Add canonical-rules-v1 verification, paired with the updated library rule lock. Safely parse the pack and hash every complete rule object. Normalize mapping keys and inventory order only; preserve inner list order, patterns, severity, languages, paths, options, messages and metadata.
Reject duplicate mapping keys/IDs, empty or changed inventories, unknown envelope fields, incomplete/malformed missed counts, unsupported modes and changed rule content. Emit expected/observed canonical digests on drift.
Pass exactly the verified JSON representation to Semgrep, preventing differences between the verifier and scanner YAML parsers. Real rule changes still require review. No licensed rule payload is committed or redistributed.
Keep mitigation selectors and policy unchanged: stable CVE/GHSA or scanner rule IDs plus affected scope. Caller inputs, release evidence and publication/deployment gates are unchanged.

Validation

54 local rule-integrity, security-policy, scanner-diagnostic and reporting tests pass; diff checks pass. Synthetic fixtures cover both encodings, reordered inventories, changed rule fields/counts, duplicate keys and incomplete/malformed packs. Source-scan integration mocks external execution and confirms changed rules never reach Semgrep and produce a failed report.

The new verifier accepts the retained reviewed JSON and all three current YAML responses as exactly 225 rules with canonical SHA256 5bc39e23a7580c9ba5c09b08f5a2b310f4648e5b41636ec49c3c08af7871aa73. Real hosted Library checks, source scans and secret-rejection fixtures remain required; no local container/scanner run or live deployment is claimed.

Release and Renovate adoption

Review this PR and require both hosted Library checks and Repository security to pass before merge.
Publish the next unused patch release through Publish workflow release from protected main. Keep existing tags immutable.
Adopt that released tag in Renovate PR #6 across paired workflow/library pins and source guards. The lock/verifier are library-owned and move together.
Verify Renovate source completes, followed by the temporary image gate; retain its required checks and resolve remaining image findings before merge/release. This PR does not make the existing blocked Renovate image pass.

Commit: 818cbdafa0eb9153f09743f0b617c1c217b5ddcb. See docs/security.md for the dated evidence and exact hashing contract.

## Failure and evidence [Renovate run 28](https://git.gitchegumi.com/Homelab/renovate/actions/runs/28/jobs/2/attempt/1) failed source security with zero blockers, zero mitigations and one Semgrep checksum error. Its image job was correctly skipped by the new source prerequisite. Tracks #4 and Homelab/renovate#6; does not close the broader security issue. The reviewed v1.25.1 response was JSON (433,040 bytes, raw SHA256 `5509b33e...`). Three current urllib downloads returned YAML (473,625 bytes, raw SHA256 `b109a039...`). Comparing every field of all 225 rules by ID finds no additions, removals or changes. JSON includes `missed: 0`; YAML omits that zero envelope count. The changing serialization explains why changing only the raw checksum is unreliable. CI did not retain its raw response, so its exact encoding is not independently established. ## Change - Add `canonical-rules-v1` verification, paired with the updated library rule lock. Safely parse the pack and hash every complete rule object. Normalize mapping keys and inventory order only; preserve inner list order, patterns, severity, languages, paths, options, messages and metadata. - Reject duplicate mapping keys/IDs, empty or changed inventories, unknown envelope fields, incomplete/malformed `missed` counts, unsupported modes and changed rule content. Emit expected/observed canonical digests on drift. - Pass exactly the verified JSON representation to Semgrep, preventing differences between the verifier and scanner YAML parsers. Real rule changes still require review. No licensed rule payload is committed or redistributed. - Keep mitigation selectors and policy unchanged: stable CVE/GHSA or scanner rule IDs plus affected scope. Caller inputs, release evidence and publication/deployment gates are unchanged. ## Validation 54 local rule-integrity, security-policy, scanner-diagnostic and reporting tests pass; diff checks pass. Synthetic fixtures cover both encodings, reordered inventories, changed rule fields/counts, duplicate keys and incomplete/malformed packs. Source-scan integration mocks external execution and confirms changed rules never reach Semgrep and produce a failed report. The new verifier accepts the retained reviewed JSON and all three current YAML responses as exactly 225 rules with canonical SHA256 `5bc39e23a7580c9ba5c09b08f5a2b310f4648e5b41636ec49c3c08af7871aa73`. Real hosted Library checks, source scans and secret-rejection fixtures remain required; no local container/scanner run or live deployment is claimed. ## Release and Renovate adoption 1. Review this PR and require both hosted Library checks and Repository security to pass before merge. 2. Publish the next unused patch release through **Publish workflow release** from protected main. Keep existing tags immutable. 3. Adopt that released tag in [Renovate PR #6](https://git.gitchegumi.com/Homelab/renovate/pulls/6) across paired workflow/library pins and source guards. The lock/verifier are library-owned and move together. 4. Verify Renovate source completes, followed by the temporary image gate; retain its required checks and resolve remaining image findings before merge/release. This PR does not make the existing blocked Renovate image pass. Commit: `818cbdafa0eb9153f09743f0b617c1c217b5ddcb`. See docs/security.md for the dated evidence and exact hashing contract.

kiyi added 1 commit

2026-09-30 20:50:02 -05:00

fix(security): verify Semgrep rules across registry encodings

Workflow acceptance / acceptance (pull_request) Successful in 1m42s

Details

Repository security / security-1 (pull_request) Successful in 6m51s

Details

Library checks / tests (pull_request) Successful in 7m13s

Details

818cbdafa0