Application Delivery Automation

Scope: reusable deployment guidance. This is not a claim that any particular environment has completed implementation. See Homelab Operations Status for this environment's actual status. Replace all example roles, hosts and namespaces with reviewed values.

Operator experience

The operator selects dev or prod and runs the deployment Action. Testing, approval and restart/persistence acceptance can remain manual. Credentials, prerequisite validation, release download, signature verification, scans, storage/edge readiness, deployment and health checks belong in the Action. A retry should be a fresh authorized dispatch when the platform cannot securely identify the actor of a job rerun.

Automatic app registration (prepared 2026-09-11, not installed)

The owner clarified that automation includes initial deployment identity setup: new apps must not require hand-written permission contracts, OpenBao roles or RBAC. The prepared implementation adds one trusted platform reconciliation job. It discovers opted-in protected Homelab repositories, reads the existing dev/prod metadata at an immutable commit, and generates separate environment identities from a versioned platform permission profile. Contracts are delivered through OpenBao, so adding an app does not require a new shared-workflow release.

The shared managed credential mode derives the app identity, waits for first registration, verifies the platform contract and effective Kubernetes permissions, then runs the existing signed-artifact deployment. The template supplies the opt-in declaration. Namespace recovery belongs to the platform job; application jobs do not administer RBAC. The one-time platform installation grants a privileged control-plane identity and requires review before enabling reconciliation.

See the platform implementation and installation and workflow integration. Draft proposals are published: platform#39, workflows#38, GitchAlerts#5, and bundles#9. Local unit and contract checks pass; installation and live acceptance remain pending. Existing Actual identities remain unchanged. Runtime database/secret setup, existing-data adoption and production routing handoff are not claimed automated by this identity change.

Credentials without recurring manual renewal

Forgejo Actions can obtain a job-specific OIDC identity token. OpenBao validates that signed identity and issues a Kubernetes token for the approved application's existing ServiceAccount. Tokens may expire quickly internally; obtaining and replacing them is automatic, so the operator has no per-deployment key maintenance.

Configure trust from observed issuer, subject, audience, repository, protected ref and workflow claims. Reusable workflow identities must be measured on the installed Forgejo version. Do not assume GitHub claim names or permissions syntax apply. Restrict token issuance to approved repositories/workflows; a caller-selected role string is not authorization by itself.

OpenBao should use its automatically rotated in-cluster ServiceAccount credential for Kubernetes TokenRequest operations, with permission only for approved deployment identities. Prefer organization-level non-secret configuration; do not store an administrator kubeconfig or bootstrap root token in organization secrets. Any genuinely necessary long-lived integration secret needs documented ownership and a maintenance schedule of at least 90 days where provider limits permit.

Build once and promote

Sign the immutable application artifact and test it in dev. Production must identify the same approved artifact digest; it must not silently build another payload. Environment-specific hostnames, SSO settings and storage references need an explicit versioned deployment contract. This is especially important when an archive already embeds environment configuration: merely changing a dropdown does not implement promotion.

Distinguish a migration of one existing data instance from parallel dev/prod installations. A stateful migration using one retained dataset must keep one writer. Separate environments need separate datasets; selecting dev must never accidentally attach a second application instance to production data.

Prerequisites and certificates

Provision or verify storage before scheduling the app. Existing volumes require identity and lifecycle checks; do not delete/rebind storage to force a retry. Cert-manager Certificate resources manage issuance and renewal, while gateways reference the managed TLS Secret. Validate readiness and renewalTime, observe a renewal, and verify the listener reloads the new certificate. Monitor failed issuance and approaching expiry. Renewal belongs to the certificate controller, not a person copying certificates into Actions secrets.

Evidence and failure behavior

Record artifact and configuration digests, environment, workflow identity, health results and failure stage without retaining tokens, kubeconfigs or secret payloads in logs/artifacts. Authentication failures stop deployment. Rollback of routing/application configuration is distinct from database restoration. Manual acceptance results should identify who confirmed them and when, rather than being labeled automated tests.

Homelab finding: Forgejo 16.0.3 (2026-09-08)

Environment-specific verified finding. Actual run #87 successfully inspected OIDC after workflows v1.4.2. Forgejo 16.0.3 hardcodes ref_protected to false in services/actions/context.go, then copies it into the signed claims. This field cannot attest real branch protection on that version. The run passed the separate authenticated API protection/owner-dispatch gate.

Platform PR #29 corrects the role generator to omit that unsupported binding while keeping exact issuer, subject, audience, repository/owner, main ref, event and workflow restrictions. The PR is merged and live OpenBao bootstrap and repeated credential issuance have succeeded. See the operations status page for deployment acceptance and remaining work. OpenBao validates signed identity but does not independently query branch protection or verify workflow contents. Protect the approved main-branch workflow and its library pin; the pinned workflow enforces protection through the API before issuance. Reassess claim support on future Forgejo upgrades. No production application change was made by inspection.

Primary references

Released environment promotion and app upgrades

The Homelab/workflows v1.5.0 contract builds reviewed dev and prod profiles into one signed release with the same image inventory. The deployment Action selects the corresponding package. Applications must adopt that contract; changing only an environment input does not retrofit an older single-environment artifact.

For an upgrade, update the pinned upstream image and profile versions in the application repository, run its validation/security checks, and merge the reviewed change. Build and publish one signed release, deploy that release to dev, then select prod with the same release after acceptance and a production backup. Disposable empty dev storage tests startup and integration; it does not test upgrading an existing production database.

Namespace cleanup and deployment permissions

Deleting a namespace also deletes its namespaced Roles and RoleBindings. Keeping deployment identities in another namespace does not preserve those grants. The current shared credential contract checks permissions for both environments, so deleting dev can block a prod dispatch. Restore the reviewed platform RBAC before retrying, or preserve the namespace and its RBAC when cleaning up disposable workloads. Restoring RBAC alone does not deploy the application.

Bringing manual changes under deployment management

A server-side apply conflict can indicate fields still owned by earlier manual kubectl operations. Inspect the exact object, field and manager, preserve a backup, and reconcile ownership with the reviewed desired configuration. A targeted one-time ownership repair may be needed after a manual migration; it is not a routine upgrade step. Keep subsequent routing, SSO and application configuration changes in the repository and Actions. Do not uninstall a healthy stateful application or delete its volume to resolve a field-ownership conflict.

A failed Helm rollback does not establish that the running application is down. Inspect pods, availability and release history separately. A retry that stops at credential preflight cannot prove the earlier deployment conflict is repaired. See Homelab Operations Status for the Actual-specific evidence and recovery command.

Namespace independence follow-up (not yet released)

Workflows #31 and platform #31 implement selected-environment checks and an optional namespace/binding bootstrap from the independently reviewed library contract. Identity and permission definitions live outside disposable namespaces. The namespace-local binding is recreated when necessary, referencing a durable ClusterRole. Production does not bootstrap dev. Credential validation mode stays read-only.

The platform bootstrap uses named grants plus Fail/Deny admission restrictions because RBAC alone cannot limit create requests to an object name. Install and verify the platform policy before granting bootstrap access, then release the library and update callers. Each future app needs reviewed namespace/role/subject entries; the workflow implementation is reusable. Existing workloads, storage and production routing are not changed by installing the bootstrap definitions. Live recreation and admission enforcement acceptance remain pending; see Homelab Operations Status for current evidence.

ComfyUI validation path (prepared 2026-09-18)

ComfyUI PRs run fast tests and source security without rebuilding a disposable CUDA image. The protected publisher builds/scans one publishable image; deploy its signed bundle to dev and validate startup, all 17 declared node imports, GPU workflows and group SSO there. Fix failures in a PR and publish a new candidate. Promote the same accepted release to production. Image security, signing and owner-dispatched publication/deployment remain unchanged.

Comfy-UI PR #3, commit 7228fb99ef, removes the redundant image/CPU-smoke job and its helpers. This is prepared configuration, not a successful image publication or deployment. Publisher network-release adoption, storage/onboarding and live acceptance remain tracked in Comfy-UI #1.

Homelab Wiki