Final Architecture

Updated 2026-10-08. Reconciled against instance architecture, platform state, and applications. The owner confirmed PostgreSQL remains outside Kubernetes and cancelled the Technitium migration in favor of keeping Pi-hole on 2026-10-08. This is a dated architecture record, not a fresh live-system audit. Instance documentation remains the source of truth; its older Technitium selection is superseded by this owner decision.

flowchart TB
    LAN["LAN clients"]
    Public["Internet clients"]
    DNS["Pi-hole — retained LAN DNS and ad blocking"]
    CF["Cloudflare — public DNS / DNS-01"]
    Relay["Sager TCP relay — target edge: 10.0.0.107"]
    LAN -->|DNS queries| DNS
    Public -. Global cutover unconfirmed .-> Relay

    subgraph Cluster["RKE2 / UDS Kubernetes cluster"]
        subgraph Nodes["Physical nodes"]
            Nuc["NucBox G5 — single control plane / etcd"]
            Sager["Sager — general worker"]
            Opti["OptiPlex 5070 — worker / game-server target"]
            Media["Media — GPU worker / RTX 3080 Ti"]
        end
        Tenant["MetalLB 10.0.0.241 — Istio tenant gateway"]
        Admin["MetalLB 10.0.0.240 — Istio admin gateway / LAN only"]
        Apps["Production: Actual Budget / GitchAlerts"]
        Comfy["ComfyUI — production / node-local models and state"]
        Auth["UDS Keycloak / GitchAuth theme"]
        Bao["OpenBao"]
        ESO["ESO — scoped Kubernetes Secrets"]
        Cert["cert-manager"]
        Storage["PV/PVC + TrueNAS CSI — NFS / iSCSI"]
        Obs["Loki / Velero"]
        Runners["KEDA Linux runners — separate ci / deploy pools"]
        Renovate["Renovate CronJob — schedule enabled Oct 4"]
        Doc["Documentation site — prepared / deployment unconfirmed"]
        Tenant --> Apps
        Tenant --> Comfy
        Tenant --> Auth
        Tenant -. Prepared .-> Doc
        Admin --> Bao
        Bao --> ESO
        ESO --> Apps
        ESO --> Runners
        ESO --> Renovate
        Cert -->|Gateway TLS| Tenant
        Cert -->|Gateway TLS| Admin
        Apps --> Storage
        Media --- Comfy
    end

    subgraph External["Outside Kubernetes"]
        subgraph NAS["TrueNAS SCALE"]
            PG[("PostgreSQL 18 + pgvector — separate databases and roles")]
            Files[("NFS / iSCSI datasets — application state / media")]
            S3[("S3-compatible storage — MinIO / AIStor")]
            Retained["Retained services — ReadMeABook / NocoDB / media acquisition"]
        end
        Forgejo["Forgejo — source / issues / Actions / registry / signed releases"]
        Windows["Windows Hyper-V runner"]
        NPM["Legacy NPM — 10.0.0.116:443"]
        DNS
    end

    LAN -->|Application HTTPS| Tenant
    LAN -->|Administrative HTTPS| Admin
    Relay -. Target TCP 80/443 .-> Tenant
    Public -. Last recorded NAT baseline .-> NPM
    Tenant -->|Explicit legacy SNI passthrough| NPM
    Auth -->|Database| PG
    Apps -->|GitchAlerts database| PG
    Forgejo -->|External database| PG
    Storage --> Files
    Apps -->|GitchAlerts assets| S3
    Obs --> S3
    Retained --> Files
    CF -->|DNS-01| Cert
    Forgejo -->|Jobs| Runners
    Forgejo -->|Windows jobs| Windows
    Runners -->|Reviewed signed UDS delivery| Apps
    Renovate -->|Dependency updates| Forgejo

    classDef external fill:#ecfdf5,stroke:#059669,color:#064e3b;
    classDef planned fill:#fffbeb,stroke:#d97706,color:#78350f,stroke-dasharray:5 5;
    class PG,Files,S3,Forgejo,Windows,NPM,Retained,DNS external;
    class Relay,Doc planned;

Dashed paths are planned or unconfirmed. The global public NAT/LAN DNS handoff is not recorded complete; the admin VIP remains LAN-only. Published or prepared runner images do not establish live adoption.

Client traffic and DNS

1. Network clients query the configured LAN resolver for application or administrative domains. Pi-hole remains the selected LAN DNS and ad-blocking service; the owner cancelled the Technitium replacement on 2026-10-08. Preserve and validate the actual client/DHCP DNS path.
2. The resolver resolves application domains to the Tenant Gateway VIP and administrative domains to the Admin Gateway VIP.
3. Clients send traffic directly to the resolved VIP. The DNS resolver does not proxy or forward the application traffic.
4. The bare-metal LoadBalancer provides the external VIPs and delivers traffic to the corresponding Istio gateway.
5. Istio routes traffic through its Envoy data plane to UDS Core applications and services.

The bare-metal load-balancing implementation uses MetalLB in Layer 2 mode for application Services. A reserved LAN pool supplies stable service VIPs, and node-scoped advertisements restrict announcements to each eligible node's wired interface. Exact addresses remain environment-specific. kube-vip may be added later for a stable RKE2 registration and Kubernetes API address without replacing MetalLB for application Services.

UDS ingress and service mesh

Normal UDS HTTP/HTTPS ingress is provided by Istio:

UDS Core
└── Istio
    ├── Tenant Gateway
    │   └── Envoy proxy
    └── Admin Gateway
        └── Envoy proxy

The diagram does not introduce a separate Envoy Gateway as the normal HTTP/HTTPS ingress controller. A separate Envoy Gateway may be evaluated later for UDP workloads if required.

Shared edge during migration

The tenant gateway is 10.0.0.241; the admin gateway is 10.0.0.240 and must remain LAN-only. Production NAT and LAN DNS are intended to target the tenant gateway. The cutover is still open in platform#1; do not infer completion from deployed routing resources.

Istio terminates TLS for native routes and dev hostnames. Known legacy production hosts use an explicit TLS/SNI allowlist and pass through to NPM at 10.0.0.116:443, which retains TLS termination for those hosts. Unknown hosts have no wildcard NPM fallback. HTTP on configured application hosts redirects to HTTPS.

The 29-host deployed baseline and validation script live under infrastructure/istio/npm-legacy in Homelab/platform. Dated records report dev DNS/valid TLS and legacy tests; cert-manager issuance and trusted chains are recorded in the current instance documentation. Future renewal and global edge handoff still require verification. See the per-host handoff and rollback procedure.

Repository, deployment and secrets boundaries

The public wiki and sanitized boilerplate remain in gitchegumi/homelab. Homelab/platform owns shared infrastructure; application repositories own application packages; workflows owns reusable CI/CD; bundles owns compositions across repositories. Homelab Roadmap shows their issues on one board.

Use tested immutable UDS artifacts for routine deployment; raw manifests are transitional migration/testing inputs. OpenBao is the selected runtime secrets backend, ESO supplies Kubernetes Secrets, and Forgejo Actions secrets supply CI authentication. OpenBao/ESO deployment and managed credential issuance are recorded in the current instance documentation; independent persistence, unseal and recovery still require their own acceptance evidence. The local TrueNAS CSI secret manifest remains a bootstrap artifact and must never be committed. See delivery architecture and Security and Secrets.

Foundational services outside Kubernetes

• LAN DNS: remains independent and provides resolution to the gateway VIPs; Pi-hole is retained; no Technitium migration is planned (owner decision, 2026-10-08).
• MinIO: remains outside Kubernetes initially as S3-compatible object storage for Loki, Velero, or other UDS dependencies where required.
• Forgejo: remains outside Kubernetes initially and authoritative for repository content, issues, and future Forgejo Actions workflows. It may still share physical NucBox hardware with the control-plane node.
• Windows Forgejo runner: remains on its Windows Hyper-V VM to provide true Windows execution.
• Linux Forgejo runners: global disposable Kubernetes pools are live, with cutover confirmed by the owner on 2026-09-10. Standard ci and deploy labels share four CI slots plus one deployment slot across Sager, Optiplex and Media, scaling to zero. Worker bundle 0.1.2 uses KEDA 2.20.2; credentials are supplied by OpenBao/ESO, with instance-admin read:admin for global queue polling. See Linux Runner Migration for acceptance and recovery.

RKE2 nodes

• NucBox G5: RKE2 server, control plane, and etcd; tainted against ordinary application scheduling.
• Sager: general worker, secondary game-server worker, Jellyfin candidate, and possible GTX 1070 workload host. Jellyfin cache/transcode remains local if placed here.
• OptiPlex 5070: primary game-server worker. Existing local PVs may be used initially, with shared TrueNAS-backed game storage planned so workloads can also schedule on the Sager where practical.
• Media server: GPU worker with the RTX 3080 Ti. ComfyUI remains bound here because its GPU and approximately 786 GB local model dataset are hardware-dependent.

Persistent storage boundaries

Kubernetes owns storage objects and drivers:

• Static PVs/PVCs: bind existing TrueNAS NFS exports.
• TrueNAS CSI driver: dynamically provisions NFS or iSCSI storage and uses the TrueNAS API.
• Local PVs: represent node-local ComfyUI assets and any initial local game-server data without arbitrary hostPath mounts.

TrueNAS remains the storage appliance and exposes:

• Existing datasets
• NFS exports backed by existing datasets
• Dynamic CSI-backed NFS and iSCSI storage

Static PV/PVC objects are Kubernetes resources; they do not reside on TrueNAS.

Application dependencies

• Keycloak → PostgreSQL: uses the established external PostgreSQL service on TrueNAS with a dedicated database and role; preserve native database backups and independent recovery.
• Loki / Velero / UDS packages → MinIO: S3-compatible object storage is consumed as required while MinIO remains outside Kubernetes initially.
• Persistent workloads → TrueNAS: use static NFS or dynamically provisioned shared-file/block storage according to workload semantics.

Intentionally unresolved

• Exact installed package/image digests and UDS flavor for reproducible adoption; baseline versions are recorded in the readiness runbook
• Remaining production DNS/NAT cutover and validation of the retained Pi-hole client/DHCP path
• Future control-plane API VIP implementation
• Independent OpenBao persistence, unseal and tested recovery coverage
• ESO recovery coverage and future cert-manager renewal verification
• Exact installed CSI artifact pin and non-secret configuration capture; the official driver and truenas-nfs/truenas-iscsi classes are already established
• Final Jellyfin placement
• Whether a separate Envoy Gateway is required later for UDP
• Linux Forgejo runner migration completed on owner acceptance (2026-09-10); routine operations are documented in platform.

These decisions should be captured in issues and architecture decision records before implementation.