MetalLB Layer 2 Load Balancing

This runbook installs and configures MetalLB in Layer 2 mode for a bare-metal Kubernetes or RKE2 cluster. It provides stable LAN addresses for Kubernetes Services of type LoadBalancer without requiring BGP-capable network equipment.

Replace every placeholder with values from the target environment before running commands or applying manifests.

Architecture and boundaries

In Layer 2 mode, one eligible MetalLB speaker announces each service VIP to the local network using ARP for IPv4 or NDP for IPv6. Kubernetes then routes traffic from the announcing node to the Service endpoints.

LAN client
  → DNS record
  → MetalLB service VIP
  → elected wired cluster node
  → Kubernetes LoadBalancer Service
  → application or ingress pods

MetalLB provides service address allocation and LAN advertisement. It does not provide:

• HTTP routing
• DNS record management
• TLS termination
• Internet-facing NAT or firewall policy
• A highly available Kubernetes API address

An ingress gateway such as the UDS/Istio tenant or admin gateway handles application-layer routing. A future kube-vip deployment may provide a stable Kubernetes API address without replacing MetalLB for application Services.

Placeholders

Placeholder	Purpose
<METALLB_VERSION>	Pinned stable MetalLB release, including the leading v
<LAN_SUBNET>	LAN CIDR containing the cluster and service addresses
<LAN_GATEWAY_IP>	Default gateway used to identify the preferred LAN interface
<DHCP_START> / <DHCP_END>	Router-managed DHCP range
<METALLB_POOL_START> / <METALLB_POOL_END>	Reserved service-address range
<NODE_1_NAME>	Kubernetes hostname label for the first eligible node
<NODE_1_WIRED_INTERFACE>	Wired LAN interface on the first eligible node
<NODE_2_NAME>	Kubernetes hostname label for the second eligible node
<NODE_2_WIRED_INTERFACE>	Wired LAN interface on the second eligible node
<MANIFEST_DIRECTORY>	Local directory containing the tracked configuration
<ASSIGNED_TEST_VIP>	Address allocated to the temporary validation Service

Add another node/interface pair for every additional node that should participate in Layer 2 failover.

Requirements

• A healthy Kubernetes or RKE2 cluster
• Administrative kubectl access
• Layer 2 connectivity between intended clients and the cluster LAN interfaces
• A service-address range excluded from DHCP and existing static assignments
• TCP and UDP port 7946 permitted between MetalLB speaker nodes
• One known wired LAN interface on every eligible announcing node
• No other controller claiming the same LoadBalancer Services or address pool

Check for existing load-balancer implementations before installation:

kubectl get services --all-namespaces
kubectl get pods --all-namespaces | grep -Ei 'metallb|kube-vip|svclb'

If kube-proxy runs in IPVS mode, review MetalLB's strictARP requirement before continuing.

Reserve the service-address range

1. Record the LAN subnet, gateway, DHCP start, DHCP end, static reservations, and manually assigned addresses.
2. Select a contiguous range large enough for ingress gateways and any Services that need distinct VIPs.
3. Change the DHCP scope so it cannot allocate addresses from the proposed MetalLB range.
4. Move or renew any existing DHCP clients that still hold leases inside the proposed range.
5. Check the router's active-client and reservation lists.
6. Scan the complete proposed range while the relevant devices are online.

Example discovery scan:

sudo nmap -sn <METALLB_POOL_START>-<METALLB_POOL_END>

Do not configure the pool until the router configuration excludes it and the range is confirmed unused. A scan alone is insufficient because offline devices and static assignments may not respond.

Install MetalLB

Use a pinned stable release instead of an unversioned branch:

kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/<METALLB_VERSION>/config/manifests/metallb-native.yaml

Wait for the controller and speakers:

kubectl rollout status deployment/controller -n metallb-system
kubectl rollout status daemonset/speaker -n metallb-system
kubectl get pods -n metallb-system -o wide

Expected state:

• One controller pod is Ready.
• One speaker pod is Ready on every schedulable cluster node.
• Existing nodes remain Ready.

Identify eligible wired interfaces

Confirm the Kubernetes hostname labels:

kubectl get nodes -L kubernetes.io/hostname

On each node, identify the interface used to reach the LAN gateway:

ip route get <LAN_GATEWAY_IP>
ip -br -4 address

Use the interface reported by the gateway route as the wired advertisement interface. The Kubernetes node IP does not, by itself, restrict which interface MetalLB uses.

Explicit interface selection is particularly important on nodes that also have Wi-Fi. Wi-Fi may remain enabled; only the wired interface is listed in the MetalLB advertisement.

Configure the address pool and advertisements

Store the configuration at:

<MANIFEST_DIRECTORY>/metallb-config.yaml

Create one IPAddressPool and one node-scoped L2Advertisement for each eligible node:

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: lan-pool
  namespace: metallb-system
spec:
  addresses:
    - "<METALLB_POOL_START>-<METALLB_POOL_END>"
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: lan-node-1
  namespace: metallb-system
spec:
  ipAddressPools:
    - lan-pool
  nodeSelectors:
    - matchLabels:
        kubernetes.io/hostname: <NODE_1_NAME>
  interfaces:
    - <NODE_1_WIRED_INTERFACE>
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: lan-node-2
  namespace: metallb-system
spec:
  ipAddressPools:
    - lan-pool
  nodeSelectors:
    - matchLabels:
        kubernetes.io/hostname: <NODE_2_NAME>
  interfaces:
    - <NODE_2_WIRED_INTERFACE>

Repeat the L2Advertisement pattern for additional eligible nodes, using a unique resource name and the correct hostname/interface pair.

Avoid leaving a broad advertisement that references the same pool without node or interface restrictions. MetalLB combines advertisements referencing a pool, so an unrestricted resource can re-enable unwanted interfaces.

Apply and inspect the configuration:

kubectl apply -f <MANIFEST_DIRECTORY>/metallb-config.yaml

kubectl get ipaddresspools,l2advertisements -n metallb-system
kubectl get l2advertisements -n metallb-system -o yaml

Verify that:

• The pool contains only the reserved service range.
• Every node selector matches an existing Kubernetes hostname.
• Every listed interface exists on its selected node.
• No wireless interface is listed.
• No unintended advertisement references the pool.

Validate with a temporary Service

Create a disposable nginx workload:

kubectl create namespace metallb-test

kubectl create deployment web   --namespace metallb-test   --image=nginx:alpine

kubectl expose deployment web   --namespace metallb-test   --name=web   --type=LoadBalancer   --port=80

Watch the pod and Service separately:

kubectl get pods -n metallb-test -o wide --watch
kubectl get services -n metallb-test -o wide --watch

After MetalLB assigns an external address, test it from a different machine on the intended LAN:

curl http://<ASSIGNED_TEST_VIP>
kubectl describe service web -n metallb-test

Successful validation includes:

• An external IP from the configured pool
• An IPAllocated event from metallb-controller
• A nodeAssigned event identifying an eligible Layer 2 announcer
• A successful HTTP response from the LAN client
• A healthy workload endpoint

Remove the test resources after validation:

kubectl delete namespace metallb-test

Optional failover validation

Perform this only during a maintenance window because clients may experience a brief interruption.

1. Deploy the temporary validation Service.
2. Identify the announcing node from kubectl describe service.
3. Find the speaker pod running on that node.
4. Continuously request the test VIP from a LAN client.
5. Restart the elected speaker pod and observe another eligible node take ownership.

kubectl get pods -n metallb-system -o wide
kubectl delete pod -n metallb-system <SPEAKER_POD_ON_ANNOUNCING_NODE>
kubectl describe service web -n metallb-test

The speaker DaemonSet recreates the pod. Confirm that connectivity returns and the Service event identifies a new eligible announcer.

UDS and ingress integration

UDS Core normally exposes separate tenant and admin Istio gateways through Services of type LoadBalancer. MetalLB allocates and advertises the corresponding VIPs.

Recommended boundaries:

• Internal tenant DNS names point to the tenant gateway VIP.
• Internal administrative DNS names point to the admin gateway VIP.
• The admin gateway remains internal unless a separate security decision explicitly approves external exposure.
• Public DNS, NAT, edge firewall policy, and certificate management remain separate decisions.
• Reserve enough service addresses for both gateways and future Services.
• Keep externalTrafficPolicy: Cluster unless there is a deliberate reason to use Local.

With externalTrafficPolicy: Cluster, an announcing node can forward traffic to gateway pods elsewhere in the cluster. If Local is selected, ensure eligible announcers also run local gateway endpoints; otherwise MetalLB may exclude nodes or the Service may become unreachable.

Routine operations

Inspect the installation:

kubectl get pods -n metallb-system -o wide
kubectl get ipaddresspools,l2advertisements -n metallb-system -o wide
kubectl get services --all-namespaces

Inspect allocation and announcement events:

kubectl describe service <SERVICE_NAME> -n <SERVICE_NAMESPACE>
kubectl get events --all-namespaces --sort-by=.lastTimestamp

When adding a cluster node:

1. Confirm the node is Ready.
2. Identify its wired LAN interface.
3. Add a dedicated node/interface L2Advertisement.
4. Apply the tracked manifest.
5. Confirm the speaker pod and advertisement.
6. Repeat reachability or maintenance-window failover validation.

When replacing a NIC or renaming an interface, update the corresponding advertisement before relying on that node for failover.

Troubleshooting

Service remains Pending

Check:

kubectl describe service <SERVICE_NAME> -n <SERVICE_NAMESPACE>
kubectl get ipaddresspools,l2advertisements -n metallb-system -o yaml
kubectl logs -n metallb-system deployment/controller

Confirm that a pool exists, auto-assignment is enabled when expected, addresses remain available, and no competing controller is claiming the Service.

VIP is allocated but unreachable

Confirm:

• The client and announcing interfaces share the intended Layer 2 network.
• The selected node is Ready and has a healthy speaker.
• The selected wired interface exists and is up.
• The address is excluded from DHCP and unused elsewhere.
• Host and network firewalls allow the Service traffic.
• Service endpoints are Ready.
• externalTrafficPolicy: Local is not excluding all eligible announcers.

Useful commands:

kubectl describe service <SERVICE_NAME> -n <SERVICE_NAMESPACE>
kubectl get endpoints <SERVICE_NAME> -n <SERVICE_NAMESPACE>
kubectl get pods -n metallb-system -o wide
kubectl logs -n metallb-system <SPEAKER_POD>

VIP is advertised from an unexpected interface

List every advertisement referencing the pool:

kubectl get l2advertisements -n metallb-system -o yaml

MetalLB uses the union of advertisements referencing a pool. Remove or restrict any older broad advertisement, then reapply the intended manifest.

Upgrade

1. Review MetalLB release notes and compatibility guidance.
2. Update <METALLB_VERSION> to a tested stable release.
3. Apply the pinned upstream manifest.
4. Wait for controller and speaker rollouts.
5. Confirm the custom resources and test an existing VIP.

kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/<METALLB_VERSION>/config/manifests/metallb-native.yaml
kubectl rollout status deployment/controller -n metallb-system
kubectl rollout status daemonset/speaker -n metallb-system

Do not deploy directly from an unversioned development branch.

Rollback

Before removing MetalLB, identify every Service using an allocated VIP and provide an alternate access path.

Remove the local configuration:

kubectl delete -f <MANIFEST_DIRECTORY>/metallb-config.yaml

Then remove the same pinned release manifest used for installation:

kubectl delete -f https://raw.githubusercontent.com/metallb/metallb/<METALLB_VERSION>/config/manifests/metallb-native.yaml

Removing MetalLB makes its external VIPs unavailable. It does not automatically redesign ingress, DNS, NAT, or application access.

Security and repository hygiene

Never publish:

• Real internal address ranges when the repository is intended to remain generic
• Credentials, kubeconfigs, private keys, or node tokens
• Router administration details
• Public DNS provider credentials
• Environment-specific firewall secrets or API tokens

Use placeholders in reusable documentation. Keep operational values in an approved private configuration source.

References

• MetalLB installation
• MetalLB Layer 2 concepts
• MetalLB advanced Layer 2 configuration
• UDS production deployment guidance