home-dc-kubernetes, Part 4: The Platform Layer — Cilium, Envoy Gateway, Ceph CSI, Cloudflare

The plumbing layer #

In Part 3 we looked at how Argo CD decides what to run and where. Now the platform layer — the pieces that make workloads actually work: networking, ingress, storage, and external access.

This is the layer where most homelab setups quietly fall apart. It’s not the apps that are hard — it’s the plumbing. A CNI that fights the node’s routes, an ingress that can’t get traffic in, storage that dies with the node it lives on. This setup was designed so each piece has one job and does it well.

1. Cilium: the network brain #

The app cluster runs Cilium with the built-in Talos CNI disabled from day one (see Part 2). Cilium here isn’t just “a CNI” — it’s doing three jobs at once:

• Native routing mode with ipv4NativeRoutingCIDR: 10.42.0.0/16 — pod traffic routes natively on the LAN, no overlay encapsulation.
• Full kube-proxy replacement (kubeProxyReplacement: true) — no kube-proxy daemonset, no iptables layer. Services and load balancers are handled by eBPF in the kernel.
• L2 announcements — a CiliumLoadBalancerIPPool claims 10.0.40.0/24, and a CiliumL2AnnouncementPolicy announces LoadBalancer IPs directly on the LAN. No MetalLB needed.

The result: LoadBalancer services get real, routable IPs on the home LAN announced by the nodes themselves.

2. Envoy Gateway: the front door #

Ingress is handled by Envoy Gateway — the Kubernetes Gateway API implementation on top of Envoy proxy:

• A wildcard certificate for *.krapulax.dev (ECDSA, Let’s Encrypt production, cert-manager-managed, SOPS-encrypted in git).
• Two replicas behind a Kubernetes service.
• The infra cluster runs a mirror (envoy-gateway-infra) with its own GatewayClass — the symmetry from Part 3, applied at the platform layer.

3. Ceph CSI: storage that survives #

Storage is where “rebuild, not restore” earns its keep. The app cluster gets storage from an external Ceph cluster via the Ceph CSI driver:

• Three monitors at 10.0.70.10/.11/.12:6789 — the dedicated storage network (the second NIC from Part 2).
• CephFS for most workloads, kernel mounts tuned (ms_mode=prefercrc).
• CSI credentials in values.sops.yaml — encrypted with age.

The result: a StorageClass backed by a replicated cluster, not a single disk on a single host. A node dying is a non-event for storage.

4. Cloudflare Tunnel + external-dns: the public face #

No port forwarding. No public IP exposure. The path to the outside world is a Cloudflare Tunnel:

• One tunnel per cluster, each with its own *.cfargotunnel.com target.
• external-dns keeps Cloudflare DNS in sync via DNSEndpoint records — DNS is a manifest, like everything else.
• Wildcard ingress: *.krapulax.dev → Envoy Gateway, with a 404 catch-all behind it.

Adding a new public app is: add a DNSEndpoint record, add an HTTPRoute, wait for Argo. No firewall rules, no NAT entries, no “which port did I use” spreadsheets.

The takeaway #

The platform layer is the difference between a homelab that demos well and one that survives. Every piece here is either declarative in git or encrypted in git — no state exists only in a running cluster. The CNI, ingress, storage, and public face can all be rebuilt from the repo alone.

Next up in Part 5: the app catalog — what actually runs on top of this, and how media, productivity, and monitoring fit together.