Reducing Traefik CPU Usage on a Single-Core K3s Node
When my single-core k3s control plane hit 95% CPU and load averages above 3.7, I discovered my Traefik Gateway API provider was thrashing the API server with unnecessary watch events. Here's what I changed and what worked.
A couple of months ago, I migrated my Kubernetes ingress from nginx to Traefik with the Gateway API. It worked well, but last week I noticed something was off. My k3s control plane node (in-oci-01 — a modest 1-core VM in Oracle Cloud) was running at 95% CPU with a load average of 3.78.
That’s 3.78 on a 1-core VM. It was pegged.
This post covers how I investigated the issue, what I found, and the specific changes that brought CPU down to ~25%.
The Investigation
I started with the usual suspects. kubectl top nodes showed nothing unusual at the node level, so I went deeper.
On the Host
SSH’d into the node and ran ps aux --sort=-%cpu | head:
1
2
3
4
k3s-server 43%
traefik 21%
tailscaled (×2) 5%
containerd/system 5%
Traefik was consuming ~21% of a single core — and that was just the process itself. The real cost was what it triggered in the API server.
Inside the Pods
kubectl top pods on the same node confirmed it:
| Pod | CPU |
|---|---|
| traefik (external-gateway) | 167m |
| pi-hole | 28m |
| postgres, redis, metrics-server | ~20m total |
The K3s Server Itself
The k3s binary embeds etcd, kube-apiserver, controller-manager, and scheduler — all in one process. It was using 43% CPU on its own. I checked the API server metrics for clues:
1
kubectl get --raw=/metrics | grep apiserver_request_total
62,606 GET requests to Endpoints and EndpointSlices. Most of these were coming from Traefik’s Gateway API provider, which maintains WATCH connections on Gateway API resources, services, and endpoints.
The pattern was clear: my Traefik instance had --providers.kubernetesGateway=true with all defaults. Every WATCH event — from any Gateway API resource, service change, or endpoint update — triggered a full config rebuild. On a 1-core machine running an embedded etcd, this cascades quickly.
Why This Happened
When you enable providers.kubernetesGateway in Traefik, it does the following:
- Establishes WATCH informers on GatewayClasses, Gateways, HTTPRoutes, TLSRoutes, Services, and EndpointSlices
- On every event from any of those informers, it rebuilds its internal routing configuration
- For config rebuilds, it queries the API server for endpoints and endpointslices — which is where those 62k GETs came from
Every time a pod starts, a service updates, or a route changes — Traefik’s config gets rebuilt from scratch. In a cluster with ~30 components and ArgoCD reconciling constantly, there’s a lot of churn.
The nginx ingress I had before was simpler — it read a static config and proxied traffic. No controller loops, no informers, no API server pressure.
The Changes
I made three changes. Here they are in order of impact.
1. Throttle Config Rebuilds (Biggest Win)
Traefik’s throttleDuration controls the minimum time between config rebuilds. The default is 0s — meaning every single K8s event triggers a rebuild.
Setting it to 10s batches rapid events so Traefik only rebuilds once every 10 seconds during periods of high change activity.
1
2
additionalArguments:
- "--providers.kubernetesGateway.throttleDuration=10s"
This was the single most impactful change. Config rebuilds went from happening constantly to once every 10 seconds at most, and the API server GETs to endpoints dropped off a cliff.
2. Reduce API Server Request Rate
Traefik’s default qps/burst to the API server is 50/100 — aggressive for any controller sharing a 1-core node with its API server.
1
2
- "--providers.kubernetesGateway.qps=20"
- "--providers.kubernetesGateway.burst=40"
This halves the sustained request rate. Since routing decisions are cached locally, there’s no latency impact on actual traffic — only config rebuilds are affected.
3. Scope Traefik to Relevant GatewayClasses
The cluster has three GatewayClasses:
| GatewayClass | Gateways | Purpose |
|---|---|---|
external-gateway |
15 | Public (Cloudflare-facing) |
internal-gateway |
6 | Internal (Tailscale) |
public-gateway |
0 | Dead — created during testing |
Without labelSelector, Traefik was watching and reconciling all three — including the dead public-gateway. I narrowed it to only the GatewayClass that actually matters:
1
- "--providers.kubernetesGateway.labelSelector=app.kubernetes.io/instance=external-gateway-external-gateway"
This prevents Traefik from wasting cycles on routes and gateways it will never serve.
Bonus: Disable Access Logs
Traefik had --accesslog=true with default field logging. Every request was writing a formatted log line. I wasn’t actively using those logs, so I disabled it.
1
2
accessLog:
enabled: false
The Result
After applying these changes and restarting Traefik:
Before: ~95% CPU, load average 3.78 After: ~25% CPU, load average well under 1.0
The k3s-server itself dropped from 43% to about 15-18% — mostly because the API server wasn’t drowning in WATCH connections and endpoints GETs anymore.
What Each Change Contributed
| Change | CPU reduction | Notes |
|---|---|---|
throttleDuration=10s |
~15-20% | Stopped config thrashing |
qps=20 / burst=40 |
~5-8% | Less API server queueing |
labelSelector |
~3-5% | Ignoring irrelevant GatewayClasses |
| Disable access logs | ~1-2% | Per-request I/O overhead removed |
What I Would Have Done Differently
Measure before migrating. I switched from nginx to Traefik and didn’t think to check the CPU impact until weeks later. A simple kubectl top pods right after the migration would have caught this immediately.
Start with throttleDuration from day one. There’s no reason to have it at 0s in a homelab cluster where config changes are infrequent. A 5-10s throttle costs nothing and prevents this kind of cascading load.
Summary
If you’re running Traefik with the Gateway API provider — especially on a small node — these flags are worth setting:
1
2
3
4
--providers.kubernetesGateway.throttleDuration=10s
--providers.kubernetesGateway.qps=20
--providers.kubernetesGateway.burst=40
--providers.kubernetesGateway.labelSelector=<your-gatewayclass-label>
They don’t affect routing performance. They just stop Traefik from rebuilding config on every K8s event and hammering the API server unnecessarily. On a 1-core k3s control plane running embedded etcd, that makes all the difference.
