kubepath

platform/traefik/values.yaml

The hostPort model

The file where "no cloud load balancer" stops being a decision and becomes a configuration. Six of its settings are unusual, and every one of them is unusual for the same reason.

Read this one first in a class, because it establishes the constraint that explains everything else. Nothing here is exotic on its own — hostPort, a DaemonSet, a disabled Service — but the combination is a coherent design, and the interesting part is that the settings only make sense together. Turn any one of them back to its default and the cluster stops serving.

Three of the annotations below are the ones worth the most time: the port pair that makes an unprivileged container answer the public internet, allowExternalNameServices and the trust boundary it moves, and the readTimeout that git needs.

Along the path: hop 2 · The node’s :80 and :443 · hop 4 · Matching the request · hop 8 · Forgejo, on :2222 and through the API

platform/traefik/values.yaml 10 annotated lines
1# Traefik v3 · chart 41.0.2 · Scaleway Kapsule, pl-waw, one worker node
2#
3# There is no cloud load balancer in front of this cluster, by choice.
4# Everything below follows from that one sentence.
5
6deployment:
8 replicas: null # meaningless for a DaemonSet; left explicit
9
10service:
12
13ports:
14 web:
16 hostPort: 80
17 expose:
19 redirectTo:
20 port: websecure
22 websecure:
23 port: 8443
24 hostPort: 443
25 expose:
26 default: false
27 tls:
28 enabled: true
29 transport:
30 respondingTimeouts:
32 writeTimeout: 0s # same argument, for a large clone
33 idleTimeout: 180s # still bounded; idle is not in-flight
34
35providers:
36 kubernetesCRD:
37 enabled: true
38 allowCrossNamespace: true # platform routes to tenant namespaces
40 kubernetesIngress:
41 enabled: true # cert-manager solvers, tenant Ingresses
42 allowExternalNameServices: true
43 publishedService:
45
46securityContext:
48 drop: [ALL]
49 readOnlyRootFilesystem: true
50 runAsNonRoot: true
51 runAsUser: 65532
52
53logs:
54 general:
55 level: INFO
56 access:

line 7

DaemonSet, so the mistake cannot be made

hostPort carries an invariant: at most one pod per node may hold a given port. A Deployment can violate it — scale to two replicas on a one-node cluster and the second pod sits Pending forever, reporting a port conflict that reads like a scheduling problem.

A DaemonSet cannot express that mistake, because one pod per node is what a DaemonSet is. The workload type is enforcing what would otherwise need an admission policy or a runbook note. That is the pattern worth naming: when a constraint can be encoded in the choice of object, encode it there.

line 11

There is no Service. Not a disabled one — none.

No ClusterIP, no NodePort, no LoadBalancer. Nothing in the cluster can reach the ingress controller by name, and nothing needs to: traffic arrives from the node’s network edge, upstream of kube-proxy entirely.

Two things follow. Anyone debugging this cluster who looks for the ingress the way they would anywhere else — by finding its Service — will conclude the ingress is not installed. And publishedService further down must also be off, because it reads a Service that does not exist.

This is the single line most likely to confuse a visiting engineer, so it is the one to put on a slide.

line 15

The pair of numbers that keeps this container unprivileged

The container binds 8000 and 8443 — unprivileged ports. The node answers on 80 and 443. The bridge between them is not a privileged bind: it is a DNAT rule that the CNI’s portmap plugin installs on the node, rewriting node:443 to pod:8443.

So this pod needs no NET_BIND_SERVICE, no hostNetwork: true, and no root. It drops every capability, runs as UID 65532 with a read-only root filesystem, and still serves the public internet. That is a better posture than the two obvious alternatives, and it falls out of the port model rather than being bolted on.

Verify it on the node rather than believing the manifest: iptables -t nat -L CNI-HOSTPORT-DNAT -n.

line 18

Nothing to expose it through

expose controls whether the chart adds this port to the Service. There is no Service. Left at its default the chart would try to render a port into a template that is not being rendered — harmless in some chart versions, an error in others, and misleading in all of them.

Setting it explicitly is documentation: it tells the next reader that the omission is deliberate.

line 21

The redirect that can eat your certificate renewals

This 301s plain HTTP to HTTPS, which is correct for browsers and dangerous for ACME. HTTP-01 challenges are answered over plain HTTP at /.well-known/acme-challenge/ by design; redirect them unconditionally and cert-manager’s solver never gets its request.

It often appears to work, because Let’s Encrypt follows redirects — right up until the target hostname has no valid certificate yet, which is exactly the situation during first issuance.

The priority here is what makes it survivable: the solver Ingress cert-manager creates carries a higher-priority match on the challenge path, so it is evaluated first. Lower this number below the solver’s and renewals begin failing silently, with a 30-day fuse before anyone notices.

line 31

The git fix — a correct HTTP default that is wrong for git

Traefik bounds how long it will spend reading a request. The default is 60 seconds: generous for an HTTP request, short for git-receive-pack, which holds a single request open while the client uploads a pack and the server indexes it.

Over 60 seconds Traefik cuts the connection and the tenant sees RPC failed; curl 92 or the remote end hung up unexpectedly. Small repositories work; large ones fail. Nothing in the error names the proxy, so every instinct points at git, at Forgejo, or at the client’s network.

Give this line real classroom time, and reproduce it on purpose: set readTimeout: 5s on a preview and push something big. A failure you have caused once is a failure you recognise in a ticket.

line 39

This moves a trust boundary onto Service creation

An ExternalName Service is a DNS alias, not an endpoint — it tells Traefik to send the request to an arbitrary hostname. Traefik v3 disables this by default because allowing it converts "can create a Service in a watched namespace" into "can make the ingress controller issue requests to a host of my choosing", with the cluster’s network position behind it. That is a server-side request forgery primitive.

It is on here for a real reason: the edge/workload split needs off-cluster backends referenced by name. So the honest framing is not that the setting is unsafe, but that it makes Service creation a privileged operation.

Which means the mistake is not turning it on. The mistake is turning it on without also constraining who can create Services in the watched namespaces — by RBAC, by scoping the providers to specific namespaces, or both. Ask a class where that boundary is enforced in this repo before telling them.

line 44

Off because there is no Service to publish

With this on, Traefik reads its own Service to find an external address and writes it back to status.loadBalancer.ingress on every Ingress it manages. With service.enabled: false there is no Service to read, so those statuses stay permanently empty.

An empty status is not cosmetic. Anything that waits on it hangs against a cluster that is working perfectly: a kubectl wait --for=jsonpath='{.status.loadBalancer.ingress[0].ip}', a CI readiness gate, an external-dns looking for a published address. The symptom is a timeout with no error anywhere, which is the hardest shape of failure to diagnose.

line 47

Every capability dropped — and it still serves :443

This is the payoff from the port pair above. Because the DNAT happens on the node and the container binds 8443, the ingress controller for a publicly reachable cluster needs no capabilities at all.

Compare it to the alternatives out loud: hostNetwork: true puts the pod in the node’s network namespace and gives it every interface; a privileged bind to 443 needs NET_BIND_SERVICE and a root-capable start. Both are common, both are worse, and the difference is two port numbers.

line 57

The only place a request is recorded

There is no load balancer log, no flow log, and no cloud console to look at. This access log is the sole record that a request reached the cluster. If it is off, a class debugging a 404 has nothing upstream of the application to read.

Worth pairing with the point from hop 5: because Traefik dials pod IPs directly, this log is also the only place the chosen backend is written down.

Check yourself

1 question. One attempt each is recorded; the explanation is the point, not the score.

An engineer new to this cluster runs kubectl get svc -n traefik and finds nothing. What have they learned?