kubepath
1 DNS → node IP The A record 2 DNAT → :8000/:8443 The node’s :80 and :443 3 ACME ← back through :80 TLS, by HTTP-01 only 4 router → service Matching the request 5 EndpointSlice → pod IP Service, and what it actually does here 6 runsc The pod, inside gVisor 7 CNPG · ClickHouse The data path, shared
The request path hop 3 of 10 traces to the constraint

TLS, by HTTP-01 only

One ClusterIssuer, one challenge type, and therefore no wildcard certificates anywhere.

cert-manager 1.21.0, one letsencrypt-prod ClusterIssuer, HTTP-01 solver. That is the entire TLS story, and the choice of solver decides more than it looks like it does.

HTTP-01 proves control of a hostname by answering a plain HTTP request at http://<host>/.well-known/acme-challenge/<token>. The important word is plain: the challenge arrives on port 80, at the exact hostname on the certificate, from Let’s Encrypt’s validation servers. Which means it comes back in through hop 2 — the same node hostPort, the same Traefik, the same DNAT rule. The renewal path is the serving path. They are not two systems that can fail independently.

No wildcards. HTTP-01 cannot validate *.example.com; only DNS-01 can, and there is no DNS-01 solver configured. So the wildcard A record from hop 1 does not buy a wildcard certificate. Every hostname the cluster serves needs its own Certificate, its own Secret, and its own Ingress or IngressRoute carrying a tls block that names it. For a multi-tenant platform where each tenant gets a hostname, that turns certificate issuance into a per-tenant provisioning step with a per-tenant failure mode.

The redirect trap. Hop 2’s web entrypoint usually redirects everything to websecure. Do that unconditionally and you have just 301’d the ACME challenge, which is answered over HTTP by design. Let’s Encrypt does follow redirects, so this often appears to work — until the target hostname has no valid certificate yet, which is precisely the case during first issuance. The fix is to let cert-manager’s solver Ingress win on the challenge path: it is created with a higher-priority match on /.well-known/acme-challenge/, and it must be evaluated before the catch-all redirect.

Rate limits become a capacity number. Per-hostname certificates mean the CA’s issuance limits stop being a footnote and start being the ceiling on how fast tenants can be onboarded. Whatever the current published limits are, the number to know is certificates per registered domain per week, and the number to compare it against is how many tenants a batch import creates. Onboarding a class of thirty tenants at once is the operation that discovers this. Staging issuer first, always.

one issuer, one solver, one certificate per host
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: ops@example.com
    privateKeySecretRef:
      name: letsencrypt-prod-account-key
    solvers:
      - http01:                     # no dns01 solver == no wildcards
          ingress:
            ingressClassName: traefik
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: tenant-acme-tls
spec:
  secretName: tenant-acme-tls       # one Secret per hostname
  dnsNames: [acme.example.com]      # never *.example.com here
  issuerRef:
    name: letsencrypt-prod
    kind: ClusterIssuer

Check yourself

2 questions. One attempt each is recorded; the explanation is the point, not the score.

The zone has a wildcard A record. Why is there no wildcard certificate?

Why can an unconditional HTTP→HTTPS redirect break certificate issuance specifically?

← The node’s :80 and :443 Matching the request →