Setting up Traefik on AWS using a Network Load Balancer

I've recently been looking at various Kubernetes ingress controllers, and have taken a bit of a shine to Traefik. This is a quick guide to installing the Traefik controller on an existing Kubernetes cluster running inside AWS, and using the AWS Network Load Balancer to terminate SSL.

Installing Traefik

We're going to use the Helm chart to install Traefik on our existing K8s cluster. In my case, the cluster has been provisioned using kops, and is additionally using the SpotInst controller to ensure the entire cluster is running on AWS Spot instances to reduce the cost.

The most important part of the Helm deployment to allow us to use the NLB is to configure the values.yaml file correctly - you can see my configuration file below:

replicas: 1

rbac:
  enabled: true

accessLogs:
  enabled: false

dashboard:
  enabled: true
  domain: traefik.example.com
  auth:
    basic:
     # admin: password
      admin: '$apr1$xd1kpMTs$qFcsWe0VjLuTSJB3MihOV0' ☠️☠️☠️

service:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:eu-west-1:123456789012:certificate/abcdef12-3456-7890-abcd-ef1234567890
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"

externalTrafficPolicy: Local

ssl:
  enabled: true
  enforced: true
  upstream: true

This configuration file enables the Dashboard with a stupidly insecure username/password pair, and configures the Service with the correct annotations to use SSL termination at the ELB layer. You will need to ensure you have an ACM certificate already provisioned for your domain(s) - I use a wildcard certificate, i.e. *.example.com to cover all services.

You could also use the built-in ACME/LetsEncrypt support in Traefik, but as all my services live under a single domain, and I'm using Amazon Web Services, I prefer to use the also “free” Amazon provided and managed certificates. This method also means you don't need persistent storage or a central KV store to be maintained, and Traefik can remain a ‘stateless’ application.

We tell Traefik that we have SSL enabled, and we want to redirect all visitors to the HTTPS (Secure) version of our sites, but also that we're terminating SSL outside of Traefik's visibility (i.e. at the NLB).

Also note that we must set externalTrafficPolicy: Local (rather than Cluster) to preserve the inbound source IP addresses. Failing to set this will mean you have a working setup, but all your client connections will originate from IP ranges of your K8s cluster, making logging/IP whitelisting/authentication challenging.

Save this file to ‘traefik.yaml’ and you're ready to deploy the Helm chart:

$ helm install traefik --namespace kube-system stable/traefik --values traefik.yaml

Once done, check Traefik is running, and obtain the Load Balancer address:

$ kubectl -n kube-system get pods -l app=traefik
NAME                       READY   STATUS    RESTARTS   AGE
traefik-6947d6699c-lchgf   1/1     Running   0          13h
                                   ^^^^^^^

$ kubectl get svc traefik --namespace kube-system
NAME      TYPE           CLUSTER-IP     EXTERNAL-IP                                                                     PORT(S)                      AGE
traefik   LoadBalancer   172.20.23.14   abcdef000000000000beef0000000000-abcdef000000f00d.elb.eu-west-1.amazonaws.com   443:30379/TCP,80:32689/TCP   13h

All we now need to do, is setup our DNS records to point to the Load Balancer address. This is easily done with an ALIAS record in Route53. If you're feeling adventurous, you can also use the Kubernetes external-dns project to automatically provision DNS records as the Ingress resources are created. There's even a Helm Chart for it!

One more thing… 👵

The version of Traefik in the stable Helm chart is 1.7.x, this is the “legacy” version of Traefik. There's been quite a number of changes in v2, but due to the complexity of the change there's not a production-ready Helm chart available yet.