Advanced Traefik 2.0 with Kubernetes

Following my earlier post about Traefik 2 and Kubernetes, here are some advanced configuration examples and a full yaml example at the end of this post:

Protecting a route with a password

Create an htpasswd file named users for a user admin

 htpasswd -c users admin

Use kubectl to create the secret (easier for multi lines file).

kubectl create secret generic admin-authsecret --from-file=users

Create a middleware for authentication:

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: admin-auth
spec:
  basicAuth:
    secret: admin-authsecret

Then apply this admin-auth middleware to each IngressRoute you want to protect.

Prometheus

To enable Traefik Prometheus metrics, add these args options to the template spec args:

			# create a dedicated entrypoints for metrics
            - --entrypoints.metrics.address=:8888
            - --metrics.prometheus=true
            - --metrics.prometheus.entryPoint=metrics

Add the scrape annotation to the deployment spec metadata:

      annotations:
        prometheus.io/scrape: 'true'
        prometheus.io/port: '8888'

Note: The older Traefik Grafana templates are incompatible with Traefik 2.0 since several metrics have been renamed.

Jaeger

To enable tracing with Jaeger add these args options to the template spec args:

        - --tracing=true
        - --tracing.jaeger=true
        - --tracing.jaeger.localAgentHostPort=jaeger-agent:6831
        - --tracing.jaeger.samplingParam=1
        - --tracing.jaeger.samplingType=const

Do not use this configuration in production since it will trace every requests, but use a probabilistic one.

Note: do not use double quote around localAgentHostPort value …

Traefik Dashboard

To expose the Traefik dashboard:

        - --api.dashboard=true

Be sure to have a traefik-admin service:

apiVersion: v1
kind: Service
metadata:
  name: traefik-admin
spec:
  ports:
    - port: 8080
      name: admin
      targetPort: admin
  selector:
    app: traefik

Then a route for the dashboard:

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: traefik-ingress
  namespace: default
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`traefik.mydomain.tld`)
      middlewares:
        - name: admin-auth
        - name: compress
      kind: Rule
      services:
        - name: traefik-admin
          port: 8080
  tls:
    certResolver: default

admin-auth is a login password middleware to protect the page (see above).

Grafana Dashboard

To expose Grafana dashboard:

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: grafana-ingress
  namespace: default
spec:
  entryPoints:
  - websecure
  routes:
  - kind: Rule
    match: Host(`grafana.mydomain.tld`)
    middlewares:
    - name: admin-auth
    - name: compress
    services:
    - name: grafana
      port: 80
  tls:
    certResolver: default

Let’s Encrypt

You’ll need a storage for the SSL certificates, it’s important because each time you’ll restart Traefik will regenerate them, possibly hitting let’s encrypt rate limits if you don’t.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  labels:
    component: "server"
    app: traefik
  name: traefik-acme-storage
spec:
  accessModes:
    - ReadWriteOnce

  resources:
    requests:
      storage: "100Mi"

Do not forget to mount the disk:

          volumeMounts:
            - name: storage-volume
              mountPath: /data
              subPath: ""
      volumes:
        - name: storage-volume
          persistentVolumeClaim:
            claimName: traefik-acme-storage

To enable let’s encrypt, add these args options to the template spec args

            - --certificatesresolvers.default.acme.email=root@mydomain.tld
            - --certificatesresolvers.default.acme.storage=/data/acme.json

Enable a DNS provider for the dns challenge: Look for your DNS provider settings.

Example with Gandi:

            - --certificatesResolvers.default.acme.dnsChallenge.provider=gandiv5
            - --certificatesResolvers.default.acme.dnsChallenge.delayBeforeCheck=5

The provider is looking for a key in the env, set in your deployment as usual:

          env:
            - name: GANDIV5_API_KEY
              valueFrom:
                secretKeyRef:
                  name: gandi-secret
                  key: secret

Overall Example

Here is a full example Traefik 2 config.

Note: this is a non HA setup, do not upscale this deployment if you are using let’s encrypt.