Introduction

At some point, I wanted to try something different from k8s and decided to have a look at Nomad from Hashicorp. And it has a pretty simple configuration to integrate with Traefik.

You could find some details on this topic in the official documentation for Nomad here.

Since this configuration is running on my personal server and resources are not publicly exposed, there would be not so many info about securing configuration.

Below you could find some of my configuration files.

Container orchestration

Nomad

data_dir = "/var/lib/nomad"
bind_addr = "0.0.0.0"

server {
  enabled = true
  bootstrap_expect = 1
}

client {
  enabled = true
  servers = ["127.0.0.1:4646"]
  options {
    "docker.volumes.enabled" = "true"
  }
}

telemetry {
  publish_allocation_metrics = true
  publish_node_metrics       = true
  prometheus_metrics         = true
}

Consul

advertise_addr = "<SERVER_IP>"
bind_addr = "<SERVER_IP>"
bootstrap_expect = 1
client_addr = "0.0.0.0"
datacenter = "DC1"
data_dir = "/var/lib/consul"
enable_script_checks = true
dns_config {
  enable_truncate = true
  only_passing = true
}
enable_syslog = true
leave_on_terminate = true
log_level = "INFO"
encrypt = "<SECRET>"
rejoin_after_leave = true
server = true
ui = true
telemetry {
  statsite_address = "127.0.0.1:2180"
}

Notes

Nomad and Consul are able automatically to bootstrap themselves without additional configuration if they are running on the same machine.

Traefik

In my setup I’m running Traefik as one of Nomad deployments. Here is my configuration for it:

job "traefik" {
  region      = "global"
  datacenters = ["dc1"]
  type        = "service"

  group "traefik" {
    count = 1

    task "traefik" {
      driver = "docker"

      config {
        image        = "traefik:<CONTAINER_VERSION_TAG>"
        network_mode = "host"

        volumes = [
          "local/traefik.toml:/etc/traefik/traefik.toml",
          "/mtw/nomad/traefik/etc/:/etc/traefik/"
        ]
      }

      env {
        CF_API_EMAIL="<MY_CF_EMAIL_ADDRESS>"
        CF_API_KEY="<MY_CF_API_KEY_TO_MANAGE_ZONE>"
      }

      template {
        data = <<EOF
[entryPoints]
    [entryPoints.web]
        address = ":80"
        [entryPoints.web.forwardedHeaders]
            trustedIPs = ["127.0.0.1","192.168.0.0/16","173.245.48.0/20","103.21.244.0/22","103.22.200.0/22","103.31.4.0/22","141.101.64.0/18","108.162.192.0/18","190.93.240.0/20","188.114.96.0/20","197.234.240.0/22","198.41.128.0/17","162.158.0.0/15","104.16.0.0/12","172.64.0.0/13","131.0.72.0/22"]
    [entryPoints.websecure]
        address = ":443"
        [entryPoints.websecure.forwardedHeaders]
            trustedIPs = ["127.0.0.1","192.168.0.0/16","173.245.48.0/20","103.21.244.0/22","103.22.200.0/22","103.31.4.0/22","141.101.64.0/18","108.162.192.0/18","190.93.240.0/20","188.114.96.0/20","197.234.240.0/22","198.41.128.0/17","162.158.0.0/15","104.16.0.0/12","172.64.0.0/13","131.0.72.0/22"]
    [entryPoints.traefik]
        address = ":8080"

[api]
    dashboard = true
    insecure  = true

[log]

[accessLog]
    format = "json"

[metrics]
    [metrics.influxdb]
        address = "localhost:8086"
        protocol = "http"
        pushinterval = "60s"
        database = "traefik"
        retentionpolicy = "traefik"

[certificatesResolvers.myresolver.acme]
    email = "<EMAIL_USED_FOR_LETSENCRYPT_REQUESTS>"
    storage = "/etc/traefik/acme.json"
    keyType = "RSA4096"
    [certificatesResolvers.myresolver.acme.dnschallenge]
        provider = "cloudflare"
        delayBeforeCheck = 90
        resolvers = ["1.1.1.1:53", "8.8.8.8:53"]
        disablePropagationCheck = true

# Enable Consul Catalog configuration backend.
[providers.consulCatalog]
    prefix           = "traefik"
    exposedByDefault = false
    requireConsistent = true

    [providers.consulCatalog.endpoint]
        address = "127.0.0.1:8500"
        scheme  = "http"
        datacenter = "dc1"
EOF

        destination = "local/traefik.toml"
      }

      resources {
        cpu    = 100
        memory = 128

        network {
          port "http" {
            static = 80
          }

          port "https" {
              static = 443
          }

          port "api" {
            static = 8080
          }
        }
      }

      service {
        name = "traefik"
        tags = [
          "traefik.http.routers.http-catchall.rule=hostregexp(`{host:[a-z-.]+}`)",
          "traefik.http.routers.http-catchall.entrypoints=web",
          "traefik.http.routers.http-catchall.middlewares=redirect-to-https",
          "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https"
        ]

        check {
          name     = "alive"
          type     = "tcp"
          port     = "http"
          interval = "10s"
          timeout  = "2s"
        }
      }
    }
  }
}

This setup enables support for the processing of X-Forwarded-* headers with trustedIPs option in entryPoints section.

As default way to process certificate requests, DNS challenge is used. To authenticate in CloudFlare API, you need to update environment variables with your credentials.

By default, this setup redirects all HTTP traffic to HTTPS port. So we don’t need to configure it separately. This configuration is a part of the service section in Nomad deployment.

Detailed Traefik config

Entry points

I’ve enabled three endpoints:

As mentioned above, there is also specified list of CloudFlare IP’s to enable proper match of real IP’s from users visiting sites behind CloudFlare and (in my case: 192.168.0.0/16) NAT.

API

This section enables Traefik dashboard and disables SSL encryption to access it.

Logs

Those two section enables Traefik system logs and access logs.

Metrics

In my setup I have also InfluxDB to collect metrics. So I enabled Traefik to push metrics to InfluxDB.

Certificate resolver

This section enables settings which are needed to request certificates from Let’s Encrypt. In my case I enabled DNS challenge.

Consul catalog

This section integrates Traefik with Consul Catalog. This is needed to enable autoconfiguration of routers and services for applications deployed with Nomad.

CloudFlare

Since we enabled HTTPS redirect from HTTP port, we also need to enforce encryption between CloudFlare and our server with Traefik.

CloudFlare Full SSL