Ory OAthKeeper with Envoy Proxy, Prometheus and Jaeger Docker Compose

Ory OAthKeeper is a great Open Source Identity and Access Proxy (IAP). It can be deployed as a reverse proxy or as a control decision engine deployed with a reverse proxy. Most of the time, services are already deployed with a proxy like an Nginx or an Envoy. If you want to deploy for production, you may want to have metrics and distributed tracing as well. So, here is a docker-compose with all those components:

docker-compose.yml

version: '3.7'

services:
  oathkeeper:
    image: oryd/oathkeeper
    container_name: oathkeeper
    ports:
      - "4455:4455"
      - "4456:4456"
      - "9000:9000"
    depends_on:
      - jaeger
    command: serve --config=/etc/config/oathkeeper/config.yaml
    environment:
      - TRACING_PROVIDER=jaeger
      - TRACING_PROVIDERS_JAEGER_PROPAGATION=jaeger
      - TRACING_PROVIDERS_JAEGER_SAMPLING_SERVER_URL=http://jaeger:5778/sampling
      - TRACING_PROVIDERS_JAEGER_LOCAL_AGENT_ADDRESS=jaeger:6831
      - TRACING_PROVIDERS_JAEGER_SAMPLING_TYPE=const
      - TRACING_PROVIDERS_JAEGER_SAMPLING_VALUE=1
    volumes:
      - ./oathkeeper:/etc/config/oathkeeper
    restart: on-failure
    networks:
      - intranet

  prometheus:
    image: prom/prometheus:v2.12.0
    ports:
      - "9090:9090"
    depends_on:
      - oathkeeper
    command: --config.file=/etc/prometheus/prometheus.yml
    volumes:
      - ./prometheus:/etc/prometheus
    networks:
      - intranet

  jaeger:
    image: jaegertracing/all-in-one
    container_name: jaeger
    environment:
      - COLLECTOR_ZIPKIN_HOST_PORT=9411
    ports:
      - "16686:16686" # The UI port
    networks:
      - intranet

  envoy:
    image: envoyproxy/envoy:v1.25-latest
    ports:
      - "8081:8081"
    depends_on:
      - oathkeeper
    volumes:
      - ./envoy/envoy.yaml:/etc/envoy/envoy.yaml
    networks:
      - intranet

networks:
  intranet:

You also need an OAthKeeper configuration. Do not forget to add your own rules file. To know how to define rules, have a look at the Ory documentation.

This example authenticates tokens issued by the Google Identity provider. If you want to authenticate against another provider, you should change this configuration and put the right JWKS URLs.

./oathkeeper/config.yaml

serve:
  proxy:
    port: 4455 # run the proxy at port 4455
  api:
    port: 4456 # run the api at port 4456
  prometheus:
    port: 9000
    metrics_path: /metrics

access_rules:
  repositories:
    - file:///etc/config/oathkeeper/your-rule.json

errors:
  fallback:
    - json
  handlers:
    json:
      enabled: true
      config:
        verbose: true

mutators:
  noop:
    enabled: true

authorizers:
  allow:
    enabled: true
  deny:
    enabled: true

authenticators:
  noop:
    enabled: true

  anonymous:
    enabled: true
    config:
      subject: guest

  jwt:
    enabled: true
    config:
      jwks_urls:
        - https://www.googleapis.com/service_accounts/v1/jwk/[email protected]
      scope_strategy: hierarchic

The prometheus scrapper configuration for OAthKeeper.

./prometheus/prometheus.yml

global:
  scrape_interval: 15s # By default, scrape targets every 15 seconds.

scrape_configs:
  - job_name: 'prometheus'
    scrape_interval: 15s
    static_configs:
      - targets: ['localhost:9090']
  - job_name: 'oathkeeper'
    scrape_interval: 15s
    metrics_path: /metrics
    static_configs:
      # The target needs to match what you've configured above
      - targets: ['oathkeeper:9000']

And an Envoy configuration file to use OAthKeeper and configure Zipkin tracing.

./envoy/envoy.yaml

admin:
  address:
    socket_address:
      protocol: TCP
      address: 0.0.0.0
      port_value: 9901
static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address:
        protocol: TCP
        address: 0.0.0.0
        port_value: 8081
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          generate_request_id: true
          tracing:
            provider:
              name: envoy.tracers.zipkin
              typed_config:
                "@type": type.googleapis.com/envoy.config.trace.v3.ZipkinConfig
                collector_cluster: jaeger
                collector_endpoint: "/api/v2/spans"
                split_spans_for_request: true
                shared_span_context: true
                collector_endpoint_version: HTTP_JSON
          stat_prefix: ingress_http
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              domains: ["*"]
              routes:
              - match:
                  prefix: "/"
                route:
                  host_rewrite_literal: www.envoyproxy.io
                  cluster: service_envoyproxy_io
          http_filters:
          - name: envoy.filters.http.ext_authz
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthz
              failure_mode_allow: false
              metadata_context_namespaces:
              - envoy.filters.http.jwt_authn
              stat_prefix: oathkeeper
              http_service:
                path_prefix: /decisions
                server_uri:
                  uri: http://oathkeeper:4456
                  cluster: oathkeeper
                  timeout: 0.25s
                authorization_request:
                  allowed_headers:
                    patterns:
                    - exact: accept
                    - exact: authorization
                    - exact: content-type
                    - exact: x-forwarded-for
                    - exact: x-forwarded-proto
                    - exact: x-forwarded-host
                authorization_response:
                  allowed_upstream_headers:
                    patterns:
                    - exact: authorization
          - name: envoy.filters.http.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
  clusters:
  - name: googleapis
    connect_timeout: 30s
    type: logical_dns
    lb_policy: round_robin
    load_assignment:
      cluster_name: googleapis
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: www.googleapis.com
                port_value: 443
  - name: oathkeeper
    connect_timeout: 30s
    type: logical_dns
    lb_policy: round_robin
    load_assignment:
      cluster_name: oathkeeper
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: oathkeeper
                port_value: 4456
  - name: service_envoyproxy_io
    connect_timeout: 30s
    type: LOGICAL_DNS
    dns_lookup_family: V4_ONLY
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: service_envoyproxy_io
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: www.envoyproxy.io
                port_value: 443
    transport_socket:
      name: envoy.transport_sockets.tls
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
        sni: www.envoyproxy.io
  - name: jaeger
    type: STRICT_DNS
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: jaeger
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: jaeger
                port_value: 9411