Proxy log collection with HAProxy

If your network configuration restricts outbound traffic, you can use a proxy to send logs from either the Datadog agent or 3rd party log collectors you utilize to the Datadog logs intake.

Unlike the metrics intake API, which listens on HTTPS 443, the logs intake utilizes TCP (Layer 4) on port 10516 (for TLS and 10514 for plaintext). In this article we're providing an example for proxying logs with HAProxy.

agent ---> HAProxy ---> Datadog

Here's a basic haproxy.conf used to proxy logs to the Datadog intake. In this example haproxy also does TLS wrapping ensuring internal plaintext logs are encrypted between your proxy and Datadog's log intake API endpoint:

global
    log 127.0.0.1 local0
    maxconn 4096
    stats socket /tmp/haproxy

# Some sane defaults
defaults
    log global
    option dontlognull
    retries 3
    option redispatch
    timeout client 5s
    timeout server 5s
    timeout connect 5s

# This declares a view into HAProxy statistics, on port 3835
# You do not need credentials to view this page and you can
# turn it off once you are done with setup.
listen stats
    bind *:3833
    mode http
    stats enable
    stats uri /

# Logs frontend
frontend logs_frontend
    bind *:10514
    mode tcp
    default_backend logs_backend

# Logs backend
# agent-intake.logs.datadoghq.com used specifically for agent logs
# intake.logs.datadoghq.com is also available for logs submitted without an agent
# ca-certificates.crt located in /etc/ssl/certs/ for Ubuntu 16.04
backend logs_backend
    balance roundrobin
    mode tcp
    option tcplog
    server datadog agent-intake.logs.datadoghq.com:10516 ssl verify required ca-file /etc/ssl/certs/ca-certificates.crt

When using the Datadog agent as the logs collector the agent itself would also need to be instructed to use the newly created proxy instead of establishing a connection directly with the logs intake. This is done with the following options in datadog.yaml:

logs_config:
  dd_url: myProxyServer.myDomain
  dd_port: 10514
  dev_mode_no_ssl: true

Notice the dev_mode_no_ssl: true line. It's ok for us to employ this parameter, because establishing the SSL/TLS connection is handled by HAProxy. Do not run with this option if you aren't intending to use a proxy, which can encrypt the connection to the logs intake. 

We've successfully testing these configurations on Ubuntu 16 with HAProxy versions 1.6 and 1.8, and Datadog agent ver. 6.3.1

This logs example configuration can also be combined with the the HAProxy example we give for metrics/process/trace agents proxying.

If using other 3rd part log shippers see this article for details on their configuration options. 

Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.