There are a number of common issues that can get in the way when sending new logs to Datadog via the log collector in the dd-agent. If you experience issues sending new logs to Datadog, this list will help you troubleshoot. If you continue to have trouble, please email firstname.lastname@example.org for further assistance.
The agent needs to be restarted
After you've made any configuration changes to the dd-agent, the changes will only take effect after you restart the dd-agent.
No new logs have been written
The dd-agent only collects logs that have been written after it has started trying to collect them (whether it be tailing or listening for them. In order to confirm whether log collection has been successfully set up, please make sure that new logs have been written.
Can you see metrics for that host?
The problem might be bigger than just the logs. Double check that you see metrics from this host.
Outbound traffic on port 10516 or 10514 is blocked
The log-agent within the dd-agent sends its logs to Datadog over tcp via port 10516. If that connection is not available, logs will fail to be sent and an error will be recorded in the
logs-agent.log to that effect.
You can test your connection by running starting a telnet command like so (port 10514 would work too, but is less secure):
openssl s_client -connect intake.logs.datadoghq.com:10516
and then by sending a log like the following:
<API_KEY> this is a test message
These are a few of the common configuration issues that are worth triple-checking in your dd-agent setup:
- You do not have
log_enabled: truein your
- You do not have any
.yamlfile in your
conf.d/directory that includes a logs section and the appropriate values, or you have added these values only to the
(N.B. you will want to keep the
logs-agent.yamlconfigurations separate from the configuration files that determine where the dd-agent should look/listen for logs.)
- You may have some
.yamlparsing errors in your configuration files. Yaml is notoriously finicky, so when in doubt, a good yaml validator is worth referencing.
Check for errors in the logs
There might be an error in the logs that would explain the issue. So just run the following command and check for errors:
sudo cat /var/log/datadog/agent.log | grep log
Permission Issues While Tailing Log Files
The dd-agent does not run as root (and we do not recommend that you make it run as root, as a general best-practice). For this reason, when you configure your dd-agent to tail log files (for custom logs or for integrations) you will need to take special care to ensure the dd-agent user has read access to tail the log files you want to collect from.
If the dd-agent user does not have read access to the files you configure it to tail, then a permissions error will be captured in the
N.B. when you add the appropriate read permissions, you will want also to make sure that these permissions are correctly set on your log rotation configuration. Otherwise, when the log rotates next, the dd-agent may lose its read permissions.