Configuring Mesosphere DC/OS with Datadog

Avatar
Kevin Walter
Follow

Mesosphere Datacenter Operating System DC/OS

Mesosphere DC/OS combines several components to run applications in a scalable, clustered infrastructure and runs on the the Apache Mesos kernel. While DC/OS is open source, Mesosphere Enterprise DC/OS is a paid service. 

At it’s basic level, DC/OS has a Master node and Slave nodes. Mesos is used under the hood to manage the resources underneath applications. DC/OS provides a Web UI and CLI to manage applications running in the cluster. DC/OS has Frameworks that provide another layer of abstraction for running applications. The Marathon Framework is installed by default to manage applications, and we have a Marathon integration. Frameworks can be easily installed in the UI and are retrieved from Universe - the application “store.”


Since Mesos uses containers for resource management, each slave node will have numerous containers running the deployed applications. In the image above, you can see that it's also possible to use a scheduler on top of Mesos/Marathon to deploy applications/containers. Here is a breakdown of how our agent collects metrics in a Mesos/Mesosphere DC/OS environment (not including use of Kubernetes):

  • Agent is deployed on each slave node
    • mesos_slave agent check collects Mesos slave metrics
    • Agent collects slave node system.* metrics
    • docker_daemon check collects container level metrics
  • Agent is deployed on master node
    • mesos_master agent check colllects Mesos master metrics
    • Agent collects master node system.* metrics

 

Installing the agent on Mesosphere DC/OS

  • Many users will use Mesosphere DC/OS on top of Mesos. If you are only using Mesos, you can run the json file manually.
  • If using DC/OS, you can install the agent on all of your slave nodes with either the CLI or Web UI by installing from Universe.
  • By default, we enable the mesos_slave and docker_daemon check on all slave nodes. An environment variable MESOS_SLAVE=true is used to flag the node accordingly
  • We recommend installing an agent on each Master, or optionally just the Leader. A user will need to SSH into the Master and run the docker-dd-agent install command manually with the env var MESOS_MASTER=yes on ONE master only - in the case where we are running an agent on each master (standbys or in a HA cluster where there are multiple masters). We also suggest setting the MARATHON_URL env var on that master as well, to enable the Marathon check. 
  • In short, only one Master should run the mesos_master check. 
  • Also, DC/OS uses Zookeeper, when enabling the mesos_master check, the zookeeper check will also be enabled.
  • You only need one host with the Marathon check enabled. We recommend the Leading Master, but really any host can have it enabled.
Have more questions? Submit a request

1 Comments

Date Votes
  • 0
    Avatar
    Scott Beamish

    The datadog-agent has a health check called probe.sh.  In the DC/OS UI you can setup a COMMAND Health Check by calling this script or add it to your JSON configuration.

    "healthChecks": [
        {
          "gracePeriodSeconds": 300,
          "intervalSeconds": 60,
          "maxConsecutiveFailures": 3,
          "timeoutSeconds": 20,
          "delaySeconds": 15,
          "protocol": "COMMAND",
          "command": {
            "value": "./probe.sh"
          }
        }
      ]

    Edited by Scott Beamish
Please sign in to leave a comment.
Powered by Zendesk