Mesos-DNS Setup on Mesos Cluster

Mesos-DNS:

Mesos-DNS supports service discovery in Apache Mesos clusters. It allows applications and services running on Mesos to find each other through the domain name system (DNS), similarly to how services discover each other throughout the Internet.

Applications launched by Marathon or Aurora are assigned names like search.marathon.mesos or log-aggregator.aurora.mesos. Mesos-DNS translates these names to the IP address and port on the machine currently running each application. To connect to an application in the Mesos datacenter, all you need to know is its name. Every time a connection is initiated, the DNS translation will point to the right machine in the datacenter.

For complete reference please look at https://mesosphere.github.io/mesos-dns/

As mentioned above applications launched by Marathon are assigned names like search.marathon.mesos.

Now let’s see how to decode this name.

search

marathon.mesos

Example :

I have created a group called mesos-dns-test in marathon and created a application called dns with in the same group.

App name : dns

Group name : mesos-dns-test

(Note: Marathon groups can have sub groups also)

Let’s see how it looks in marathon GUI.

tmptmp

To access above marathon application using mesos-dns we can use following name dns.mesos-dns-test.marathon.mesos. This is how marathon names the application once it gets launched by marathon,

Here if you observe the naming of an application it’s started with application name followed by group name and marthon-mesos extension.

Using the same name one can easily extract the IP address of the box where the above application got launched,

Command to extract IP address from marathon application name using dns server is:

dig +short

Exp: dig +short dns.mesos-dns-test.marathon.mesos

tmp

If application is running on multiple nodes (with marathon scale) than above command will return multiple comma separated IP addresses.

Now let’s see how to setup Mesos-DNS :

Assuming mesos, marathon cluster was already installed if not please use my mesos and marathon cluster setup guide (Mesos & Marathon Cluster Setup Guide).

Note: Make sure to have port 53 is unblocked to allow traffic between nodes.

  1. Launch mesos cluster.As mentioned if setup was not available prior to follow these steps use my mesos and marathon cluster setup guide.
  2. Install mesos-dns.

You can install mesos dns on individual node or use existing mesos slave nodes.

Access the node using ssh

ssh xx.xx.xx.xxx (Replace ‘x’ with your IP address)

Pull binaries for your local system from GitHub.

  • sudo mkdir /usr/local/mesos-dns
  • In the same directory (created above) create a config.json file with below content.
    cat /usr/local/mesos-dns/config.json
    {
    “zk”: “zk://zookeeper_IP1:2181,zookeeper_IP2:2181,zookeeper_IP3:2181/mesos”,
    “masters”:[“mesos_master1:5050″,”mesos_master2:5050″,”mesos_master3:5050”],
    “ttl”: 60,
    “domain”: “mesos”,
    “port”: 53,
    “resolvers”: [“RESOLVER_IP”],
    “timeout”: 5,
    “httpon”: true,
    “dnson”: true,
    “httpport”: 8123,
    “externalon”: true,
    “listener”: “0.0.0.0”,
    “SOAMname”: “ns1.mesos”,
    “SOARname”: “root.ns1.mesos”,
    “SOARefresh”: 60,
    “SOARetry”: 600,
    “SOAExpire”: 86400,
    “SOAMinttl”: 60,
    “IPSources”: [“netinfo”, “mesos”, “host”]
    }

Replace zookeper_IP, masters IP’s and Resolver_ip with relevant IP’s according to your setup.

If your IP address is 192.168.1.XXX than your resolver is 192.168.1.1 or If your IP address is 10.0.10.XXX than your resolver is 10.0.10.1

With above we are done with configuration part and now we can start mesos dns with below command.

sudo /usr/local/mesos-dns/mesos-dns -v -config=/usr/local/mesos-dns/config.json &

Above will run mesos-dns in os background.

  • To launch the same mesos-dns with marathon use below json file,
    Create a mesos-dbs.json file with below content in required location. This file is like a template to invoke mesos-dns application from Marathon.

cat mesos-dns.json

{
“cmd”: “sudo /usr/local/mesos-dns/mesos-dns -v=1 -config=/usr/local/mesos-dns/config.json”,
“cpus”: 0.5,
“mem”: 256,
“id”: “mesos-dns”,
“instances”: 1,
“constraints”: [[“hostname”, “CLUSTER”,”dns_node_IP”]]
}

Here dns_node_IP is the box address where these mesos dns binaries are installed.

Now launch mesos-dns via Marathon using :

curl -X POST -H “Content-Type: application/json” http://<Marathon_Box_IP&gt;:8080/v2/apps -d@mesos-dns.json

  • Modify resolv.conf and dhclient.conf files to have name server IP as mesos-dns server IPEdit and add “nameserver ” to /etc/resolv.conf
    Edit and add “prepend domain-name-servers ” to /etc/dhcp/dhclient.confNow run “dig google.com” to test the functionality of mesos-dns.dig command should return response from mesos-dns server

    dig command will return o/p like below.

    ;; Query time: 8 msec
    ;;SERVER: #53()
    ;;WHEN: Wed Jan 25 14:35:39 SGT 2017
    ;;MSG SIZE rcvd: 44

  • Similarly modify resolv.conf and dhclient.conf file content on all nodes to point to mesos-dns server.
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s