6

Deploying services with Mesos, Marathon, Zookeeper and Docker

 3 years ago
source link: http://www.linux-admins.net/2016/04/deploying-services-with-mesos-marathon.html
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

Deploying services with Mesos, Marathon, Zookeeper and Docker

In this post I'll demonstrate how to run a scalable and highly available service inside of Docker containers, managed by Marathon, Mesos and Zookeeper.

Here's a quick overview of the different components involved:

- Apache Zookeeper acts as the master election service, state store and high-performance coordination service for distributed applications such as Mesos. [1]

- Apache Mesos abstracts CPU, memory, storage, and other compute resources in a similar way to Open Stack Compute, effectively acting as a cluster manager and intelligent task distributor across a cluster of machines. The development community describes it as the kernel for distributed systems.[2]

- Marathon is a container orchestration platform and is one of the supported frameworks for Apache Mesos. It acts as a process and service manager. The development community describes it as systemd/upstart for the Mesos kernel.[3]

In a previous post I showed how to deploy Apache Zookeeper from source, but in this tutorial I'll do it from a package.
The setup will consist of 3 nodes running the Zookeeper cluster, and 2 nodes in a master/slave setup running Mesos and Marathon.

First let's install and configure the zookeeper service on the 3 nodes:

[server1,2,3]$ sudo rpm -Uvh http://archive.cloudera.com/cdh4/one-click-install/redhat/6/x86_64/cloudera-cdh-4-0.x86_64.rpm [server1,2,3]$ sudo yum -y install zookeeper zookeeper-server [server1,2,3]$ sudo -u zookeeper zookeeper-server-initialize --myid=1 # reaplce with 2 and 3 for server2 and server3 [server1,2,3]$ sudo service zookeeper-server start [server1,2,3]$ sudo chkconfig zookeeper-server on

Make sure that /var/zookeeper/data/myid has a unique id on each of the 3 servers.

To test if zookeeper is working properly lets insert and retreive a key/value (this is somewhat similar to etcd):

[server1]$ cd /usr/lib/zookeeper [server1]$ java -cp zookeeper.jar:lib/slf4j-api-1.6.1.jar:lib/slf4j-log4j12-1.6.1.jar:lib/log4j-1.2.15.jar:conf:lib/jline-0.9.04.jar org.apache.zookeeper.ZooKeeperMain -server 127.0.0.1:2181 create /mytest 1 Created /mytest get /mytest 1 cZxid = 0x50cfe ctime = Fri Apr 08 15:39:21 UTC 2016 mZxid = 0x50cfe mtime = Fri Apr 08 15:39:21 UTC 2016 pZxid = 0x50cfe cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 1 numChildren = 0 quit Quitting... 2016-04-08 15:39:39,949 [myid:] - INFO [main:ZooKeeper@684] - Session: 0x153f61f2f490000 closed

[server1]$ # Or use the provided script [server1]$ /usr/lib/zookeeper/bin/zkCli.sh [zk: localhost:2181(CONNECTED) 0] get /test 1 cZxid = 0x50cfe ctime = Fri Apr 08 15:39:21 UTC 2016 mZxid = 0x50cfe mtime = Fri Apr 08 15:39:21 UTC 2016 pZxid = 0x50cfe cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 1 numChildren = 0 [zk: localhost:2181(CONNECTED) 1] quit Quitting... 2016-04-08 15:41:43,043 [myid:] - INFO [main:ZooKeeper@684] - Session: 0x153f61f2f490001 closed 2016-04-08 15:41:43,128 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@512] - EventThread shut down

Installing Apache Mesos on the master and slave nodes is next:

[server1]$ sudo rpm -Uvh http://repos.mesosphere.com/el/7/noarch/RPMS/mesosphere-el-repo-7-1.noarch.rpm [server1]$ sudo yum -y install mesos [server1]$ echo "zk://192.168.0.10:2181/mesos,192.168.0.11:2181/mesos,192.168.0.12:2181/mesos" > /etc/mesos/zk # Replace the IP's with those of the Zookeeper nodes [server1]$ sudo service mesos-master start [server1]$ sudo service mesos-slave start [server1]$ sudo chkconfig mesos-slave on [server1]$ sudo chkconfig mesos-master on

[server2]$ sudo rpm -Uvh http://repos.mesosphere.com/el/7/noarch/RPMS/mesosphere-el-repo-7-1.noarch.rpm [server2]$ sudo yum -y install mesos [server2]$ echo "zk://192.168.0.10:2181/mesos,192.168.0.11:2181/mesos,192.168.0.12:2181/mesos" > /etc/mesos/zk # Replace the IP's with those of the Zookeeper nodes [server2]$ sudo service mesos-slave start [server2]$ sudo chkconfig mesos-slave on [server2]$ sudo chkconfig mesos-master off

By pointing both Mesos master and slave to the Zookeeper nodes, we delegate election to the Zookeeper service in case the Mesos master fails.
You should be able to access the Mesos GUI on port 5050:

To schedule a test task run:

[server1]$ mesos-execute --master=192.168.0.10:5050 --name="simple_test" --command="sleep 300" I0408 16:14:33.173717 21479 sched.cpp:222] Version: 0.28.0 I0408 16:14:33.179813 21480 sched.cpp:326] New master detected at [email protected]:5050 I0408 16:14:33.180215 21480 sched.cpp:336] No credentials provided. Attempting to register without authentication I0408 16:14:33.224674 21480 sched.cpp:703] Framework registered with 745f2733-768d-4bf3-b78e-d6efd00e4c8f-0000 Framework registered with 745f2733-768d-4bf3-b78e-d6efd00e4c8f-0000

For full list of the configuration parameters for Mesos refer to [4].

With Mesos deployed installing Marathon is next:

[server1,2]$ sudo yum -y install marathon [server1,2]$ sudo service marathon start [server2]$ ps axfww | grep marathon 24865 ? Ssl 0:11 java -Djava.library.path=/usr/local/lib:/usr/lib:/usr/lib64 -Djava.util.logging.SimpleFormatter.format=%2$s%5$s%6$s%n -Xmx512m -cp /usr/bin/marathon mesosphere.marathon.Main --zk zk://192.168.0.10:2181/marathon --master zk://192.168.0.10:2181/mesos

Marathon uses --master to find the Mesos masters, and --zk to find ZooKeepers for storing state. The Marathon GUI is now accessible on port 8080.
For full list of the configuration parameters for Marathon refer to [5].

To run an application on Marathon you can either schedule it using the GUI on port 8080, or the REST API:

[server2]$ mkdir marathon_apps && cd marathon_apps [server2]$ cat <<EOF > my_app.json { "id": "my_app", "cmd": "python -m SimpleHTTPServer $PORTS", "cpus": 0.5, "mem": 32, "instances": 2, "healthChecks": [ { "protocol": "HTTP", "portIndex": 0, "path": "/", "gracePeriodSeconds": 5, "intervalSeconds": 20, "maxConsecutiveFailures": 3 } ], "ports": [8888], "constraints": [ ["hostname", "UNIQUE", ""] ] } EOF [server2]$ curl -X POST "192.168.0.10:8080/v2/apps" -d @my_app.json -H "Content-type: application/json"

To list and delete the application through the API:

[server2]$ curl http://192.168.0.10:8080/v2/apps | python -mjson.tool { "apps": [ { "acceptedResourceRoles": null, "args": null, "backoffFactor": 1.15, "backoffSeconds": 1, "cmd": "python -m SimpleHTTPServer $PORT0", "constraints": [ [ "hostname", "UNIQUE", "" ] ], "container": null, "cpus": 0.5, "dependencies": [], "deployments": [ { "id": "d9f6584e-7894-4515-9cfe-5b720af66082" } ], "disk": 0, "env": {}, "executor": "", "fetch": [], "healthChecks": [ { "gracePeriodSeconds": 5, "ignoreHttp1xx": false, "intervalSeconds": 20, "maxConsecutiveFailures": 3, "path": "/", "portIndex": 0, "protocol": "HTTP", "timeoutSeconds": 20 } ], "id": "/my_app", "instances": 2, "ipAddress": null, "labels": {}, "maxLaunchDelaySeconds": 3600, "mem": 32, "ports": [ 4000 ], "requirePorts": false, "storeUrls": [], "tasksHealthy": 2, "tasksRunning": 2, "tasksStaged": 0, "tasksUnhealthy": 0, "upgradeStrategy": { "maximumOverCapacity": 1, "minimumHealthCapacity": 1 }, "uris": [], "user": null, "version": "2016-04-08T16:52:35.848Z", "versionInfo": { "lastConfigChangeAt": "2016-04-08T16:52:35.848Z", "lastScalingAt": "2016-04-08T16:52:35.848Z" } } ] }

[server2]$ curl -X DELETE http://192.168.0.10:8080/v2/apps/my_app { "deploymentId": "137f8969-97cb-4116-95a8-09901f6d3ad2", "version": "2016-04-08T17:06:04.442Z" }

For a full list of available json config fields refer to [6].

And finally to run a web app from a Docker container:

[server1,2]$ sudo yum install -y device-mapper-event-libs docker [server1,2]$ sudo chkconfig docker on [server1,2]$ sudo service docker start [server1,2]$ echo 'docker,mesos' | sudo tee /etc/mesos-slave/containerizers [server1,2]$ sudo service mesos-slave restart [server1,2]$ docker pull docker.io/httpd [server1]$ cat << EOF > http_docker_app.json { "id": "http-docker-app", "cpus": 0.2, "mem": 32.0, "instances": 1, "constraints": [["hostname", "UNIQUE", ""]], "container": { "type": "DOCKER", "docker": { "image": "docker.io/httpd", "network": "BRIDGE", "portMappings": [ { "containerPort": 8080, "hostPort": 0, "servicePort": 0, "protocol": "tcp" } ] } } } EOF

[server1]$ curl -X POST http://192.168.0.10:8080/v2/apps -d @http_docker_app.json -H "Content-type: application/json" { "acceptedResourceRoles": null, "args": null, "backoffFactor": 1.15, "backoffSeconds": 1, "cmd": null, "constraints": [ [ "hostname", "UNIQUE", "" ] ], "container": { "docker": { "forcePullImage": false, "image": "docker.io/httpd", "network": "BRIDGE", "parameters": [], "portMappings": [ { "containerPort": 8080, "hostPort": 0, "protocol": "tcp", "servicePort": 0 } ], "privileged": false }, "type": "DOCKER", "volumes": [] }, "cpus": 0.2, "dependencies": [], "deployments": [ { "id": "c7d68104-98be-4055-8a59-88830c4a786c" } ], "disk": 0, "env": {}, "executor": "", "fetch": [], "healthChecks": [], "id": "/http-docker-app", "instances": 1, "ipAddress": null, "labels": {}, "maxLaunchDelaySeconds": 3600, "mem": 32, "ports": [ 0 ], "requirePorts": false, "storeUrls": [], "tasks": [], "tasksHealthy": 0, "tasksRunning": 0, "tasksStaged": 0, "tasksUnhealthy": 0, "upgradeStrategy": { "maximumOverCapacity": 1, "minimumHealthCapacity": 1 }, "uris": [], "user": null, "version": "2016-04-08T17:18:30.387Z" }

[server1]$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 5e3dc19cfa84 docker.io/httpd "httpd-foreground" 3 minutes ago Up 3 minutes 80/tcp, 0.0.0.0:31372->8080/tcp mesos-cfccf3bb-3168-4ec5-99da-0bb9fa618a9c-S0.b5487c02-e054-4013-9bfc-eeb6c4387b15

Resources:
[1] https://zookeeper.apache.org/doc/r3.4.8/
[2] http://mesos.apache.org/documentation/latest/
[3] https://mesosphere.github.io/marathon/docs/
[4] http://mesos.apache.org/documentation/latest/configuration/
[5] https://mesosphere.github.io/marathon/docs/command-line-flags.html
[6] https://mesosphere.github.io/marathon/docs/rest-api.html#post-v2-apps

report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK