Introduction to Docker and Using Percona Server for MongoDB
source link: https://www.tuicool.com/articles/BzUN7n6
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
In this blog post, I’ll walk through using Docker containers and discuss the advantages of using them to run multiple versions of Percona Server for MongoDB (PSMDB) instances in the same server. This is particularly useful for local testing. Lots of companies have started to use containers for their applications, due to the advantages of this model, and due to their hassle-free deployment. I’ll show you how to configure variables for MongoDB® when using docker containers. Hopefully, this will be a useful introduction that’ll help you to start working with Dockers yourself. Let’s get into the topic…
Experiments and testing…
It is always been fun to work with test cases for our databases. When I do that, I generally use VMs, Dockers or a test a machine – hmm! I know what you think… your boss should give you one if he wants you to solve problems by testing. There are still more things that you could try. As we are working inside Percona, this might be MongoDB operators (see early release here: https://www.percona.com/blog/2019/02/21/percona-server-for-mongodb-operator-0-2-1-early-access-release-is-now-available/ ) which is light weight packed MongoDB instance and these could be used with kubernetes + minikube or docker too to deploy and using it. You can expect more information about how this might work from our folks very soon!
What is Docker?
Here’s a basic Docker architecture diagram:
If you want to start your application quickly using lightweight software that packages up code and all its dependencies, then you need to go for containers. These use the host operating system and share the relevant libraries. In simple terms, a Docker engine is a platform that helps you to run multiple containers easily within the same host operating system. It allows you to quickly initialize databases & their applications and start to start to use them almost immediately. From the above diagram, you can see that we have a host OS, and Docker runs on that, sharing its kernel. We can start containers on top of Docker. For example, on my Macbook if I use Docker I can start using PSMDB 3.6 running with CentOS 7, Percona Server MySQL 5.7, Ubuntu Xenial etc all together. These containers don’t contradict each other with dependencies even though they have different configurations and you can customise them as per your requirement.
Vinodhs-MBP:~ vinodhkrish$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 758de9eda080 percona/percona-server-mongodb:3.6.10 "/entrypoint.sh --po…" 26 hours ago Up 6 minutes 27017/tcp, 0.0.0.0:27021->27021/tcp psmdb36-20a933d72874c percona:ps-5.7 "/docker-entrypoint.…" 27 hours ago Up 5 minutes 3306/tcp, 0.0.0.0:3310->3310/tcp ps57
Docker basics:
You should install Docker first. I’ve used this link for Mac OS. Then download the related image – which consists of the necessary software and its dependencies packaged together – as per your requirement. Now let’s see how to download a simple image and check how it works. Hmm… “ hello world ” is handy to test anything anywhere!
Vinodhs-MBP:~ vinodhkrish$ docker --version Docker version 18.09.3, build 774a1f4 Vinodhs-MBP:~ vinodhkrish$ docker run hello-world Unable to find image 'hello-world:latest' locally latest: Pulling from library/hello-world 1b930d010525: Pull complete Digest: sha256:92695bc579f31df7a63da6922075d0666e565ceccad16b59c3374d2cf4e8e50e Status: Downloaded newer image for hello-world:latest Hello from Docker! This message shows that your installation appears to be working correctly. To generate this message, Docker took the following steps: 1. The Docker client contacted the Docker daemon. 2. The Docker daemon pulled the "hello-world" image from the Docker Hub. (amd64) 3. The Docker daemon created a new container from that image which runs the executable that produces the output you are currently reading. 4. The Docker daemon streamed that output to the Docker client, which sent it to your terminal. To try something more ambitious, you can run an Ubuntu container with: $ docker run -it ubuntu bash Share images, automate workflows, and more with a free Docker ID: https://hub.docker.com/ For more examples and ideas, visit: https://docs.docker.com/get-started/
Check the list of images
This command tells you all the images available. These images are used to create a container and build your application. When you start a container, and if no related image exists, then the Docker engine will pull them from the Docker hub where lot of images are shared. An aside: you could build an image and share it there too if you wanted too. You can see lot of images below that I’ve downloaded and and have been using already for my own purposes. So don’t worry if you can’t find this exact list.
Vinodhs-MBP:~ vinodhkrish$ docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE percona/pmm-client-dock latest ee5230d0a1b5 2 days ago 429MB percona/pmm-client latest 52972ab72568 2 days ago 502MB percona/percona-server-mongodb 3.6.11 db2aad78e8c0 8 weeks ago 483MB percona ps-5.7 69377a52e49a 8 weeks ago 583MB centos latest 9f38484d220f 8 weeks ago 202MB ubuntu 16.04 9361ce633ff1 2 months ago 118MB ubuntu xenial 9361ce633ff1 2 months ago 118MB percona/pmm-server 1 deac61541b67 2 months ago 1.08GB percona/pmm-server latest deac61541b67 2 months ago 1.08GB docker/kube-compose-controller v0.4.18 47df7579c5fc 4 months ago 30.6MB docker/kube-compose-api-server v0.4.18 e73645df5dc6 4 months ago 47.8MB hello-world latest fce289e99eb9 4 months ago 1.84kB
Check the list of all containers (running and stopped)
You can see the containers that I have created so far on my Mac:
Vinodhs-MBP:~ vinodhkrish$ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 3b3af3ad561d percona:ps-5.7 "/docker-entrypoint.…" 27 hours ago Exited (1) 27 hours ago ps57-5 676752882d2e percona:ps-5.7 "/docker-entrypoint.…" 27 hours ago Exited (1) 27 hours ago ps57-4 45849aaefc15 percona:ps-5.7 "/docker-entrypoint.…" 27 hours ago Exited (1) 27 hours ago ps57-3 6ee0689cb3ea percona:ps-5.7 "/docker-entrypoint.…" 27 hours ago Exited (1) 27 hours ago ps57-2 0a933d72874c percona:ps-5.7 "/docker-entrypoint.…" 27 hours ago Up 5 minutes 3306/tcp, 0.0.0.0:3310->3310/tcp ps57 204ee0b19ad1 ubuntu:xenial "/bin/bash" 28 hours ago Exited (255) 7 minutes ago ubuntuOSxenial f63412062bbf ubuntu:16.04 "/bin/bash" 28 hours ago Exited (255) 7 minutes ago ubuntuOS
Start PSMDB with the Docker
Now it’s time to start PSMDB with Docker. After the preliminary tests above, you’re good to go. Let’s start pulling the image for PSMDB 3.6.10:
Vinodhs-MBP:~ vinodhkrish$ docker pull percona/percona-server-mongodb:3.6.10 Unable to find image 'percona/percona-server-mongodb:3.6.10' locally 3.6.10: Pulling from percona/percona-server-mongodb a02a4930cb5d: Already exists baaac93628af: Pull complete 9fa06d7f2f7b: Pull complete 3b1fa6c4024d: Pull complete 4a7531f5f348: Pull complete e3c3a566fa23: Pull complete Digest: sha256:173ad141b88adec95e39848284b752a85f5b4370fccc3c4e33a773d378663c0b Status: Downloaded newer image for percona/percona-server-mongodb:3.6.10
Now start your database, a docker instance of PSMDB 3.6.10 , using the image downloaded. Here’s the command:
Vinodhs-MBP:~ vinodhkrish$ docker run -d --name psmdb36-1 -p 27051:27051 percona/percona-server-mongodb:3.6.10 a5f81bd47f85cf3638a65586813d536784d075ea3c0eb8bf74517066b73a32c7 Vinodhs-MBP:~ vinodhkrish$ docker exec -it psmdb36-1 mongo --version Percona Server for MongoDB shell version v3.6.10-3.0 git version: da6faf62946711bd29a9c30806dc9c535eb1becb OpenSSL version: OpenSSL 1.0.2k-fips 26 Jan 2017 allocator: tcmalloc modules: none build environment: distarch: x86_64 target_arch: x86_64
List the running Docker containers and check whether the container you’ve created is running. You should see all those that are currently running:
Vinodhs-MBP:~ vinodhkrish$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 758de9eda080 percona/percona-server-mongodb:3.6.10 "/entrypoint.sh --po…" 26 hours ago Up 6 minutes 27017/tcp, 0.0.0.0:27021->27021/tcp psmdb36-2
Now login into the database as shown below:
Vinodhs-MBP:~ vinodhkrish$ docker exec -it psmdb36-1 mongo
Percona Server for MongoDB shell version v3.6.10-3.0
connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("9ab230db-35b9-47ee-9aad-cb759f8d14af") }
Percona Server for MongoDB server version: v3.6.10-3.0
Welcome to the Percona Server for MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
https://www.percona.com/doc/percona-server-for-mongodb
Questions? Try the support group
https://www.percona.com/forums/questions-discussions/percona-server-for-mongodb
2019-04-27T14:58:55.400+0000 I STORAGE [main] In File::open(), ::open for '/home/mongodb/.mongorc.js' failed with No such file or directory
Server has startup warnings:
2019-04-27T14:58:25.939+0000 I STORAGE [initandlisten]
2019-04-27T14:58:25.939+0000 I STORAGE [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2019-04-27T14:58:25.939+0000 I STORAGE [initandlisten] ** See http://dochub.mongodb.org/core/prodnotes-filesystem
2019-04-27T14:58:26.511+0000 I CONTROL [initandlisten]
2019-04-27T14:58:26.511+0000 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.
2019-04-27T14:58:26.511+0000 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.
2019-04-27T14:58:26.511+0000 I CONTROL [initandlisten] ** You can use percona-server-mongodb-enable-auth.sh to fix it.
2019-04-27T14:58:26.511+0000 I CONTROL [initandlisten]
> show dbs
admin 0.000GB
config 0.000GB
local 0.000GB
You can run the Linux commands as follows:
Vinodhs-MBP:~ vinodhkrish$ docker exec -it psmdb36-1 uname -a Linux a5f81bd47f85 4.9.125-linuxkit #1 SMP Fri Sep 7 08:20:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Vinodhs-MBP:~ vinodhkrish$
If you would like to work with the command prompt within the Docker instance, then you can start bash and work through that shell:
Vinodhs-MBP:~ vinodhkrish$ docker exec -it psmdb36-1 /bin/bash bash-4.2$ bash-4.2$ df -kh Filesystem Size Used Avail Use% Mounted on overlay 59G 4.6G 51G 9% / tmpfs 64M 0 64M 0% /dev tmpfs 1000M 0 1000M 0% /sys/fs/cgroup /dev/sda1 59G 4.6G 51G 9% /data/db shm 64M 0 64M 0% /dev/shm tmpfs 1000M 0 1000M 0% /proc/acpi tmpfs 1000M 0 1000M 0% /sys/firmware bash-4.2$
Check mongod help
If you would like to change the parameters for MongoDB while starting the instance, you can check the options available to you as follows: start an instance, show the output, and destroy the container. In fact, this blog arose from a particular question from a community member: “How do I check and change mongodb settings in Docker?”
There are many ways to achieve that requirement and this is one way:
Vinodhs-MBP:~ vinodhkrish$ docker run -it --rm percona/percona-server-mongodb:3.6.10 mongod --help Options: General options: -h [ --help ] show this usage information --version show version information -f [ --config ] arg configuration file specifying additional options -v [ --verbose ] [=arg(=v)] be more verbose (include multiple times for more verbosity e.g. -vvvvv) --quiet quieter output --port arg specify port number - 27017 by default --bind_ip arg comma separated list of ip addresses to listen on - localhost by default --bind_ip_all bind to all ip addresses --ipv6 enable IPv6 support (disabled by default) --listenBacklog arg (=128) set socket listen backlog size --maxConns arg max number of simultaneous connections - 1000000 by default --logpath arg log file to send write to instead of stdout - has to be a file, not directory --syslog log to system's syslog facility instead of file or stdout --syslogFacility arg syslog facility used for mongodb syslog message --logappend append to logpath instead of over-writing --logRotate arg set the log rotation behavior (rename|reopen) --timeStampFormat arg Desired format for timestamps in log messages. One of ctime, iso8601-utc or iso8601-local …. ….
Custom mongod settings:
To start the new docker instance with custom MongoDB settings, you can start the container afresh like this:
Vinodhs-MBP:~ vinodhkrish$ docker run -d --name psmdb36-custom -v /mongodb/data:/var/lib/mongo -v /mongodb/log:/var/log/mongodb/ -p 27051:27051 percona/percona-server-mongodb:3.6.10 mongod --dbpath /var/lib/mongo --logpath /var/log/mongodb/mongod.log --logappend --engine wiredtiger —port 27041 79176f0a9b9073c0c379fec697286c455d3c8dc946fabeafbc1621f0f8e4b6b8 Vinodhs-MBP:~ vinodhkrish$ Vinodhs-MBP:~ vinodhkrish$ Vinodhs-MBP:~ vinodhkrish$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Vinodhs-MBP:~ vinodhkrish$
The container that’s been started is not showing in the running containers process list. This suggests that there is some issue with the container.
Let’s check the log (the eagle eyed might have already noticed the problem in the command that I first used):
Vinodhs-MBP:~ vinodhkrish$ docker logs psmdb36-custom Error parsing command line: unrecognised option '--engine' try 'mongod --help' for more information
It’s the parameter name. Let’s use the correct parameter and start again.
Note: Even though there was an error, the docker command created the container with the name “psmdb36-custom” anyhow. So, to be able to use it again we need to remove it first and then restart with the same name:
Vinodhs-MBP:~ vinodhkrish$ docker rm psmdb36-custom psmdb36-custom Vinodhs-MBP:~ vinodhkrish$ Vinodhs-MBP:~ vinodhkrish$ docker run -d --name psmdb36-custom -v /mongodb/data:/var/lib/mongo -v /mongodb/log:/var/log/mongodb/ -p 27041:27041 percona/percona-server-mongodb:3.6.10 mongod --dbpath /var/lib/mongo --logpath /var/log/mongodb/mongod.log --logappend --storageEngine wiredTiger --port 27041 d0803cd7c6dcbd252533784faea00abc635d8f2042ca66360421d5acadebca91 Vinodhs-MBP:~ vinodhkrish$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d0803cd7c6dc percona/percona-server-mongodb:3.6.10 "/entrypoint.sh mong…" 4 seconds ago Up 3 seconds 27017/tcp, 0.0.0.0:27041->27041/tcp psmdb36-custom Vinodhs-MBP:~ vinodhkrish$
Now check your setting in the mongod instance:
Vinodhs-MBP:~ vinodhkrish$ docker exec -it psmdb36-custom mongo localhost:27041/admin --eval "db.adminCommand({getCmdLineOpts:1})"
Percona Server for MongoDB shell version v3.6.10-3.0
connecting to: mongodb://localhost:27041/admin?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("5b68bc78-621a-461a-a58e-524e588d8a0c") }
Percona Server for MongoDB server version: v3.6.10-3.0
{
"argv" : [
"mongod",
"--dbpath",
"/var/lib/mongo",
"--logpath",
"/var/log/mongodb/mongod.log",
"--logappend",
"--storageEngine",
"wiredTiger",
"--port",
"27041",
"--bind_ip_all"
],
"parsed" : {
"net" : {
"bindIpAll" : true,
"port" : 27041
},
"storage" : {
"dbPath" : "/var/lib/mongo",
"engine" : "wiredTiger"
},
"systemLog" : {
"destination" : "file",
"logAppend" : true,
"path" : "/var/log/mongodb/mongod.log"
}
},
"ok" : 1
}
You’ll see that above we have used the option - v / mongodb / data : / var / lib / mongo - v / mongodb / log : / var / log / mongodb / This allow you to access data files from a local machine and map that local storage as your docker image partitions
Vinodhs-MBP:~ vinodhkrish$ docker exec -it psmdb36-custom df -kh Filesystem Size Used Avail Use% Mounted on overlay 59G 4.6G 51G 9% / tmpfs 64M 0 64M 0% /dev tmpfs 1000M 0 1000M 0% /sys/fs/cgroup /dev/sda1 59G 4.6G 51G 9% /data/db shm 64M 0 64M 0% /dev/shm osxfs 113G 87G 19G 83% /var/lib/mongo tmpfs 1000M 0 1000M 0% /proc/acpi tmpfs 1000M 0 1000M 0% /sys/firmware
Available locally, the files exist in the file system like this, below. Be careful that you don’t alter them manually as they are db files:
Vinodhs-MBP:~ vinodhkrish$ ls -l /mongodb/data/ total 376 -rw------- 1 vinodhkrish staff 45 May 9 14:47 WiredTiger -rw------- 1 vinodhkrish staff 21 May 9 14:47 WiredTiger.lock -rw------- 1 vinodhkrish staff 1103 May 9 14:50 WiredTiger.turtle -rw------- 1 vinodhkrish staff 57344 May 9 14:50 WiredTiger.wt -rw------- 1 vinodhkrish staff 4096 May 9 14:48 WiredTigerLAS.wt -rw------- 1 vinodhkrish staff 16384 May 9 14:49 _mdb_catalog.wt -rw------- 1 vinodhkrish staff 16384 May 9 14:49 collection-0-3582398392962612942.wt -rw------- 1 vinodhkrish staff 16384 May 9 14:49 collection-2-3582398392962612942.wt -rw------- 1 vinodhkrish staff 4096 May 9 14:48 collection-4-3582398392962612942.wt drwx------ 4 vinodhkrish staff 128 May 9 14:51 diagnostic.data -rw------- 1 vinodhkrish staff 16384 May 9 14:49 index-1-3582398392962612942.wt -rw------- 1 vinodhkrish staff 16384 May 9 14:49 index-3-3582398392962612942.wt -rw------- 1 vinodhkrish staff 4096 May 9 14:48 index-5-3582398392962612942.wt -rw------- 1 vinodhkrish staff 4096 May 9 14:49 index-6-3582398392962612942.wt drwx------ 5 vinodhkrish staff 160 May 9 14:48 journal -rw------- 1 vinodhkrish staff 2 May 9 14:48 mongod.lock -rw------- 1 vinodhkrish staff 16384 May 9 14:50 sizeStorer.wt -rw------- 1 vinodhkrish staff 114 May 9 14:48 storage.bson Vinodhs-MBP:~ vinodhkrish$ ls -l /mongodb/log/ total 16 -rw------- 1 vinodhkrish staff 7588 May 9 14:51 mongod.log
Conclusion
I hope this post helps you to get going with Docker containers and to start using them straightaway. You can see how it could allow you to start different versions of MongoDB for the purposes of testing. As you start to explore, you’ll find you have other options available such as Docker swarm for maintaining Docker Engine clusters, and technologies, such as Kubernetes with Docker, that can help you to run entire applications and maintain nodes automatically. And I’ve already mentioned, Percona likes to useMongoDB operators which are in early release at the time of writing this blog. Using a similar approach, you could test other databases such as Percona Server for MySQL.
Let me know your thoughts and share any tricks you might know that would help users to run Docker containers smoothly.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK