8

Let’s build something Outrageous – Part 9: Docker | Max De Marzi

 3 years ago
source link: https://maxdemarzi.com/2021/08/03/lets-build-something-outrageous-part-9-docker/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

Let’s build something Outrageous – Part 9: Docker

In the movie “Field of Dreams“, a voice can be heard saying “If you build it, he will come“. For some reason people got that mixed up with “if you build it, they will come” and it became a bit of a trap that many engineers fall for. The myth being that if you build a better contraption everyone will want to use it. But that is not how the world works. You have to win the hearts and minds of the people who may want to use your product… just ask Apple which posted record results yet again.

Distribution is the name of the game. How do we get RageDB in the hands of people to try it out, give feedback and help improve it? Well one thing we can do is make it easy to try. Nobody wants to spend 30 minutes installing a bunch of dependencies and compiling software. This is where Docker comes in.

I put together a multi-stage docker image to build and deploy RageDB. I am not a DevOps engineer or much of a Docker user, if you have a way to improve it, please send me a pull request! Let’s take a look:

FROM ubuntu:latest as build
ARG DEBIAN_FRONTEND=noninteractive
RUN echo "deb http://mirrors.kernel.org/ubuntu hirsute main universe" | tee -a /etc/apt/sources.list

We start off with Ubuntu latest which is the LTS currently at 20.04, ’cause it’s my favorite Linux variant. We go into noninteractive mode so none of the steps ask us any questions since we won’t be around to answer them. Then I add the hirsute repository (which is Ubuntu 21.04 for some newer dependencies). Once Ubuntu 22.04 LTS comes out that won’t be needed.

RUN apt-get -qq update && apt-get -qq install -y build-essential git sudo pkg-config ccache python3-pip luajit luajit-5.1-dev \
valgrind libfmt-dev gcc-11 g++-11 ninja-build ragel libhwloc-dev libnuma-dev libpciaccess-dev libcrypto++-dev libboost-all-dev \
libxml2-dev xfslibs-dev libgnutls28-dev liblz4-dev libsctp-dev gcc make libprotobuf-dev protobuf-compiler python3 systemtap-sdt-dev \
libtool cmake libyaml-cpp-dev libc-ares-dev stow
RUN sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 60 --slave /usr/bin/g++ g++ /usr/bin/g++-11
RUN sudo update-alternatives --auto gcc

Then I install a whole bunch of dependencies for the building of Seastar and RageDB, then set our compiler to 11.

RUN pip install --user conan
RUN sudo ln -s ~/.local/bin/conan /usr/bin/conan
RUN git clone https://github.com/scylladb/seastar.git /data/seastar
WORKDIR /data/seastar
RUN ./configure.py --mode=release --prefix=/usr/local
RUN sudo ninja -C build/release install
RUN rm -rf /data/seastar/*

Next we grab Conan which we will use later to build RageDB, but first we need to clone seastar, configure it, build it and install it. This step takes a while… like a long while. Once it’s installed we can delete it (but it doesn’t seem to do anything… help me Docker experts).

RUN git clone https://github.com/ragedb/ragedb.git /data/rage
RUN mkdir /data/rage/build
WORKDIR /data/rage/build
RUN cmake .. -DCMAKE_BUILD_TYPE=Release
RUN cmake --build . --target ragedb

With all the dependencies ready to go, we can now clone ragedb and build it. This concludes the first part of our Dockerfile, now on to the second part which runs RageDB. The first time I built this Docker image I just ran it from there, but it created a 13 GB image which was way too big. Then I decided to split it and just install the dependencies I needed which brought it down to just over 1 GB. But then Avi told me I just needed to just copy the ragedb binary and the result of ldd ragedb  into the container. So we’re giving that a try here:

FROM ubuntu:latest
RUN echo "deb http://mirrors.kernel.org/ubuntu hirsute main universe" | tee -a /etc/apt/sources.list
RUN apt-get -qq update && apt-get -qq install -y luajit libc6

I once again start from Ubuntu latest, add hirsute and install dependencies. In this case, I think I just need luajit and libc6. Then I copy the results of ldd ragedb I am going to spare you the long list, take a look at the Dockerfile for the rest.

COPY --from=build /lib/x86_64-linux-gnu/libboost_program_options.so.1.74.0 /lib/x86_64-linux-gnu/
COPY --from=build /lib/x86_64-linux-gnu/libboost_thread.so.1.74.0 /lib/x86_64-linux-gnu/
COPY --from=build "/lib/x86_64-linux-gnu/libcrypto++.so.8" /lib/x86_64-linux-gnu/
...

Now I’m ready to copy the ragedb binary, expose the port and set the entry point:

WORKDIR /ragedb
COPY --from=build /data/rage/build/bin/ragedb ragedb
EXPOSE 7243/tcp
ENTRYPOINT ./ragedb

I pushed to Docker Hub under ragedb/ragedb:latest and now I can run it from my laptop:

Let’s run it and when we go take a look at http://localhost:7243/db/rage/health_check we see:

["Shard 0 is OK","Shard 1 is OK","Shard 2 is OK","Shard 3 is OK",
"Shard 4 is OK","Shard 5 is OK","Shard 6 is OK","Shard 7 is OK"]

Sweet! If you wan to play along at home give this a shot:

docker run --cap-add=sys_nice -p 7243:7243 --name ragedb -t ragedb/ragedb:latest

The sys_nice part is to get passed this warning message:

seastar - Unable to set SCHED_FIFO scheduling policy for timer thread; latency impact possible.

You may still get these warnings on MacOs (and maybe Windows, I haven’t tried it, please do and report back):

WARNING: unable to mbind shard memory; performance may suffer:

So it seems you are better off running it from a Linux host or building it manually, but if you just want to play with it without hassle then Docker is the easiest way to go. Don’t forget to join me on Slack… and for a preview of our next installment checkout ragedb.com.


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK