How to Install GlusterFS on Rocky Linux

GlusterFS or Gluster File System is a free and open-source distributed file system developed by RedHat. GlusterFS is a scalable file system formed from several servers into one entity file system that allows users to connect and mount the GlusterFS volume.

GlusterFS is a modern file system that can handle petabytes of data, also it's easy to install and maintenances, and also easy to scale the file system.

In this tutorial, you will learn how to install GlusterFS on two Rocky Linux systems and set up the clustering between two servers. We will also cover how to mount GlusterFS volume on the client machine.

Prerequisite

• Two Rocky Linux servers with an additional disk on each server.
• A root password is configured on both servers

Seting Up FQDN and /etc/hosts

Before starting, you will need to set up the FQDN and /etc/hosts file on each server.

Below is the detailed server that we will be using to create the GlusterFS cluster:

• server1.localdomain.lan with IP address 192.168.10.15
• server2.localdomain.lan with IP address 192.168.10.20

To set up the FQDN on each server, run the hostnamectl command below.

On server1, run the hostnamectl command below.

sudo hostnamectl set-hostname server1.localdomain.lan

And on server2, run the following command.

sudo hostnamectl set-hostname server2.localdomain.lan

Next, edit the configuration /etc/hosts using nano editor.

sudo nano /etc/hosts

Paste the following configuration.

192.168.10.15 server1.localdomain.lan
192.168.10.20 server2.localdomain.lan

Save the file by pressing Ctrl+X and entering Y to confirm.

Now test the connectivity between server1 and server2 using the ping command below.

ping -c3 server1.localdomain.lan
ping -c3 server2.localdomain.lan

Each local domain name will be resolved to the correct IP address of the server.

Setting Up Partition

To deploy the GlusterFS, it is recommended to use different disk storage, especially for the production environment.

For this tutorial, both server1 and server2 have secondary disk /dev/vdb1 with a size of 5GB.

Now edit the /etc/fstab configuration using nano editor to add your disk.

sudo nano /etc/fstab

For server1, the /dev/vda1 disk will be mounted to the /data/vol1 directory.

/dev/vda1 /data/vol1 ext4 default 0 0

And for server2, the /dev/vda1 disk will be mounted to the /data/vol2 directory.

/dev/vda1 /data/vol2 ext4 default 0 0

Next, mount the /dev/vda1 disk using the command below.Advertisement

sudo mount -a

If there is no error message, verify the list mounted disk on each system by running the df command as below.

sudo df -h

You should see the /dev/vda1 disk is mounted to the /data directory.

Now before start installing and configuring the GlusterFS cluster, run the below command to create the brick0 directory inside the /data directory.

On server1, run the below command.

sudo mkdir -p /data/vol1/brick0

For server2, run the below command.

sudo mkdir -p /data/vol2/brick0

Adding GlusterFS Repository for Rocky Linux

After successfully setting up the disk, now you will be installing GlusterFS packages to your Rocky Linux servers.

The current Rocky Linux still not provided GlusterFS server packages, but you can use the CentOS release GlusterFS packages.

Add the GlusterFS repository to your Rocky Linux system using the following command.

sudo dnf install centos-release-gluster9

Type y to confirm the installation and press ENTER to continue.

Next, go to the /etc/yum.repos.d/ directory and edit the repository confioguration CentOS-Gluster-9.repo using nano editor.

cd /etc/yum.repos.d/
sudo nano CentOS-Gluster-9.repo

Change the baseurl line using the Rocky Linux repository and comment the mirrorlist line as below

# CentOS-Gluster-9.repo
#
# Please see http://wiki.centos.org/SpecialInterestGroup/Storage for more
# information

[centos-gluster9]
name=CentOS-$releasever - Gluster 9
#mirrorlist=http://mirrorlist.centos.org?arch=$basearch&release=$releasever&repo=storage-gluster-9
baseurl=https://dl.rockylinux.org/vault/centos/8.5.2111/storage/x86_64/gluster-9/
gpgcheck=1
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Storage

[centos-gluster9-test]
name=CentOS-$releasever - Gluster 9 Testing
baseurl=http://buildlogs.centos.org/centos/$releasever/storage/$basearch/gluster-9/
gpgcheck=0
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Storage

Save the file using Ctrl+X and Y to confirm.

Now check the available repository on your Rocky Linux system using the following command.

sudo dnf repolist

You will see the CentOS release repository for GlusterFS 9.Advertisement

Installing GlusterFS Server

Now install GlusterFS packages on the Rocky Linux system using the dnf command below.

sudo dnf install glusterfs glusterfs-libs glusterfs-server

Type Y to confirm the installation and press ENTER to continue.

After installation is completed, start the glusterfsd service and add it to the system startup using the following command.

sudo systemctl enable glusterfsd.service
sudo systemctl start glusterfsd.service

Now verify the glusterfsd service status using the command below.

sudo systemctl status glusterfsd.service

You will see the glusterfsd service is active with status exited, which means it's running, but the systemd manager cannot find any daemon to monitor.

Opening Port with Firewalld

Before starting to configure the GlusterFS cluster, you will need to open the port for the GlusterFS service.

On RHEL Linux-based operating systems, the default firewall application is firewalld, which provides the command line tool firewall-cmd for managing the firewall.

Add the service glusterfs to your firewalld configuration and reload the firewalld using the command below.

sudo firewall-cmd --add-service=glusterfs --permanent
sudo firewall-cmd --reload

After that, verify the list of services on the firewalld using the following command.

sudo firewall-cmd --list-services

You will see the glusterfs service is added to the firewalld.

Initializing GlusterFS Cluster

The next step will be to initialize and set up the GlusterFS cluster on the Rocky Linux systems.

First, initialize the GlusterFS cluster using the following command. This command must be run on the server1 server and only can be run once.

By using this command, you will be adding the server2.localdomain.lan to the GlusterFS peer on server1. Also, server2 will be automatically initialized after the following command is completed.

sudo gluster peer probe server2.localdomain.lan

Now you should see the output message 'peer probe: success'.

Next, verify the peer status on server1 using the following command.

sudo gluster peer status

You should see there is only one peer on server1.

Also, run the command below on server2 to check available peers on server2.

sudo gluster peer status

You should see that there is only one peer on server2.

Creating GlusterFS Volume

After you have successfully initialized the GlusterFS cluster, you will be creating a new GlusterFS volume that will be used by client machines.

In this tutorial, you will be creating a new volume with the name myvolume with the type replica from 2 disks from server1 and server2.

sudo gluster volume create myvolume replica 2 server1.localdomain.lan:/data/vol1/brick0 server2.localdomain.com:/data/vol2/brick0

Now type y to confirm and press ENTER to create the GlusterFS volume.

When the process is a success, you will see the output like the following screenshot.

Next, start the Glusterfs volume myvolume using the following command.

sudo gluster volume start myvolume

You will see the output message 'volume start: myvolume: success', which means the myvolume is started.

Now run the command below to verify the GLusterFS volume status.

sudo gluster volume status

Below you will see the status of myvolume is online and ready to use for clients.

Optionally, you can also verify the status of the GlusterFS volume using the following command.

sudo gluster volume info

You will see the volume myvolume with type replicate and status is started. The myvolume is based on the disk on server1 and server2.

Mounting GlusterFS Volume on Client Machine

Now you have created the GlusterFS volume myvolume. Next, you will learn how to set up the cline machine and mount the GlusterFS volume.

In this tutorial, we will use the client machine with the name client.

Edit the /etc/hosts configuration on your client machine using nano editor.

sudo nano /etc/hosts

Paste the following configuration.

192.168.10.15 server1.localdomain.lan
192.168.10.20 server2.localdomain.lan

Save the file with Ctrl+X and enter Y to confirm.

To verify the connection between the client machine to the GlusterFS server, run the ping command below.

ping -c3 server1.localdomain.lan

you will see the client command can connect to the GlusterFS server.

Next, install the GlusterFS client package from the AppStream Rocky Linux repository using the following command.

sudo dnf install glusterfs-client

Type y to confirm the installation and press ENTER to continue.

After the installation is completed, create a new mount directory /data using the command below.

sudo mkdir /data

Mount the GlusterFS volume myvolume to the /data directory using the mount command below.

sudo mount.glusterfs server1.localdomain.lan:/myvolume /data

If there is no error, the GlusterFS volume is successfully mounted to your client machine.

Run the following command to verify the GlusterFS volume.

df -h

You will see the GlusterFS myvolume is successfully mounted to the /data directory on the client machine.

Testing Write Access and High Availability

On the client machine, go to the /data directory and create a new file using the below command.

With this command, you will be creating 5 files on the /data directory.

cd /data
touch file{1..5}.md

Now move to server1 and check the directory /data/vol1/brick0 using the following command.

ls /data/vol1/brick0

You should see the file is available on server1.

Next, turn off or shut down server1 using the following command.

sudo shutdown -r now

Move again to server2 and verify the status of GlusterFS on server2 using the following command.

sudo gluster peer status

You will see the status of peer server1 is disconnected.

Verify available files on the /data/vol2/brick0 directory from server2.

ls /data/vol2/brick0

You will see all files from the client is available on server2 too, which means the data is replicated from the peer server.

Now if you go back to the client machine, you will see the status of the connection to the GlusterFS server is still active and running, even when server1 is down. This means the High Availability of GlusterFS is successful.

Conclusion

Congratulation! You have now successfully installed and configured the GlusterFS cluster storage on Rocky Linux systems. Also, you have learned how to set up GlusterFS volume and mount the GlusterFS volume on the client machine.