Install Graylog Server on Debian 11/10 with Let's Encrypt | ComputingForGeeks

In this guide, we will chew through how to install Graylog server on Debian 11/10 with Let’s Encrypt SSL. But before we dive into the crux of this tool, let’s get to understand what it is all about. Graylog is a free and open open-source log management and aggregation tool used to store, analyze and send alerts from collected logs. It can be used in the analysis of both structured and unstructured data. Graylog is based on the following components:

• Java /OpenJDK– which is used as a runtime environment for ElasticSearch.
• ElasticSearch– this is the log analysis tool for the Graylog Server.
• MongoDB – it stores the data and configurations.
• Graylog Server– The sever that passes logs for visualization using the provides a built-in-web Interface.

Setup Requirements.

1. Memory above 4 GB.
2. Storage above 20 GB.
3. 4 CPU cores
4. Debian 10/11 installed and updated.
5. All packages upgraded.

With all the above requirements met, let us dive into the installation process.

Step 1: Install Java on Debian 11/10

In Graylog installation, the recommended Java version is any version above Java 8. In this guide, we will use Java OpenJDK 11. Install Java OpenJDK 11 on Debian 11/10 using the below command.

sudo apt update
sudo apt install -y apt-transport-https openjdk-11-jre-headless uuid-runtime pwgen curl dirmngr

Verify the installed version of Java.

$ java -version
openjdk version "11.0.12" 2021-07-20
OpenJDK Runtime Environment (build 11.0.12+7-post-Debian-2)
OpenJDK 64-Bit Server VM (build 11.0.12+7-post-Debian-2, mixed mode, sharing)

Step 2: Install ElasticSearch on Debian 11/10.

Here Elasticsearch is used to store and offer real-time analysis of logs with a RESTful web interface.

It stores data sent from the Graylog server and displays messages when requested by the user in a web interface.

Let’s add first add the Elasticsearch GPG key.

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

Then add the Elasticsearch repository on Debian 11/10

echo "deb https://artifacts.elastic.co/packages/oss-7.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-7.x.list

With the repository added, install Elasticseach as below.

sudo apt update
sudo apt install -y elasticsearch-oss

Then do configurations to the YAML file and set the cluster name to graylog as below.

sudo apt install vim
sudo vim /etc/elasticsearch/elasticsearch.yml

Find and set the cluster name and add the lines below :

cluster.name: graylog
action.auto_create_index: false

Reload and start Elasticsearch service as below.

sudo systemctl daemon-reload
sudo systemctl start elasticsearch
sudo systemctl enable elasticsearch

Check the status of the service:

$ systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
     Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2021-08-26 20:41:39 UTC; 2s ago
       Docs: https://www.elastic.co
   Main PID: 16515 (java)
      Tasks: 50 (limit: 4678)
     Memory: 1.1G
        CPU: 22.106s
     CGroup: /system.slice/elasticsearch.service
             └─16515 /usr/share/elasticsearch/jdk/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -X>

Aug 26 20:41:28 logs.techwizpro.com systemd[1]: Starting Elasticsearch...
Aug 26 20:41:39 logs.techwizpro.com systemd[1]: Started Elasticsearch.

In case Elasticsearch fails to start, you will need to edit etc/elasticsearch/jvm.options to suit your available RAM.

sudo vim /etc/elasticsearch/jvm.options

In the file, find and replace the options below if your RAM is below 4GB.

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space
-Xms512m
-Xmx512m

Then start Elasticsearch and proceed as below.

By default, Elasticsearch runs on port 9200, verify this using the below cURL command.

curl -X GET http://localhost:9200

Sample Output:

{
  "name" : "logs.techwizpro.com",
  "cluster_name" : "graylog",
  "cluster_uuid" : "BGPuR_1OQaOF_YWgvXDxeQ",
  "version" : {
    "number" : "7.10.2",
    "build_flavor" : "oss",
    "build_type" : "deb",
    "build_hash" : "747e1cc71def077253878a59143c1f785afa92b9",
    "build_date" : "2021-01-13T00:42:12.435326Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Step 3: Install MongoDB on Debian 11/10

We will install MongoDB by adding repositories as below.

wget -qO - https://www.mongodb.org/static/pgp/server-5.0.asc | sudo apt-key add -
sudo apt update

Add the MongoDB repositories as below.

echo "deb http://repo.mongodb.org/apt/debian buster/mongodb-org/5.0 main" | sudo tee /etc/apt/sources.list.d/mongodb-org-5.0.list

Update and install MongoDB as below.

sudo apt-get update
sudo apt-get install -y mongodb-org mongodb-org-database mongodb-org-server mongodb-org-shell mongodb-org-mongos mongodb-org-tools

Then start and enable MongoDB service to run on boot.

sudo systemctl start mongod
sudo systemctl enable mongod

Verify that the service is running.

$ systemctl status mongod
● mongod.service - MongoDB Database Server
     Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2021-08-26 10:52:37 UTC; 10s ago
       Docs: https://docs.mongodb.org/manual
   Main PID: 22633 (mongod)
     Memory: 66.9M
        CPU: 715ms
     CGroup: /system.slice/mongod.service
             └─22633 /usr/bin/mongod --config /etc/mongod.conf

Aug 26 10:52:37 techwizpro.com systemd[1]: logs.techwizpro.com systemd[1]: Started MongoDB Database Server.

Step 4: Install Graylog Server on Debian 11/10

Graylog server accepts and processes the machine logs then displays them on a web interface on request.

Download and install Graylog repository package on Debian 11/10.

wget https://packages.graylog2.org/repo/packages/graylog-4.1-repository_latest.deb
sudo dpkg -i graylog-4.1-repository_latest.deb

Update the cache repositories and install Graylog server.

sudo apt update
sudo apt install -y graylog-server

Then we will use the pwgen command below to generate a secret to secure user passwords as below.

pwgen -N 1 -s 96

Output:

98KM6k7W6CtfQPc0EFKS3EMsb3bgYK1qPwDZcNezkqx4usSOMZE1rbKtuHuRwllkzm37cAp5U07jD9Hv6hCybkk3vJdVlC38

Copy the secret code and use it below.

Edit the .conf file

sudo vim /etc/graylog/server/server.conf

In the file, paste the secret as below:

password_secret = 98KM6k7W6CtfQPc0EFKS3EMsb3bgYK1qPwDZcNezkqx4usSOMZE1rbKtuHuRwllkzm37cAp5U07jD9Hv6hCybkk3vJdVlC38

In the .conf file, also add the below lines.

rest_listen_uri = http://127.0.0.1:9000/api/
web_listen_uri = http://127.0.0.1:9000/

Save and exit. then proceed and create a sha256 password for the administrator. The password created will be used to login into the web interface.

echo -n Str0ngPassw0rd | sha256sum

In the above command, replace “Str0ngPassw0rd” with your preferred password. Alternatively, you can set a password with the command below.

$ echo -n "Enter Password: " && head -1 </dev/stdin | tr -d '\n' | sha256sum | cut -d" " -f1
Enter Password: Str0ngPassw0rd

The output will be as the below:

ab38eadaeb746599f2c1ee90f8267f31f467347462764a24d71ac1843ee77fe3

Copy this output and use it in the below step.

Edit the /etc/graylog/server/server.conf file as below.

sudo vim /etc/graylog/server/server.conf

In the file, add root_password_sha2 =”the above output” as below.

root_password_sha2 = ab38eadaeb746599f2c1ee90f8267f31f467347462764a24d71ac1843ee77fe3

With this done, the Graylog server is now ready for use. Start and enable the service as below.

sudo systemctl daemon-reload
sudo systemctl restart graylog-server
sudo systemctl enable graylog-server

Check the log as below.

sudo tail -f /var/log/graylog-server/server.log

If the server is running successfully, you should see the output as below.

................
021-08-26T12:44:31.839Z INFO  [ServerBootstrap] Graylog server up and running.
2021-08-26T12:44:31.839Z INFO  [ServiceManagerListener] Services are healthy
....................

If you want to access the Graylog Web Interface with a server IP and port, then set http_bind_address to your public hostname or public IP address as below

sudo vim /etc/graylog/server/server.conf

Then add the below line:

http_bind_address = 0.0.0.0:9000

Restart the service for the changes made to apply.

sudo systemctl restart graylog-server

With this done, you can access the Graylog server Web UI using the URL http://server_IP:9000 or http://hostname:9000

Step 5: Setup Graylog Server using Let’s Encrypt Certificate

This step is optional and you should only proceed after meeting the below requirements.

• Fully qualified domain name(FQDN)
• Nginx
• Let’sEncrypt certificate

Here we want to set SSL so that we can be able to access Graylog Web Interface using HTTPS. First, install and configure Nginx as below.

sudo apt install nginx

Configure the firewall if you are using ufw run:

sudo ufw allow 'Nginx Full'

Next, we will create a VirtualHost with your domain name as below.

sudo vim /etc/nginx/sites-available/graylog.domain.com.conf

In the file, add the below content replacing appropriately.

server {
  listen 80;
  server_name           logs.example.com;

  return 301            https://$host$request_uri;
  access_log            /var/log/nginx/graylog_access.log combined;
  error_log             /var/log/nginx/graylog_error.log;
}

In the file, replace logs.example.com with your own FQDN.

Create a symbolic link between /etc/nginx/sites-available/graylog.yourdomain.com.conf and /etc/nginx/sites-enabled as below.

sudo ln -s /etc/nginx/sites-available/graylog.domain.com.conf /etc/nginx/sites-enabled/

Check the syntax of the created file.

$ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Now, let’s install Let’sEncrypt with certbot on Debian 11/10

sudo apt install certbot python3-certbot-nginx

Then run Certbot for Nginx as below.

sudo certbot --nginx

Output:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator nginx, Installer nginx
Enter email address (used for urgent renewal and security notices)
 (Enter 'c' to cancel): Enter Your Email Address

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. You must
agree in order to register with the ACME server. Do you agree?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: y

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Would you be willing, once your first certificate is successfully issued, to
share your email address with the Electronic Frontier Foundation, a founding
partner of the Let's Encrypt project and the non-profit organization that
develops Certbot? We'd like to send you email about our work encrypting the web,
EFF news, campaigns, and ways to support digital freedom.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: y
Account registered.

Which names would you like to activate HTTPS for?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: logs.techwizpro.com
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel): 1
Requesting a certificate for logs.techwizpro.com
Performing the following challenges:
http-01 challenge for logs.techwizpro.com
Waiting for verification...
Cleaning up challenges
Deploying Certificate to VirtualHost /etc/nginx/sites-enabled/graylog.domain.com.conf
Redirecting all traffic on port 80 to ssl in /etc/nginx/sites-enabled/graylog.domain.com.conf
...............

When this is successful, you should see congratulations as below.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations! You have successfully enabled https://logs.techwizpro.com
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Subscribe to the EFF mailing list (email: [email protected]).

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/logs.techwizpro.com/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/logs.techwizpro.com/privkey.pem
   Your certificate will expire on 2021-11-24. To obtain a new or
   tweaked version of this certificate in the future, simply run
   certbot again with the "certonly" option. To non-interactively
   renew *all* of your certificates, run "certbot renew"
 - If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

Next, configure reverse proxy on Nginx serving Graylog running on the same 9000 port. This is achieved by editing the .conf file created earlier as below

sudo vim /etc/nginx/sites-available/graylog.domain.com.conf

In the file add the location segment as below.

  ..................................
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

location /
    {
      proxy_set_header Host $http_host;
      proxy_set_header X-Forwarded-Host $host;
      proxy_set_header X-Forwarded-Server $host;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Graylog-Server-URL https://$server_name/;
      proxy_pass       http://127.0.0.1:9000;
    }
}

Now ensure that your configuration file appears as below.

$ cat /etc/nginx/sites-available/graylog.yourdomain.com.conf
server {
  server_name           logs.example.com;

  return 301            https://$host$request_uri;
  access_log            /var/log/nginx/graylog_access.log combined;
  error_log             /var/log/nginx/graylog_error.log;
}
server {
    listen 80;
    root /var/www/html;       
     index index.html index.htm index.nginx-debian.html;
     server_name    logs.example.com;

    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/logs.example.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/logs.example.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

location /
    {
      proxy_set_header Host $http_host;
      proxy_set_header X-Forwarded-Host $host;
      proxy_set_header X-Forwarded-Server $host;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Graylog-Server-URL https://$server_name/;
      proxy_pass       http://127.0.0.1:9000;
    }
}

Check the syntax of the file.

$ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Now restart the Nginx service.

sudo systemctl restart nginx

Step 6: Access Graylog Web interface

We are now set to access the Graylog Web Interface using HTTPS. Use the URL https://logs.example.com replacing logs.computingforgeeks.com with your FQDN.

The default username is admin and the password is what we configured earlier in step 4. In this case, my password is “Passw0rd”

Now you are set to start using the Graylog Web dashboard, here you have a visual representation of the logs.

Conclusion

This is the end of this guide, we have gone through how to install Graylog server on Debian 11/10 with Let’s Encrypt SSL. Now you can collect, index, and analyze logs from a centralized location. This makes it easy to monitor unusual activities and debug applications. I hope you found this guide interesting.

See more on this page.