Menu Close

ElasticSearch cluster upgrade

ElasticSearch cluster upgrade from 5.5.1 to 6.8.1

  • Author : Miroslav Kohútik
  • Operating System : Ubuntu 16.04

In this guide we will show you how to upgrade an ElasticSearch cluster located on a single machine.
As an example we will use our ElasticSearch cluster that consists of five ES nodes.

All nodes need to be stopped before upgrading

sudo systemctl stop elasticsearch_data1
sudo systemctl stop elasticsearch_data2
sudo systemctl stop elasticsearch_data3
sudo systemctl stop elasticsearch_ingest
sudo systemctl stop elasticsearch_master

Download the installation package for ElasticSearch version 6.8.1

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.8.1.deb

Install the new version

sudo dpkg -i elasticsearch-6.8.1.deb

ElasticSearch should now be successfully updated to version 6.8.1. However, we cannot start up our cluster just yet. First, we need to update the Linux services for each node since service definition in 6.x is slightly different from version 5.x.
Our Cluster’s nodes’ services are located in /usr/lib/systemd/system/

Here is an excerpt from /usr/lib/systemd/system/elasticsearch_master.service:

[Service]
Environment=ES_HOME=/usr/share/elasticsearch
Environment=CONF_DIR=/etc/master
Environment=DATA_DIR=/var/lib/elasticsearch/master
Environment=LOG_DIR=/var/log/elasticsearch/master
Environment=PID_DIR=/var/run/elasticsearch
EnvironmentFile=-/etc/default/elasticsearch

WorkingDirectory=/usr/share/elasticsearch

User=elasticsearch
Group=elasticsearch

ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec

ExecStart=/usr/share/elasticsearch/bin/elasticsearch 
                                                -p ${PID_DIR}/elasticsearch.pid 
                                                -Edefault.path.logs=${LOG_DIR} 
                                                -Edefault.path.data=${DATA_DIR} 
                                                -Edefault.path.conf=${CONF_DIR}

Here is the same excerpt from the same service file updated for version 6.x:

[Service]
Environment=ES_HOME=/usr/share/elasticsearch
Environment=PID_DIR=/var/run/elasticsearch
EnvironmentFile=-/etc/default/elasticsearch
LimitMEMLOCK=infinity
RuntimeDirectory=elasticsearch
PrivateTmp=true
Environment=ES_PATH_CONF=/etc/master

WorkingDirectory=/usr/share/elasticsearch

User=elasticsearch
Group=elasticsearch

ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet

Make sure that for every single variable you have set in your elasticsearch_.service files you have also commented out its equivalent in /etc/default/elasticsearch. Otherwise, values in the latter file will override the changes you have made in the former.

Service files of the remaining nodes (in our case the following files: elasticsearch_ingest.service, elasticsearch_data1.service, elasticsearch_data2.service and elasticsearch_data3.service) need to be updated in a similar manner.

Each node’s service also requires its own elasticsearch.yaml file. This file should be located on the path set in ES_PATH_CONF in the service file as seen above (in the case of master node it is /etc/master/).
Here is an example of elasticsearch.yaml located in /etc/master/. Note the attributes node.master, node.data, and node.ingest, these need to be set in respect to the role of the node in particular and are different for nodes of other types.

# ---------------------------------- Cluster -----------------------------------
# Use a descriptive name for your cluster:
cluster.name: elastic
# ------------------------------------ Node ------------------------------------
# Use a descriptive name for the node:
node.name: master
# Add custom attributes to the node:
node.master: true
node.data: false
node.ingest: false
node.max_local_storage_nodes: 5
# ----------------------------------- Paths ------------------------------------
# Path to directory where to store the data (separate multiple locations by comma):
path.data: /data/elasticsearch/data_master
# Path to log files:
path.logs: /var/log/elasticsearch/master
# ----------------------------------- Memory -----------------------------------
# Lock the memory on startup:
bootstrap.memory_lock: true
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.

Each node also uses a distinct pair of HTTP and TCP ports specified by attributes http.port and transport.tcp.port.

# ---------------------------------- Network -----------------------------------
# Set the bind address to a specific IP (IPv4 or IPv6):
network.host: 192.168.1.1
# Set a custom port for HTTP:
http.port: 9200
transport.tcp.port: 9300

Master node needs to bo able to discover other nodes in the cluster, therefore, attribute discovery.zen.ping.unicast.hosts contains a list of IPs and transport ports of all the other nodes. On nodes other than master it will contain only the master’s IP and transport port [“192.168.1.1:9300”]:

# --------------------------------- Discovery ----------------------------------
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
discovery.zen.ping.unicast.hosts: ["192.168.1.1:9301","192.168.1.1:9302","192.168.1.1:9303","192.168.1.1:9304"]

You should now be able to get the ElasticSearch cluster up and running:

sudo systemctl stop elasticsearch_master
sudo systemctl stop elasticsearch_ingest
sudo systemctl stop elasticsearch_data1
sudo systemctl stop elasticsearch_data2
sudo systemctl stop elasticsearch_data3

Installing full Linux from Windows on USB with persistent storage

This guide will describe how to install a full Linux OS on an USB flash disk with permanent storage feature, i.e. the system does not lost your files after a reboot as usual live distribution does. The system is fully operable OS and transferable to any PC with all your installed and saved stuff.

There are several ways how to do it, for example using two usb keys, or making a bootable live USB key and then performing the installation (as is for example decribed here How to Install Linux OS on USB Drive and Run it On Any PC ).

KIS 2019 network dataset

KIS 2019 network dataset

  • Authors : Jana Uramová, Tomáš Mokoš, Patrik Rodina, Peter Seemann, Miroslav Kohútik

This article describes the KIS 2019 network dataset. If you wish to access this dataset, contact us by e-mail at dataset@kis.fri.uniza.sk.

The KIS 2019 dataset was created by Tomáš Mokoš as a part of his Diploma thesis at the Department of Information Networks on the University of Žilina’s Faculty of Management Science and Informatics.

Tools for a quick SIP diagnostics – ngrep, sipgrep and sngrep

Sometimes there is a need for simple and quick analysis of a SIP server and its call functions. Of course, we should use the well-known tcpdump, mentioned in the article Using tcpdump for SIP diagnostics. However, for some occasional Linux users this may be too difficult and unclear. But actually there exist some simpler utilities, that could work fine, as ngrep, and for me newer, sipgrep and sngrep (love at first sight).

All utils are directly available and can be installed online from Debian repo using apt-get install ngrep sipgrep sngrep.

SIP clients – security features analysis

Table provides the overview of security features of nine analysed open-source SIP clients (some sources call them the RTC communicator).

Source: P. Segeč, M. Moravčík, J. Hrabovský, J. Papán and J. Uramová, “Securing SIP infrastructures with PKI — The analysis,” 2017 15th International Conference on Emerging eLearning Technologies and Applications (ICETA), Stary Smokovec, 2017, pp. 1-8.
doi: 10.1109/ICETA.2017.8102525
URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?rp=&arnumber=8102525&isnumber=8102457

Enabling USB Wi-Fi dongle adapter on VirtualBox VM

When we have Wi-Fi adapter in our PC, we may use it as a network interface of VM, but the VM will recognize it just as another wired interface. But there is possibility to connect an USB Wi-Fi dongle adapter and enable it to the VM. VM then will see dongle as native USB Wi-Fi adapter and it allows to work with WiFi networks natively.

For example, we have the TP-LINK Archer T2UH USB dongle, Windows 10 as base system, Lubuntu 19.04 as the virtual machine and VirtualBox 6.0.8.

Set language on Cisco SPA 502G

This post is relevant to all Cicso SPA phones, which have “G” letter in their names. “G” means global (that you can put there languages), not gigabit ethernet (all phones have 100 Mbps interfaces).

First of all, you need to download language files from cisco support page. They are XML files and look like “spa50x_30x_en_v756.xml”. This is for english language. Important is, that you must have two files – for your desired language and for english language. Then set up TFTP server and put there files.