Menu Close

Author: Miroslav Kohútik

Network traffic dataset PCAP anonymization

Network traffic dataset PCAP anonymization

  • Author: Miroslav Kohútik

Sometimes you may need to provide PCAP files to third-party organizations or perhaps, in our case, publish a network traffic dataset. In order to not reveal your network infrastructure and/or other sensitive data, you must anonymize these files before sharing them with anyone outside of you organization.


We use TraceWrangler for network data anonymization on OSI Layers 2 through 4. TraceWrangler is very easy to use and has an intuitive GUI:

Moloch Upgrade

Moloch Upgrade

  • Authors: Tomáš Mokoš, Miroslav Kohútik

Upgrading Moloch to the latest version is not possible from all versions. Some older versions require installation of newer versions in an exact order.

Upgrading to Moloch 1.1.0

The oldest version of Moloch we have had in active use was version 0.50.
Upgrading Moloch from version 0.50 to version 1.0 and higher requires reindexing of all session data due to the major changes introduced in version 1.0. Reindexing is done in the background after upgrading, so there is little downtime before the server is back online.

Server monitoring with Elastic Stack

Server monitoring with Elastic Stack

  • Author: Miroslav Kohútik
  • Elastic Stack Version: 6.7.0
  • Operating system : Ubuntu 16.04

Elastic stack is a group of products from the Elastic company built around the Elasticsearch database designed to process data from any type of source.

In this article we will show you how to monitor the state of the Elasticsearch service and server load using the Elastic Stack services.

Installation of Scirius CE

Installation of Scirius CE

  • Author: Miroslav Kohútik
  • Operating system : Ubuntu 16.04

Scirius Community Edition is a web interface dedicated to Suricata ruleset management. It handles the rules file and updates of the associated files.

This guide will walk you through the installation of Scirius Community Edition on Ubuntu 16.04 operating system.
Before proceeding with installation of Scirius CE, you need to have IDS Suricata installed. Installation guide for Suricata can be found here.

Installation of Zabbix 4.0

Installation of Zabbix 4.0

  • Author: Miroslav Kohútik
  • Operating system : Ubuntu 16.04

This guide describes the individual steps of the installation process of Zabbix version 4.0 on Ubuntu 16.04 operating system.

Zabbix is a free open-source monitoring software. Zabbix provides monitoring of many metrics about the state of the administered network and its devices and services (including virtual machines).

Installation of Suricata

Installation and basic setup of Suricata

First, add the latest stable Suricata repository to APT:

sudo add-apt-repository ppa:oisf/suricata-stable
sudo apt-get update

Now you can either install Suricata with:

sudo apt-get install suricata 

or the Suricata package with built-in (enabled) debugging

sudo apt-get install suricata-dbg

Basic setup

Start with creating a directory for Suricata’s log information.

sudo mkdir /var/log/suricata

To prepare the system for using it, enter:

sudo mkdir /etc/suricata

The next step is to copy classification.config, reference.config and suricata.yaml from the base build/installation directory (ex. from git it will be the oisf directory) to the /etc/suricata directory. Do so by entering the following:

sudo cp classification.config /etc/suricata
sudo cp reference.config /etc/suricata
sudo cp suricata.yaml /etc/suricata

Auto setup

You can also use the available auto setup features of Suricata:

The make install-conf option will do the regular “make install” and then automatically create/setup all the necessary directories and suricata.yaml.

 ./configure && make && make install-conf

The make install-rules option will do the regular “make install” and it automatically downloads and sets up the latest ruleset from Emerging Threats available for Suricata.

./configure && make && make install-rules

The make install-full option combines everything mentioned above (install-conf and install-rules) – and will present you with a ready to run (configured and set up) Suricata

./configure && make && make install-full


Suricata – Ubuntu installation

Moloch/Arkime- Installation

Installation of Moloch/Arkime

  • Author : Miroslav Kohútik
  • Tested version : 1.7.0
  • Operating system : Ubuntu 16.04

Installation of Moloch is no trivial matter, that is why we have prepared this guide on how to set up the system in cloud environment.

Setup before installation

Before installing Moloch itself, you need to install the Elasticsearch database and make the following changes in configuration of the operating system.

Add Java repository

sudo add-apt-repository ppa:webupd8team/java 

Perform an update of the list of packages and packages themselves to the latest versions

sudo apt-get update -y && sudo apt-get upgrade -y

Download and install the public GPG signing key

wget -qO - | sudo apt-key add -

Add Elastic Repository

echo "deb stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-5.x.list

Perform another package update

sudo apt-get update -y && sudo apt-get upgrade -y && sudo apt-get dist-upgrade -y 

Clean-up (Optional)

sudo apt-get autoremove

Disable swap

sudo swapoff -a
sudo nano /etc/fstab

Edit fstab – comment out the following:

#/dev/mapper/logs--vg-swap_1 none     swap   sw      0     0


#/dev/mapper/user--vg-swap_1 none     swap   sw      0     0

Install Java 8

sudo apt-get install oracle-java8-installer

Install Elasticsearch

sudo apt-get install elasticsearch

Install Moloch/Arkime

Install additional necessary packages

sudo apt-get install wget curl libpcre3-dev uuid-dev libmagic-dev pkg-config g++ flex bison zlib1g-dev libffi-dev gettext libgeoip-dev make libjson-perl libbz2-dev libwww-perl libpng-dev xz-utils libffi-dev

Download the  newest version of Moloch/Arkime (Arkime downloads)


Install Moloch

Note: when asked whether or not to install Elasticsearch choose no, since you have already installed Elasticsearch earlier and this script offers only the demo version.

sudo dpkg -i moloch_1.7.0-1_amd64.deb

Install dependencies (If the previous step halts due to errors)

sudo apt-get -f install

Configure Moloch/Arkime

Start Elasticsearch on startup

sudo systemctl enable elasticsearch.service

Configure Elasticsearch (OPTIONAL) (Configure as needed [max RAM allocation is 32GB])

It is recommended Elasticsearch be installed on a separate machine

sudo nano /etc/elasticsearch/jvm.options

Start Elasticsearch

sudo systemctl start elasticsearch.service

Check Elasticsearch Status

sudo systemctl status elasticsearch.service

To configure Moloch, you can either download a configuration file from or you can configure Moloch yourself using the following two commands

Before configuring Moloch manually, delete the config.ini file from /data/moloch/etc/

sudo rm /data/moloch/etc/config.ini 

Configure Moloch as needed

sudo /data/moloch/bin/Configure

Initialize Elasticsearch Database

sudo /data/moloch/db/ http://localhost:9200 init

Install and update npm

sudo apt install npm
npm update

Add Moloch User

sudo /data/moloch/bin/ admin admin PASSWORDGOESHERE --admin

Start Moloch Capture Service

sudo systemctl start molochcapture.service

Check Moloch Capture Service status

sudo systemctl status molochcapture.service

Start Moloch Viewer Service

sudo systemctl start molochviewer.service

Check Moloch Viewer Service status

sudo systemctl status molochviewer.service

Provided you have done everything right so far, you should be able to access the web interface at http://IPADDRESSOFINTERFACE:8005


Analysis of the ISCX dataset from june 15th

Dataset 2012 – ISCX – Elsevier

In this article we take a closer look at the ISCX IDS 2012 dataset created by the Canadian Institute for Cybersecurity.

Network datasets serve for the purpose of training of network security systems, namely IDS and IPS.

Analysis of the ISCX dataset from June 15th in Moloch

The size of PCAP data from this day is 24.5 GB. The dataset is described in three XML files, with the attack being described in the file TestbedTueJun15-3Flows. The description implies a DDoS attack using an IRC botnet.

According to PCAP data, the most intense part of the attack lasted for one hour from 21:05 to 22:05. In the XML file, the attack is recorded at 16:04, therefore there is a 5-hour delay between the data.

The attack originated from infected devices in a private network, with the target being the device with IP address Other sessions marked as attacks were of too low intensity to be visibly displayed. According to the XML description, the attack commenced roughly one hour before the start of the most intense part and lasted for five more hours after its end.

This is an illustration of the most intense part of the attack. IP address with the highest traffic representing the device being attacked is located in the center of the graph. Network communication that was not a part of the attack is being displayed too

This illustration shows only the originating addresses of sessions with destination address – target of the attack.

Analysis of the dataset in IDS Suricata

Immediately, Suricata detected high amounts of P2P bittorrent traffic (this does not necessarily imply an attack, rather than a violation of network terms).

In the early morning hours (3 a.m. to 4 a.m. ) of the following day (16.6.), several brute force attack attempts on the aforementioned IP address ( were detected. Several attempts of the same attack in the opposite direction were also detected (internal IP address was attempting to reach an external IP address via SSH).

In addition, Suricata detected several possible Trojans and Malwares (about 60), e.g. Blue Botnet for attack generation and Sality for infection of files in OS Windows. In the afternoon and evening hours, an access to website was also detected. This site is linked to typical scammers from India who pose as MS support and demand your credit card number for cleaning of a purportedly infected computer.

Suricata failed to directly detect an ongoing DDoS attack, the only sign which was generation of “STREAM 3way handshake with ack in wrong dir” alert between IP addresses and 150x per second. However, since the alert always regarded the same IP addresses, we should have been dealing with a DoS attack, rather than a DDoS attack. The aforementioned TCP anomaly occurred for unknown reasons, if it were not for this, there would be no sign of an attack.

Suricata supports rule thresholding, which can be used to detect DDoS attacks. These thresholds have parameters which define the number of sessions, timeframe, maintaining count by source or destination IP address etc. A signature for detecting DDoS attacks using this rule is located online. However, the test performed on this dataset was unsuccessful even after editing of the aforementioned parameters. Dataset analysis revealed that the malicious packets contain TCP flags PUSH and ACK, while the signature expected packets with SYN flag (TCP SYN flood detection). After the removal of the rule, the signature was unusable, since its rules have been met even for normal traffic. I have tried editing the rule’s TCP flags to PUSH and ACK and tested its functionality.

It was necessary to find the marginal value of packets per second low enough to trigger the alert for a potential DDoS attack and high enough not to trigger the alert for common traffic and make sure that alert is triggered only for IP addresses involved in the attack.

My work was complicated by the fact that there were not just the packets incoming from the attacking IP addresses, but also packets outbound in the opposite direction because the attacked server tried to respond to all those HTTP requests. I could have set a signature rule to match destination IP address to server IP, this would, however, render the signature useless for other purposes. The solution to this was to consider only the source IP addresses of the alerts during text formatting (elimination of duplicities and other superfluous data) and ignore the destination addresses.

While counting packets by destination IP, I needed to get at least one alert. I have concluded that precisely one alert is generated at traffic exceeding 8500 packets per second. Minimal traffic involving only the attacking IP addresses was ca. 800 packets per second.

In contrast, while counting packets by source IP, it is necessary to generate at least one alert for each attacking IP address. The maximum flow at which all the attacking IP addresses were detected was ca. 200 packets per second. Minimum flow at which no attacking IP addresses were detected was ca. 100 packets per second.

Summary of attacks (excluding DDoS attacks) detected by IDS Suricata:

PCAP time Attack Src. IP and port Dest. IP and port
05:58:34.34 Blue Botnet
06:52:16.11 Blue Botnet
07:00:26.37-07:01:49.75 Blue Botnet
07:26:06.70 SSH Scan
09:13:55.36 Sality
09:52:16.48 Sality
11:52:36.59-12:20:50.57 Blue Botnet
12:22:13.68 Blue Botnet
13:42:37.50 Blue Botnet
13:43:17.58 Blue Botnet
13:43:26.02 Sality
13:50:04.74 Blue Botnet
13:51:41.43 Blue Botnet
13:52:37.86 Blue Botnet
13:52:40.68 Blue Botnet
13:52:41.26 Blue Botnet
13:53:53.14 Sality
13:54:08.33 Blue Botnet
13:55:32.63 Sality
13:57:59.03 Sality
14:01:57.07 Sality
14:30:42.14 Sality
15:00:18.78 Sality
15:52:56.48 Sality
16:19:14.08 Sality
16:36:36.10 Blue Botnet
16:39:18.07 Blue Botnet
16:54:38.75 Blue Botnet
16:55:15.07 Sality
17:21:56.68 Blue Botnet
17:56:24.79 Sality
18:54:46.43 Sality
19:17:27.72 Sality
20:27:20.45- 22:06:47.20 IRC správy * *
23:12:26.69 Sality
23:13:43.47- 23:14:04.25 Blue Botnet
00:03:06.58 Sality
03:56:02.22- 03:57:15.40 SSH Scan
04:01:24.27 MSIL/Karmen Rans.
04:36:30.46 SSH Scan

* -> -> -> -> -> -> -> -> -> -> -> -> -> ->

Dataset analysis from XML file and PDF article (JU)

  • Test period: 00:01:06 Friday, 11.06.2011 – 00:01:06 Friday, 18.06.2011

  • We have analyzed Tuesday, 15.6.2011
    • Time difference between XML and PCAP is 5 hours
      (i.e. 16:00 in XML = 21:00 in PCAP)

    How we proceeded:

        1. Information gathering and reconnaissance (passive and active)
    1. Vulnerability identification and scanning
    2. Gaining access and compromising a system
    3. Maintaining access and creating backdoors
    4. Covering tracks

    Scenario 1: infiltrating the network from the inside


    • We have used a DNS request to discover mail server IP address. Messages containing a virus were sent to e-mail addresses from the server, this enabled the attackers to access the network. Using the Metasploit tool and reverse TCP shell 5555 the attackers created connection to the devices inside the network.
      • The devices that were misused belonged to network:

      Scenario 2: HTTP DoS

      • The Slowloris tool was used to overload the Web server by sending incomplete HTTP requests to keep the socket open. Since the number of sockets available to the Web server is finite, it is only a matter of time until they are exhausted and server becomes inaccessible.

      Scenario 3: DDoS using an IRC Botnet (Internet Relay Chat)

      • This bot was sent to users as an update message
      • Subsequently, DoS launched HTTP GET attack from each infected device, creating hundreds of requests
      • The attack lasted for 60 minutes


      • AppName marked in XML: HTTPweb
      • IP address of the targeted web server:
      • Beginning and end of the attack:
        • PCAP time: approximately 21:00 – 22:00
        • XML time: 16:04:42 – 17:05:49

        First and last session that were part of the attack:

        • 16:04:42 – 16:04:43
          • IPsource: /Ps: 2677
          • IPdest: /Pd: 80
          • Duration: 1s
          • Inbound packet count (source): 103
          • Outbound packet count (dest.): 270

          17:05:48 – 17:05:49

          • IPsource: /Ps: 4131
          • IPdest: /Pd: 80
          • Duration: ?
          • Inbound packet count (source): 78
          • Outbound packet count (dest.): 170

          Scenario 4: Brute Force SSH


          • Dictionary attack on users