Menu Close

Problem with a VoIP phone behind NAT – disabling FortiGate SIP ALG

Initial state and observed problems

Observed problems

We had observed a problem, where a SIP phone is registering, but the AOR record indicates, that as a Contact IP address the incorrect and strange private IP address is used. As is shown on following listing:

voip*CLI> pjsip show aor 1765
   Aor:  <Aor..............................................>  <MaxContact>
   Aor:  1765                                                 1 Contact:  1765/sip:1765@              f123d14d1c NonQual         nan
 ParameterName        : ParameterValue
  authenticate_qualify : false
  contact              : sip:1765@
  default_expiration   : 7200
  mailboxes            :
  max_contacts         : 1
  maximum_expiration   : 7200
  minimum_expiration   : 60
  outbound_proxy       :
  qualify_frequency    : 0
  qualify_timeout      : 3.000000
  remove_existing      : true
  support_path         : false
  voicemail_extension  :

This cause a problem, where incoming phone calls (call on 1765 number) are not reaching the SIP phone. We had tried to solve the situations on the phone only modifying its NAT configuration and using STUN, but with no success. Then we setup the lab with two Cisco NAT to simulate the topo. It works perfectly. This indicate on a problem with the Fortigate firewall. Several posts indicates that it could be the SIP ALG problem, which is on Fortigate devices turned on by default and it modifies SIP messages.

Installation of Suricata

Installation and basic setup of Suricata

First, add the latest stable Suricata repository to APT:

sudo add-apt-repository ppa:oisf/suricata-stable
sudo apt-get update

Now you can either install Suricata with:

sudo apt-get install suricata 

or the Suricata package with built-in (enabled) debugging

sudo apt-get install suricata-dbg

Basic setup

Start with creating a directory for Suricata’s log information.

sudo mkdir /var/log/suricata

To prepare the system for using it, enter:

sudo mkdir /etc/suricata

The next step is to copy classification.config, reference.config and suricata.yaml from the base build/installation directory (ex. from git it will be the oisf directory) to the /etc/suricata directory. Do so by entering the following:

sudo cp classification.config /etc/suricata
sudo cp reference.config /etc/suricata
sudo cp suricata.yaml /etc/suricata

Auto setup

You can also use the available auto setup features of Suricata:

The make install-conf option will do the regular “make install” and then automatically create/setup all the necessary directories and suricata.yaml.

 ./configure && make && make install-conf

The make install-rules option will do the regular “make install” and it automatically downloads and sets up the latest ruleset from Emerging Threats available for Suricata.

./configure && make && make install-rules

The make install-full option combines everything mentioned above (install-conf and install-rules) – and will present you with a ready to run (configured and set up) Suricata

./configure && make && make install-full


Suricata – Ubuntu installation

Moloch – Web interface specification

Viewer is a feature of Moloch that makes processing of captured data easier with the use of web browser GUI. Viewer offers access to numerous services, most notably :

  • Sessions
  • SPI View
  • SPI Graph
  • Connections
  • Upload
  • Files
  • Users
  • Stats


The main tab illustrated below contains:

  • Search field – here you can enter different search filtering rules. Being similar to Wireshark, it enables, for example, to search for results containing source/destination IP address (ip.dst== X.X.X.X && ip.src== X.X.X.X), port(port.dst 80 || port.src 80), web URL (http.uri ==, time of delivery, etc.
  • Country map – results are cross-referenced with GeoIP database and the number of occurrences from a given country is displayed on a map using shades of the color purple.
  • Occurrence frequency chart – this bar chart represents traffic in a given time period. The y-axis can be set to represent the number of sessions, packets or bytes of data.

Okno vyhladavania

  • Record window – displays individual connections and their parameters, including times of beginning and end of session, source and destination IP addresses, source and destination port, packet count and session size in bytes. After unpacking, the session can be exported to a PCAP file, where you can add your own tag and see detailed information about the nature and origin of the packet. A DNS request is Illustrated below

Okno zaznamov

SPI View

SPI (Session Profile Information) View is used to take a deeper look at connection metrics. Instead of writing queries manually, queries can be expanded by clicking on the respective actions in SPI view, which adds the requested item to a query by using AND or AND NOT operators. This tab also provides a quick view of occurrence frequency of all items requested by the user. Furthermore, SPI view also offers a quick summary of monitored IP addresses in a given time period, HTTP response codes, IRC NICKs/channels, etc.

SPI View

SPI Graph

SPI Graph provides user with visualization of any given item contained in SPI View. This tab is very useful for displaying activity for a type of SPI, as well as in-depth analysis. The illustration below displays the demand for HTTP with http.method view selected. The displayed bar charts represent the amount of sessions/packets/databytes captured. To the right of each chart, after clicking View Map button, a map containing origins of the sessions is displayed. The methods GET, POST, HEAD OPTIONS and CONNECT are also displayed below. Maximum number of charts displayed can be set by the Max Elements field, with the default value being 20 and the maximum value being 500. Charts can also be sorted by number of sessions or by name. Moloch also offers a refresh rate setting from 5 to 60 seconds.

SPI Graph


The connections tab provides the user with a tree graph rooted in a source or destination node of the user’s own selection. The illustration below displays the relations between source IP address and destination IP address and port, with 500 sessions displayed using the query size setting. The minimum query size is 100 and the maximum size is 100000. Furthermore, the minimum number of connections can be set to values from 1(like in the case below) to 5. Node dist defines the distance of nodes in pixels. Any node in the graph can be moved after clicking the Unlock button. Source nodes are marked in violet and destination nodes are marked in yellow. By clicking a node, information about the node is displayed: Type (source/destination), Links (number of connected nodes), number of sessions, bytes, databytes and packets. This view also allows for nodes to be added with AND/OR logic as well as hiding nodes by clicking the Hide Node tab. This tab is appropriate for users who prefer node data analysis over visualization.



This tab is used to import PCAP files selected by clicking on Search button and clicking upload. Tags (separated by commas) can be appended to the imported packets, however, this option must be enabled in data/moloch/etc/config.ini. The Upload option is in experimental state and can be enabled by the following command

/data/moloch/bin/moloch-capture --copy -n {NODE} -r {TMPFILE} 

NODE is the node name, TMPFILE is the file to be imported, CONFIG is the configuration file and TAGS are the tags to be appended.


The files tab displays the table of archived PCAP files. Details include: file number, node, file name, whether the file is locked, upload date and file size. File size (in GB) is defined in data/moloch/etc/config.ini using the maxFileSizeG parameter. If a file is locked, Moloch cannot delete it and it must be deleted manually.


The users tab defines access rights to individual users. It allows to add/remove user accounts and to edit their passwords.


The stats tab provides visual representation and table display of metrics for each node. Other display options can also be used:

Packets/Sec Sessions/Sec Active TCP Sessions Total Dropped/Sec Free Space (MB)
Free Space (%) Fragments Queue Active UDP Sessions Input Dropped/Sec Bytes/Sec
Memory Active Fragments Active ICMP Sessions Active Sessions Bits/Sec
ES Queue Overload Dropped/Sec Disk Queue CPU Memory (%)
Waiting Queue Fragments Dropped/Sec Closing Queue Packet Queue ES Dropped/Sec

Statistics for each node display: current time on the node, number of sessions, remaining free storage, CPU usage, RAM usage, number of packets in queue, number of packets per second, number of bytes per second, number of lost packets per second, number of packets dropped due to congestion and number of packets dropped by Elasticsearch per second. Charts depicting visualization of these stats can be displayed by clicking the plus(+) icon. Elasticsearch statistics contain number of files saved under unique ID, HDD size, Heap size, OS usage, CPU usage, bytes read per second, bytes written per second and number of searches per second.


  • CRZP Komplexný systém pre detekciu útokov a archiváciu dát – Moloch
  • Report Projekt 1-2 – Marek Brodec



Port mirroring is used on a network switch to send a copy of network packets seen on one switch port (or an entire VLAN) to a network monitoring connection on another switch port. This is commonly used for network appliances that require monitoring of network traffic such as an intrusion detection system, passive probe or real user monitoring (RUM) technology that is used to support application performance management (APM).

In our particular case, the faculty provided us with a Cisco Catalyst 2960 switch. We have configured this switch to mirror all internet-bound data traffic traversing the interface connected to network gateway, to the interface connected to Moloch server. As a result, we can now monitor all inbound and outbound lab traffic.

Switch(config)#monitor session 1 source fa0/1 both  

– This command specifies source interface as fa 0/1. The parameter “both” specifies both directions to be monitored.

Switch(config)#monitor session 1 destination interface fa0/24 

– This command defines the destination interface of mirrored traffic



  • CRZP Komplexný systém pre detekciu útokov a archiváciu dát – Moloch

Moloch – Usage possibilities of Moloch

Usage possibilities of Moloch

  • Author : Tomáš Mokoš

Moloch offers many distinct usage possibilities, the set of which is not limited to the ones mentioned down below and can be expanded by individual users, provided they can find other applications of this service:

    • DOS attacks – Analysis of connections suspected of originating DOS attacks.
    • Geolocation – Identification of connection’s country of origin.
    • Access Intelligence – Helps with the analysis of authorized/non-authorized access to system resources, applications, servers, system operation and different functions. You can also perform depth analysis (with the use of tagging) of a particular system, application or service running in the network
    • Port connection usage – amount of connections on a particular port.
    • URL connection usage – amount of connections tied to a particular URL by requests.
    • Data volumes

    As an example, we will show you the use of Moloch for analysis of the CICIDS 2017 dataset, where we analyze a DDoS Hulk attack. First, we filter the traffic. Using the command tags == CICIDS2017_WEDNESDAY && ip.dst == we extract the traffic from the day of the attack with the webserver’s IP as the destination address.


    Afterwards, in the  SPI Graph tab, we can look up the source IP addresses that communicated with this web server by setting SPI Graph to ip.src.

    As we can see, the IP address generated 84315 of the 85268 sessions, making it likely to be the address of the attacker.


    In the SPI View tab, we can see that the network communication did not originate from just one port, but several thousands and almost all of these were bound for the port 80. Furthermore, we can see that most of the communication was bound for miscellaneous URIs, which is characteristic of a Hulk attack. By generating random URIs, Hulk attack causes resource depletion of the web server, making the server inaccessible.Moloch3


    • CRZP Komplexný systém pre detekciu útokov a archiváciu dát – Moloch

Moloch – Network interface configuration

Considering the possibility of packet loss at high traffic flows, it is recommended for the packet capture interface to NOT be the same as the interface connected to the internet, in this case, the interface assigned with static IP address. On the server in our lab there are two interfaces, one for packet capture and one for “outside” communication. To prevent packet loss, it is recommended to increase the Moloch-side interface’s buffer to maximum and turn off most of the NIC’s services by using the following commands:

ethtool –G enp0s9 rx 4096 tx 4096 
ethtool –K enp0s9 rx off tx off gs off tso off gso off 

You can find out the maximum buffer size using the ethool -g command, to check NIC’s services use the ethtool -k command. Disable most of NIC’s services, since you want to capture network traffic instead of what the OS can see, they are not going to be used anyway.


  • CRZP Komplexný systém pre detekciu útokov a archiváciu dát – Moloch

Moloch – Load Testing

  • Author : Tomáš Mokoš, Marek Brodec

In our topology, the server running Moloch was connected to a 100Mbps switch, therefore, even though the generated network traffic reached 140Mbps, the flow was subsequently limited on switch.

Single source to single destination test

At first, while generating packets with a generated IP address from cloud to a lab PC, we have had a problem with the cloud’s security policies. These policies prevented the sending of packets with source IP address different from the one assigned to the hosting cloud instance, therefore we have only generated traffic from a single source IP address to a single destination IP address.

We have observed the volume of incoming data and its effect on performance using network monitoring tools mentioned in the previous chapters. The graphs below show that in the moment of traffic spike, CPU usage spiked as well, however it did not exceed 20%. The graph titled “Mem” shows the RAM usage of all running services including the instances of Moloch, Snort and many others with the addition of cache memory.


The following graph shows utilization of RAM allocated exclusively to Elasticsearch. Having been allocated with 25 GB of RAM, Elasticsearch uses 4.2 GB in this instance. This test revealed that Elasticsearch uses unnecessary amount of RAM and after the test we have decreased the amount of allocated RAM to 18 GB.


N sources to single destination test

The previous packet generation was not ideal because the relations had many shared characteristics. All test cases had identical IP addresses, number of packets and payload, which made indexing much easier. To approximate real network traffic as precisely as possible, we have tried to address the issue with source IP address generation. The solution was packet generation from our own laptop with OS Kali Linux running in VirtualBox as a source of attacks. The laptop was connected to our switch and generated traffic towards the lab. The packets passed through the interface which was mirrored towards our server.

Test results showed CPU utilization rise in the range from 20 to 30%.


“Heap Mem” graph shows that, of the allocated 19 GB of RAM, 6.3 GB is used .


The results revealed that the amount of allocated RAM can be further decreased, freeing up space for other processes. Graph titled “Mem” shows that all running processes use 31 GB of RAM, therefore it would be advisable to stop all unnecessary processes before testing. This, however, was not an option for us, because in parallel with us, other students have been working on their respective bachelor’s theses.

Testing evaluation

Restart of services was performed before the single source to single destination test, but service instances were not restarted before other tests. We have concluded that higher network traffic mainly influences CPU utilization. Graphs of both Elasticsearch RAM usage and overall RAM usage do not show any significant spikes during the arrival of generated test traffic. The table shows that during normal traffic (4.5 Mbps at the time), RAM usage is higher in both cases above than before the first test, where traffic was 20 times higher than normal. These two graphs are mainly affected by the time elapsed from service start and possibly the amount of captured data. These tests were performed in the same order as they were mentioned above, with one week between the individual tests. Even though the graphs indicate utilization of 31 out of 32 GB of RAM, from the 18 GB allocated to Elasticsearch, only one third is used. This can be solved by reducing the amount of RAM allocated to Elasticsearch, thus freeing up space for other processes.

Cache can be cleared by the following command:

free && sync && echo 3 > /proc/sys/vm/drop_caches && free 

This drops RAM usage from 31 GB to 20 GB.


  • Report Project 1-2 – Marek Brodec

Moloch – CPU, RAM and HDD usage

  • Author : Tomáš Mokoš, Marek Brodec

Considering the fact that the formulas that we used to calculate for how many days can Moloch archive network traffic and what hardware should we use were only approximate, we have decided to measure some statistics to help us clear up these values.

From the Elasticsearch node quantity calculation formula: ¼ * [average network traffic in Gbit/s] * [number of days to be archived], we get that at 2 Mbit/s, one node should suffice.

Using the formula 86400 * [average network traffic per second] * [number of days to be archived], we can calculate that 1Gbit/s of traffic takes up 11TB of disk space daily, therefore 2Mbit/s of traffic will take up 22GB per day. At this rate we can archive approximately 113 days’ worth of raw data.

Since our lab server is not under heavy load, only 7GB (22%) of RAM is used on average. This is due to the the fact that during the night the network traffic is minimal. Non-uniform network traffic creates distortions, therefore long-term observation would be desirable.


Moloch by itself uses about 5% of total CPU utilization and 1.0 to 1.3 GB of RAM (3-3.5%).

Utilization of disk capacity was 340 GB (17%) on the first week, 220 GB (11%) on second week and 140 GB (7%) on third week.


Thanks to our use of data trimming we have managed to archive 6 months’ worth of traffic, although the actual value is closer to 4 months, since during the two months of the exam period, network traffic was very low. The captured data took up 52% (1.3 TB) of storage.



  • Report Projekt 1-2 – Marek Brodec