Dataset 2012 – ISCX – Elsevier

In this article we take a closer look at the ISCX IDS 2012 dataset created by the Canadian Institute for Cybersecurity.

Network datasets serve for the purpose of training of network security systems, namely IDS and IPS.

Analysis of the ISCX dataset from June 15th in Moloch

The size of PCAP data from this day is 24.5 GB. The dataset is described in three XML files, with the attack being described in the file TestbedTueJun15-3Flows. The description implies a DDoS attack using an IRC botnet.

According to PCAP data, the most intense part of the attack lasted for one hour from 21:05 to 22:05. In the XML file, the attack is recorded at 16:04, therefore there is a 5-hour delay between the data.

The attack originated from infected devices in a private network, with the target being the device with IP address 192.168.5.122. Other sessions marked as attacks were of too low intensity to be visibly displayed. According to the XML description, the attack commenced roughly one hour before the start of the most intense part and lasted for five more hours after its end.

This is an illustration of the most intense part of the attack. IP address with the highest traffic representing the device being attacked is located in the center of the graph. Network communication that was not a part of the attack is being displayed too

This illustration shows only the originating addresses of sessions with destination address 192.168.5.122 – target of the attack.

Analysis of the dataset in IDS Suricata

Immediately, Suricata detected high amounts of P2P bittorrent traffic (this does not necessarily imply an attack, rather than a violation of network terms).

In the early morning hours (3 a.m. to 4 a.m. ) of the following day (16.6.), several brute force attack attempts on the aforementioned IP address (192.168.5.122) were detected. Several attempts of the same attack in the opposite direction were also detected (internal IP address was attempting to reach an external IP address via SSH).

In addition, Suricata detected several possible Trojans and Malwares (about 60), e.g. Blue Botnet for attack generation and Sality for infection of files in OS Windows. In the afternoon and evening hours, an access to website Regnow.com was also detected. This site is linked to typical scammers from India who pose as MS support and demand your credit card number for cleaning of a purportedly infected computer.

Suricata failed to directly detect an ongoing DDoS attack, the only sign which was generation of “STREAM 3way handshake with ack in wrong dir” alert between IP addresses 192.168.5.122 and 192.168.4.120 150x per second. However, since the alert always regarded the same IP addresses, we should have been dealing with a DoS attack, rather than a DDoS attack. The aforementioned TCP anomaly occurred for unknown reasons, if it were not for this, there would be no sign of an attack.

Suricata supports rule thresholding, which can be used to detect DDoS attacks. These thresholds have parameters which define the number of sessions, timeframe, maintaining count by source or destination IP address etc. A signature for detecting DDoS attacks using this rule is located online. However, the test performed on this dataset was unsuccessful even after editing of the aforementioned parameters. Dataset analysis revealed that the malicious packets contain TCP flags PUSH and ACK, while the signature expected packets with SYN flag (TCP SYN flood detection). After the removal of the rule, the signature was unusable, since its rules have been met even for normal traffic. I have tried editing the rule’s TCP flags to PUSH and ACK and tested its functionality.

It was necessary to find the marginal value of packets per second low enough to trigger the alert for a potential DDoS attack and high enough not to trigger the alert for common traffic and make sure that alert is triggered only for IP addresses involved in the attack.

My work was complicated by the fact that there were not just the packets incoming from the attacking IP addresses, but also packets outbound in the opposite direction because the attacked server tried to respond to all those HTTP requests. I could have set a signature rule to match destination IP address to server IP, this would, however, render the signature useless for other purposes. The solution to this was to consider only the source IP addresses of the alerts during text formatting (elimination of duplicities and other superfluous data) and ignore the destination addresses.

While counting packets by destination IP, I needed to get at least one alert. I have concluded that precisely one alert is generated at traffic exceeding 8500 packets per second. Minimal traffic involving only the attacking IP addresses was ca. 800 packets per second.

In contrast, while counting packets by source IP, it is necessary to generate at least one alert for each attacking IP address. The maximum flow at which all the attacking IP addresses were detected was ca. 200 packets per second. Minimum flow at which no attacking IP addresses were detected was ca. 100 packets per second.

Summary of attacks (excluding DDoS attacks) detected by IDS Suricata:

*

192.168.1.103:4889 -> 192.168.2.112:6667

192.168.1.105:22348 -> 192.168.2.112:6667

192.168.2.109:2969 -> 192.168.2.112:6667

192.168.2.110:3311 -> 192.168.2.112:6667

192.168.2.112:6667 -> 192.168.1.103:4889

192.168.2.112:6667 -> 192.168.1.105:22348

192.168.2.112:6667 -> 192.168.2.109:2969

192.168.2.112:6667 -> 192.168.2.110:3311

192.168.2.112:6667 -> 192.168.2.113:2581

192.168.2.112:6667 -> 192.168.4.118:3761

192.168.2.112:6667 -> 192.168.4.120:4784

192.168.2.113:2581 -> 192.168.2.112:6667

192.168.4.118:3761 -> 192.168.2.112:6667

192.168.4.120:4784 -> 192.168.2.112:6667

Dataset analysis from XML file and PDF article (JU)

• Test period: 00:01:06 Friday, 11.06.2011 – 00:01:06 Friday, 18.06.2011

• We have analyzed Tuesday, 15.6.2011
  • Time difference between XML and PCAP is 5 hours
    (i.e. 16:00 in XML = 21:00 in PCAP)

  How we proceeded:

      1. Information gathering and reconnaissance (passive and active)

  1. Vulnerability identification and scanning
  2. Gaining access and compromising a system
  3. Maintaining access and creating backdoors
  4. Covering tracks

  Scenario 1: infiltrating the network from the inside

  1.

  • We have used a DNS request to discover mail server IP address. Messages containing a virus were sent to e-mail addresses from the server, this enabled the attackers to access the network. Using the Metasploit tool and reverse TCP shell 5555 the attackers created connection to the devices inside the network.
    • The devices that were misused belonged to network: 192.168.3.0/24

    Scenario 2: HTTP DoS

    • The Slowloris tool was used to overload the Web server by sending incomplete HTTP requests to keep the socket open. Since the number of sockets available to the Web server is finite, it is only a matter of time until they are exhausted and server becomes inaccessible.

    Scenario 3: DDoS using an IRC Botnet (Internet Relay Chat)

    • This bot was sent to users as an update message
    • Subsequently, DoS launched HTTP GET attack from each infected device, creating hundreds of requests
    • The attack lasted for 60 minutes

    1.

    • AppName marked in XML: HTTPweb
    • IP address of the targeted web server: 192.168.5.122
    • Beginning and end of the attack:
      • PCAP time: approximately 21:00 – 22:00
      • XML time: 16:04:42 – 17:05:49

      First and last session that were part of the attack:

      • 16:04:42 – 16:04:43
        • IPsource: 192.168.2.113 /Ps: 2677
        • IPdest: 192.168.5.122 /Pd: 80
        • Duration: 1s
        • Inbound packet count (source): 103
        • Outbound packet count (dest.): 270

        17:05:48 – 17:05:49

        • IPsource: 192.168.2.110 /Ps: 4131
        • IPdest: 192.168.5.122 /Pd: 80
        • Duration: ?
        • Inbound packet count (source): 78
        • Outbound packet count (dest.): 170

        Scenario 4: Brute Force SSH

        1.

        • Dictionary attack on users