{"id":3799,"date":"2018-11-09T15:18:50","date_gmt":"2018-11-09T14:18:50","guid":{"rendered":"http:\/\/nil2.kis.fri.uniza.sk\/?p=3799"},"modified":"2021-03-19T10:48:40","modified_gmt":"2021-03-19T09:48:40","slug":"moloch-cyber-defense-monitoring-course-suite","status":"publish","type":"post","link":"https:\/\/nil.uniza.sk\/en\/moloch-cyber-defense-monitoring-course-suite\/","title":{"rendered":"Moloch &#8211; Cyber Defense Monitoring Course Suite"},"content":{"rendered":"<ul>\n<li><strong>Authors<\/strong> : Tom\u00e1\u0161 Moko\u0161, Marek Brodec<\/li>\n<li><strong>Operating system<\/strong> : Ubuntu 16.04<\/li>\n<li><strong>Elasticsearch version<\/strong> : 5.5.1<\/li>\n<li><strong>Suricata version<\/strong> : 4.0.1<\/li>\n<\/ul>\n<p><strong>This article is outdated, see the newer installation guides below.<\/strong><\/p>\n<p><a href=\"https:\/\/nil.uniza.sk\/en\/installation-of-suricata\/\">Installation of Suricata<\/a><\/p>\n<p><a href=\"https:\/\/nil.uniza.sk\/en\/arkime-installation\/\" target=\"_blank\" rel=\"noopener\">Akime (former Moloch) Installatioon<\/a><\/p>\n<p><a href=\"https:\/\/nil.uniza.sk\/en\/integrating-moloch-and-suricata\/\">Integrating Moloch and Suricata<\/a><\/p>\n<p><img decoding=\"async\" title=\"Graf\" src=\"https:\/\/i.imgur.com\/KrBtFSz.png\" alt=\"Graf\" \/><\/p>\n<h1><a id=\"Elasticsearch_10\"><\/a>Elasticsearch<\/h1>\n<p>Elasticsearch is an open source tool, with its primary purpose being the fast and effective fulltext browsing of its indexed data. It is mostly used to browse document databases.<\/p>\n<p>Download the Elasticsearch version currently supported by Moloch:<\/p>\n<pre><code>wget https:\/\/artifacts.elastic.co\/downloads\/elasticsearch\/elasticsearch-5.5.1.deb \n<\/code><\/pre>\n<p>Unpack and install the archive:<\/p>\n<pre><code>sudo dpkg -i elasticsearch-5.5.1.deb \n<\/code><\/pre>\n<h1><a id=\"Suricata_21\"><\/a>Suricata<\/h1>\n<p>Suricata is a very fast, robust and continually developed free open source detection tool. It is capable of detecting access violations in real time, providing intrusion prevention, monitoring network safety and offline PCAP file processing.<\/p>\n<p>Set the variable containing the installed version number.<\/p>\n<pre><code>VER=4.0.1 \n<\/code><\/pre>\n<p>Download and unpack the installation package.<\/p>\n<pre><code>wget http:\/\/www.openinfosecfoundation.org\/download\/suricata-$VER.tar.gz \ntar -xvzf \"suricata-$VER.tar.gz\" \n<\/code><\/pre>\n<p>Installation and configuration<\/p>\n<pre><code>.\/configure --enable-nfqueue --prefix=\/usr --sysconfdir=\/etc --localstatedir=\/var \n.\/configure --prefix=\/usr --sysconfdir=\/etc --localstatedir=\/var \n<\/code><\/pre>\n<p>Now you can choose one of the following options:<\/p>\n<ul>\n<li>Create and setup only the necessary directories and the suricata.yaml configuration file.<\/li>\n<\/ul>\n<pre><code>.\/configure &amp;&amp; make &amp;&amp; make install-conf \n<\/code><\/pre>\n<ul>\n<li>Automatically download and setup the latest accessible rules for Suricata packet threat evaluation.<\/li>\n<\/ul>\n<pre><code>.\/configure &amp;&amp; make &amp;&amp; make install-rules \n<\/code><\/pre>\n<ul>\n<li>Combination of both the previous options: all necessary files are created and configured and the latest accessible threat evaluation rules are downloaded and installed.<\/li>\n<\/ul>\n<pre><code>.\/configure &amp;&amp; make &amp;&amp; make install-full \n<\/code><\/pre>\n<ul>\n<li>Edit the configuration file for the needs of this guide. These changes include: eve.json logging configuration, suricata enp7s0f0 interface definition and the default rule path (\/usr\/local\/etc\/suricata\/rules). The following lines will be added to the tail of the file:<\/li>\n<\/ul>\n<pre><code>cat &gt;&gt; \/usr\/local\/etc\/suricata\/suricata.yaml &lt;&lt;EOF \nstats: \n  enabled: no \noutputs: \n  - fast: \n      enabled: no \n  - eve-log: \n      enabled: yes \n      filename: eve.json \n      types: \n        - alert: \n            tagged-packets: no \n            xff: \n              enabled: no \naf-packet: \n  - interface: enp7s0f0 \n    cluster-id: 98 \n    cluster-type: cluster_flow \n    defrag: yes \ndefault-rule-path: \/usr\/local\/etc\/suricata\/rules \nsensor-name: moloch-singlehost \nEOF \n<\/code><\/pre>\n<h1><a id=\"GeoLite_78\"><\/a>GeoLite<\/h1>\n<p>GeoLite is a free geolocation database. It contains a database of allocated IP addresses listed with country of allocation along, in some cases, with organization to which the given address has been allocated and\/or IP block size. The IP address database is regularly updated on the first Tuesday of every month.<\/p>\n<p>Download archives and unpack the database<\/p>\n<pre><code>echo \"$(date) installing GeoLite2\" \n[[ -f 'GeoLite2-City.mmdb.gz' ]] || wget -q  -4 http:\/\/geolite.maxmind.com\/download\/geoip\/database\/GeoLite2-City.mmdb.gz \nmkdir -p \/usr\/local\/share\/GeoIP \ngunzip GeoLite2-City.mmdb.gz --stdout &gt; \/usr\/local\/share\/GeoIP\/GeoLite2-City.mmdb \n<\/code><\/pre>\n<h1><a id=\"Evebox_88\"><\/a>Evebox<\/h1>\n<p>EveBox is a web based UI management tool for alerts and events generated by the Suricata network threat detection engine. EveBox works closely with Elasticsearch, with its secondary role being the integration of Suricata logs with Elasticsearch.<\/p>\n<p>Download the latest EveBox installation package.<\/p>\n<pre><code>wget -q -4 https:\/\/evebox.org\/files\/development\/evebox-latest-amd64.deb \n<\/code><\/pre>\n<p>Unpack and install the archive<\/p>\n<pre><code>dpkg -i evebox-latest-amd64.deb \n<\/code><\/pre>\n<p>Set up the ELASTICSEARCH_INDEX and SURICATA_EVE variables, and an URL for Elasticsearch Access.<br \/>\nAfter calling ELASTICSEARCH_INDEX, the data is indexed from Suricata to Elasticsearch under index names found in Suricata. The SURICATA_EVE variable contains the absolute path to Suricata alerts and events source file.<\/p>\n<pre><code>cat &gt;\/usr\/local\/etc\/default\/evebox &lt;&lt;EOF \nELASTICSEARCH_URL=\"-e http:\/\/localhost:9200\" \nELASTICSEARCH_INDEX=\"--index suricata\" \nSURICATA_EVE=\"--end \/var\/log\/suricata\/eve.json\" \nEOF \n<\/code><\/pre>\n<p>Creation of this file allows EveBox server launch without the need to define additional files and options every time.<\/p>\n<pre><code>cat &gt; \/lib\/systemd\/system\/evebox.service &lt;&lt;EOF \n[Unit] \nDescription=EveBox Server \n[Service] \nExecStart=\/usr\/bin\/evebox \\$ELASTICSEARCH_URL \\$ELASTICSEARCH_INDEX \\$CONFIG \\$EVEBOX_OPTS \nEnvironmentFile=-\/usr\/local\/etc\/default\/evebox \n[Install] \nWantedBy=multi-user.target \nEOF \n<\/code><\/pre>\n<p>With intention similar to the one in previous step, create this file for launching of an EveBox process which imports alerts from Suricata logs.<\/p>\n<pre><code>cat &gt; \/lib\/systemd\/system\/evebox-esimport.service &lt;&lt;EOF \n[Unit] \nDescription=EveBox-EsImport \n[Service] \nExecStart=\/usr\/bin\/evebox esimport \\$ELASTICSEARCH_URL \\$ELASTICSEARCH_INDEX \\$SURICATA_EVE \nEnvironmentFile=-=-\/usr\/local\/etc\/default\/evebox \n[Install] \nWantedBy=multi-user.target \nEOF \n<\/code><\/pre>\n<p>Enable the services configured in previous steps.<\/p>\n<pre><code>systemctl enable evebox-esimport \nsystemctl enable evebox \n<\/code><\/pre>\n<p>Use the following commands to start\/restart\/stop or print status of the given service.<\/p>\n<pre><code>systemctl start|restart|stop|status evebox-esimport \nsystemctl start|restart|stop|status evebox \n<\/code><\/pre>\n<p>After any changes made in service configuration file, daemon reloading and enabling of the service is needed.<\/p>\n<pre><code>systemctl daemon-reload \nsystemctl enable .... \n<\/code><\/pre>\n<h1><a id=\"Moloch_147\"><\/a>Moloch<\/h1>\n<p>Add apt repository and install JAVA.<\/p>\n<pre><code>add-apt-repository ppa:webupd8team\/java \napt-get update \napt-get -y install oracle-java8-installer \n<\/code><\/pre>\n<p>Install packages necessary for running Moloch.<\/p>\n<pre><code>apt-get install wget curl libpcre3-dev uuid-dev libmagic-dev pkg-config g++ flex bison zlib1g-dev libffi-dev gettext libgeoip-dev make libjson-perl libbz2-dev libwww-perl libpng-dev xz-utils libffi-dev \n<\/code><\/pre>\n<p>Download Moloch installation package for Ubuntu 16.04.<\/p>\n<pre><code>wget https:\/\/files.molo.ch\/builds\/ubuntu-16.04\/moloch_0.20.1-1_amd64.deb \n<\/code><\/pre>\n<p>Unpack and install the package<\/p>\n<pre><code>dpkg -i moloch_0.20.1-1_amd64.deb \n<\/code><\/pre>\n<p>Run Moloch configuration, since you have already installed Elasticsearch, do not allow Elasticsearch Demo installation.<\/p>\n<pre><code>sudo .\/data\/moloch\/bin\/Configure \n<\/code><\/pre>\n<p>Continue the installation by running Elasticsearch and initializing the database.<\/p>\n<pre><code>systemctl start elasticsearch.service \n\/data\/moloch\/db\/db.pl http:\/\/127.0.0.1:9200 init \n\/data\/moloch\/db\/db.pl http:\/\/127.0.0.1:9200 upgrade \n<\/code><\/pre>\n<p>Add user to web GUI.<\/p>\n<pre><code>\/data\/moloch\/bin\/moloch_add_user.sh admin user password \u2013admin \n<\/code><\/pre>\n<p>Create the configuration file of wiseService components and set parameters of both the service itself and of Suricata (EveBox access IP address, fields displayed in Moloch, etc.)<\/p>\n<pre><code>cd \/data\/moloch\/etc\/ \ncp \/data\/moloch\/wiseService\/wiseService.ini.sample \/data\/moloch\/etcwise.ini \ncat &gt; \/data\/moloch\/etc\/wise.ini &lt;&lt;EOF \n[wiseService] \nport=8081 \n[suricata] \nevBox=http:\/\/127.0.0.1:5636 \nfields=severity;category;signature;flow_id;_id \nmustHaveTags=escalated \nmustNotHaveTags=archived \nEOF \n<\/code><\/pre>\n<p>Create a symlink in wiseService folder referencing the configuration file created in the previous step.<\/p>\n<pre><code>cd \/data\/moloch\/wiseService\/ \nln -s \/data\/moloch\/etc\/wise.ini wiseService.ini \n<\/code><\/pre>\n<p>Always run from wiseService directory<\/p>\n<pre><code>node \/data\/moloch\/bin\/node wiseService.js -c wiseService.ini \n<\/code><\/pre>\n<h1><a id=\"Kibana_203\"><\/a>Kibana<\/h1>\n<p>Download and unpack the archive, choose the version supported by the installed Elasticsearch version.<\/p>\n<pre><code>wget https:\/\/artifacts.elastic.co\/downloads\/kibana\/kibana-5.5.3-amd64.deb \ndpkg -i kibana-5.5.3-amd64.deb \n<\/code><\/pre>\n<p>Start the service<\/p>\n<pre><code>service kibana start \nservice kibana status \n<\/code><\/pre>\n<p>Location of the configuration file<\/p>\n<pre><code>cat \/etc\/kibana\/kibana.yml \n<\/code><\/pre>\n<p>To gain web access, you need to enable communication on the port number of Kibana. The standard port is 5601.<\/p>\n<pre><code>iptables -A INPUT -m udp -p udp --dport 5601 -j ACCEPT \niptables -A INPUT -m tcp -p tcp --dport 5601 -j ACCEPT \n<\/code><\/pre>\n<p>To access Elaticsearch you can use services provided by Kibana. First, you need to set the values of indices to be searched. Set index pattern to \u201csession-\u201d for Moloch and \u201csuricata-\u201d for Suricata, these settings can be found in the Management menu item.<\/p>\n<p><img decoding=\"async\" title=\"Kibana\" src=\"https:\/\/i.imgur.com\/s87cmEm.png\" alt=\"Kibana\" \/><\/p>\n<h3><a id=\"Sources_227\"><\/a>Sources<\/h3>\n<p><a href=\"https:\/\/github.com\/ccdcoe\/CDMCS\">CDMCS<\/a> Cyber Defence Monitoring Course Suite<\/p>","protected":false},"excerpt":{"rendered":"<p>Authors : Tom\u00e1\u0161 Moko\u0161, Marek Brodec Operating system : Ubuntu 16.04 Elasticsearch version : 5.5.1 Suricata version : 4.0.1 This article is outdated, see the newer installation guides below. Installation of Suricata Akime (former Moloch) Installatioon Integrating Moloch and Suricata Elasticsearch Elasticsearch is an open source tool, with its primary purpose being the fast and&#8230;<\/p>","protected":false},"author":6,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_seopress_robots_primary_cat":"","_seopress_titles_title":"","_seopress_titles_desc":"","_seopress_robots_index":"","_kad_blocks_custom_css":"","_kad_blocks_head_custom_js":"","_kad_blocks_body_custom_js":"","_kad_blocks_footer_custom_js":"","_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"footnotes":""},"categories":[872],"tags":[908,900,910,912,874,914],"class_list":["post-3799","post","type-post","status-publish","format-standard","hentry","category-network-security-moloch-en","tag-cdmcs","tag-elasticsearch","tag-evebox","tag-geolite","tag-moloch","tag-suricata"],"taxonomy_info":{"category":[{"value":872,"label":"Moloch"}],"post_tag":[{"value":908,"label":"CDMCS"},{"value":900,"label":"elasticsearch"},{"value":910,"label":"evebox"},{"value":912,"label":"geolite"},{"value":874,"label":"Moloch"},{"value":914,"label":"suricata"}]},"featured_image_src_large":false,"author_info":{"display_name":"Tom\u00e1\u0161 Moko\u0161","author_link":"https:\/\/nil.uniza.sk\/en\/author\/tomas-mokos\/"},"comment_info":9,"category_info":[{"term_id":872,"name":"Moloch","slug":"network-security-moloch-en","term_group":0,"term_taxonomy_id":870,"taxonomy":"category","description":"","parent":707,"count":14,"filter":"raw","cat_ID":872,"category_count":14,"category_description":"","cat_name":"Moloch","category_nicename":"network-security-moloch-en","category_parent":707}],"tag_info":[{"term_id":908,"name":"CDMCS","slug":"cdmcs","term_group":0,"term_taxonomy_id":906,"taxonomy":"post_tag","description":"","parent":0,"count":1,"filter":"raw"},{"term_id":900,"name":"elasticsearch","slug":"elasticsearch","term_group":0,"term_taxonomy_id":898,"taxonomy":"post_tag","description":"","parent":0,"count":5,"filter":"raw"},{"term_id":910,"name":"evebox","slug":"evebox","term_group":0,"term_taxonomy_id":908,"taxonomy":"post_tag","description":"","parent":0,"count":1,"filter":"raw"},{"term_id":912,"name":"geolite","slug":"geolite","term_group":0,"term_taxonomy_id":910,"taxonomy":"post_tag","description":"","parent":0,"count":1,"filter":"raw"},{"term_id":874,"name":"Moloch","slug":"moloch","term_group":0,"term_taxonomy_id":872,"taxonomy":"post_tag","description":"","parent":0,"count":10,"filter":"raw"},{"term_id":914,"name":"suricata","slug":"suricata","term_group":0,"term_taxonomy_id":912,"taxonomy":"post_tag","description":"","parent":0,"count":6,"filter":"raw"}],"_links":{"self":[{"href":"https:\/\/nil.uniza.sk\/en\/wp-json\/wp\/v2\/posts\/3799","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nil.uniza.sk\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nil.uniza.sk\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nil.uniza.sk\/en\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/nil.uniza.sk\/en\/wp-json\/wp\/v2\/comments?post=3799"}],"version-history":[{"count":0,"href":"https:\/\/nil.uniza.sk\/en\/wp-json\/wp\/v2\/posts\/3799\/revisions"}],"wp:attachment":[{"href":"https:\/\/nil.uniza.sk\/en\/wp-json\/wp\/v2\/media?parent=3799"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nil.uniza.sk\/en\/wp-json\/wp\/v2\/categories?post=3799"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nil.uniza.sk\/en\/wp-json\/wp\/v2\/tags?post=3799"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}