{"id":3801,"date":"2018-11-09T15:22:17","date_gmt":"2018-11-09T14:22:17","guid":{"rendered":"http:\/\/nil2.kis.fri.uniza.sk\/?p=3801"},"modified":"2019-10-13T14:16:00","modified_gmt":"2019-10-13T12:16:00","slug":"moloch-components-and-architecture","status":"publish","type":"post","link":"https:\/\/nil.uniza.sk\/en\/moloch-components-and-architecture\/","title":{"rendered":"Moloch &#8211; Components and architecture"},"content":{"rendered":"<h1><a id=\"Components_4\"><\/a>Components<\/h1>\n<p>Moloch consists of three components:<\/p>\n<ul>\n<li><strong>Elasticsearch<\/strong> \u2013 search engine powering the Moloch system. It is distributed under the terms of Apache license. Requests are handled using HTTP and results are returned in JSON file format. Elasticsearch supports database sharding, making it fast and scalable.<\/li>\n<li><strong>Capture<\/strong> \u2013 C language based application for real-time network traffic monitoring. Captured data is written to disk in PCAP format. Alternatively, it can be used to import PCAP files for analysis and archiving manually through command line. The application analyzes protocols of OSI layers three through seven and creates SPI data which it sends to the Elasticsearch cluster for indexing.<\/li>\n<li><strong>Viewer<\/strong> \u2013 The viewer uses a number of node.js tools. Node.js is an event-based, server-side Javascript platform with its own HTTP and JSON communication. Viewer runs on each device with running Capture module and it provides a web UI for searching, displaying and exporting of PCAP files. GUI\/API calls are carried out using URIs, enabling integration with security information and event management (SIEM) systems, consoles or command line for PCAP file obtaining.<\/li>\n<\/ul>\n<h1><a id=\"Architecture_13\"><\/a>Architecture<\/h1>\n<p>All of the components can be located and run on a single node, however this is not recommended for processing of larger data flows. Whether the data flow is too large can be determined by requests taking too long to respond, in that case, transition to multi-node architecture is advised. The individual components have distinct requirements, Capture requires large amounts of disk space to store received PCAP files, by contrast, Elasticsearch requires large amount of RAM for idexing and searching. The viewer has the smallest requirements of the three, allowing it to be located anywhere.<\/p>\n<p><img decoding=\"async\" title=\"Single node architekt\u00fara\" src=\"https:\/\/i.imgur.com\/vKfjNfw.png\" alt=\"MolochS\" \/><\/p>\n<p>Moloch can be easily scaled to multiple nodes for Capture and Elasticsearch components. One or several instances of Capture can run on a single or multiple nodes, while sending data to the Elasticsearch database. Similarly, single one or multiple instances of Elasticsearch can run on either one or several nodes to increase the amount of RAM capacity for indexing. This architecture type is therefore recommended for data flow capture and real-time indexing.<\/p>\n<p><img decoding=\"async\" title=\"Multi node architekt\u00fara\" src=\"https:\/\/i.imgur.com\/payTYka.png\" alt=\"MolochM\" \/><\/p>\n<p>We recommend deploying Moloch behind a mirrored switch interface, in our case a Cisco SPAN port. Click <a href=\"https:\/\/nil.uniza.sk\/en\/span\/\">here<\/a> for more information on port mirroring.<\/p>\n<h3><a id=\"Sources_24\"><\/a>Sources<\/h3>\n<ul>\n<li><a href=\"http:\/\/opac.crzp.sk\/?fn=detailBiblioFormChild3&amp;sid=B7A4F0D5DE0EE4D8F05E77CD7EE5\">CRZP<\/a> Komplexn\u00fd syst\u00e9m pre detekciu \u00fatokov a archiv\u00e1ciu d\u00e1t &#8211; Moloch<\/li>\n<\/ul>","protected":false},"excerpt":{"rendered":"<p>Components Moloch consists of three components: Elasticsearch \u2013 search engine powering the Moloch system. It is distributed under the terms of Apache license. Requests are handled using HTTP and results are returned in JSON file format. Elasticsearch supports database sharding, making it fast and scalable. Capture \u2013 C language based application for real-time network traffic&#8230;<\/p>","protected":false},"author":6,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_seopress_robots_primary_cat":"","_seopress_titles_title":"","_seopress_titles_desc":"","_seopress_robots_index":"","_kad_blocks_custom_css":"","_kad_blocks_head_custom_js":"","_kad_blocks_body_custom_js":"","_kad_blocks_footer_custom_js":"","_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"footnotes":""},"categories":[872],"tags":[918,916,874],"class_list":["post-3801","post","type-post","status-publish","format-standard","hentry","category-network-security-moloch-en","tag-architectue","tag-components","tag-moloch"],"taxonomy_info":{"category":[{"value":872,"label":"Moloch"}],"post_tag":[{"value":918,"label":"architectue"},{"value":916,"label":"components"},{"value":874,"label":"Moloch"}]},"featured_image_src_large":false,"author_info":{"display_name":"Tom\u00e1\u0161 Moko\u0161","author_link":"https:\/\/nil.uniza.sk\/en\/author\/tomas-mokos\/"},"comment_info":57,"category_info":[{"term_id":872,"name":"Moloch","slug":"network-security-moloch-en","term_group":0,"term_taxonomy_id":870,"taxonomy":"category","description":"","parent":707,"count":14,"filter":"raw","cat_ID":872,"category_count":14,"category_description":"","cat_name":"Moloch","category_nicename":"network-security-moloch-en","category_parent":707}],"tag_info":[{"term_id":918,"name":"architectue","slug":"architectue","term_group":0,"term_taxonomy_id":916,"taxonomy":"post_tag","description":"","parent":0,"count":1,"filter":"raw"},{"term_id":916,"name":"components","slug":"components","term_group":0,"term_taxonomy_id":914,"taxonomy":"post_tag","description":"","parent":0,"count":1,"filter":"raw"},{"term_id":874,"name":"Moloch","slug":"moloch","term_group":0,"term_taxonomy_id":872,"taxonomy":"post_tag","description":"","parent":0,"count":10,"filter":"raw"}],"_links":{"self":[{"href":"https:\/\/nil.uniza.sk\/en\/wp-json\/wp\/v2\/posts\/3801","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nil.uniza.sk\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nil.uniza.sk\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nil.uniza.sk\/en\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/nil.uniza.sk\/en\/wp-json\/wp\/v2\/comments?post=3801"}],"version-history":[{"count":0,"href":"https:\/\/nil.uniza.sk\/en\/wp-json\/wp\/v2\/posts\/3801\/revisions"}],"wp:attachment":[{"href":"https:\/\/nil.uniza.sk\/en\/wp-json\/wp\/v2\/media?parent=3801"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nil.uniza.sk\/en\/wp-json\/wp\/v2\/categories?post=3801"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nil.uniza.sk\/en\/wp-json\/wp\/v2\/tags?post=3801"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}