An implementation of a context-based malware detection tool. The tool's modular architecture allows for flexible deployment. It can be integrated with other network monitoring solutions such as IPFIX probes and collectors, SIEM and other tools.
The following diagram shows the architecture of the solution integrated with the SIEM system.
┌─────────────────────────────────────────────────────────────────────────────┐
│ │
┌─────────────┐ │ ┌───────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ │ 7811│ FLOW │ 7821│ CONTEXT │ 7831│ MALWARE │ │
│ ├───┼────────► READER ├───────► COLLECTOR ├─────────► DETECTOR │ │
│ IPFIX │ │ │ │ │ │ │ │ │
│ │ │ └───────────┘ └──────────────┘ └──────┬───────┘ │
│ (FLOWMON, │ │ │ │
│ IPFIXCOL, │ │ │ │
│ SURICATA) │ │ │ │
│ │ │ ┌────────────────┼────────────────┘
│ │ │ │ │
└─────────────┘ │ │ │
│ ┌─────────────────────┐ │ ┌──────▼───────────┐
│ │ │ │ │ │
│ │ WEB APP ◄────────────┼─────────► SIEM │
│ │ (BLAZOR+TELERIK) │ │ │ (QRADAR) │
│ └─────────────────────┘ │ │ │
│ │ │ │
│ │ │ │
│ DOCKER │ └──────────────────┘
└───────────────────────────────────────────┘
The architecture diagram shows the integration of a sophisticated security solution with a Security Information and Event Management (SIEM) system, specifically QRadar. At the core of this solution is the IPFIX protocol, supported by various tools such as FLOWMON, IPFIXCOL and SURICATA, which are central to the collection of network flow data.
The collected data is first passed to the FLOW READER component, which is responsible for reading and interpreting the flow data. From the FLOW READER, the data journey continues to the CONTEXT COLLECTOR. This component enriches the flow data with additional context, making it more meaningful and useful for security analysis.
The enriched data is then sent to the MALWARE DETECTOR. This critical component analyzes the data to identify potential malware activity on the network. It uses sophisticated algorithms and patterns to detect anomalies that may indicate the presence of malware.
An essential part of this architecture is the WEB INTERFACE, built using BLAZOR and TELERIK technologies. This interface serves as a user-friendly portal for administrators to interact with the system, allowing configuration, monitoring and analysis of security data in an intuitive manner.
Finally, the processed and analyzed data is integrated into the SIEM system, in this case QRadar. QRadar uses this data to provide comprehensive security insight, enabling organizations to more effectively detect, investigate, and respond to security threats.
The entire solution is containerized using DOCKER, ensuring scalability, portability and ease of deployment across multiple environments. This modern approach to deploying security solutions increases the flexibility and efficiency of managing security operations.
The malware detection toolchain consists of a processing pipeline designed to read, process, and analyze source data in the form of JSON-represented IPFIX records. The pipeline includes three main components:
-
- Reads IPFIX JSON records from the specified source (e.g., Flowmon).
- Converts the data into a format suitable for further processing.
- Outputs the data to the next tool in the pipeline.
-
- Receives data from FlowReader.
- Computes the context by aggregating and analyzing the IPFIX records.
- Uses a custom configuration to define context parameters.
- Outputs the computed context to the next tool in the pipeline.
-
- Analyzes the computed context for potential malware indicators.
- Uses predefined malware profiles to match observed indicators.
- Outputs the analysis results in a specified format (e.g., Markdown).
These tools can communicate via standard input/output streams or using GRPC, providing flexibility in data handling and integration:
- Standard Input/Output Streams: Tools can be connected directly via standard input and output streams, enabling seamless data flow through the pipeline.
- GRPC: Tools can also communicate using GRPC for more robust and scalable integration, especially in distributed environments.
The following example demonstrates a standard I/O connected pipeline that reads Flowmon's IPFIX JSON records from a specified file, computes the context, and produces a Markdown-formatted output of the context analysis:
..\Source\FlowReader\bin\Debug\net8.0\FlowReader.exe read-input -i "${inUri}?format=json&source=flowmon" -o "stdout://" |
..\Source\ContextCollector\bin\Debug\net8.0\ContextCollector.exe read-input -i "stdin://" --config ContextCollector.custom.json -o "stdout://" |
..\Source\MalwareDetector\bin\Debug\net8.0\MalwareDetector.exe scan-input -m Malware\Models\triage.mal.zip -t 0.5 -i "stdin://" -o "${outUri}?format=markdown"-
FlowReader:
- Command:
FlowReader.exe read-input -i "${inUri}?format=json&source=flowmon" -o "stdout://" - Reads IPFIX JSON records from the specified input URI.
- Outputs the data to the standard output stream.
- Command:
-
ContextCollector:
- Command:
ContextCollector.exe read-input -i "stdin://" --config ContextCollector.custom.json -o "stdout://" - Reads the input from the standard input stream.
- Computes the context based on the provided custom configuration.
- Outputs the context to the standard output stream.
- Command:
-
MalwareDetector:
- Command:
MalwareDetector.exe scan-input -m Malware\Models\triage.mal.zip -t 0.5 -i "stdin://" -o "${outUri}?format=markdown" - Reads the context from the standard input stream.
- Analyzes the context using the specified malware profiles and threshold score.
- Outputs the analysis results to the specified output URI in Markdown format.
- Command:
This toolchain provides a flexible and efficient workflow for detecting malware by leveraging a series of interconnected tools, each performing a specific role in the data processing pipeline.
This project is licensed under the BSD-3-Clause License, which grants broad permission to use, modify, and distribute the software, provided that the following conditions are met:
-
Redistributions of source code must retain the above copyright notice, this list of conditions, and the following disclaimer.
-
Redistributions in binary form must reproduce the above copyright notice, this list of conditions, and the following disclaimer in the documentation and/or other materials provided with the distribution.
-
Neither the name of Brno University of Technology nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
This license does not require you to release the source code of any modifications you make to the software, but it does not permit proprietary licensing, meaning any derivative work must also be licensed under BSD-3.
© 2024 Brno University of Technology. All rights reserved.
This copyright notice pertains to all the software, documentation, and other materials included in this repository, unless otherwise stated.
This project was developed in the frame of research initiative Analysis of encrypted traffic using network flows.