Issue description

Network flow and packet analysis is the ultimate data source for understanding network performance trends, organization productivity, and the detection of abnormal and malicious activity.

The traditional approach to network flow/packets analysis is based on deterministic algorithms. Good examples are setting the threshold for utilization at a static value (75% for example) or looking at very particular patterns in the flow to detect a known virus or attack.

A deterministic approach, while efficient to a certain degree, cannot adopt itself to the dynamic nature of the flows going through the network. Patterns in these flows are often too complex to be efficiently detected by strict algorithms based on traditional logic.

Solution

NFVgrid utilizes a Machine Learning/AI approach to analyze networking flows and packets. Based on this analysis behavioral models are built to determine patterns and thresholds. These models then applied to ongoing networking flows to threshold crossing and abnormalities in the on-going network flow.

The patterns are dynamic in nature and constantly getting automatically adjusted by the models based on present conditions or user input. This non-deterministic approach allows to efficiently adopt to ever changing nature of network activity.

Fully dynamic, self-learning analysis algorithms bring network management efficiency to a completely different level. They detect not only known issues and threads, but also new types of abnormalities, simply by comparing them to the historical behavior of the network.

Functions

• Super-granular data collection. In addition to a traditional netflow-sflow, NFVgrid gets data derivatives from packets as well. More than 15 extra values are collected from IP packets header and body. More can be collected based on users input;
• Big-Data pre-processing. Collection directly from network packets creates serious volume of data. NFVgrid processes them using Big-Data architecture and can handle large amount of data with minimal amount of compute necessary and without any impact on the network’s performance;
• Graph topology analysis. The determination of patterns in the flow is based on connections between network nodes, which NFVgrid represents internally as a graph. This allows for abnormalities in network connections pattern;
• Machine-learning/AI-based analysis. NFVgrid uses Machine-Learning and AI algorithms for network flow pattern recognition. It builds dynamic patterns for threshold determination. When threshold is crossed – an alarm is generated;
• UI for abnormalities management. NFVgrid provides rich UI functions for view of the generated abnormalities and troubleshooting based on them;
• Orchestration feedback loop. When an abnormality is detected, the NFVgrid analytical module sends it back to NFVgrid orchestration for further processing. NFVgrid orchestration, based on business rules, determines the necessary actions, such as increasing resources for VNF or blocking IP with malicious activity using policy management.

Scenarios

• Performance management using dynamic thresholds;
• Malicious network activity detection based on flows/packet analysis;
• Active policy management by instant quarantining IPs showing abnormal activity.