Threat detection with NetFlow: IP reputation

Mike Patterson speaks out
Learn more about Mike Patterson...
Archive

Aironet

Competitive Lab Tests

Subscribe to Bloggers speak out on BradReese.Com

Threat detection with NetFlow: IP reputation

NetFlow threat detection is a great secondary security measure to detect threats such as DDoS, network scans and proves useful for investigating threats detected by the IPS.

Thu, 1/19/12 - 11:59pm View comments

I'm Mike Patterson, the Founder and CEO of network performance monitoring software vendor Plixer International, a registered Cisco developer within the systems management technology category.

Can threat detection with NetFlow become the coup de grace of most Internet threats?

Well, it certainly provides another layer of valuable threat detection and insight, but it's not going to become a company's primary threat detection and mitigation solution. As a huge NetFlow and IPFIX supporter, I tend to agree that flow technologies can augment security practices, but they can't replace them.

Recently I authored a blog on TMCnet about NetFlow Behavior Analysis Systems. Although NetFlow and IPFIX collection can be leveraged to detect abnormal traffic patterns such as odd ratios of flows to unique hosts, much of the security industry still relies on deep packet inspection and IP reputation as a first line of defense.

Threats Not Detected by NetFlow

Although NetFlow can certainly be used to detect some security threats, the inherent limitation of the technology is that it lacks the entire packet. Because of this, NetFlow products such as my Plixer Scrutinizer or those of my competitors Lancope Stealthwatch and Arbor Networks, rely on flow patterns and baselines to detect behavior anomalies. Although effective at detecting some problems, without the entire packet, many attacks get past even the most sophisticated NetFlow analysis.

"Lets consider HTTP Fragmentation attacks. In this attack, a non-spoofed attacker establishes a valid HTTP connection with a web server. The attacker then proceeds to fragment legitimate HTTP packets into tiny fragments, sending each fragment as slow as the server time out allows, holding up the HTTP connection for a long time without raising any alarms" said Lori MacVittie, who is responsible for education and evangelism of application services available across application delivery networking vendor F5 Network's entire product suite.

NetFlow will often export all of these individual packets in a single flow which could prevent NetFlow analysis from detecting the anomaly.

MacVittie continued, "The attacker creates multiple HTTP requests, not by issuing them one after another during a single session, but by forming a single packet embedded with multiple requests. This allows the attacker to maintain high loads on the victim server with a low attack packet rate. This low rate makes the attacker nearly invisible to NetFlow anomaly detection techniques."

How is NetFlow used in Threat Detection?

In practice, NetFlow is used to follow up on an alarm often detected by the IPS or possibly the NetFlow Analyzer. Mainly for investigative purposes, NetFlow reporting can tell us who the machine was talking to at the time of the alarm, which applications or ports were in use and how long the behavior went on. Follow-up on an event is something NetFlow analysis is great for. IPFIX and NetFlow are also handy for routines like verifying that an ACL entry which blocks all traffic except ports 80 and 53. If any traffic outside of these ports comes in, alarm!

NetFlow Threat Detection

NetFlow threat detection can be done with NetFlow for things like DDoS attacks, FIN, XMAS and RST/ACK Scans. Actually it is quite good for detecting many traffic anomalies. The screen capture below is a partial list of the different threats detected by my product, Scrutinizer NetFlow Analyzer with Flow Analytics:

In the above screen capture, you not only see DDoS detection, but also the "Internet Threats Monitor" which checks the IP Reputation of the internet hosts internal users are communicating with.

IP Address Reputation and NetFlow

IP host reputation is one area where NetFlow can provide tremendous value in NetFlow threat detection. As flows come into the NetFlow collector, the source and destination addresses are compared to an updated listed of known internet bad guys. Internal computers communicating with other hosts that have poor reputations trigger events where upon action can be taken or at the very least, noted.

As pointed out by Mike Schiffman at Cisco, "We've learned that NetFlow can tell us who is talking to who across our network, but how can we tell if either who is a bad actor? By checking the reputation of the IP addresses at both ends of the conversation."

Intrusion Protection Systems are the Best Defense

Intrusion Protection Systems (IPS) are the best security defense against internal and internet threats because they have the ability to perform deep packet inspection. NetFlow (excluding technologies such as NetFlow-Lite and Smart Logging Telemetry) does not export the entire packet. We are however seeing next generation firewalls that do perform deep packet inspection which can export NetFlow or IPFIX details. Vendors such as Cisco, SonicWALL and Palo Alto Networks are all exporting flow information with much greater details than that provided by NetFlow v5. User names, applications (e.g. Skype, Webex, Flash, etc.), URLs, Jitter, Latency, syslogs (NSEL) are currently being exported by these vendors. Expect more packet details in flow exports to follow suit.

"IPS (or deep packet inspection) is our #1 security defense; Netflow is a very close #2 (of course now with version 9 you have some limited DPI too)," said Gavin Reid, Manager of Cisco CSIRT.

Gavin added, "If we identify a security issue with IPS, we can query Netflow to find out exactly what IP's accessed the host, at what time they accessed the host, and also what that host did on the network after the issue."

Mike Schiffman reinforced Gavin's above comment by stating "...if a Cisco IPS fires on a malevolent packet, NetFlow data can provide much needed context on the flow or flows that precede, contain, and follow the attack."

NetFlow Security Summary

The experts agree that NetFlow behavior analysis augments existing security measures. Primary security is still done with IPS equipment due to its ability to perform deep packet inspection. NetFlow threat detection is a great secondary security measure to detect threats such as DDoS, network scans and proves useful for investigating threats detected by the IPS.

Related blogs:

Systrax High-Impact Network Monitoring

TMCnet Advanced NetFlow Traffic Analysis