How is Machine Learning Used in Cyber Security?

Also posted at Quora

Let's start with 2 points:

1. The objective of cyber security (strategy) is not to avoid 100% the attacks, something unattainable; but to reduce the "attack surface" to a minimal.
2. the number of attack perpetrators will be always bigger than the number of people trying to protect against attacks.

With that in mind, several companies discovered soon enough that fighting for protection was becoming an ever increasing ($$) exercise. The biggest security/infrastructure firms (symantec, mcafee, palo alto, checkpoint etc) united to work in common initiatives, such as developing web apps against DDoS attacks (web apps not in the sense of website but in firewall webapps, also called next generation firewalls).

 

The SecIntel Exchange

Now, a very important concept here to remember: They do not exchange their solutions, they do exchange their attacks. That's a very important point. This is called SecIntel Exchange. The whole idea behind this is: To understand how attacks are done and what types of exploits are there, we need to increase our catch network, so they can be aware of attacks BEFORE they become a real worry.

OK, now that these companies found a common protocol to receive and analyze their attacks, and are able to collect information about what's going on out there in the wild, each company go about and find solutions appropriate for their own products. This is a great strategy, defend as a stronghold, attack as a militia. However, another challenge comes up: Slowly but surely this process is also becoming time-consuming and expensive. In short, it does not scale. Remember, while a company has a team of 10 people to protect, the world always will have thousands working 24x7 trying to break it. (that also explains why Linux/Unix systems don't have as many vulnerabilities as Windows for example, but that's another topic)

 

 

The Machine Learning Angle

Good. Now that's when machine learning (ML) comes in nicely!  In conjunction with other technologies (virtual machines, test simulators, honey pots etc) machine learning algorithms can pick up the information collected by the SecIntel and QUICKLY SCALE the analysis process. What used to take 2 days for an InfoSec team to understand, takes 1 day for an ML algorithm to understand...but that's not the main benefit of ML. The main benefit is that the ML algorithms will learn and predict based on experience and results. It means that today it takes 1 day, tomorrow it will take 20 hours, the next day it will take 12 hours and so on. ML by "learning and predicting" effectively scale the effort to a level human teams cannot do, specially when dealing with automated tasks.

 

A Real World Analogy

Imagine when you do blood tests if your blood had to be analysed individually. It would take weeks before you get your results not because the process is slow but mainly because the queue to get to you will be too large and by then the effort to get the results could be potentially wasted. By scaling the effort, ML will free up the InfoSec teams to focus in the higher ground and strategy trying to be one step ahead of the game.

It is about scale and quick response to market.

images credits: @msau