Our analog world is full of vulnerabilities. I enjoy watching shows like Tiger Team and The Real Hustle because they offer good examples of their great numbers, and how easily they can be exploited. Interestingely enough, we don’t put alot of effort into minimising them either. Instead we accept risks, or transfere them with for instance insurance. We are more focused on detection and response rather than prevention. Likely because it’s evolved as the most cost effective method.

I believe paramount to why this works in the analog world, is because incidents are naturally detectable. If someone steals your juwelery you’ll eventually miss it. We know how much the loss cost, and since all of us have insurance we (or at least the insurance companies) have good averages on what a normal person is likely to loose per year and bases their premiums on that.

But a digital asset can be stolen in a number of ways without the owner knowing about it. Digital incidents aren’t naturally detectable, and we have no real numbers on the number of incidents and the average costs associated with them. In fact, there are insurances for digital security breaches and issues, but if we fail to detect them, when are we ever going to make us of our insurance? It will never be the most cost effective method. It seems like we have no option other than employing prevention and chasing vulnerabilities. But then again, since there are no perfect security, this approach will always leave us with an inaccurate view of incidents and no other options. Just another hamster wheel of pain. Find vulnerability, Patch and Proceed.

Below is a brief taxonomy of various events that I like to use when thinking about security monitoring. Their naming can be questioned, but I believe they give a decent structure to various events and their relationships. It like to think that it aligns well with NSM.

Security Events

Security events are any form of warnings, or what I like to treat them as, “indications” of attacks and/or malicious activities. From a security monitoring perspective, these events can direct the analysis and response activities in a direction where it is likely that something security relevant is happening. Security events can be an IDS/IPS alert from triggered signatures indicating that someone is targeting exploits at our public web services, or a network traffic anomaly alert indicating that an unusual high amount of traffic is leaving our network in the middle of the night. Security events need not be limited to digital ones, but can also be a user calling the help desk saying that his/her computer is slow.

Typical information sources are network/host intrusion detection/prevention systems, web proxies with malware features, endpoint antivirus systems etc. Any system that “looks” for attacks.

Communication Events

Communication events are anything that tell us who is talking to who. Collecting these events allows for establishing, depending on the type events of course, granular “audit trails”. A complete picture of how a certain host has been communicating can assist the response process by limiting the scope of a possible compromise, or to completely dismiss alerts in the case no traffic or data has been exchanged between the target and the source.

Typical information sources for informing us about communication patterns are Firewalls in the forms of Accept and Deny events; and Routing and Switching infrastructure by exporting meta data about sessions and traffic in the form of traffic flow information (i.e. netflow, sflow etc). Session data and network traffic flow information is favourable before firewalls. The ultimate information source for Communication Events are systems dedicated to sampling of full content data from traffic at strategic positions in the network. Full content data offers complete transaction records of sampled traffic, and can often completely verify or dismiss a suspected attack (unless it’s encrypted), but at the cost of alot of diskspace.

Identity Events

Identity events are any form of event that ties an IP-adress to another type of identity, i.e. a user name, a mac adress, a dns-name, a physical location such as a room-numer, or even a telephone number or in the case of an external host complete whois and owner/abuse information. Identitity events allows for establishing relationships between different identities and assets. A relationsship between an IP-adress and a Username aids the analysis and response process by making it possible to ask questions to the person in question.

Typical information source for Identity events are ordinary clients and servers, central authentication systems, dhcp server and dns server.

Activity Events

Activity events are information about activities, and not necessarily strictly security related ones. If communication events informs about who is talking to who, activity events informs about what they are saying and what they are talking about. Activity events allows us to know what is happening in a given situation/session/point in time - i.e. what actions hosts, users or processes are making. Having a complete picture of the HTTP requests a client have been making towards the internet can support the reponse of a possible malware infection of a client. Having a complete picture of what the IIS- and SQL server are up to eases the investigation of a possible XSS or SQL-injection attack.

Typical informations sources for Activity events are any strategic system that has a certain service. Web Proxies tracks any web requests to the internet. A web server, or a web application firewall, tracks interactions with the web server. Collecting Event logs from a Windows server or syslog feeds from an important Solaris servers allow you to track interactions with those resources.

To summarise:

Security Events point us to likely malicious activity. Communication events tells us who is talking to who. Identity Events tells us who the source and target is and Activity events tells us what actions a system, a users or a process is doing. My experience tells me that a successfull security monitoring requires a degree of all four of these.

I favor the idea of trying to measure progress and success of the security process - Any security investment should have a quantifiable outcome in terms of risk reduction. Related to this, some months ago I read Security Metrics by Andrew Jaquith, and somewhere around the same time I read Measuring the Return on IT Security Investments authored by Intel researcher Matthew Rosenquist. The later defines a methodology for measuring the return of a security investment (ROSI) which involves the following steps,

• Evaluating cyber-attack incident data averages over time.
• Measuring the reduction of incidents from implementing new security programs.
• Valuating the impact of avoided incidents.
• Applying the results to similar areas to estimate future value.

The first step involves comparing incident counts between different months, years etc. Security is about avoiding bad things to happen, so a decrease of bad things happening must mean progress. Well, that is only true if we have an accurate value of the number of incidents. Our method of detecting incidents must be exactly as effective and accurate as last year, even though threats targeting us, vulnerabilities in our defences and assets of importance might have completely changed. If our defences (more specifically the detection process) doesn’t follow this change, we will have a inaccurate value of the number of incidents. Without measuring the accuracy of the detection process the “Number of Incidents” comparison is irrelevant. It becomes an oxomoron where we are measurably secure while being insecure.

The “number of incidents” dilemma also haunts the second step which involves measuring the reduction of incidents from implementing new security investments (technical, processes, people etc). So if we make an investment and our incident count drops then that’s considered a success, right? What if that security investment causes incidents to go undetected? Perhaps turning on IPSEC between all our servers and clients are considered a security investment - but what happened with all the alerts from IDSs that contributed to defining incidents when all traffic flyes by encrypted? Again, the model must take detection accuracy as a parameter.

The second and third step involves “valuing impact of avoided incidents” and “estimate future value” of investments. These steps aim to define how much the investments have really paid of in terms of loss avoidance. But seriously, how do you define the cost coming from an incident when it has not yet happend? Take a malware infection as an example: you can define the potential loss to be anything from the consumption of 10% cpu and memory at an individual workstation to information disclosure and theft of scetches for the next stealh aircraft. This calculation allows anyone to creatively come up with numbers that are favorable for the situation.

Measuring security progress and how security investments actually pays of is an excellent idea. But it’s difficult since both good and bad security might equal zero incidents. Somehow we must also measure how well we detect incidents and malicious activity. 

There has been some debate and discussion lately on the current state of nation-wide cyber security efforts here in Sweden. Recently published reports has highlighted Swedens inability to resist attacks such as those targeting Estonia earlier this year. The reports led to an interpellation and a debate in the parliament earlier this week where concrete suggestions and answers were in the line with: there is currently “work in progress in defining the responsibilities and requirements on government agencys, and once these are ready they will become mandatory”. First of all, why reinvent the wheel? Why not just require government agencys to meet an all-ready mature, well defined and frequently “used” standard such as iso 27001? Too easy..

Related to that, a year ago (2006) the Swedish government arm KBM published a report that highligheted the current state of Swedish cyber security efforts. From the report (”freely” translated),

It can be established that Sweden today lack a national system for discovering, alerting, terminating and in a coordinated way respond to [incidents].

Hopefully, the “requirements” mentioned in the debate will attempt to address this. Efforts such as giving SITIC - the national incident response organistation - additional funding and more responsibilities are excellent steps in the right direction. Other countries are also moving in this direction, such as the US for instance whom are currently planning to reduce the number of internet connections to be able to monitor them more efficently. I’d like to see Sweden following similar tactics.

I’ve spent alot of time lately writing about how various security technologies fail. My idea is that the problem boils down to the fact that most of our security efforts are vulnerability oriented. Fixing the vulnerabilities seem to be the smartest thing to do, right? Problems is, we can’t keep up with patching and we aren’t aware of all our vulnerabilities, or even a small part of them. To some degree, we will always be vulnerable. In order to complement the natural patch management process I favor efforts and technologies that are threat centric instead of vulnerability centric; Meaning efforts that focus on preventing threats instead of instances of attacks at vulnerabilities.

In an attempt to make an example, have a look at authentication systems. Most of us understand that all passwords can be broken if the necessary time and resources are put into it. The security of the authentication system relies on the strength of passwords, and in order to higher the bar for brute forcing it we

• create a basic policy that defines the minimum strength of the password (eight chars, with numbers etc).
• we force the user to change the password every four, six or eight weeks.
• we only let the user make three unsuccessfull attempts before disallowing any further attempts (or we incrementaly increase the delay between each possible attempt).

The first two points are vulnerability oriented, meaning they attempt to increase the strength of the password and as such decrease the vulnerability of the password. They increase the level of effort the threat must put into brute forcing the password. The last one, however, focuses on preventing any attempts of guessing the password. It doesn’t increase the strength of the password, but drastically limits the threats possibility to repeatably try passwords. This measure is threat oriented and no matter how much money or time he or she puts into buying equipment for crunching numbers, all they have is three attempts (there are other ways of defeating authentication systems though).

This kind of threat centric approach is what most Intrusion Prevention technologies lack. Common IPS:s functions as authentication systems without the threat centric part; They block individual attacks but passively lets the intruder keep trying until he crafts an exploit that slips through. Good examples are solutions that correlates alerts with vulnerability information of the target (which ISS does) in order to decide what to block. They forget to question whether the activity itself is acceptable. Just because the webserver runs Apache, all attempts at exploiting it as an IIS is ok? Or just because a perticular vulnerability isn’t present in this version of Apache, the activity is ok? This kind of thinking is, for example, what causes IPS implementation to fail at repelling penetration tests.

To be fair, some IPS:s have the necessary functionallity for being a threat centric tool. Juniper, for instance, has the option to block individual offending packets, an offending session and ultimately an offending IP for a configurable amount of time. Other IPS:s has similar functionality. Problem is that the functionallity isn’t used in an automated fashion due to the risk of consequences from creating long lasting blocks of legitimate traffic sources. This is why I advocate the involvement of a human to make the ultimate decision to block an offending IP for a given time, or use a good SIEM system (Security Information and Event Management) that correlates alerts with other information sources to get the knowledge of context necessary to make accurate decisions.

Expect to see more posts related to threat centric security in the future.

In response to a post on risk versus uncertainty.

“people spend too much time trying to reduce uncertainty and too little time focusing on reducing risk.”

My impression is the exact opposite. Companies spend to much time (and money!) on ad-hoc attempts in reducing risk with no control of where their biggest risks are or how these countermeasures actually pays of in terms of risk reduction. There is too much focus on headline-threats and efforts resembling “fire-fighting” and “socker-goal security”. Companies buy firewalls, intrusion prevention systems, data leak prevention solutions for millions so the can put them into place and forget them. There is too much uncertainty in daily security operations, which is why I think that reducing uncertainty is crucial. Companies often can’t answer the simplest questions. I say implement solutions that give you insights in vulnerabilities, threats, assets and ultimately risks (no, the answer isn’t an annual risk-analysis paper exercise). Then (!) implement measures for risk reduction.

A common question when discussing intrusion detection versus intrusion prevention is “why on earth would we want to do only detection?”. The pretty simple reason is (which I have written about tons of times) the inaccuracy of finding and identifying something that is a very small part of the entire set looked at. I think we can learn alot in performing security in the digital world from the analog. For instance, why is it that we have very few systems in the analog world that claims to prevent something once detected? The security cameras in the subway isn’t hooked up with stun guns right? The “X-ray station” at the airport is monitored by real personel, why is that? And why not having a way of automatically stopping a car that is violating the traffic control signs? Because it doesn’t work. Why would the digital world be any different?

I learnt from Schneier of a good example of the difficulty of performing monitoring and incident detection, in the real world. From the article,

The 178 video cameras that keep watch on San Francisco public housing developments have never helped police officers arrest a homicide suspect even though about a quarter of the city’s homicides occur on or near public housing property, city officials say.

Why is that? Could it be because “Nobody monitors the cameras, and the videos are seen only if police specifically request it from San Francisco Housing Authority officials”. No shit.

Another interesting comment:

The cameras have occasionally managed to miss crimes happening in front of them because they were trained in another direction, and footage is particularly grainy at night when most crime occurs, according to police and city officials.

In the digital world we don’t have darkness clouding our detection instruments, but our “cameras” (intrusion detection systems) can be just as grainy and badly positioned. I agree with Schneier when he says that the cameras, who intends to have a scaring affect on criminals, also causes a false sense of security for the inhabitants of San Fransisco. Since we normally don’t disclose that we have “digital cameras”, aka intrusion detection or prevention systems, in the digital world we can pretty much rule out the scare off effect. What about the false sense security?

I learned by Slashdot that some of the Ubuntu servers have been hacked. As some of the comments to the Slashdot post suggests, that is no catastrophe itself (but it quickly can be). Everyone makes mistakes, and prevention eventually fails. They are not alone. What matters is wheather or not they have the routines, tools and processes in place to investigate, analyse and limit the damage.

What is interesting is that we actually get to know that they have been compromised, and that raises some questions: Does knowing about the compromise increase or decrease my trust in the Ubuntu source? Let us look at what the announcement gives us:

Knowledge of the breach lets us

• Choose wheather or not to apply future updates of fear that they might contain backdoors, at least until the scope of the compromise is fully determined.
• If considered necessary by -us-, choose wheather to withdraw all ubuntu boxes and replace them with another distribution or OS
• Watch and see how the Ubuntu community and sponsors, e.g. Conical, reacts to this. How serious they are with security and how well their processes and routines are for handling these kind of emergency situations.
• be somewhat assured that the Ubuntu servers and its administrators are competent enough to detect the compromise, and yeah, that can be hard.

If things were as yesterday, with no breach, what knowledge does that give us?

• Can they detect a compromise if there were one? Do they have the necessary tools and processes to do so?
• Are the servers compromised? The source code? Are there backdoors in the software?
• Should we react? How? What might be compromised?
• …. etc

The point is, without any other information, an announcement of a compromise yields us with more information to make decisions than a company that has no apparent security issues. This is a fundamental problem with security. It’s similar with vulnerabilities: a company anouncing a vulnerability in their software at least proves that they are looking, the absence of one proves nothing. I’m not saying I wish more companies were compromised, but I wished they reported once they were. The companies that are not can have other things to show that their customers can be assured that they have the necessary tools and processes in place. I come to think of information security frameworks such as ISO 17799 and similar standards witch at least proves that a companies is putting some effort into security.

To sum the post up. The following weeks will tell more of Ubuntu’s security-priorities than some of their un-compromised rivals.

Apperently, the growth of the number of disclosed vulnerabilities are declining. As usuall, these kind of statistics follows with alot of speculations of what this means for security. A common idea is that fewer disclosed vulnerabilities must mean we are more secure, right?

While at it, let me speculate some too. From a hackers perspective, the most valuable vulnerability is the one no-one else knows, right? Such a vulnerability, if in the “right” (from our perspective, wrong) piece of code, would let a hacker walk in through the front door of many targets. With money to gain from stealing company secrets, identity information and credit card details, it’s relatively obvious that vulnerabilities themselves have more value today than earlier. The decline in disclosed vulnerabilities could just as easily be a result of an increased care, focus and determination within the underground. For use in targeted, more sophisticated attacks. There is no easy way for us to know exactly, and that is why drawing conclusions of the state of security from statistics of vulnerabilities is far to simplistic.

This kind of mindset reminds me of a speech I heard from the CSO of a large financial company here in Sweden: “We measure our security on the number of security incidents we have a year”. I instantly thought, “That says nothing. How hard are you looking?”.