The Center for Education and Research in Information Assurance and Security (CERIAS)
The Center for Education and Research inInformation Assurance and Security (CERIAS)
I haven’t posted an update lately of new content on our site, so here’s a bit of a make-up post:
A look at the National Vulnerability Database statistics will reveal that the number of vulnerabilities found yearly has greatly increased since 2003:
| Year | Vulnerabilities | %Increase |
|---|---|---|
| 2002 | 1959 | N/A |
| 2003 | 1281 | -35% |
| 2004 | 2367 | 85% |
| 2005 | 4876 | 106% |
| 2006 | 6605 | 35% |
Average yearly increase (including the 2002-2003 decline): 48%
6605*1.48= 9775
So, that’s not quite 9999, but fairly close. There’s enough variance that hitting 9999 in 2007 seems a plausible event. If not in 2007, then it seems likely that we’ll hit 9999 in 2008. So, what does it matter?
MITRE’s CVE effort uses a numbering scheme for vulnerabilities that can accomodate only 9999 vulnerabilities: CVE-YEAR-XXXX. Many products and vulnerability databases that are CVE-compatible (e.g., my own Cassandra service, CIRDB, etc…) use a field of fixed size just big enough for that format. We’re facing a problem similar, although much smaller in scope, to the year-2000 overflow. When the board of editors of the CVE was formed, the total number of vulnerabilities known, not those found yearly, was in the hundreds. A yearly number of 9999 seemed astronomical; I’m sure that anyone who would have brought up that as a concern back then would have been laughed at. I felt at the time that it would take a security apocalypse to reach that. Yet there we are, and a fair warning to everyone using or developing CVE-compatible products.
Kudos to the National Vulnerability Database and the MITRE CVE teams for keeping up under the onslaught. I’m impressed.
One of the most convincing arguments for full disclosure is that while the polite security researcher is waiting for the vendor to issue a patch, that vulnerability MAY have been sold and used to exploit systems, so all individuals in charge of administering a system have a right to know ALL the details so that they can protect themselves, and that right trumps all other rights.
That argument rests upon the premise that if one person found the vulnerability, it is possible for others to find it as well. The key word here is “possible”, not “likely”, or so I thought when I started writing this post. After all, vulnerabilities can be hard to find, which is a reason why products are released with vulnerabilities. How likely is it that two security researchers will find the same vulnerability?
Mathematically speaking, the chance that two successful security researchers (malicious or not) will find the same flaw is similar to the birthday problem. Let’s assume that there are X security researchers, each finding a vulnerability out of N vulnerabilities to be found. In 2006, 6560 vulnerabilities were found, and 4876 in 2005 (according to the national vulnerability database). Let’s assume that the number of vulnerabilities available to be found in a year is about 10 000; this is most surely an underestimation. I’ll assume that all of these are equally likely to be found. An additional twist on the birthday problem is that people are entering and leaving the room; not all X are present at the same time. This is because we worry about two vulnerabilities being found within the grace period given to a vendor.
If there are more successful researchers in the room than vulnerabilities, then necessarily there has been a collision. Let’s say that the grace period given to a vendor is one month, so Y = X/12. Then, there would need to be 120,000 successful security researchers for collisions to be guaranteed. For fewer researchers, the likelihood of two vulnerabilities being the same is then 1- exp(-(Y(Y-1))/2N) (c.f. Wikipedia). Let’s assume that there are 5000 successful researchers in a given year, to match the average number of vulnerabilities reported in 2005 and 2006. The probability that two researchers can find the same vulnerability over a given time period is:
| Grace Period | Probability |
|---|---|
| 1 month | 0.9998 |
| 1 week | 0.37 |
| 1 day | 0.01 |
In other words, nowadays the grace period given to a vendor should be on the order of one or two days, if we only take this risk into account. Has it always been like this?
Let’s assume that in any given year, there are twice as many vulnerabilities to be found than there are reported vulnerabilities. If we make N = 2X and fix the grace period to one week, what was the probability of collision in different years? The formula becomes 1- exp(-(X/52(X/52-1))/4X), where we take the ceiling of X/52.
| Year | Vulnerabilities Reported | Probability |
|---|---|---|
| 1988-1996 | 0 | |
| 1997 | 252 | 0.02 |
| 1998 | 246 | 0.02 |
| 1999 | 918 | 0.08 |
| 2000 | 1018 | 0.09 |
| 2001 | 1672 | 0.15 |
| 2002 | 1959 | 0.16 |
| 2003 | 1281 | 0.11 |
| 2004 | 2363 | 0.20 |
| 2005 | 4876 | 0.36 |
| 2006 | 6560 | 0.46 |
So, according to this table, a grace period of one week would have seemed an acceptable policy before 2000, perhaps fair in 2000-2003, but is now unacceptably long. These calculations are of course very approximative, but they should be useful enough to serve as guidelines. They show, much to my chagrin, that people arguing for the full and immediate disclosure of vulnerabilities may have a point.
In any case, we can’t afford, as a matter of national and international cyber-security, to let vendors idly waste time before producing patches; vendors need to take responsibility, even if the vulnerability is not publicly known. This exercise also illustrates why a patch-it-later attitude could have seemed almost excusable years ago, but not now. These figures are a serious problem for managing security with patches, as opposed to secure coding from the start: I believe that it is not feasible anymore for traditional software development processes to issue patches before the threat of malicious disclosure and exploits becomes significant. Finally, the grace period that we can afford to give vendors may be too short for them to issue patches, but that doesn’t mean it should be zero.
Note: the astute reader will remark that the above statistics is for any two vulnerabilities to match, whereas for patching we are talking about a specific vulnerability being discovered independently. The odds of that specific ocurrence are much smaller. However, we need to consider all vulnerabilities in a systematic management by patches, which reverts to the above calculations.
