Symposium Transcript: Complexity vs. Security—Choosing the Right Curve, Morning Keynote Address
Transcribed and edited by Jacques Thomas.
The speaker was introduced by Joel Rasmus.
Dr. Ron Ritchey and Booz Allen Hamilton (BAH), the company he works for, have had a relationship with CERIAS for the last couple of years now. This has been a very good relationship; one that has continued to grow. BAH does contracting for the government and for the IATAC, which does IA consulting and testing for governmental agencies and vendors. His work is of interest to a lot of us, with the unique perspective of working inside the government. Dr Ritchey, in addition to his duties with BAH and as chief scientist for the IATAC, also occasionally teaches at George Mason University in Fairfax, Virginia.
We want to thank Dr. Ritchey for readily accepting our invitation to speak at the Symposium on his first visit to Purdue. As many of you know, one of Dr. Ritchey's colleagues, Admiral Mike McConnell, was supposed to participate in yesterday's fireside chat. Unfortunately, he could not attend it. Dr. Ritchey kindly substituted for him. We thank Dr. Ritchey for going above and beyond his commitments in helping this symposium be a success. Today's talk is a talk that Dr. Ritchey had in the works and proposed to finalize for the Symposium. When we asked him if he would like to come give a talk, we did not have to prompt him. He said he had this talk that he had been working on, and that would be good for me to go ahead and finish this talk. The gist of the talk clearly showed that it matched CERIAS's interest in security. We are sure that this talk will raise some eyebrows in the audience.
Teaching as an adjunct faculty member, even at George Mason is a labor of love: the pay is not great, but the job definitely has rewards. One of these rewards is to see how bad a job we are doing as an industry in teaching people about security awareness. I teach a course on how to develop secure software. In my opinion, this is a “101” level course. At George Mason, it is listed as 600. The quality of the code at the beginning of each semester is interesting. Some code is very well structured and most of the code is not that bad, but the security awareness is not there. When you teach security awareness to the students, the quality of the code improves. Especially when you give them the techniques on what to watch for, and the methods and the mechanisms that can be used to avoid those challenges. This is not what this talk is about.
This talk is about the end result, and what happens after you do that [give them the tools and techniques], even when we spend this time, a full semester, 15 weeks of concentrated effort. By the way, they write a ton of code, which is one of the challenges about the class—let me just describe this one problem, since this talk is likely to run somewhat shortly anyway:
Right now, I have students writing programs. I give them a specification and they write a program back to me given that specification. And the only way I can think of to grade it is to actually do a code analysis. To actually look at it and do the testing myself. Then I come back and identify the vulnerabilities at the code level, because if I don’t do it at the code level, how could they possibly learn what they messed up ? And that’s “killing me”. I spend at least 40 hours every time a program is turned in by the class. And I guarantee I am not hitting every vulnerability. If anybody has a way of structuring these assignments so that I can get the same points across, I am all ears. We will leave this as an aside.
At the end of the class, even with the folks that I consider to be good coders, there are still vulnerabilities that are creeping in. Especially into the more interesting and full featured assignements that we give out.
The students spend the second half of the semester writing something big. This year they had two choices for the final assignment. The first choice was a network enabled game, which had to be peer-to-peer, between two machines, running across the network. They could choose the game. I am always interested when people choose really bad games.
Quick thing: in any game, the idea is that there is an adversary actively trying to compromise the game, and the software has to defend against that. Don’t choose random ! Random is hard in that environment. You want to choose chess. Chess has a defined state: there is no randomness in chess. If people choose games like poker, it is a bad idea: who controls the deck ? There are ways to work around that though.
The second choice of assignment was a remote backup service which, surprisingly enough, about half the class chose to do. I think conceptually they can get their head around it. I thought it would be a harder assignment.
These projects turn out to be non-trivial: there is a lot of code written. I think the most lines of code we saw was on the order of 10,000 lines of code, which isn’t huge by the standards of industry, but certainly is pretty big for a classroom project. Some of the submissions were much smaller. Some people try to abstract away the difficulty by using whatever Microsoft libraries they can find to implement the project functionalities, which caused its own difficulties. Most of these projects, by the way, were the worst from a security perspective. Not because the Microsoft technology wasn’t good; actually Microsoft has done some pretty cool stuff when it comes to security APIs, but the students did not understand what the APIs had to offer, so they called them in ways that were truly insecure. This is another challenge that we get: understand the underlying base technology that we rely on is a major challenge.
So, this talk —back to this talk…
This talk is about what we do after we’ve already done some level of training and started focusing on security, to keep our products more secure over time. Because, frankly, our economy (this is a bad time to say this of course) isn’t a big smoking hole; some people would say it is. The impact of software vulnerabilities definitely impacts on the economy. There is a great book called Geekonomics by David Rice. The major thesis of the book is that the cost of vulnerabilities in code is actually a tax that we all pay. So when you buy, say, Microsoft Office or Adobe Illustrator, whatever tool suite you’re on, all the vulnerabilities and bugs in that are paid by every user and every person that gains service through the use of these tools. We are paying a huge cost. Because all of us are being taxed, we don’t see it. It is a very big number. Every once in a while it becomes specific. I have worked for the banking industry. You have to love these guys, because you do not have to make an ROI argument with them when it comes to cybercrime. They’re either experiencing fraud or they’re not. If they’re experiencing fraud, they know how much. In the case of fraud, the numbers are high enough that, even as a CEO used to seeing big numbers, you would pay attention to them. By the way those banks are still profitable, some of them.
The thing is, if all that fraud is happening, who is paying for that ? Eventually, the cost of fraud has to be passed on to the consumer for the bank to remain profitable. So, again, the point I want to make is that we are all paying this price.
Some of the pre-conceptions I had before preparing this talk turned out to be false.
The main question of this talk is “How are we doing as an industry?” Is software trending towards a better place or a worse one? My position when I started preparing this talk is that we are going towards a worse place. The thought around that is that we are trending towards more complex systems. Why is that happening ? I’ll argue with numbers later; these number show that software is getting at least larger if not more complex.
Question: “How do you sell version 2?” Actually, “How do you sell version 9?” (since nobody has version 2 anymore). So how do you sell the next version of a software? You are not going to sell version (n+1) with less features than version (n) in this market. You have to add something. But most of the time we add something, that means more code. There is also all that engineering work that goes into maintaining that code over time. There is no economic incentive to buy version (n+1) if it does not provide something new besides a new pretty GUI. I guess this did not work too well with Vista. In theory it works. Actually, have you seen the Windows 7 interface ? That is actually pretty cool. I am currently being blown away by how the interface on my Mac increased my productivity, after just a few weeks. I think 7 might give Microsoft a leg up in that area. But I digress.
So, software is getting bigger because the vendors who produce the software for us, by and large, need to add features to the products to sell it to us. That’s one reason. A couple of other things though:
- If we are getting bigger, are we getting more complex. And if we are getting more complex, does that actually affect the security question? That was the underlying thought behind this talk.
To the left, this is the only code I can write and guarantee it comes out bug-free. [points to “Hello, World”, with character encoding issues]. There is a joke there…
But that’s not like average code. Average code looks more like the code on the right [points to cryptic code: deeply nested, and dense]. By the way, in fairness, I “borrowed” from the recent winners of the C obfuscation project. But I have been handed code that is at least that ugly. When you start trying to understand that code, you are going to find out that it is nearly impossible to do so. It really matters how much you focus on the readability and legibility of your code. Architecture matters here. But as you get bigger, chances are that even with good controls, you are still going to wind up with increasing complexity.
How many people think they can pick up bugs in the right hand side given an hour to do it? How confident would you be?
I read a lot of source code. By the way, one of the reasons I teach the class is to keep touching source code. Even with the course I teach and all the time I spend doing it, I am not going to find it. Now, there are tricks you can do from a vulnerability perspective. Following the inputs is always a good idea. I do not necessarily have to read every line to find the most tasty vulnerabilities in a piece of code. Nonetheless, that is kind of ugly [obfuscated code on the right] and there is a lot of ugly in the products we use every day. So we have to account for that.
Here is what a few bright guys in the industry have to say about this. [list of a few quotes on the slide including one from Dr. Gene Spafford]
This is a general acknowledgment that complexity is the driving force behind some of the problems that we have. What do you all think ? Is it real, is it fake ? There are pretty outrageous claims being made here. I like the second one: “Every time the number of lines is doubled, the company experiences four times as many security problems.”
Do you buy that ?
If that is the case, pardon the language, but we are screwed. We can’t survive that because the level of changes that we are experiencing in the code base that we are sitting on are following something like Moore’s law. The period at which code bases are doubling is much longer than a year and a half [Moore’s law], but it is occurring. So, if we project at five years, that would be a fairly reasonable time to expect that we will be sitting on something twice as big as what we’re sitting on today, unless it all comes down and crashing and we decide we have to get rid of legacy code.
If we are increasing by four times the rate of security problems, we are in serious trouble. So the questions are:
- “Is that true?”
- “If it is true, or even partially true, what do we do about it ?”
There is research that tries to correlate complexity and security faults. Surprisingly, this is an underexplored area of research. There was not as much literature in this area as I was expecting,but there is some. Some of it comes from Industry and some of it comes from Academia.
The first one comes from academia and it does find some correlation between complexity and security. One of the interesting things about that paper is that it actually went in and looked at a number of different complexity metrics. A lot of people use source lines of code but these guys actually looked at several different ones. Some perform much better than others at predicting vulnerabilities in the experiment that was run, which is a relatively small experiment.
The next one is an industry line. These guys sell a tool that looks for, in an automated sense, for software vulnerabilities. By the way, I really wish that these tools were much cheaper, because they do work ! They actually are cool products. Do they find every bug? Absolutely not: they have very little concept of the context. So I find vulnerabilities in the code of students who run Fortify and Coverity and Klocwork against their code all the time. The thing is, these tools do find vulnerabilities, so they reduce the total count of vulnerabilities. And everytime we close one of these vulnerabilities, we close a window to the attack community. So I wish these tools were cheaper and therefore more commonly used.
So, these guys [next reference] run their experiment over a fairly large codebase. They pulled a number of open source products and ran their tools against them. They found a very large correlation in their experiments between lines of code and not just the number but the rate of software vulnerabilities that were being discovered.
So there are some folks on the pro- side.
We have some folks on the con side. Very interesting paper out of MIT where they went through multiple versions of OpenBSD and looked at what happened to the vulnerability rates over time. And they showed pretty much the opposite. They showed a trending of vulnerabilities down over time. I have a theory for why this is. This is one of the surprising things as I got into this paper I was trying to figure out what was going on there, because that seems counter to my intuition.
There are some other folks at Microsoft who would say “Look at XP and look at Vista, it’s night and day. We got religion, we got our security development lifecycle going on.” By the way, I believe they deserve a huge amount of credit for doing that. You could say it was a response to a market demand, and that’s probably true. But they did it and they have been a pretty strong force in the industry for pushing this idea of secure development forward. I’m not in any way, shape, or form an anti-Microsoft guy, although I really love my Mac. They’re showing a system (Vista) which is larger but has fewer vulnerabilities.
This is interesting: we have people looking at this problem hard and they are coming up with entirely different results. That got me thinking, as I was preparing this talk, “What is actually going on here?” I think I have a few ideas as to why we are getting these divergences.
One thing is that source lines of code is probably a very poor measure of complexity. If I have a program that never branches and is a thousand lines long, chances are it is going to be pretty easy to analyze. But if I have a program that has a bunch of recursive loops in it, and they are chosen based on random values and interact with threads that have to synchronize themselves over a shared database. Someway, it is going to be more difficult to get your head around that. So there are lots of different ways to measure complexity. Examples include:
- Halstead metrics
- McCabe metrics
I think this is an area that ought to receive more focused attention in research. We ought to spend more time trying to understand what metrics should be applied, and in what context should they be applied to understand when we’re getting past our ability to successfully control quality in a software project. I think this is a very interesting research question. I do not know how you feel about that, if that is something that people should chew on. I spent a lot of time crawling through IEEE and ACM in preparation for this talk. Frankly, i was a little bit underwhelmed by the number of papers that were out there on the subject. So, to anybody working on their doctorate, anytime you find a white space, this is a good thing. And this is a very focused research area. It is very easy in one sense to run an experiment. The hard part is actually in understanding the difference between discovering a vulnerability and understanding how many vulnerabilities are likely in a particular population of code.
So, let’s talk a little bit about size. Again, source lines of code are probably not the best measure. And by the way some vendors publish the source lines of code for their products and some don’t. So some of these numbers are better than others; some of them are estimates. Microsoft has started publishing the numbers for Windows. So we have: * 40 million LOC (MLOC) in XP, depending on which version we are talking about * 60 MLOC in Vista, although some people would say higher.
These are pretty large numbers. If you told me my car had 60 MLOC on it, I probably wouldn’t drive it. As an interesting aside, has anyone driven a BMW 7 series? Nice car, right? They had huge quality problems when they were first shipping, mainly the German version, before it came over. Anyone know the story behind this? The cars would crash. I don’t mean they would hit something. The computers in the car would crash. And at that point you could not re-enable them. They were dead; they had to be towed off the road and the computers replaced.
Prof. Gene Spafford: “I had the first series and the windows system onboard, while the car was sitting in the garage, thought it had been in an accident and disabled the car and actually blew the explosive bolt on the battery on the starter cable. So I had the distinction of having the first BMW 7 series in the Midwest towed to the dealership”
Dr. Ron Ritchey: “Don’t you find it ironic that somebody in you position, with your standing in industry, is the guy that has the BMW that blows itself up because of a software fault ?”
Prof. Gene Spafford: “Somehow, it seems very appropriate… to most of my students”
Dr. Ron Ritchey: “I think that’s brilliant.”
Someone in the audience: “Was it actually a security fault ? Wasn’t it actually one of the students who didn’t want you to come to class ?”
Dr. Ron Ritchey: “Well, a failure mode is a failure mode.”
So, how fast are these things growing ? Between versions, quite a bit. Windows, if we go way way back to NT4, some numbers show around 8MLOC; the number I got for this talk was 11, so I went with it. XP, 40MLOC. That’s twice twice. [discusses also Mac OS X, Linux, Internet Explorer]
The point is that a lot of different products are getting a lot more complex. I shouldn’t say that. They are getting larger. But there is strong evidence to suggest that structurally, if you do not focus on it, larger does actually equal more complex. I do need to be a little more careful here. One of the pieces of belief I’m attempting to prove is that the reason why SLOC as a metric are showing that sometimes things get better and sometimes things get worse, is probably because SLOC is not the measure we should be focusing on. Unfortunately, it’s very hard for an outside researcher, unless you work for Microsoft, or you work for Sun, or you work for one of these companies that produces these large code blocks, to have access to the source code in order to run these different metrics and track the performance of these metrics over time. I think that it would help a lot if we could get these metrics published and used better, to correlate these tools.
This is a simple experiment that we ran for the talk. We basically took a number of different versions of Windows and we went into the national vulnerability database NVD, which is a wonderful program run by NIST. They track all vulnerabilities they can get their hands on. They do analysis of each vulnerability, and they put them in this database that is accessible to anybody (to you, to anyone), to do research. This is really useful data. So we pulled the data from NVD.
This slide shows vulnerability discovery over time and each one of the different lines is a different product. [# of vulnerabilities vertical per year, years in horizontal; one curve per product, with the product’s release year as the origin of the curve]. You need to take a second and look a this to realize what is going on. The bottom one where the line jumps up and then kind of bounces at the bottom a little bit, that’s 95. With 95, you saw a little bit of bump when the product was released and then a flat line. What’s going on there? Is Windows 95 a really solid secure product? No. But keep that question in your mind for a little bit. When we start getting to more modern operating systems and look at the curve for XP for instance, it goes up. Notice that it spikes up initially and then it continues to spike over time. In my opinion that is an unusual curve, because that was when Microsoft was starting to develop an appreciation for security being important. If you look at Vista, unfortunately this hasn’t been out that long, we have a huge spike at the beginning. If we look at NT, we see a more classic curve: a lot of spikes in the early lifecycle of the product and then it dwindles down. That dwindle could happen for two reasons. One, nobody cares about it because they are not running it, so vulnerability researchers go away, which I think is a factor to keep in mind. But I also think that the code base that these products are sitting on is being mined out.
For the vulnerabilities that are being discovered, there is this concept of foundational code versus new code that I’ll get into. It’s really interesting that in the first two years we are seeing this spike in vulnerabilities. The count at two years for Vista was around 75 vulnerabilities. In fairness, I did not distinguish between root-level compromises and more minor issues. This is just raw numbers. You might get a much different analysis if the only thing you cared about was root level compromise. This is not an argument that there aren’t techniques to reduce your risk. Fortunately, there are. This is an argument about our ability to control the number of faults that we write into our software, that result in vulnerabilities over time.
[graph showing the cumulative number of vulnerabilities (vertical axis) for Windows products over time (horizontal axis)]
This is an interesting graph: it represents the cumulative number of vulnerabilities over time. The fact that you have numbers going up left to right in this graph isn’t really that scary. But what you want to see is the rate of change over time [the rate of discovery of vulnerabilities] decreasing. You want to see these curves flattening as they reach the right. That’s a good thing. When you see stuff that is accelerating forward, when the curve of your graph is going positive, that’s not a good thing. What that says is that you are not managing complexity well, you are not managing the security well. Vista hasn’t been long enough to figure out which way it is going to curve. XP had a pretty bad history, as did 2K, though you can see it starting to flatten out. And NT is following the curve that we would like to see: it is starting to flatten out as we go to the right.
The last thing that we want to show is cumulative vulnerabilities versus complexity. So this is the slide that attempts to answer the question are we getting worse or better. And, again, because I do not have better data, I have to use source lines of code. As we get to more source lines of code, what is happening to the rate of vulnerabilities that we’re experiencing? They are pretty clearly going up, and the rate at which they are going up is not linear in many cases. There is a curve occurring here. Whether when we got to 120 MLOC it would flatten out or not, we don’t know. What this data tends to suggest is that as complexity goes up, the total count of security vulnerabilities goes up at a rate greater than the linear extrapolation of the number of source lines of code.
I am not going to try to make a proof out of this; it’s a thought exercise. The thought is “Hey, this might be true.” The sum of the evidence seems to suggest that’s true.
So the result suggests that complexity does matter. And actually it contradicts some of the data that has been coming out by industry saying that “We’ve got religion, SDL [software development lifecycle] is working”, which, by the way I think is probably a true statement, but it’s not true enough to completely eliminate this problem. Interestingly enough, why are some of the numbers different ? Let me get into that.
There is this notion of foundational code. Foundational code is the code that is shipped in the initial release or at least exists in a particular version of a product. As that product moves forward and more things are added to it, that new code, those patches, the things that we change, those aren’t in the foundational code. Those are in the new areas. We need to be careful figuring out what is going on when we have foundational code versus new code. We can have two or three different scenarios.
In the first scenario, you have foundational code, say 10 MLOC. Version (n+1) comes out and you have 20 MLOC, and the new code is just a superset of the old code [diagram with two concentric circular areas]. In other words, all of the foundational code still exists in the product.
The other extreme you could have is when new code merely replaces foundational code, which is no longer being used in the product [diagram with a small and a large circle that barly overlap, small being the foundational code]. The first case is much more likely to happen, because once you have a library written … I don’t know if you maintain your own personal libraries of code; I probably have code that I wrote when I was a senior in high school, because it just works. So foundational code tends to be sticky. There are probably lines of code running in Windows 7 that Gates wrote back in his graduate days.
The interesting thing with foundational code is that when we find problems in it we fix them. Sometimes when we fix them we break other things, but over time the quality of foundational code improves. It has to: we’re not that bad. In fact, once we find a problem, if it’s a class of problems, we often go looking for other examples of that same class of problems in our code. So one fault in one section of the code can actually make larger chunks of the code better. So this notion of foundational code is somehing that we need to keep in our heads. As we go out we figure, “Hey, I’m gonna ship version (n+1)”, I might want to ask myself: “Did I only change my code at the margins that make the user experience a little bit better? But most of the code that I’m shipping is actually code that has really gotten through the rigors and the experience of a lot of peer review, a lot of time to stress the code” Or, “did I throw out everything else and start from scratch because maybe my foundational code was so bad that I needed to do that?” Maybe there are other marketing reasons to do that.
But I have to have a notion of how much of the product I am shipping is foundational code versus new, because that is going to affect the vulnerability rate that it is going to experience over time.
Another issue that we need to worry about, say that we are about to ship version (n+1), how many vulnerabilities are in it? 2, 12, 2,000? What you have to worry about is how many people care, how many people are looking. If you have a non-popular product, a niche product, the number of people that will focus on it is probably directly correlated to the number of people that really don’t like you. What I mean by that is that people need to have a reason to target the smaller set of groups that are using that tool.
An unpopular product is not going to experience the same level of scrutiny.
Now, if you are Internet Explorer, there will be a lot of folks focused on you. And this skews our data horribly. If you are trying to compare the fault rate between IE and Opera, that’s a difficult thing to do because, while people care about the security of Opera, people in the attack community are business people. [the day before, the “fireside” conversation touched on the “professionalization of the attack community”]. They are business people that like not to follow the rules. (Of course, some of these people live in countries where the rules are written differently.) They invest in what they can get a return on; they are very good at this. And, if you are going to choose where you are going to invest your money, as an attacker, what do you choose? Do you choose IE or do you choose Opera? Well, up and until the point that Opera has a market share that is similar to IE, you are going to choose IE, because you get a bigger bang for your buck. So, this is something that we need to keep in mind.
Another thing that we need to worry about is that having the vulnerability reporting, itself, turns out to be really difficult to get right. One problem that we have now, is that the vulnerabilities are not being published. There are commercial organizations and a black market that are perfectly happy to pay you to release your vulnerability only to them, and it does not get into tools like the NVD [national vulnerability database]. This is one of the challenges that we have. A lot of the really good vulnerabilities that are being discovered are being sat on. So, we are not learning about them at the same rate that we used to. It used to be that tools like metasploit would be up to date very quickly with new vulnerabilities, but it’s slowing. It’s slowing because you can hand that vulnerability over to the black market and get 10.000 dollars, sometimes more depending on what product is being targeted and how “cool” the attack is —cool being an operational term.
There are other problems: it’s very hard sometimes to [identify] two different bug reports, two different fault reports, that actually are the same vulnerability but they were expressed in differet ways. So, the data that we collect, itself, can be somewhat challenged. Also, we don’t always know when these things are truly closed up. Sometimes the fix dates are published because, when you get to “Patch Tuesday” and your box reboots, they might have told you about all of the vulnerabilities they just patched; they might not. The same thing happens with all of the folks. That’s an operational security decision. So, again, not all of the data is getting to us, so you have to take some of this with a grain of salt. But, again, I am making an argument about the large numbers; not trying to make sure that every single fact that I have is 100% correct. It;s more of a trend that I think demonstrates the concept.
There are some industries that are much more careful about their development process than others. And when you look at the fault rates, their fault rates are actually much lower than our experience by industry as a whole. In this case I am refering to safety critical [faults], and aerospace is a good example of this. Now, there was an incident with one of fly-by-wire airplanes that recently crashed in London. [Prof. Gene Spafford helped clarify that it was an Airbus 340. It is actually a Boeing 777, flight BA038. So, there have been accidents caused by software faults in the aviation industry. But, by and large, their fault rate is pretty darn low.
In fact, their is a project that my academic advisor was involved in, which was TCAS II. TCAS II is a system where, if two airplanes are on a collision path, there are transponders in both aircrafts that negociate with each other and give the airplane directions to deconflict themselves. So, the TCAS alert will go off and say “This airplane, dive, go left”, and the other will say “Climb, go right”. The thing is, you have to make sure that both instructions are right. There has been one accident, where the TCAS system went off and announced directions, and air-traffic control said “No, the TCAS box is wrong, do what I said”. The pilot chose to follow air-traffic control and, indeed, collided. It was a pretty bad accident. The TCAS system was actually correct; it came with the right decision. There has never been a case where TCAS has given the wrong answer in an operational setting. So, the point is that if we focus on this, and train for it, and actually run complexity metrics against our code and measure it today, we can get better. Unfortunately, it is costly: there is a price to be paid. And can we, as a society, afford to pay that cost for every product that we rely on. The answer is probably no. But, I get back to the point I made earlier: are we truly calculating the cost of faults in the products that we rely upon today ?
There are lots of different reasons why managing complexity is hard. I’ve just listed a few of the examples. [slide would help here].
You can’t manage complexity in an ad-hoc development environment. Just throwing code together without any idea of how you are going to collaborate, how you are going to compare, how you are going to define interfaces, how you are going to architect your code, is … madness. It does not work. That’s another thing I learned from my class. Most of the assignments are written as individuals, and the quality of that code is directly tied to the quality of that individual developer. As soon as they get into project teams, things get a lot more interesting, because the faults start showing up at the integration layer, as opposed to the unit level. So, when we bring development shops together, especially large development shops, if you do not have a lot of good process behind your development, you are not going to manage complexity well.
Here is the point that I wanted to get to: if you really focus on security and in general software faults management, you can actually write code that is of higher complexity, without necessarily injecting a lot of security faults into it. But the problem is, you have to know how good you are before you do that. If you have really good development processes, if you have smart developers that thave been trained, they’re security-aware, they’re quality-aware, they’re writing code. [the slide presents curves with different slopes and asymptotic behaviors] You might be able to have —the bottom axis here is complexity, driving up in complexity— [the vertical axis is the number of faults] …
The bottom line is the mystical beast: you write larger code and actually get higher quality. [the curve decreases from left to right]. I would say that if you could be on these first two lines [constant curve and slow linear increase], you are doing very very well. The idea here is that you want to measure the complexity of your code and you want to measure the fault rate that you are experiencing once you ship the product. This is going to take years to get good numbers for any particular organization. But once you know where you are at, from a maturity standpoint, and once you have decided what you are willing to pay for in terms of your development processes, you can place yourself somewhere on this graph. Based on that you are going to have to make a decision of how far along on that complexity curve you are willing to tolerate [faults increase with the complexity], and your customers are willing to tolerate from a vulnerability perspective. If you are that top line [at least quadratic growth] or worse, you don’t want to be doubling the complexity of your code in the next five years, because youre vulnerabilities number is not going to double but to quadruple. You are going to be that “four times” number that we saw a couple of slides ago. If you’re instead on one of these slow-sloped lines, maybe you can double the complexity of your code and still successfully manage the vulnerabilities that you’re exposing. But unless you measure that, unless you’ve looked at it, it’s all ad-hoc. It is faith versus true.
So I am making, I hope, a plea here. As you go out, as you start looking and working on real world software projects that are going to translate into code that large numbers of people run, that you know where you are on this graph.
One thing is that the initial code release, the foundational code, that’s where most of the bugs are introduced. The lesson here is, as you’re releasing large chunks of new, that’s when you need to focus really tightly on this question. When you’re just doing the maintenance-side of things, that’s a little easier. But that initial chunk, all the numbers show —the research is in fact pretty consistent in this area— that large chunks of foundational code is usually where most of the action is from a vulnerability perspective.
Complexity does impact vulnerability. I strongly believe that. I don’t believe that source lines of code (SLOC) are a way to go. I believe that we need better complexity measures that correlate better to security faults. I do believe complexity drives security.
This is an important one: foundational code is increasing in size rather rapidly. So, again, not Moore’s law, but about every five years it appears to be doubling, at least with the large products that we are using every day. And our ability to prevent vulnerabilities from increasing at the same rate is directly related to our ability to put ourselves in the right curve [referring to the graph on previous slide] and then potentially make some hard choices. Because, I have two large options.
If I discover that I am on one of these high-slope curves, one approach is that I can invest more heavily in my software development processes. I could invest more heavily in training my developers. I could invest more heavily in measuring the complexity of code as I am doing it, and maintaining it over time. I could walk closer to the kind of standards that are used in the safety critical world. So I could put myself on a slower slope, and I could say “OK, my customers need more complexity because that’s what is going to sell. But to do that I am going to need to invest more heavily in my software development process.”
The other option is: I have to choose to release products that have the same complexity as version (n). And really what that means is that I am going to choose to give up some functionnality that i would normally like to release. And that’s a very hard decision to make. But, as consumers, we have some ability to control that as well. By the way, one of the things that the software industry has been doing over the last year and a half to two years is moving away from charging you for software. Software as a service: big move. You are not being charged for software; you are being charged for the use of the software. I actually think this is going to be a good thing in the long term for security. Why ? Because there is a profit motive for maintaining that software and keeping it up and reliable over time.
Same token: anybody bought Oracle or any of these large products recently ? They’ll add: you want some new Oracle feature ? Fine, they are almost not going to charge you for it, but they are going to charge you for the maintenance. 25 percent a year. That might be annoying to you but it is changing the economics behind software development pretty dramatically. I think it’s in the right way: a way that starts shifting the focus to what the customers actually need as opposed to adding features just so that you will buy the next version. (somewhat of an aside).
The last option which, I’m afraid, is the default one, is that you don’t care. You go ahead and you release the product that has the increased complexity. You probably don’t know what part of the curve you’re on. And because of that, we are moving towards a world where the complexity [vulnerability?] rates are increasing as opposed to decreasing. Frankly I think that some of the reason the curve of the vulnerability rate flattens out is because there are only so many vulnerabilities that you need as an attacker to get where you want to be. So there are a numbers of reasons why that curve might flatten, which are beyond bugs simply disappearing. So with that, I would open up to thoughts and questions.
- Booz Allen Hamilton
- Geekonomics, by David Rice
- Static code analysis tools for security:
- National Vulnerability Database
- The Metasploit Project
- British Airline’s Boeing 777 crash in Heathrow Note: My curiosity got spiked. Actually, the crash was due to icing of the fuel lines, as reported by the Air Accidents Investigation Branch
- TCAS II