A keynote summary by Gaspar Modelo-Howard.

Dr. Ronald W. Ritchey, Booz, Allen and Hamilton

Ronald Ritchey is a principal at Booz Allen Hamilton, a strategy and technology consulting firm, and chief scientist for IATAC, a DoD initiative to provide authoritative cyber assurance information to the defense community. He spoke about software complexity and its relation to the number of vulnerabilities found in software.

Ritchey opened the talk sharing his experience as a lecturer for a secure software development course he gives at George Mason University. The objective of the course is to allow students to understand why emphasis on secure programming is so important. Using the course dynamics, he provided several examples on why secure programming is not easy to achieve: much of the code analysis to grade his course projects includes manual evaluation which makes the whole process long, even students with good development skills usually have vulnerabilities in their code, and some students insert vulnerabilities by calling secure-sounded libraries in insecure ways. All these examples allowed Ritchey to formulate the following question: How hard can it be to write good/secure software?

Ritchey then moved on to discuss software complexity. He presented the following statement: software products tend toward increasing complexity over time. The reason is that to sell the next version of a program, market is expecting to receive more features, compared to previous version. To add more features, more code is needed. Software is getting bigger, therefore more complex. So in light of this scenario: Does complexity correlate to software faults? Can we manage complexity for large development projects? And, should development teams explicitly limit complexity to what they have demonstrated they can manage?

Several security experts suggest that complexity increases security problems in software. Quoting Dan Geer, “Complexity is the enemy”. But Ritchey mentioned that researchers are divided on the subject. Some agree that complexity is a source of vulnerabilities in code. The latest Scan Open Source Report[1] found strong linear correlation between source lines of code (SLOC) and number of faults, after the analysis of 55 million SLOC from 250 open source projects. Shin & Williams[2] suggest that vulnerable code is more complex than faulty code after analyzing the Mozilla JavaScript engine.

Some researchers suggest there is no clear correlation. Ozment and Schechter[3] found no correlation after analysis of the OpenBSD operating system which is known for its developers’ focus on security. Also, Michael Howard of Microsoft Corp. pointed out that even though Windows Vista’s SLOC is higher than XP, Vista is experiencing a 50% reduction in its vulnerability count and this is attributed to their secure development practices.

Regardless of the relationship between complexity and security, Ritchey mentioned it is likely that SLOC is a weak metric for complexity and suggested potential replacements in terms of code structure (cyclomatic complexity, depth of inheritance), computational requirements (space, time), and code architecture (number of methods per class, lack of cohesion of methods).

Looking at different popular programs, it is clear that all are becoming larger as new versions are released. MacOS X v10.4 included 86M SLOC and Ubuntu Linux has 121M. Browser applications also follow this trend, with Internet Explorer v6 included 7M SLOC and Firefox v3 has 5M. A considerable percentage of these products doubled their sizes between versions: Windows NT4 has more than 11M SLOC and its later version XP has 40M, Debian v3 has 104M and v4 jumped to 283M.

In light of the different opinions and studies presented, Ritchey analyzed the Microsoft Windows operating system by counting the vulnerabilities listed on the National Vulnerabilities Database[4] for different versions of this popular system. No distinction was made between the root level compromise and other levels. From the results presented, a large number of vulnerabilities were found after the initial release of the different Windows versions. Such trend represents the initial interest shown by researchers to find vulnerabilities who later moved to newer versions or different products. Ritchey also commented on the impact of the foundational (initial release) code, which seems to have a higher vulnerability rate than later added code from updates. From the cumulative vulnerability count vs. complexity (SLOC) graph shown, lines go up so it might be true that complexity impacts security. He alerted though on need to be careful on how to judge these numbers since factors such as quantity and quality of resources available to development team, popularity of software, and operational and economic incentives might impact these numbers.

Throughout his talk, Ritchey emphasized that managing complexity is difficult. It requires a conscious cultural paradigm shift from the software development team to avoid and remove faults that lead to security vulnerabilities. And as a key point from the talk, a development team should know at a minimum how much complexity can be handled.

Ritchey then concluded that complexity does impact security and the complexity found in code is increasing, at a plausible rate of 2x every 5 to 8 years. The foundational code usually contributes to the majority of vulnerabilities reported. The ability to prevent vulnerability rates from increasing is tied to the ability to either limit the complexity or improve how we handle it. The speaker (calls himself an optimist and) believes that shift from software as a product to software as a service is good for security since it will promote sound software maintenance and move industry away from adding features just to sell new versions.

References

1. Coverity, Inc. Scan Open Source Report 2008. Available at http://scan.coverity.com/.
2. Shin, Y. and Williams, L.: Is complexity really the enemy of software security? In: 4th ACM workshop on Quality of protection, pp. 47—50. ACM, New York, NY, USA.
3. Ozment, A. and Schechter, S.: Milk or Wine: Does Software Security Improve with Age? In: 15th USENIX Security Symposium, pp. 93—104. Usenix, Berkeley, CA, USA.
4. National Institute of Standards and Technology. National Vulnerability Database. Available at http://nvd.nist.gov.