Posts in Secure IT Practices
Page Content
A Serious Threat to Online Trust
There are several news stories now appearing (e.g., Security News) about a serious flaw in how certificates used in online authentication are validated. Ed Felten gives a nice summary of how this affects online WWW site authentication in his Freedom to Tinker blog posting. Brian Krebs also has his usual readable coverage of the problem in his Washington Post article. Steve Bellovin has some interesting commentary, too, about the legal climate.
Is there cause to be concerned? Yes, but not necessarily about what is being covered in the media. There are other lessons to be learned from this.
Short tutorial
First, for the non-geek reader, I'll briefly explain certificates.
Think about how, online, I can assure myself that the party at the other end of a link is really who they claim to be. What proof can they offer, considering that I don't have a direct link? Remember that an attacker can send any bits down the wire to me and may access to faster computers than I do.
I can't base my decision on how the WWW pages appear, or embedded images. Phishing, for instance, succeeds because the phishers set up sites with names and graphics that look like the real banks and merchants, and users trust the visual appearance. This is a standard difficulty for people -- understanding the difference between identity (claiming who I am) and authentication (proving who I am).
In the physical world, we do this by using identity tokens that are issued by trusted third parties. Drivers licenses and passports are two of the most common examples. To get one, we need to produce sufficient proof of identity to a third party to meet its standards of proof. Then, the third party issues a document that is very difficult to forge (almost nothing constructed is impossible to forge or duplicate -- but some things require so much time and expenditure it isn't worthwhile). Because the criteria for proof of identity and strength of construction of the document are known, various other parties will accept the document as "proof" of identity. Of course, other problems occur that I'm not going to address -- this USACM whitepaper (of which I was principal author) touches on many of them.
Now, in the online world we cannot issue or see physical documents. Instead, we use certificates. We do this by putting together an electronic document that gives the information we want some entity to certify as true about us. The format of this certificate is generally fixed by standards, the most common one being the X.509 suite. This document is sent to an organization known as a Certificate Authority (CA), usually along with a fee. The certificate authority is presumably well-known, and performs a check (to their own standards) that the information in the document is correct, and it has the right form. The CA then calculate a digital hash value of the data, and creates a digital signature of that hash value. This is then added to the certificate and sent back to the user. This is the equivalent of putting a signature on a license and then sealing it in plastic. Any alteration of the data will change the digital hash, and a third party will find that the new hash and the hash value signed with the key of the CA don't match. The reason this works is that the hash function and encryption algorithm used are presumed to be so computationally difficult to forge that it is basically not possible.
As an example of a certificate , if you visit "https://www.cerias.purdue.edu" you can click on the little padlock icon that appears somewhere in the browser window frame (this is browser dependent) to view details of the CERIAS SSL certificate.
You can get more details on all this by reading the referenced Wikipedia pages, and by reading chapters 5 & 7 in Web Security, Privacy and Commerce.
Back to the hack
In summary, some CAs have been negligent about updating their certificate signing mechanisms in the wake of news that MD5 is weak, published back in 2004. The result is that malicious parties can generate and obtain a certificate "authenticating" them as someone else. What makes it worse is that the root certificate of most of these CAs are "built in" to browser and application trust lists to simplify look-up of new certificates. Thus, most people using standard WWW browsers can be fooled into thinking they have connected to real, valid sites -- even through they are connecting to rogue sites.
The approach is simple enough: a party constructs two certificates. One is for the false identity she wishes to claim, and the other is real. She crafts the contents of the certificate so that the MD5 hash of the two, in canonical format, is the same. She submits the real identity certificate to the authority, which verifies her bona fides, and returns the certificate with the MD5 hash signed with the CA private key. Our protagonist then copies that signature to the false certificate, which has the same MD5 hash value and thus the same digital signature, and proceeds with her impersonation!
What makes this worse is that the false key she crafts is for a secondary certificate authority. She can publish this in appropriate places, and is now able to mint as many false keys as she wishes -- and they will all have signatures that verify in the chain of trust back to the issuer! She can even issue these new certificates using a stronger hash algorithm than MD5!
What makes this even worse is that it has been known for years that MD5 is weak, yet some CAs have continued to use it! Particularly unfortunate is the realization that Lenstra, Wang and de Weger described how this could be done back in 2005. Methinks that may be grounds for some negligence lawsuits if anyone gets really burned by this....
And adding to the complexity of all this is the issue of certificates in use for other purposes. For example, certificates are used with encrypted S/MIME email to digitally sign messages. Certificates are used to sign ActiveX controls for Microsoft software. Certificates are used to verify the information on many identity cards, including (I believe) government-issued Common Access Cards (CAC). Certificates also provide identification for secured instant messaging sessions (e.g., iChat). There may be many other sensitive uses because certificates are a "known" mechanism. Cloud computing services , software updates, and more may be based on these same assumptions. Some of these services may accept and/or use certificates issued by these deficient CAs.
Fixes
Fixing this is not trivial. Certainly, all CAs need to start issuing certificates based on other message digests, such as SHA-1. However, this will take time and effort, and may not take effect before this problem can be exploited by attackers. Responsible vendors will cease to issue certificates until they get this fixed, but that has an economic impact some many not wish to incur.
We can try to educate end-users about this, but the problem is so complicated with technical details, the average person won't know how to actually make a determination about valid certificates. It might even cause more harm by leading people to distrust valid certificates by mistake!
It is not possible to simply say that all existing applications will no longer accept certificates rooted at those CAs, or will not accept certificates based on MD5: there are too many extant, valid certificates in place to do that. Eventually, those certificates will expire, and be replaced. That will eventually take care of the problem -- perhaps within the space of the next 18 months or so (most certificates are issued for only a year at a time, in part for reasons such as this).
Vendors of applications, and especially WWW browsers, need to give careful thought about updates to their software to flag MD5-based certificates as deserving of special attention. This may or may not be a worthwhile approach, for the reason given above: even with a warning, too few people will be able to know what to do.
Bigger issue
We base a huge amount of trust on certificates and encryption. History has shown how easy it is to get implementations and details wrong. History has also shown how quickly things can be destabilized with advances in technology.
In particular, too many people and organizations take for granted the assumptions on which this vast certificate system is based. For instance, we assume that the hash/digest functions in use are computationally difficult to reverse or cause collisions. We also assume that certain mathematical functions underlying public/private key encryption are too difficult to reverse or "brute force." However, all it takes is some new insight or analysis, or maybe new, affordable technology (e.g., practical quantum computing, or massively parallel computing) to violate those assumptions.
If you look at the way our systems are constructed, too little thought is given to what happens to existing infrastructure when something breaks. Designs can include compensating and recovery code, but doing so requires some cost in space or time. However, all too often people are willing to avoid the investment by putting off the danger to "if and when that happens." Thus, we instance such as the Y2K problems and the issues here with potentially rogue CAs.
(I'll note as an aside, that when I designed the original version of Tripwire and what became the Signacert product, I specifically included simultaneous use of several different message digest functions in different families for this very reason. I knew it was a matter of time before one or two were broken. I still believe that it is beyond reason to find files that will match multiple, different algorithms simultaneously.)
Another issue is the whole question of who we trust, and for what. As noted in the USACM whitepaper, authentication is always relative to a third party. How much do we trust those third parties? How much trust have we invested in the companies and agencies issuing certificates? Are they properly verifying identities? How good is there internal security? How do we know, and how much is at risk from our trust in those entities?
Let me leave you with a final thought. How do we know that this problem has not already been quietly exploited? The basic concept has been in the open literature for years. The general nature of this attack on certificates has been known for well over a decade, if not two. Given the technical and infrastructure resources available to national agencies and organized criminals, and given the motivation to use this hack selectively and quietly, how can we know that it is not already being used?
[Added 12/31/2008]: A follow-up post to this one is available in the blog.
Word documents being used in new attacks
I have repeatedly pointed out (e.g., this post) to people that sending Word files as attachments is a bad idea. This has been used many, many times to circulate viruses, worms, and more. People continue to push back because (basically) it is convenient for them. How often have we heard that convenience trumps good security (and good sense)?
Now comes this story of yet another attack being spread with Word documents.
There are multiple reasons why I don't accept Word documents in email. This is simply one of the better reasons.
If you want to establish a sound security posture at your organization, one of the things you should mandate is no circulation of executable formats -- either out or in. ".doc" files are in this category. I am unsure if the new ".docx" format is fully immune to these kinds of things but it seems ".rtf" is.
Failures in the Supply Chain
[This is dervied from a posting of mine to Dave Farber's Interesting People list.]
There is an article in the October Businessweek that describes the problem of counterfeit electronic components being purchased and used in critical Defense-related products.
This is not a new threat. But first let's reflect on the past.
Historically, the military set a number of standards (MIL-SPEC) to ensure that materials they obtained were of an appropriate level of quality, as well as interoperable with other items. The standards helped ensure a consistency for everything from food to boots to tanks to software, as well as ensuring performance standards (quality).
The standards process was not without problems, however. Among issues often mentioned were:
- Standards were sometimes not revised often enough to reflect changes in technology. The result was that the military often had to acquire and use items that were generations behind the commercial marketplace (esp. in software/computers);
- Knowing and complying with so many standards often caused companies considerable extra time and effort in supplying items, thus raising their cost well above comparable commercial equivalents;
- Incompatible standards across military agencies and services, especially when compared with commercial items used by civilian agencies, led to waste and increased cost, and lack of flexibility in implementation;
- Imposition of rigid standards cut down on innovation and rapid development/acquisition/deployment cycles;
- The rigidity and complexity of the standards effectively shut out new vendors, especially small vendors because they could not match the standards-compliance efforts of large, entrenched defense vendors.
Thus, in June of 1994, William Perry, the then Secretary of Defense, issued a set of orders that effectively provide a pathway to move away from the standards and adopt commercial standards and performance goals in their place. (cf. the Wikipedia article on MIL-SPEC). One of the rationales expressed then, especially as regarded computing software and hardware, was that the competition of the marketplace would lead to better quality products. (Ironically, the lack of vendor-neutral standards then led to a situation where we have large monocultures of software/hardware platforms throughout government, and the resultant lack of meaningful competition has almost certainly not served the goals of better quality and security.)
In some cases, the elimination of standards has indeed helped keep down costs and improve innovation. I have been told, anecdotally, that stealth technology might not have been fielded had those aircraft been forced within the old MIL-SPEC regime.As a matter of cost and speed many MIL-SPEC standards seem to have been abandoned to choose COTS whenever possible without proper risk analysis. Only recently have policy-makers begun to realize some of the far-reaching problems that have resulted from the rush to abandon those standards.
As the Businessweek article details, counterfeit items and items with falsified (or poorly conducted) quality control have been finding their way into critical systems, including avionics and weapons control. The current nature of development means that many of those systems are assembled from components and subsystems supplied by other contractors, so a fully-reputable supplier may end up supplying a faulty system because of a component supplied by a vendor with which they have no direct relationship. One notable example of this was the "Cisco Raider" effort from a couple of years ago where counterfeit Cisco router boards were being sold in the US.
As noted in several press articles (such as the ones linked in, above) there is considerable price motive to supply less capable, "grey market" goods in large bids. The middlemen either do not know or care where the parts come from or where they are being used -- the simply know they are making money. The problem is certainly not limited to Defense-related parts, of course. Fake "Rolex" watches that don't keep time, fake designer shoes that fall apart in the rain, and fake drugs that either do nothing or actually cause harm are also part of the "gray market." Adulteration of items or use of prohibited materials is yet another aspect of the problem: think "lead paint" and "melamine" for examples. Of course, this isn't a US-only problem; people around the world are victimized by gray-market, adulterated and counterfeit goods.
These incidents actually illustrate some of the unanticipated future effects of abandoning strong standards. One of the principal values of MIL-SPEC standards was that it established a strict chain of accountability for products. I suspect that little thought has been given by policy-makers to the fact that there is considerable flow of items across borders from countries where manufacturing expertise and enforcement of both IP laws and consumer-protection statutes may not be very stringent. Buying goods from countries where IP violations are rampant (If there is little fear over copying DVDs, then there is little fear over stamping locally-produced items as "Cisco"), and where bribes are commonplace, is not a good strategy for uniform quality.
Of course, there are even more problems than simply quality. Not every country and group has the same political and social goals as we do in the US (or any other country -- this is a general argument). As such, if they are in a position to produce and provide items that may be integrated into our defense systems or critical infrastructure, it may be in their interests to produce faulty goods -- or carefully doctored goods. Software with hidden 'features" or control components with hidden states could result in catastrophe. That isn't fear-mongering -- we know of cases where this was done, such as to the Soviets in the 1980s. Even if the host country isn't subtly altering the components, it may not have the resources to protect the items being produced from alteration by third parties. After all, if the labor is cheaper in country X, then it will also be cheaper to bribe the technicians and workers to make changes to what they are producing.
The solution is to go back to setting high standards, require authentication of supply chain, and better evaluation of random samples. Unfortunately, this is expensive, and we're not in a state nationally where extra expense (except to line the pockets of Big Oil and Banking) is well tolerated by government. Furthermore, this alters the model where many small vendors acting as middlemen are able to get a "piece of the action." Their complaints to elected representatives who may not understand the technical complexities adds even further pressure against change.
In some cases, the risk posed in acquisition of items may warrant subsidizing the re-establishment of some manufacturing domestically (e.g., chip fabs). This doesn't need to be across the board, but it does required judicious risk-analysis to determine where critical points are -- or will be in the future. We must realize that the rapid changes in technology may introduce new patterns of production and acquisition that we should plan for now. For instance, once elements of nanotechnology become security-critical, we need to ensure that we have sufficient sources of controlled, quality production and testing.
I'm not going to hold my breath over change, however. Some of us have been complaining about issues such as this for decades. The usual response is that we are making a big deal out of "rare events" or are displaying xenophobia. The sheer expense frightens off many from even giving it more than a cursory thought. I know I have been dismissed as an "over-imaginative academic" more times than I can count when I point out the weaknesses.
One of the factors that allegedly led to the decline of the Roman empire was the use of lead in pipes, and lead salts to make cheap wine more palatable for the masses. The Romans knew there was a health problem associated with lead, but the vendors saw more profit from using it.
Once we have sufficiently poisoned our own infrastructure to save money and make the masses happier, how long do we last?
[If you are interested in being notified of new entries by spaf on cyber and national security policy issues, you can either subscribe to the RSS feed for this site, or subscribe to the notification list.]
PHPSecInfo talk at OSCON 2008
If you're at OSCON, and you love security, you may or may not enjoy my talk on PHPSecInfo, a security auditing tool for the PHP environment. I'm actually going to try to show some new code, so if you've seen it before, you can see it again – for the first time.
The talk is at 1:45pm Thursday, 07/24/2008.
Virtualization Is Successful Because Operating Systems Are Weak
- Availability
- Minimized downtime for patching OSes and applications
- Restart a crashed OS or server
- Scalability
- More or different images as demand changes
- Isolation and compartmentalization
- Better hardware utilization
- Hardware abstraction for OSes
- Support legacy platforms
- Availability
- Minimized downtime for patching applications
- Restart crashed applications
- Scalability
- More or different processes as demand changes
- Isolation and compartmentalization
- Protected memory
- Accounts, capabilities
- Better hardware utilization (with processes)
- Hardware abstraction for applications
VMWare ESX Server runs its own OS with drivers. Xen and offerings based on it have a full, general purpose OS in domain 0, in control and command of the VMM (notwithstanding disaggregation). Microsoft's "Hyper-V" requires a full-blown Windows operating system to run it. So what we're doing is really exchanging an untrusted OS for another, that we should trust more for some reason. This other OS also needs patches, configuration and maintenance. Now we have multiple OSes to maintain! What did we gain? We don't trust OSes but we trust "virtualization" that depends on more OSes? At least ESX is "only" 50 MB, simpler and smaller than the others, but the number of defects/MB of binary code as measured by patches issued is not convincing.
I'm now not convinced that a virtualization solution + guest OS is significantly more secure or functional than just one well-designed OS could be, in theory. Defense in depth is good, but the extent of the spread of virtualization may be an admission that we don't trust operating systems enough to let them stand on their own. The practice of wiping and reinstalling an OS after an application or an account is compromised, or deploying a new image by default suggests that there is little trust in the depth provided by current OSes.
As for ease of management and availability vs patching, I don't see why operating systems would be unable to be managed in a smart manner just like ESX is, migrating applications as necessary. ESX is an operating system anyway... I believe that all the special things that a virtualization solution does for functionality and security, as well as the "new" opportunities being researched, could be done as well by a trustworthy, properly designed OS; there may be a thesis or two in figuring out how to implement them back in an operating system.
What virtualization vendors are really doing is a clever way to smoothly replace one operating system with another. This may be how an OS monopoly could be dislodged, and perhaps would explain the virtualization-unfriendly clauses in the licensing options for Vista: virtualization could become a threat to the dominance of Windows, if application developers started coding for the underlying OS instead of the guest. Of course, even with a better OS we'd still need virtualization for testbeds like ReAssure, and for legacy applications. Perhaps ReAssure could help test new, better operating systems.
(This text is the essence of my presentation in the panel on virtualization at the 2008 CERIAS symposium).
Related reading:
Heiser G et al. (2007) Towards trustworthy computing systems: Taking microkernels to the next level. ACM Operating Systems Review, 41
Tanenbaum AS, Herder JN and Bos H (2006) Can we make operating systems reliable and secure? Computer, 39



