Posts in Reviews

If you are intrigued by OVAL, the Open Vulnerability and Assessment Language, and are considering developing rules in it or just want to understand the technical details of how it works, you will need to look at its schemas.  The OVAL Design Document, Version 5.0 states that there are 3 schemas, for “representing configuration information of systems for testing; analyzing the system for the presence of the specified machine state (vulnerability, configuration, patch state, etc.); and reporting the results of this assessment.” However, when looking at the downloads available (here’s version 5.3) you’ll notice that there are 28 pdf documents.  Which ones do you need?

The document “Introduction to the OVAL Language, Version 5.0” contains this figure:
Conceptual Breakdown of the OVAL Language
-----------------------------------------
OVAL Definition Schema
|
|--> Core Schema
|
|--> Independent Schema (family_test, variable_test, xmlfilecontent_test, etc.)
|
|--> UNIX Schema (file_test, process_test, uname_test, etc.)
| |
| |--> Solaris Schema
| |--> HP-UX Schema
| |--> MacOS Schema
| |
| |--> Linix Schema (dpkg_test, rpminfo_test, etc.)
| |
| |--> RedHat Schema
| |--> Debian Schema
|
|--> Windows Schema (file_test, wmi_test, etc.)
|
|--> Apache Schema


This isn’t a 1-to-1 match with the pdfs available for download, but presents the idea.  One difference is that the downloads are organized by rows for “Core”, others for “Common”.  One of the downloads is titled inside as “Core Common”—which row would you guess it’s in?  What’s the difference between Core, Common and Independent?

Note that “Core Common” is present in all of the schemas, from “OVAL Definition Schema” to “OVAL Variables Schema” (hmm, that’s a 4th schema!?).  So, “Core Common” is what is common to the core of all schemas.  The “core” of a schema is therefore composed of the “Core Common” and the part of the “Core” that is specific to that schema.  Then, the “Independent” part of the schema describes things that are common to all operating systems, and serves to avoid duplication between the documents specific to OSes.  So, the difference between “Independent” and “Core” is that the OS-specific parts of the schema have dependencies on the “Core” but not on the “Independent” part.

To get a complete schema, you assemble it from “Core Common”, the “Core” that is specific to that schema, the “Independent” part of the schema, and then the parts specific to your system.  So, if you are interested in Solaris definitions, you would need to download the following parts: “Core Common”, “Core”, “Independent”, UNIX and Solaris.

As a more concrete example, the test to find if Solaris is running in 64-bit mode uses the “isainfo_test”, “isainfo_object”, and “isainfo_state” elements defined in the Solaris part of the definitions schema:

uses this object and this state:

From what I can tell, the OVAL interpreter is required to know that given an “isainfo_object” and an isainfo_state with a bits child, it needs to run “isainfo -b”.  Information about a scanned system can be stored in a “isainfo_item” which would also have a “bits” child (defined in the Solaris System Characteristics document).

To check if a service is running you would use the element “process_test” (defined in the UNIX Definition schema, as in:

which would do a pattern matching operation to the output of “ps”:

So again the OVAL interpreter has to know that it needs to run “ps” for this test, and to store the result in a “process_item” element (predictably defined in the Unix System Characteristics schema).

This is nifty but I still feel uncomfortable that all this gathered information is falsifiable in a difficult-to-detect manner.  An attacker who compromised the system and wants to evade detection, and therefore might avoid doing suspicious or more difficult things such as replacing a closed-source OVAL interpreter, could write something resembling a rootkit that would fetch up-to-date OVAL definitions and return “correct” answers when the OVAL tool runs.  A rogue administrator that just wants to pass some tests while doing other things and operating under configurations that aren’t compliant could write a fake “ps” or “inetd” to pass one or two specific rules; that would be much easier than to mess with the OVAL interpreter itself (in their simplest form the fake programs could simply print a constant string).  An attacker could want to indicate that all patches have been applied to avoid the possibility of a patch breaking a “modified” application or closing a backdoor.  It could be done by simply modifying or creating the appropriate registry keys.

Regardless of the likelihood of these scenarios, my point is that the evaluation is “subjective” in the sense that it is done by the subject of the evaluation.  Anyone wanting to evade compliance (as SCAP and XCCDF currently rely on OVAL, even though they are independent from it in theory) could do so easily, in concept anyway, so the software doesn’t necessarily need to fool the administrator, just the OVAL tool; it doesn’t need to be a full rootkit.  OVAL therefore provides a weak proof of compliance.  Using external tests whenever possible, conducted from a trusted machine, would be much stronger, but I see no evidence of efforts in that direction.  I realize that not all tests can be conducted externally, but emphasizing possible external tests would strengthen OVAL.  I have communicated those concerns to MITRE years ago, but it seems that the easy “practical” way is still favored.  One could argue that perhaps it’s “good enough”, and provides an opportunity for vendors to deliver value with more robust auditing methods.  But if the government is the main market for OVAL products, will it buy those better products or the cheap ones?  I understand that the government “needs yesterday” a manageable solution to handle compliance issues.  However, I’m afraid that once the weaker standard of proof will be in place, it will be very difficult to upgrade.

The open source Purdue Vulnerability Scanning Cluster based on Nessus is an example of the external approach to the evaluation of compliance (disclosure: I contributed to the design of the VSC).  A mixed solution, using external scanning when possible, as well as internal, would be very powerful, especially if discrepancies were flagged.