Security in computer systems is important to ensure reliable operation and protect the integrity of stored information.  Faults in the implementation can be exploited to breach security and penetrate an operating system.  These faults must be identified, detected, and corrected to ensure reliability and safe-guard against denial of service, unauthorized modification of data, or disclosure of information. We define a classification of security faults in the Unix operating system.  We state the criteria used to categorize the faults and present examples of the different fault types. We present the design and implementation details of a database to store vulnerability information collected from different sources.  The data is organized according to our fault categories.  the information in the database can be applied in static audit analysis of systems, intrusion detection, and fault detection.  We also identify and describe software testing methods that should be effective in detecting different faults in our classification scheme.

Computing systems have evolved from stand-aone mainframes to comlex, interconnected open systems, and this evolution has lead to proliferation of avenues of attack.  With the knowledge that system misusers have open avenues for attack, misuse detection provides an important line of defense.  For a misuse detection system to be effective, there needs to be an audit trail of system activity that was designed to support misuse detection needs. A major challenge in misuse detection is that audit data is inadequate.  The data supplied by current auditing systems lack content useful for misuse detection, and there is no widely accepted audit trail standard. This thesis presents a comparison of the needs of host-based misuse detection with the capabilities of auditing facilities of convential operating systems.  Host-based misuse detection systems are examined, and the audit data used by each are outlined.  Auditing systems of convential operating systems are also examined, and the data colected by each are outlined.  A comparison of the needs of the misuse detection systems and the capabilities of existing auditing facilities is then presented.  the results of this study aid in the determination of what data content should be provided by auditing systems for the support of misuse detection goals.

This dissertation studies one particular aspect of providing communication security: firewall technology. Our work provides a framework in the form of a waterfall model within which firewall systems and their components can be designed and evaluated. We introduce a reference model that captures existing firewall technology and allows for an extension to networking technologies to which it was not applied previously. The essential components of the reference model are authentication, integrity assurance, access control, audit, and their enforcement. All components are governed by a centralized security policy, and they can be deployed in a distributed fashion to achieve scaling. We introduce a formalism that is based on Hierarchical Colored Petri Nets (HCPN)to descibe the functionality of mechanisms used by firewall technology. HCPN’s provide us with a means of descriptions, composition, simulation, and analysis of firewall systems. The implementation of a firewall depends on its underlying network technologies. We describe the concept of authenticated signaling and report on the design, implementation, and exploration of its realization for asynchronous transfer mode (ATM) signaling, using above reference model. The resulting security mechanism can be used as a building block in the construction of firewall systems.