Tuesday, April 5, 2011

Panel Members:

• Victor Raskin, Purdue
• Greg Shannon, CERT
• Edward B. Talbot, Sandia National Labs
• Marcus K. Rogers, Purdue

Panel Summary by Pratik Savla

Edward Talbot initiated the discussion by presenting his viewpoint on Cyber security. He described himself as a seasoned practitioner in the field of cyber security. He highlighted his concerns for cyber security. The systems have become too complicated to provide an assurance of having no vulnerabilities. It is an asymmetrical problem. For an intruder, it may just take one door to penetrate the system but for the person managing the system, he/she would need to manage a large number of different doors. Any digital system can be hacked and any digital system that can be hacked will be hacked if there is sufficient value in that process. Talbot described problems in three variations: near-term, mid-term and long term. He used a fire-fighting analogy going back two centuries when on an average a U.S. city would be completely gutted and destroyed every five years. If the firefighters were asked about their immediate need, they would say more buckets are required. But, if they were asked what to do to prevent this from happening again, they had no answer. Talbot placed this concern into three time-frames: near-term, mid-term and long term. The first time frame involves the issue of what to do today to prevent this situation. The second timeframe tries to emphasize that it is important to be ahead of the game. The third timeframe involves the role of science. In this context, the development of a fire science program in academia. To summarize, he pointed out that the thinking that gets one into a problem is insufficient to get one out of the problem.

Talbot quoted a finding from the JASON report on the science of cyber security which stated that the highest priority should be assigned to the establishment of research protocols to enable reproducible experiments. Here, he stated that there is a science of cyber security. He concluded by comparing the scenario to being in the first step of a 12-step program (borrowing from Alcoholics Anonymous). It means to stop managing an unmanageable situation and instead developing a basis to rethink what one does.

Rogers focused on the the question: Do we have foundations that are scientifically based that can help answer some of the questions in form of research? Are we going in the right direction? This lead to a fundamental question: how we define a scientific foundation? What defines science? He highlighted some common axioms or principles such as body of knowledge, testable hypotheses, rigorous design and testing protocols and procedures, metrics and measurements, unbiased results and their interpretation, informed conclusions, repeatability as well as feedback into theory that are found across different disciplines. The problems that one comes across are non-existence of natural laws, man-made technologies in constant flux, different paradigms of research such as observational, experimental and philosophical, non-common language, extent of reliability and reproducibility of metrics, difference in approach such as applied versus basic, studying symptoms as opposed to causes. Cyber security is informed by a lot of disciplines such as physics, epidemiology, computer science, engineering, immunology, anthropology, economics and behavioral sciences.

The JASON report on the science of cyber security came out with strategies that are areas such as modeling and simulation which involved biological, decisional, inferential, medical as well as behavioral models that could be considered when viewing it on a scientific foundation. He emphasized that cyber security problems lend themselves to a scientific based approach. He stressed that there will be a scientific foundation for cyber security only if it is done correctly and only when one is conscious about what constituted a scientific foundation. Even solutions such as just-in-time, near-term and long-term can be based on a scientific foundation.

He pointed out that currently the biggest focus was on behavioral directive. In other words, how do we predict what will happen 20 years from now if employee ‘X’ is hired?

Shannon addressed the question: How do we apply the scientific method? Here, he presented the software engineering process. He discussed its various components by describing the different issues each one addresses. Firstly, what data do we have? What do we know? What can we rely on? What is something that we can stand on which is reasonably solid? Secondly, why do we have data that is prone to exploitation? He highlighted reasons such as lack of technology as well as mature technology, lack of education and lack of capacity. Here, he concluded that these hypotheses do not seem to stand the test of data as the data indicated we have always had problems. He then stated some alternative hypothesis such as market forces, people and networks that can be considered. He stressed on the point that solutions are needed based on what people and systems do, not what we wish they would do. The stumbling block for such a case is the orthodoxy of cyber security which means being in the illusion that by just telling people to do the right thing and using the right technology would lead to a solution to a problem. It is analogous to an alchemist who would state that just by telling the lead to turn gold, it would become gold. He stressed that we need to understand what is going on and what is really possible. The key message was that if there is a science that is built on data, it would involve much more than just theory.

Raskin took a more general view of cyber science by offering some of his thoughts on the subject. He said that he did not agree to the “American” definition of science which defines it as a small sub-list of disciplines where experiments can be run and immediate verification is possible as he considered it to be too narrow. He conformed to the notion of science wherein any academic discipline that is well-defined is a science. He presented a schematic of the theory-building process. It involved components such as phenomena which corresponded to a purview of the theory, theory, methodology and the description, which is a general philosophical term for results. The theory is connected to the methodology and a good theory would indicate why it can help guide the methodology. He asked why we were not questioning what we were doing. The first thought was related to the issue of data provenance i.e. why are you doing what are you doing? The second thought focused on the question of how we deal with different sciences that all part of cyber science. A mechanism that can help address that is that of rigorous application. He disagreed with the notion that combining two things without any import/export of sub-components leads to some worthy result. He stated that from the source field, components such as data, theory and methods should be imported to the target field. Only the problems of the source field should be excluded from being imported. The second thought emphasized about forming a linkage between the two fields; source and target by a common application. He concluded that without a theory, one does not know what one is doing and one does not know why one is doing it? It does not imply that there is no theory in existence. On the contrary, anything that is performed has an underlying theory and one may not be having any clue about that theory.

A question about complexity theory brought up an example of a bad scientific approach wherein the researcher adds more layer of complexity or keeps changing the research question but does not ever question the underlying theory which may be flawed.