I recenty was having a discussion with someone about the Ph.D. option for a degree here. The person said “I don’t want a Ph.D. because I don’t ever intend to do research at a university.” Thus began a conversation about how the Ph.D. may be a requirement for most faculty positions, but it is not a sentence connected to the degree! Furthermore, not all faculty positions are primarily research positions.
As an example, of the 23 Ph.D. graduates for whom I have been primary (co)advisor to date, 11 have spent some time as faculty members but only four are still full-time faculty. Six of them currently reside outside the U.S., and six (an overlapping group) have started their own companies. Seven are C-level executives, and another 10 are in senior director/partner-type positions. It is certainly not the case they are all doing academic research at a university!
The Ph.D. is a way of learning how to focus on a narrow problem, develop a comprehensive plan to solve it, and then present the problem and its solution in a formal, convincing manner. Thus, completing a Ph.D. is a way to hone time management and research skills, dive into an area of interest, and prove one’s capability to manage a big task. That is useful not only for academic research, but for managing projects, running an agency, and solving problems in “the real world.”
I’m proud of all of these graduates for what they did while completing their degrees and then going on to do interesting and important things in their careers. Here’s a list with mention of their most recent position:
I am working with five Ph.D. advisees currently. Four of them are employed outside of academia and intend to stay in those positions after getting their degrees.
If you’re interested in getting a Ph.D. (or an MS) at Purdue related to cyber security, take a look at our information page.
(As a matter of trivia, even though the majority of my former students didn’t go into university positions, there are at least 53 more people who received the Ph.D. with one of the above 23 as primary advisor. Maybe we should start a “Spaf number” similar to the Erdös Number?)
I recently found myself in a conversation where someone made a comment about "Being so old I've programmed in Pascal!" I'm considerably older than that person, and actually did some of my first programming on plugboards and punchcards. I declined the opportunity to point that out at the time.
Upon some reflection, I realize I've had the opportunity (and sometimes, the necessity) to use many, many different languages during my 48 years of programming. I used to find it empowering and instructive to try different programming paradigms and approaches, so I actively sought out new ones. As my workload and schedule have evolved over time, I have not really picked up many new ones. I’d like to learn Swift and Rust (for example) but I'll need to carve out the time and obtain a compiler, first.
For grins, I thought I'd make a list of programming languages where I wrote at least one non-trivial program, where "non-trivial" means that there were subroutines/functions/methods/etc. I may have left a few out, but... (you can find most of these documented on Wikipedia if you haven't run across them before).
I also wrote one small program in Intercal, to prove to myself that I could. I never worked up the courage to tackle Malbolge.
I've also written and debugged patches in microcode on several machines, but I won't claim that I really mastered any of the associated languages.
There may be a few I left out plus dialect/version variations, but that is almost 60 languages as is. I'm sure there are people who have programmed in more; those of us who have been around for a while have needed to adapt.
I don't program very much anymore. I occasionally will whip up a ksh or Perl script, and very rarely, a C program. Those are sort of my default programming tools. If I needed to, I suppose a weekend or two with some language manuals would get me somewhat back up to speed with these others. Thankfully, no one has a pressing need for me to write code for anything, although I'm still pretty good at debugging (errors tend to be the same in any language). I have written four complete compilers and three full operating systems using some of these languages, including one each in assembly language. Thankfully, that is also not on my agenda to do again.
So when "kids these days..." complain about having to learn a 3rd programming language for class, well, I am amused.
Last week, an article appeared in the Washington Examiner that contained a couple of quotes from me. The context of how the quotes were obtained is explained below.
Apparently, some people took exception to aspects of the article and/or my quotes. This was all manifested on Twitter, with outrage, some ad hominem attacks, bombastic comments about unfollowing me, and more. After all, it is Twitter, and what would Twitter be without outrage and posturing?
(Interestingly, despite some unfollows, my Twitter account as of Sunday has more followers than before this happened. Draw your own conclusions about that. As for me, I don't care much how many people follow or not -- I still post things there I decide I want to post.)
I decided it might be worth a short post on how the quotes came about and perhaps addressing a few things from the article.
Earlier in the week, I received a request to contact a reporter. This is not unusual. I regularly am asked by the press to comment on cybersecurity, privacy, cybercrime, and so forth. The university encourages that kind of outreach. I generally try to provide useful background to reporters.
I called the reporter. He told me he was working on a story but couldn't share details. He gave me a very vague description -- basically, that he had some evidence that someone working in cybersecurity for one of the presidential campaigns had a history of associating with racist organizations, trolling, and breaking into computers. He wanted to know what I thought of that.
As I expressed to him, if true, I thought that was a poor choice. I explained that generally speaking, someone in such a position should have been more thoroughly vetted. He then outlined how the person likely had a history of hacking into other people's accounts and asked me what I thought. I stated -- with that as context -- that people with that kind of history are usually a poor choice for positions of trust. A history of breaking the law suggests they may be (note: may) more likely to do it again, thus posing a risk to their employer. Furthermore, I noted that a past that is concealed from the employer opens up the possibility of extortion. Both of these imply an "insider" risk. Given the high stakes of this election cycle coupled with foreign interference, that seemed like a real problem.
My conversation with the reporter was over 20 minutes in length. He quoted two of my statements in the published article. This should not be a surprise to anyone who has ever spoken to a reporter...or to anyone who has written for the press. Lots of material isn't used, including material that may set useful bounds on what is published.
Unfortunately, "hacking" and "hacker" have divergent meanings. One usage means someone who explores systems and capabilities, often finding new and unexpected features or problems. A second usage means someone who breaks into systems without permission, illegally, often causing harm. This dichotomy has been a problem for over 30 years now, and we still haven't resolved it in general usage. There have been attempts to qualify the term ("white hat" and "black hat," terms which have other problems), and using labels such as "ethical hacking," which implies everything else is not ethical. These are not satisfactory solutions.
In the conversation with the reporter, he was continually using "hacking" in the pejorative sense, such as "hacking into other people's computers without their consent." My replies were to that usage and in that context.
To be clear, I understand the difference. I have taught and worked with people who are hackers in a positive sense. At one time, when I had more free time and less arthritis in my hands, I did my own share of system hacking. When performed with care and consent, the hobbyist/exploratory form of hacking is often fun and educational. Hacking of others' systems without consent, to cause damage or harm, is a bad thing.
The people who take umbrage over use of "hacking" should to pay close attention to context to moderate their blood pressure. Furthermore, they should realize that 30 years of use by journalists to denote unauthorized access means that the general public only understands that one definition of "hacking" no matter how they define it. It is now similar to any malware being labeled "computer virus" -- it is unlikely that the term will ever get a more precise definition for public use.
I have worked in the area of professional ethics for over 3 decades. I wrote one of the first articles on the ethics of computer intrusion and contributed to many textbooks in the area. I helped develop the last two iterations of ACM's Code of Professional Ethics. I am chair of ACM's Committee on Publishing Ethics & Plagiarism. I have lectured on the topic at major companies and government agencies. I teach aspects of ethics in classes. It isn't simply a word to me.
Professional ethics have a vital role in defining a profession. They help practitioners distinguish among choices. They help guide us in knowing the difference between what we can do and what we should do. Every major professional organization, across multiple professions, has some form of professional code of behavior.
In the context of this issue, breaking (hacking!) into other peoples' systems without permission is unethical. It is also usually illegal. Trolling people in the form described to me by the reporter is unethical and harmful. And being a bigot is wrong, although a too common evil in society today.
Those of us who work in computing -- and especially in security-related positions -- should be very concerned about how we are viewed by the public. If we want to be trusted, then we need to act in a trustworthy manner. Ethical behavior and knowledge of the law are important, and distinguish professionals from everyone else.
It is in this context that I made this comment: "People who are well respected don't come from trolling or hacking groups. There's been a culture shift there. Companies don't want to hire people with sketchy backgrounds." That is true. The companies I work with -- banks, aerospace, defense, telecommunications -- do not want people who have a history of breaking into systems (note the version of "hacking" here) or abusing others. It is a liability for them. It is also evidence of poor judgment and a willingness to do unethical things, at least at some time in the past. Those activities are grounds for termination from many positions. A history of those things is often an automatic disqualification from hiring -- and is questioned as a standard part of polygraph exams. (No, I'm not going to have a side conversation about polygraph exam accuracy here, but you can see one of my blog posts from 2006.)
Can people who did unethical things reform? Of course! Sometimes people do foolish things and later regret and repent. However, it is also the case that people who do foolish and illegal things usually deny they did them, or they claim to have reformed so they can get a shot at doing them again. Whether one accepts the apparent reformation of the individual is a matter of faith (religious or otherwise) and risk management. As I noted, "Somebody who shows up with red flags would not be allowed to occupy a position of sensitivity." Maybe this denies someone reformed and talented a position. However, it also is a matter of practical risk reduction and is part of the standard of due care by organizations dealing with information of great value.
I was never given the name or specifics of the person mentioned in the article during the interview. I only learned her name after the article appeared. To my knowledge, I have never met her. I have no personal knowledge of her activities. I made no statements attributing any activities to her. So, if you are a friend of hers and bent out of shape because of the article, you really shouldn't take it out on me.
TL;DR. People will bluster and posture on Twitter. I was quoted as saying some things that set a few people off, either because they don't pay attention to context, don't understand how insider threats are minimized, or perhaps because they are friends of the person the article is about. I guess it is also possible they don't like the venue or the political campaign. Whatever the reason, I don't care if people unfollow me, but if people are abusive in their comments I will block them. However, the people who want to try to understand the overall context may find the above useful.
Meanwhile, here is some reading for you:
Guest Blog by Joe Weiss, Applied Control Systems, Inc
The statistics from the call include:
There were 183 pre-registrations of which 119 attended. The registrations were from 16 countries – Australia, Austria, Brazil, China, Germany, India, Israel, Kuwait, Lithuania, Mexico, Netherlands, New Zealand, Saudi Arabia, Singapore, UK, US. Actual attendees were from India, Israel, Kuwait, Lithuania, Mexico, Netherlands, Saudi Arabia, Singapore, UK, US.
For those unable to attend, the recording will be on the Purdue Cerias website at: https://ceri.as/weiss
After 20 years, control system cyber security has made significant strides in monitoring and securing OT (control system) networks. However, after 20 years, control system cyber security has made minimal strides in monitoring or securing the actual control system devices (e.g., process sensors, actuators, drives, analyzers, etc.) and lower level device networks which is where you go “boom in the night”. Much of this is because of the culture clash between the engineering community who understand the devices but generally have been removed form control system cyber security efforts and the network security personnel who do not understand control system devices or control system processes. The impact of the culture gap is at least partially due to network security’s erroneous assumptions:
There were 10 questions raised that I did not have a chance to answer on the webinar. I thought the questions and answers would be of general interest.
1). Q: Joe, this is great. You said "Our information sharing doesn't work." What do you think needs to be improved, and how would you improve it?
Answer: Information sharing on cyber network vulnerabilities are addressed in DHS and vendor disclosures as well as industry ISACs. The information sharing that is missing is about actual cyber-related incidents. NERC Lessons Learned don’t address incidents as being cyber-related. The various industry ISACs have not addressed cyber incidents occurring within their industry. The sharing on control system incidents to date most often has been by engineers who have seen incidents that were out of the ordinary. Informally, my old conference (ICS Cyber Security Conference which no longer exists) served as an informal information sharing vehicle for the engineers to discuss real control system cyber-related incidents. Unfortunately, I don’t believe the government can help because of the private organizations concern about making these incidents public. I wrote a chapter in my book, Protecting Industrial Control Systems from Electronic Threats Chapter 9 “Information Sharing and Disclosure”. I will have more to say about this subject in a separate blog at www.controlglobal.com/unfettered.
2). Q: What is your view on Executive order for going back to analog system? We are all driving through zero carbon and digitalization- How to achieve the balance between them?
Answer: Hardwired analog systems with no “intelligence” such as electromechanical float switches as part of a hard-wired relay ladder logic system would be independent of a cyber attack from external threat agents, including hardware backdoors. However, adding any intelligence and modern communication capabilities would make the analog systems as vulnerable as digital systems to a backdoor sensor attack. Both smart and dumb systems would be potentially vulnerable with respect to a physical, hands on insider attack. That is the reason for defense-in-depth. The only way to get the balance between zero carbon and digitalization (or manufacturing and digitalization) is to have engineering and network security work the issues together.
3). Q: What approach do we take to secure level 0 and level 1 equipment?
Answer: I mentioned in my presentation that a major misconception is that all process sensor communications have traverse the Ethernet IP network and that network monitoring can identify any sensor anomalies. Consequently, there is a need to develop control system policies, procedures, and use existing equipment (as well as network) monitoring technologies. However, existing equipment or network monitoring technologies likely will not be sufficient to identify counterfeit devices or address hardware backdoors. This would most likely require new sensor monitoring technology that address the “physics” of the sensors which would be the input to both equipment and network monitoring. This new sensor monitoring technology has been proven in laboratory testing against multiple sensor vendors. In addition, there needs to be an industry recognition that the requirements within standards like ISA 62443 apply to the entire control system, level 0 through to the DMZ. Part of this understanding is that the control system and its network is owned by engineering personnel (operations, engineering support, maintenance) rather than the IT personnel, who should be used in a support role as a service provider.
4). Q: So to keep validating the low-level sensor data real time, we will need to know the algorithm that computes the result (e.g., temperature, pressure, etc.) but manufacturers may not wish to share their proprietary algorithms. Then, what can be done?
Answer: The sensors need to be identified by physics “fingerprinting” (see above). This would identify counterfeits as well as identify any sensor deviations agnostically. That is, it will identify deviations whether from sensor drift, loose screws, calibration issues, hardware problems, or cyber compromises. Once the deviation is identified, there are available tools that should be capable of determining the cause. It is a shame to say in 2020 we still don’t know when to use our diagnostic toolboxes because of the lack of awareness.
5). Q: Could you also have an engineer's SOC rather than an IT/OT SOC.? They would focus on the engineering aspects.
Answer: Without being flippant, that is the control room or control center.
6). Q: How to mitigate supply chain risks?
Answer: This is a very complex question because supply chain risks can range from an entire device to software, to firmware, to microprocessors, as well as integration of multiple instances of these. It requires the cooperation of procurement, physical security, cyber security, test shops/labs, engineering, and maintenance. Sensor monitoring to detect counterfeit or hardware backdoors would be a critical piece of the solution. Asset owners should also require their product and service providers to comply with standards like ISA 62443-2-4 and then to vet them against those requirements. I would be happy to further discuss my thoughts offline.
7). Q: Two questions : Is there any validated OT architecture that may hinder the possibility of backdoor attacks where the device would look for a master to trigger?
Answer: I don’t think so as the backdoor could bypass the OT architecture – the reason for the Presidential Executive Order.
8). Q: I had a question about the levels. Do you think there is an advantage in separating level-0 devices to continuously-monitored devices (PLCs, HMIs) and smart IO Devices (IO Link based devices, Ethernet IP devices/Profinet devices)
Answer: Two years ago, we created a special ISA99 Working Group on Level 0,1 devices – ISA99.04-07 to address Level 0,1 devices. The working group concurred that “legacy“ Level 0,1 devices cannot be secured to current cyber security requirements. Additionally, the Purdue Reference Model was identified as being out-of-date for addressing modern control system device technology for cyber and communication capabilities as there no longer are clear distinctions between levels even for the same device. There is an advantage to segregating sensors based on the zone they are located. Each zone should have its security requirements based on risk and countermeasures that are unique to that zone. For instance, a safety-instrumented system (SIS) involves sensors, logic solver, final elements as well as an engineering workstation. Having a SIS zone makes it easier to accomplish least privilege from both a physical and logical access perspective.
9). Q: Are Controls devices companies taking any action to certify programmable hardware electronics to validate no malicious logic is included on logic or printed circuit hardware?
Answer: I think that is the ultimate intent of ISASecure and commercial test/certification companies. The devices certified to date are controllers and master stations. None of the Level 0,1 devices has completed cyber security certifications.
10). Q: Another questions I had was about a recent change in the industry direction, to put all devices on the IP network now.¬ I bring new machines to our plant, and 100% of our machines have an Ethernet network and a NAT gateway to expose device.
Answer: Unfortunately, that is becoming a common practice especially with Industry4.0 and other digital upgrade initiatives. However, I believe there is a real need to question whether safety devices should be part of the overall integrated plant Ethernet network. Moreover, I think there needs to be a reassessment of the need to connect control system devices directly to the Internet without some form of proxy to shield them from public access.
I received considerable feedback from people who read the last post on the history of the COAST Lab. Several people asked for more history, and a few former students volunteered some memories.
I'll do a few posts with some specific recollections. If others want to send stories to me or enter them in the comments, we may document a little history. Eventually, I'll get around to the formation of CERIAS and some history of that effort.
In the earliest days, we had limited funding to apply to our research infrastructure; my priority for funding was student support. Everyone had an account on CS departmental machines, but we were limited in what we could do -- especially those requiring kernel configuration. Recall that this was in the era of 1992-1997, so neither "cheap" PCs running a Linux clone nor VMs were available. We needed access to workstations and a server or two.
I had contacts at several companies, and Purdue -- having the oldest degree-granting CS department in the world -- was also reasonably well-connected with vendors. I reached out to several of them.
HP stepped up to donate a workstation, but it was underpowered, and we didn't have the money for expansion. As I recall, HP at the time wasn't interested in making a donation beyond what they had already provided. Later, we also got a steep discount on an office laser printer. HP had some very clear divisions internally, so even though several groups wanted to engage, the ones with spending authority weren’t going to help.
I also recall donations of some Intel-based machines (from Intel). Other big vendors of the time -- Sequent, IBM, Pyramid, DEC -- indicated that they weren't concerned with security, so we got nothing from them. (3 of the 4 are now out of business, so go figure.) [Correction: in 1997 we were loaned a Dec ALPHA workstation for about 6 months, but weren't allowed to keep it. It was the primary computation engine for the work that led to the Kerberos 4 flaw paper.]
The company that helped the most was Sun Microsystems. (The late) Emil Sarpa was one of the people at Sun who took particular interest in what we were doing, although there were quite a few others there who interacted with us. (Mark Graff, head of their response team was one I remember, in particular.)
I don't recall if Emil was among our first contacts at Sun, but he quickly became an internal champion for us as their Manager of External Research Relations. He helped arrange some donations of equipment in return for (a) research results, and (b) access to potential hires. (That has long been the standard quid pro quo for collaboration with universities.).
Over time, including time as CERIAS, we received many workstations, a server, a lab of Sun Rays, a SunScreen firewall, and even some Java rings and readers. In return, Sun got quite a few reports of issues they could fix in their systems, and dozens of hires.
With upwards of two dozen machines in the lab we needed hostnames for all the computers. CS used names from the Arthurian legends for their machines. We knew that the CS department at Wisconsin used names of cheeses, one university (Davis?) used names of wine varieties, and there were other themes in use elsewhere. I decided that we would use the names of places from myth, legend, and science fiction/fantasy. Not only were there many candidates, but the idea of us working from places that didn't exist seemed like a good inside joke. (This also related to my long-standing interest in using deception defensively.)
Thus, we started naming machines after non-existent places: yavin, narnia, dorsai, trantor, solaria, barnum, xanadu, atlantis, lilliput, and more. We had a few disagreements in the lab when new machines came in ("I want to have Endor!"), but they all resolved amicably. I bought an atlas of imaginary places to serve as additional source material. We never really lacked for new names. Many of those names are still in use today, although the machines have been replaced many times.
COAST received a server-class machine from Sun in the mid-1990s. It had lots more space and memory than anything we had seen before, so naturally, it was named "brobdingnag." It became our central file server and mail machine. However, it soon became apparent that some of the lab denizens couldn't recall how to spell it, and petitioned for an alias. Thus, an alternate name in the host table came into being: "basm," for "big-assed server machine." A server named "basm" still exists at CERIAS to this day.
We decided to use a different naming scheme for printers and named them after Lands in the Oz mythos. Kansas, Oz, and Ix were the three I remember, but we had more.
A few machine names, in particular, have a story associated with them. One of the Intel machines we received was running Windows, and we named it "hades." (We were not Windows fans at the time.) A few years into COAST -- I don't recall when -- we attracted attention and support of Microsoft, in the form of David Ladd. He was (at that time) involved in academic outreach.
David was visiting us and saw all the Sun machines. He asked if we had anything running Windows. Someone pointed to "hades." He didn't say anything about that, but a few weeks later, we received two new Windows machines, fully configured. They went online as "nifilheim" and "tartarus." On his next visit, David quietly noted the machines. A few weeks later, two more showed up. I think those became "hel" and "duzkah." At his next visit, I observed that we were at a university, and I had access to scholars of history, religion, and sociology. I think we got a few more machines periodically to test us, but they all got named in the same scheme.
That isn't to imply that our relationship with Microsoft was adversarial! To the contrary, it was collaborative. In fall 1996, when Windows Server NT 4 came out, I offered a special-topics penetration testing class. About two dozen people enrolled. Under NDA with Microsoft, we proceeded to poke and prod the OS while also reading some of the classic literature on the topic.
Within two days, the class had discovered that NT 4 failed spectacularly if you exhausted memory, disk space, or file descriptors. By the end of the semester, everyone had found at least 4 significant flaws -- significant meaning "crashed the system" or "gained administrative privileges." We thus reported about 100 security flaws to the Windows support team. At that time, Microsoft was not as concerned about security as they are today, so we were told (eventually) that about 80 of the reports were for "expected but undocumented behavior" that would not be addressed. (Those numbers are not exact as they are based on the best of my recollection, but they are about right on the ratio.) That class provided several grads who went to work for Microsoft, as well as at least two who went to work for national agencies. I have not offered the class since that time as there have always been higher-priority needs for my teaching.
Over the years, many COAST (and eventually, CERIAS) graduates went to work at Microsoft. David --and MS -- remained supportive of our efforts until he moved into a new position well into the CERIAS days.