Are students prepared for real-world cyber curveballs?

With a projected “skills gap” numbering in the millions for open cyber headcount, educating a diverse workforce is critical to corporate and national cyber defense moving forward. However, are today’s students getting the preparation they need to do the cybersecurity work of tomorrow?

To help educators prepare meaningful curricula, the National Institute of Standards and Technology (NIST) has developed the National Initiative for Cybersecurity Education (NICE) Cybersecurity Workforce Framework. The U.S. Department of Energy (DOE) is also doing its part to help educate our future cybersecurity workforce through initiatives like the CyberForce Competition,™ designed to support hands-on cyber education for college students and professionals. The CyberForce Competition™ emulates real-world, critical infrastructure scenarios, including “cyber-physical infrastructure and lifelike anomalies and constraints.”

As anyone who’s worked in cybersecurity knows, a big part of operational reality are the unexpected curveballs ranging from an attacker’s pivot while escalating privileges through a corporate domain to a request from the CEO to provide talking points for an upcoming news interview regarding a recent breach. In many “capture the flag” and “cyber-range exercises,” these unexpected anomalies are referred to as “injects,” the curveballs of the training world.

For the CyberForce Competition™ anomalies are mapped across the seven NICE Framework Workforce Categories illustrated below:

NICE Framework Workforce categories, NIST SP 800-181.

Students were assessed based on how many and what types of anomalies they responded to and how effective/successful their responses were.

Tasks where students excelled

• Threat tactic identification—Students excelled in identifying threat tactics and corresponding methodologies. This was shown through an anomaly that required students to parse through and analyze a log file to identify aspects of various identifiers of insider threat; for example, too many sign-ins at one time, odd sign-in times, or sign-ins from non-standard locations.
• Log file analysis and review—One task requires students to identify non-standard browsing behavior of agents behind a firewall. To accomplish this task, students had to write code to parse and analyze the log files of a fictitious company’s intranet web servers. Statistical evidence from the event indicates that students are comfortable writing code to parse log file data and performing data analysis.
• Insider threat investigations—Students seemed to gravitate towards the anomalies and tasks connected to insider threat identification that maps to the Security Provision pillar. Using log analysis techniques described above, students were able to determine at a high rate of success individuals with higher than average sign-in failure rates and those with anomalous successful logins, such as from many different devices or locations.
• Network forensics—The data indicated that overall the students had success with the network packet capture (PCAP) forensics via analysis of network traffic full packet capture streams. They also had a firm grasp on related tasks, including file system forensic analysis and data carving techniques.
• Trivia—Students were not only comfortable with writing code and parsing data, but also showed they have solid comprehension and intelligence related to cybersecurity history and trivia. Success in this category ranked in the higher percentile of the overall competition.

Pillar areas for improvement

• Collect and Operate—This pillar “provides specialized denial and deception operations and collection of cybersecurity information that may be used to develop intelligence.” Statistical analysis gathered during the competition indicated that students had hesitancies towards the activities in this pillar, including for some tasks that they were successful with in other exercises. For example, some fairly simple tasks, such as analyzing logs for specific numbers of entries and records on a certain date, had a zero percent completion rate. Reasons for non-completion could be technical inability on the part of the students but could also have been due to a poorly written anomaly/task or even an issue with sign-ins to certain lab equipment.
• Investigate—Based on the data, the Investigate pillar posed some challenges for the students. Students had a zero percent success rate on image analysis and an almost zero percent success rate on malware analysis. In addition, students had a zero percent success rate in this pillar for finding and identifying a bad file in the system.

Key takeaways

Frameworks like NIST NICE and competitions like the DOE CyberForce Competition™ are helping to train up the next generation of cybersecurity defenders. Analysis from the most recent CyberForce Competition™ indicates that students are comfortable with tasks in the “Protect and Defend” pillar and are proficient in many critical tasks, including network forensics and log analysis. The data points to areas for improvement especially in the “Collect and Operate” and “Investigate” pillars, and for additional focus on forensic skills and policy knowledge.

Bookmark the Security blog to keep up with our expert coverage on security matters. Also, follow us at @MSFTSecurity for the latest news and updates on cybersecurity.

The CyberForce work was partially supported by the U.S. Department of Energy Office of Science under contract DE-AC02-06CH11357.