Testing human-auto cyber team resilience uncovering latent failures with a novel, decision-centered approach

The increased use of automation has been necessary to keep up with the growing “cyber-pace” and complexity of threats. However, automation benefits do not come for free; the addition of these automated support agents has added additional coordination and supervisory work to the human operator´s domain, bringing hidden brittleness along with the benefits. This increased human-automation teamwork calls for the need to complement traditional testing approaches and holistically assess the net decision-making effectiveness and decision-making resilience of the Human-Auto cyber team. Decision Centered Testing (DCT) uncovers problems that would otherwise go unnoticed. By uncovering this joint HumanAutomation brittleness, system designers can make necessary system interface and automation design and development improvements to create a more resilient joint cognitive team. Based on Cognitive Systems Engineering (CSE), DCT uses test scenarios built around key decision-making demands of the domain, to place “pressure” on potential areas of weakness between the user and their decision support system (“edges”) to discover latent design flaws prior to being released into the fight. We present the success of DCT in the Control Domain (The Silent Automation & the “Right Sized Hole” Case), a DCT methodology introduction and how it can be used in the cyber domain.