Applying Gestural Interfaces to Command-and-Control

Last fall, I was fortunate enough to be tasked with a fascinating challenge by the US Army's Command-and-Control Directorate (C2D). Along with Martina Balestra, a co-worker at MITRE and fellow Human Factors Engineer, I had the opportunity to explore the possible application of gesture-based interface technologies to command-and-control (C2) workflows and environments. The work that we produced, entitled "Applying Gestural Interfaces to Command-and-Control (C2)" was subsequently submitted and accepted to be presented at HCI International 2011 in Orlando, Florida this July. Since public release was required for this work to be presented, I now have the ability to share it. 

The following is a short abstract of the paper in addition to some concept sketches developed as part of the project. Please contact me if you're interested in hearing more about our process! I'm proud of this work as it was one of the most rewarding challenges of my professional career. 


Abstract (Approved for Public Release: 10-3988. Distribution Unlimited)

This whitepaper examines the applicability of gesture-based user interfaces in notional Command-and-Control (C2) environments of the United States Army. It was authored by a team of Human Factors Engineers at The MITRE Corporation, a not-for-profit research and development organization funded by the United States Government. Since MITRE resides in a not-for-profit advisory position to their federal sponsors, the research team was able to take an unbiased perspective driven solely by identified issues, the search for improved workflows, and practical opportunities for technology development. The goal of the effort was to inform the US Army community so that it can make responsible, needs-driven decisions regarding gestural interface technologies, and avoid the potential pitfalls that may arise from technology-centered or profit-driven decisions.

The problems focused upon by this research primarily revolved around the collaborative human workflows that occur within Command-and-Control environments. Specifically, the effort targeted US Army-based C2 environments, such as a notional fixed command center, a mobile command center, and the environment of the dismounted soldier in the battlefield. The primary issue is that the currently-implemented technologies, while independently sufficient, present constraints when distributed personnel are collaborating across them. The research team addressed this cross-platform issue by adhering to a Systems Engineering framework that required a holistic approach to the “system” of distributed C2 personnel and their technologies. The goal for the final output was to demonstrate how these technologies may come together as a system to support a more efficient, dynamic, and effective operational workflow than today’s reality.

After carefully examining the field of current and emerging gestural interface technologies, and mapping them against available HCI-related research findings, the team concluded that US Army personnel may indeed benefit from effectively and appropriately implemented technologies from this domain. At a high level, gestural technologies offer C2 personnel an ability to conduct more efficient and collaborative workflows across distributed environments. The exact details of these workflows, including the key users, actions, and technology paradigms, are outlined in the content of the whitepaper. In an effort to be as prescriptive as possible, the research team decided that it would be valuable to include a sizable section within the whitepaper dedicated to instructing the user on how to implement gestural technologies for C2 application. In this section, they outline the key design patterns to selecting proper solutions and developing effective interaction design frameworks. The nature of this instructional portion ranges from high-level design principles and best practices down to detailed visual demonstrations of recommended gestures. 

(Approved for Public Release: 10-3988. Distribution Unlimited)