Sign In

Communications of the ACM

Contributed articles

Sketch-Thru-Plan: A Multimodal Interface For Command and Control

Sketch-Thur Plan, illustration

Credit: Justin Metz

In 2000, Oviatt and Cohen25 predicted multimodal user interfaces would "supplement, and eventually replace, the standard GUIs of today's computers for many applications," focusing on mobile interfaces with alternative modes of input, including speech, touch, and handwriting, as well as map-based interfaces designed to process and fuse multiple simultaneous modes. In the intervening years, basic multimodal interfaces employing alternative input modalities have indeed become the dominant interface for mobile devices. Here, we describe an advanced fusion-based multimodal map system called Sketch-Thru-Plan, or STP, developed from 2009 to 2011 under the DARPA Deep Green program, enabling rapid creation of operational plans during command and control (C2) for military ground operations. As background, we describe the challenges posed by ground operations for C2 systems and their user interfaces. We discuss how C2 GUIs have led to inefficient operation and high training costs. And to address them, we cover STP's multimodal interface and evaluations. Finally, we discuss deployment of the system by the U.S. Army and U.S. Marine Corps. This case study involves the user-centered design-and-development process required for promising basic research to scale reliably and be incorporated into mission-critical products in large organizations.

Back to Top

Command and Control

Command-and-control software must meet the needs of the commander and many types of staff, ranging from higher-echelon commanders (such as of an Army division or brigade) and their own dedicated staff to relatively inexperienced commanders of smaller units.1 Across this range, there is great need for a planning tool that is easy to learn and use for both actual and simulated operations while being functional in field and mobile settings with varying digital infrastructure and computing devices. No military C2 system currently meets all these requirements, due in part to GUI limitations.


No entries found

Log in to Read the Full Article

Sign In

Sign in using your ACM Web Account username and password to access premium content if you are an ACM member, Communications subscriber or Digital Library subscriber.

Need Access?

Please select one of the options below for access to premium content and features.

Create a Web Account

If you are already an ACM member, Communications subscriber, or Digital Library subscriber, please set up a web account to access premium content on this site.

Join the ACM

Become a member to take full advantage of ACM's outstanding computing information resources, networking opportunities, and other benefits.

Subscribe to Communications of the ACM Magazine

Get full access to 50+ years of CACM content and receive the print version of the magazine monthly.

Purchase the Article

Non-members can purchase this article or a copy of the magazine in which it appears.
Sign In for Full Access
» Forgot Password? » Create an ACM Web Account