Technical Perspective: Visualization, Understanding, and Design

Photographs capture the moment; paintings convey perception, impression, and feeling; illustrations tell stories. Computer graphics aims to enrich all these artistic practices through technology. The following paper is a watershed in depiction, creating imagery that gets ideas across. Mitra et al. describe an interactive system that analyzes the operation of mechanical devices and explains them visually to users. Their compelling results showcase an innovative synthesis of newly mature techniques for robust analysis of 3D geometry and for domain-specific information design.

Like many watershed papers in computer graphics, this paper takes its cues from the work of a master artist—in this case, RISD Professor and MacArthur Fellow David Macaulay. While Macaulay may be best known for the engaging and richly informative visual storytelling of his architectural history books Cathedral and Castle, the authors here base their system on his book The Way Things Work. It is a fascinating compendium of explorations of everyday artifacts, like the lock on your front door, illustrated through lucid visual explanations that communicate a deep understanding. Such pictures are the authors' inspiration.

They get their understanding of the world through geometry. They do not simulate physics directly. They use symmetries to infer how components might move; they use correspondences to recognize components that can drive one another. The geometric computations allow approximate matches, by exploiting the robust statistical principle that reliable symmetries and correspondences leave lots of evidence. So a first step searches for candidate matches, and a second step tabulates the results to find consistent patterns. Such techniques have a long history, but recent bridges to computer graphics have had an enormous influence on the practical analysis of 3D shape, as you see here.

Geometry is a powerful cue to the behavior of everyday artifacts. The reciprocating rack and pinion at the end of their paper is a mesmerizing example. But geometry goes only so far. The system must assume its input is a working machine. And since parts like levers and belts do not have a distinctive geometric signature, the user has to label them interactively.

The following paper is a watershed in depiction, creating imagery that gets ideas across.

Making an effective picture takes more than just the right representation of the mechanism. It requires the right design principles. Design principles are domain-specific rules for using specific visual techniques to make the information in a display easy to see and understand. Maneesh Agrawala has built a wide range of influential systems for depiction by codifying such design principles, implementing them as algorithms, and evaluating their effectiveness; his research program is described in his co-authored Communications article "Design Principles for Visual Communication" (Apr. 2011). For explaining mechanical assemblies, the key principles are to focus on showing causal chains of motion, and to do this by selecting key frames and annotating them with diagrammatic arrows. Here as elsewhere, the design principles come from analysis of exemplary hand-made work, from artists' reflection on their practice, from psychological theories of how people understand these visualizations, and from the researchers' own experimentation with the possibilities of technology.

Realizing these design principles involves a judicious choice of visual techniques. Non-photorealism in computer graphics offers diverse ways to stylize appearances and guide the viewer's attention. Examples include modulations of detail and weight in rendering objects, the use of cutting and transparency to depict objects in multiple layers, and even selective choices about which elements to render at all. The use here of simple line drawings, with arrows for annotation, and a constrained set of highlighted parts and exploded views, is a choice that reliably leads to clear and uncluttered imagery. To create accessible imagery with more richly varying rendering techniques, or with visualizations of additional information (forces, for example), it might be necessary to develop much more nuanced design principles. The pictures here, however, are clearly a success.

It is never easy to endow computers with a deep and interesting understanding that they can share with their users. But that does not mean we should regard inference as hopeless or design as magic. As this work shows, general tools and methodologies are making it easier and easier for systems to communicate the understanding they have through clear and compelling visualizations. The results here thus take on particular significance as a benchmark in visual explanation, and a model for future systems.