Toward an Image Indistinguishable from Reality

Computer graphics imagery increasingly represents real physical-world scenes, objects, and processes. We see these images not just in movies, television, and advertising, but in applications in which visual accuracy is critical. Engineers and scientists use them in automotive and aircraft design, bioengineering, surgical simulation and medical training, architectural walkthroughs, and aerial reconnaissance. Game developers use them in interactive virtual reality games. Someday, we might even see them as legal evidence in a court of law. Computer graphics imagery even helps deliver seemingly visceral experiences equivalent to interactions with physical things in real-time 3D environments.

Realistic images can include the kind of intricate detail and texture people expect only in real things. Despite this precision, however, graphics practitioners and viewers still ask: How do I know when the physical and perceptual accuracy of the images is guaranteed? And how can the fidelity of the rendering algorithms be improved enough to guarantee that accuracy?

The more I’ve learned about realistic computer graphics preparing this special section, the more I’ve come to view these images as astonishingly persuasive illusions—especially those involving human senses besides vision, such as touch. Still, the deeply knowledgeable and imaginative authors might argue I’m missing a key idea behind the images: That they actually enhance a sense of reality, seemingly more real than physical reality.

But how do graphics practitioners know what an object really looks like before trying to generate a realistic simulation of it? Producing such clear and detailed images forces them to look ever more closely at the textures, shapes, materials, and other properties of the objects they want to render—as well as the functioning of their own eyesight—before delivering that information to their algorithms. Such a deep understanding of the physical world and the psychophysical basis of human perception are needed to create images believable to the observer—ultimately emulating and predicting reality.

Images can even be rendered with such completeness they emerge as self-animating synthetic graphical characters. Terzopolous describes the process of applying artificial life to virtual animals and plants, giving them built-in motivations and life-and-death needs and behavior. His emulated fish, for example, which represent a compromise between realism and computational efficiency, seem to come alive on the screen, swimming away into a vast virtual ocean, evolving into increasingly accomplished forms—apparently independent of the people who modeled and animated them. I was reminded of what Sid Phillips in Pixar’s Toy Story said in a panic about Sheriff Woody, Buzz Lightyear, and their friends: "The toys are alive!" Terzopoulos’s creatures may not be quite self-aware, though given enough time, who’s to say what their limits can or should be.

Images "correct" enough to predict reality are, for Greenberg, the holy grail of computer graphics, emphasizing how the framework for realistic image synthesis developed at Cornell University’s Program of Computer Graphics maintains fidelity between rendered images and their physical counterparts. Related renderings are realistic because they incorporate physically based lighting models, perceptually based rendering procedures, and the psychophysical process of human vision.

Because fidelity is such a critical aspect of rendering realistic images, graphics practitioners have to be able to see, measure, and define the objects they’re assembling on the screen. The kind of input technology needed for this high-fidelity sampling includes a new generation of input devices for collecting information needed to synthesize realistic images, as described by Fitzmaurice et al., of graphics software company Alias | Wavefront. Such technology represents a dramatically new method and user interface for creating computer graphics imagery.

Why stop at mere physical objects—even those for which the illusion is so persuasive they seem to be alive? Human figures remain the most fascinating of visual objects (and species). Badler et al. of the University of Pennsylvania describe how to animate 3D virtual humans in real time. Following their virtual human modeling architecture, these characters perform realistic humanlike movements, actions, and self-motivated decision making for the sake of interactive applications. Promising applications are in e-commerce and how we represent ourselves as we most want to be represented—in the form of avatars—interacting through the World-Wide Web. These virtual humans can be made more human by animating them through natural-language instructions—someday perhaps by way of spoken voice commands.

Extending the interactive experience to the other human senses—like touch—Salisbury and his team at the Massachusetts Institute of Technology have devised haptic output devices that make graphics physically tangible, not merely visible. The apparently illusory 3D imagery of displayed physical objects triggers these interfaces to exert forces directly on users’ hands and fingers, providing an experience as simple as feeling a wall or as complex as simulating the physical sensation of a surgical procedure. The resulting sensations profoundly alter users’ nominal sense of reality.

Moreover, the images described and shown in these articles can involve vast amounts of data painstakingly collected over years. Bryson et al. describe how NASA uses gigabyte data sets to produce complex computer graphics images called scientific visualizations. In this case, they support engineers’ understanding by simulating in 3D how computational fluid dynamics are used for, say, designing F/A-18 aircraft. Looking past the numbers and the code, the authors deliver insight and lessons for anyone trying to make visual and computational sense of large amounts of data derived from real-world physical phenomena.

We asssembled this section for the same reasons: to offer a multifaceted, multisensory view of the increasingly realistic and compelling directions computer graphics is taking us. Still ahead is the increasingly fine line separating real things from their virtual equivalents.

Back to Top