You can’t miss the influence of computer graphics on filmmaking this summer. Purely computer-animated films like Cars and Over the Hedge come quickly to mind, along with the annual visual effects extravaganzas like Poseidon and Mission: Impossible III. Even those that lack apparent computer-generated imagery were most likely edited and sound-mixed using computers. But few audience members give much thought to how and where the technologies behind these films were developed. In the early days of computer graphics in film (the early 1980s), much of the technology was developed directly by the studios. Notably, Lucasfilm, George Lucas’s privately held production company, created its own research division, developing many of the core technologies still used throughout the industry [1]. That division was later spun off as Pixar Animation Studios, while Lucasfilm retained many of the technologies for the fledgling computer graphics department of its own Industrial Light & Magic. As the industry has evolved, however, academic research has played an ever-larger role in filmmakers’ use of the technology.
I began working full time at ILM in 1995 and was promoted to computer graphics supervisor three years later. My job involved researching technologies that would improve the workflow and efficiency of our computer graphics artists, mainly animators and technical directors working on such films as Galaxy Quest, The Perfect Storm, and Star Wars Episode II. This environment offered a rich variety of engineering problems I found incredibly stimulating. I also found lots of inspiration working day to day with the production artists. Because they routinely focus on a single task for days or weeks on end, their work processes are extremely sensitive to the slightest software or algorithm design decisions, amplifying what might seem to be unimportant details and crystallizing the criteria for improvement. But most production artists are not researchers or programmers, focusing instead on daily results and relying on existing technologies.
As diligent as I might have been, I had a difficult time keeping up with the expanding universe of academic research solely by attending conferences and reading the occasional paper. Production demands limited the amount of time I could spend on any one project, and I found myself wishing I could work on longer-term projects.
In 2002 I began an experiment to help bridge the gap between the immediate demands of commercial production and the longer-term visions of academic research. I split my time in six-month shifts between the Bay Area working for ILM and Seattle pursuing a Ph.D. in computer science at the University of Washington. This unconventional arrangement allowed me to alternately immerse myself in both computer graphics research and computer animation production. Returning to an academic environment was a thrill; sitting next to people with seemingly unrelated research problems often results in surprisingly fruitful revelations about my own work.
For the moment, I’m primarily an academic. In 2004, I left ILM to focus on completing my Ph.D. But over the long run, I aim to maintain relationships with both industry production artists and academic researchers. The experience of living in both worlds has reinforced my belief that academics and industry practitioners in computer graphics and animation need to spend more time in each other’s domains.
But today, few animation and visual effects companies have any formal relationship with their academic counterparts, and technology transfer is haphazard. A typical scenario goes like this: A production engineer in a film studio returns from the annual SIGGRAPH computer graphics conference with a neat research idea. He then steals time between projects to implement the technology from scratch. This window—perhaps a day or two here and there—is rarely enough to do a decent job, but, with luck, he’ll get a chance to prove the technology to his manager and get a budget to work on it further. Otherwise, that’s the end of it. Opportunities are rare for giving direct feedback to academic researchers about the practical issues they face day to day. Successful collaborations require persistence to move beyond the status quo—regardless of which side of the fence participants come from. A true collaboration requires a significant investment of time, as well as a meeting of the minds, not just throwing technology over the fence.
A true collaboration requires a significant investment of time, as well as a meeting of the minds, not just throwing technology over the fence.
In order to encourage a wider discussion about the relationship between academic institutions and production studios, I brought together a number of industry veterans with experience bridging the gap for a panel discussion at last year’s SIGGRAPH. Tony DeRose, head of the Pixar Research Group and a former professor at the University of Washington, listed the criteria needed for successful technology transfer. The first was that technologies are most easily adopted if they fit into an existing, extensible part of the animation production pipeline; for example, a variety of local illumination algorithms can be implemented through interchangeable programmable shaders. Second, graphics technology must be "directable"; that is, artists must be able to modify the results of an algorithm. DeRose said that "fully automatic methods that adhere strictly to physics are unlikely to be adopted." On the flip side, he said, a technology must also be controllable; it should not have so many control parameters that the system’s behavior becomes unpredictable. Moreover, the meaning of each control should be intuitive for nontechnical users. Technologies should be designed, he said, to enhance the user’s artistry, not replace it.
To DeRose’s criteria I would add approximation and scalability. An algorithm that runs slowly can still be useful in production if part of the solution can be approximated; for example, even though rendering an entire scene using global illumination can take hours, some algorithms include quality controls allowing production in minutes of a rough rendering of a portion of a scene. Thus, an artist might plausibly determine how the final rendering will appear when the quality controls are adjusted. Rendering algorithms are often processed independently per frame, making it possible to render an entire scene quicker by harnessing multiple processors. Such attributes are rarely the case for dynamic physical simulations in which changing the resolution of a mesh or grid produces dramatically different simulation behavior.
The barriers to collaboration often boil down to misperceptions. For example, to researchers who want to achieve a broad impact in terms of peer recognition, as well as in the wider public arena, the film industry appears to be a niche market for collaboration. However, this view underestimates the industry’s influence on future technology; the high volume of digital media produced and consumed inside studio facilities makes them ideal incubators for technologies and computing resources that will be widespread beyond filmmaking in all kinds of computer devices. Many technologies pioneered in the film industry (such as shading languages and antialiasing) are now incorporated into video cards and game consoles selling millions of units, bringing unprecedented computing power into the household.
On the industry side, academic researchers are often perceived as pie-in-the-sky thinkers or even closet pirates out to steal proprietary technologies. Perhaps the most daunting obstacle to flexible collaborations is the burden of sorting out intellectual property issues. I can acknowledge that firsthand. In trying to make all parties happy about where intellectual property ends up, I have spent more time talking to lawyers than I ever would have wanted. But in most cases, patience and persistence have led to successful collaborations that bring something positive to all parties. Moreover, each successful collaboration forms an intellectual property model for the next one, so the burden may ease up over time as collaboration becomes more mainstream.
To film studios, collaboration means a partnership to help chart the corporate technology roadmap. Although they rely on new technologies to reduce the cost of doing business, as well as to produce "something nobody’s ever seen before," few studios have the resources to fund novel research. They must therefore view longer-term development as a collective effort, reaching out to academic researchers to help set their long-term technological compass.
For researchers, the principal benefit is as a rich source of real-world problems. Without collaboration, they easily lose sight of the demands of such problems. The fast pace of visual effects and animation production quickly reveals inefficient practices that also represent a gold mine of research directions spanning the gamut not only of computer graphics but of systems, networking, and HCI.
The rising tide of collaborations over the past few years is an encouraging sign. The SIGGRAPH proceedings feature an increasing number of papers with co-authors working in both academic laboratories and in the production studios of the film industry. For example, in 2002, Henrik Wann Jensen of Stanford University (now at the University of California, San Diego) collaborated with PDI’s Juan Buehler to write a paper on a practical method for subsurface scattering. Ron Fedkiw, a professor at Stanford, has an ongoing consulting arrangement with ILM and has co-authored a number of papers on physical simulation with collaborators there.
For any collaboration to succeed, industry and academic researchers must understand some fundamental truths about one another:
- Artistic control is critical in film production;
- Academic researchers live and die by publishing their results;
- Computer animation and visual effects studios are not (yet) accustomed to funding basic research in the same way many larger companies fund theirs; the resources they are willing to share more often involve access to artists and internal developers and exposure to novel and exciting problems;
- Film industry practitioners must recognize the different pace and direction of academic research; for every collaboration that results in a direct benefit to a studio’s bottom line, perhaps two others reveal their value at a much later date; and
- A middle ground is needed for defining ownership of intellectual property rights from collaborative research.
Describing the ideal relationship between researchers in film production and their counterparts in academic laboratories, Andy Hendrickson, head of production technology at DreamWorks Animation, said: "The film industry can push the envelope of necessity, while academia pushes the envelope of possibility."
Join the Discussion (0)
Become a Member or Sign In to Post a Comment