Few movie scenes have had such an effect on display-technology research and development as the droid R2D2 projecting a three-dimensional (3D) image of Princess Leia pleading for help in 1977’s block-buster film Star Wars. Numerous engineers have wondered just how they might achieve that effect, of an image you can see from any angle, in real life. Even The Walt Disney Company, which bought Lucasfilm and the distribution rights for the movie franchise in 2012, is among those with engineers working on the idea.
Two years ago, Daniel Joseph and colleagues in entertainment giant Disney’s Burbank, CA-based research and development operation filed for a patent on a projector intended to display floating 3D images. The U.S. patent points to an anticipated implementation of having the 3D image seem to be standing on an illuminated pedestal, similar to the game table on the Millennium Falcon that appears in a scene later in Star Wars.
The Disney system suffers from a problem that is shared with similar systems: the image is formed from an array of light sources fed through beam splitters and mirrors some distance behind the pedestal, which limits the viewing angle to those looking toward the projection optics, and so cannot emulate the movies.
Daniel Smalley, an assistant professor of electrical and computer engineering at Brigham Young University, says, “Like many in the holography field, I felt that holograms would provide the 3D images of the future, but the annoying issue is you have to be looking in the direction of the screen that generates them. It’s counter to what you expect 3D displays to do in the future.”
Builders of volumetric displays that can be viewed from any angle face their own challenge. “Fundamentally, you have the problem that photons will just keep traveling until they bounce off something,” says V. Michael Bove, principal research scientist and head of the object-based media group at the Massachusetts Institute of Technology.
Systems such as the VX1 built by Australian company Voxon Photonics use a fast-moving sheet to provide a reflective surface for photons. At a high-enough speed, the sheet will seem to disappear, but bright lights bounced off it will persist to the viewer; the result is the illusion of a slightly translucent 3D object floating in space. Bove says the need to move the sheet at high speed makes this an intrinsically noisy option, and one likely to suffer from mechanical wear.
Another option is to disperse particles into the air and illuminate them. A team led by John Howell, a professor of physics and optics based at the University of Rochester, used cesium vapor to create the voxels in their experimental volumetric display; the cesium atoms glow where the light from two steerable lasers cross. Yet in these displays, moving parts and poisonous particles need to be encapsulated in a transparent dome or sphere.
All volumetric displays to date share the same problem, Smalley says. “You don’t have the self-occlusion to make objects that look realistic.”
“What’s of increased interest is not have a display in the table but to interact with it in a meaningful way. Volumetric displays do have this talking-head-in-a-jar character that works against that. You have the sense that this imagery is bottled up,” Bove says.
Smalley also sees interaction as key, citing another Disney movie franchise, Iron Man, as additional inspiration for his move away from holographic technologies. In the first installment of the movie series, protagonist Tony Stark uses a 3D projector not just to visualize the elements of his powered suit, but also to create a virtual gauntlet around his hand.
Smalley’s team overcame the need to encapsulate their display by trapping and moving a single dust-sized particle. The prototype uses an ultraviolet laser taken from a Blu-ray player to capture and move the piece of dust. A visible-light source tracks and illuminates it. Physicists have yet to develop a theory that fully explains the process of such photophoretic trapping, but it appears to rely on local heating from being struck by photons. Gas molecules hitting the hotter surface acquire more kinetic energy as they bounce off, pushing the particle away.
Says Smalley, “On average it doesn’t work very well at all, but in the [statistical] tails you see incredible behavior. The particle just stays there. You can even blow on it gently. We had one particle trapped in there for 15 hours. It could have stayed for longer: we had to switch the machine off.”
The particle’s composition seems to be crucial. Smalley’s team settled on black liquor—a by-product of the paper-making process—after trying numerous candidates. “I do not believe we can say this is definitively the best material. It seems unlikely that it is,” he says.
It is possible to produce freestanding volumetric images without injecting particles into the air. More than a decade ago, Hidei Kimura, founder and CEO of Japanese company Burton Inc., and Taro Uchiyama of Keio University found that when focused on specific points, microsecond bursts of high-intensity infrared light could cause air molecules to become glowing plasma. Kimura envisaged the technology being used to create levitating signs above head height for use in emergencies; the bursts would be intense enough to burn the hand of a user foolish enough to try to touch the glowing voxels.
Much shorter pulses could yield a safer system. Yoichi Ochiai of the University of Tsukuba and Kota Kumagai of the University of Utsunomiya in Japan showed at the ACM SIGGRAPH conference in 2015 the results of a prototype based on lasers that fire bursts no more than 100 femtoseconds long. According to Ochiai, users would simply get a tingling sensation from touching the plasma voxels, though users would need to be careful to not let their eyes get too close to the images, as retinal damage is a distinct possibility. Robert Stone, professor of interactive multimedia systems at the University of Birmingham in the U.K., says he has concerns over the eye forming strong afterimages because of the brightness of the plasma.
The plasma projector has the advantage of being far more resistant to disturbance by moving hands than the particle-based option. However, all volumetric displays to date have a common problem, Smalley says: “It is like taking a bunch of fireflies and organizing them into patterns. Everything looks like a ghost. You don’t have the self-occlusion to make objects that look realistic.
“We want to be able to take a point and have it shine light in only one direction. That would mean it begins to look solid.”
The lack of self-occlusion in the optical-trap display is, for the moment, a secondary issue. It is difficult to move the single particle that flies around the Brigham Young display any faster than is possible today; that limits its coverage to a volume the size of a ping-pong ball, and the results demonstrated so far are based on long-exposure images that took up to a minute to generate.
“The general public has for 40 years been seeing cinematic depictions of physically impossible things, and when they do see what’s possible, they’re disappointed.”
Says Barry Blundell, senior lecturer in computing at the University of Derby in the U.K. and a researcher into volumetric displays since the late 1980s, “With the optical-trap display, I would have to see images generated a lot faster. The only way to do that is parallelism; you’ve got to have more lasers surrounding the display, and more particles. The problem could be that you need to have so much physical apparatus that you lose the viewing freedom.”
Smalley claims the technology exists to drive and illuminate a collection of particles in the shape of the spatial light modulator, the same kind of device as that used to research holographic displays and optical computers. Bove argues the laser and light-modulator components needed for scaled-up displays are now relatively cheap.
Still, expectations may be set too high.
“The general public has for 40 years been seeing cinematic depictions of physically impossible things, and when they do see what’s possible, they are disappointed,” says Bove.
Smalley concedes, “At this stage, you don’t have to be an expert to realize that this isn’t the Princess Leia display you are looking for. But, if given the opportunity to be developed further, I don’t think you would be disappointed.”
Researchers may be trying too hard to make fact out of fiction. “What some of the people working on volumetrics haven’t realized is that the key elements are complex movement and dynamics, not super-high resolution,” Blundell argues.
Smalley envisages applications where the user needs to inspect the shape closely and move around it. The ability to produce mid-air streamers in fluid-dynamics simulations and models of organs to help with planning medical operations seem good examples. “A lot of 3D technologies can’t give you a strong spatial sense when you get up close. With ours, you can,” he says.
Bove says by looking closely at requirements for target applications and working with user-interface designers, the developers of volumetric displays can move from experiment to market more easily. “Can it be behind a transparent barrier? Is it important that it be viewable from any angle or is 90 degrees OK? Is it acceptable for it to have moving parts?” he suggests as questions to be asked.
Developing volumetric technologies for specific applications may lead to the problem of no individual market being large enough to support research and development, but such displays look more technologically feasible, Bove says. “The problem with the Leia display is that it needs all of the boxes to be ticked.”
Further Reading
Smalley, D.E. et al
A Photophoretic-Trap Volumetric Display, Nature, 553, pp486–490 (25 January 2018), doi:10.1038/nature25176
Ochiai, Y., Kumagai, K., Hoshi, T., Rekimoto, J., Hasegawa, S., and Hayasaki, Y.
Fairy Lights in Femtoseconds: Aerial and Volumetric Graphics Rendered by Focused Femtosecond Laser Combined with Computational Holographic Fields, ACM Transactions on Graphics, Volume 35, Issue 2, (May 2016), doi:10.1145/2850414
Blundell, B.
On the Uncertain Future of the Volumetric 3D Display Paradigm, 3D Research, 8 (2) p11, doi:10.1007/s13319-017-0122-2
Joseph, D.M., Smoot, L.S., Smithwick, Q.Y., and Ilardi, M.J.
Retroreflector Display System for Generating Floating Image Effects, U.S. Patent Application 2018/0024373 A1 (25 January 2018)
Join the Discussion (0)
Become a Member or Sign In to Post a Comment