Behind the scenes of ‘Hubble 3D’

11.09.10 -

by Barbara Jewett

"Imagine the dark of deep space, interspersed with blazing and multi-hued stars. In the distance we see the brilliant colors of the nebula. I expect us to move toward the photo and see it in stunning detail, but we keep going, far beyond anything I believed possible. We get closer and closer. Stars between Earth and Orion slide past just as they would if warp speed was a reality. ...We are able to swoop and glide as we dive down into the glowing gases where stars and planets, whole solar systems, are forming before our eyes. ...When the sequence ends, it takes a moment to remember to breathe."

That's Michelle Evans, writing for Space Daily after the premiere of the movie Hubble 3D in March. She wasn't alone in her response. Since the movie debut viewers continue to say "wow" to the scenes created by NCSA's Advanced Visualization Laboratory (AVL) with the same awe as if they were watching fireworks on the Fourth of July. But just like with a fireworks extravaganza, behind the scenes of Hubble 3D were months of intense work designing and preparing just the right mix of elements, and choreographing the spectacular display.

Narrated by Leonardo DiCaprio, this documentary film takes viewers through distant galaxies as it tells the story of the repair and upgrade of the Hubble telescope. While most of the film is shot with IMAX 3D cameras, including live-action footage from the space shuttle, nearly a quarter of the movie's run time is devoted to AVL's two dramatic voyages. Comprising nearly 10 minutes of the 43-minute film, the sequences use real Hubble, astronomical, and computational data in visualizations that make audiences feel they are on a space journey. AVL's sequences are extraordinarily luminous and richly detailed, the team says, because of their extremely highresolution: 5,616 x 4,096 pixels per frame, compared to 1,920 x 1,080 pixels for high-definition (HD) TV.

The first scene AVL created flies to the Orion Nebula and through its "star nursery." The second leaves the Milky Way galaxy and flies to the Virgo Cluster, through the Hubble Ultra Deep Field of galaxies embedded within a supercomputer numerical model of the cosmic web.

AVL has worked with virtual reality, 3D rendering, and display technology for many years and has a long history of producing astronomy and astrophysics visualizations for digital planetariums, museums, and HD broadcast television, in addition to a prior IMAX documentary that was nominated for an Oscar. Hubble 3D was one of their most intensive efforts, with over 2.4 terabytes of final composited images that required six months of production and the equivalent of 28,000 PC machine hours to produce.

"Partnering with IMAX and Baltimore's Space Telescope Science Institute (STScI) on the production of the astrophysical scientific visualizations for Hubble 3D was a wonderful opportunity to share ultra-high-resolution renderings of the cosmos with the public in the immersive IMAX 3D format," says Donna Cox, director of AVL, who acted as visualization producer/designer for NCSA's contributions. Other AVL staffers involved in the production were Robert Patterson, Stuart Levy, Alex Betts, AJ Christensen, and Matthew Hall.

The IMAX team—Toni Myers (producer/director), Judy Carroll (associate producer), and Hugh Murray (stereography consultant)—visited NCSA on several occasions, along with Frank Summers from STScI, to develop the concept and treatment for NCSA's contribution to the film. The scenes were based on a host of scientific data, including galactic models and Hubble image data provided by STScI, animation elements provided by the Spitzer Science Center, and simulation data provided by Princeton University's Department of Astrophysical Science.

AVL's state-of-the-art ultra-high-resolution 3D display allowed them to interactively design the scenes and get a sense of what they would look like when transferred to the IMAX screen. Computer processing was done on AVL's visualization cluster and NCSA's Abe supercomputer.

The technology

Working with Myers and the team, Patterson created two seamless camera moves that take the audience across millions of light-years in a matter of minutes. This was achieved with AVL's Virtual Director™ software, which allowed them to work interactively in an immersive 3D "virtual cosmic set" based on the scientific models and data.

Levy, creator of Partiview, used his application to interactively preview the galaxy models, and Alex Betts used his custom interactive Maya scene file renderer for previewing the Orion nebula. Patterson and Myers were then able to make immediate, finely tuned changes to the audience's journey and preview these scenes in 3D. Hall, a primary developer of Virtual Director™, added new navigation and 3D capabilities to the software, which enabled the multi-scale flight paths to be created in real time.

Using a spaceball controller, Patterson could navigate, record, and edit flight paths through the cosmic content. Once the camera paths were designed, the AVL team could start the elaborate rendering process.

Journey to Orion Nebula

To create the voyage to the Orion Nebula, AVL collaborated with the STScI team, which provided the Orion base geometric model and Hubble image textures along with star nursery data. AVL used this data as a starting point and developed methods for creating the elaborate realistic renderings of the nebula.

Betts developed new volume modeling capability by adding a "voxelizer" to AVL's Maya data visualization plug-in. The voxelizer uses the Orion polygonal model and textures as source input along with a host of other parameters to create a three-dimensional volume model that, when rendered, creates the realistic nebula imagery. Betts and Patterson worked together to create layers of detailed volume-rendered elements for the scene.

Levy and Betts worked on a method of rendering stars that could suggest the millionfold range of star brightness through the Orion scene. Stars are shown with glare artifacts such as the human eye might see when looking at brilliant points of light. The brightest stars have fine radial lines, as if from light scattered by fibers within the eye's lens, and diffraction-related speckle patterns.

Christensen and Betts created the rendering treatments of Orion's veil, proplyds, and bowshocks based on the STScI model. Christensen created numerous veil elements that the camera passes through and refined the look of the proplyds and their internal disks and jets to match the Hubble imagery.

At the end of the Orion shot, viewers see a close-up of an elaborate proto-planetary disk, which was rendered by Robert Hurt of the Spitzer Space Telescope Group and composited at NCSA. Patterson composited the 52 layers of imagery rendered and layered together over the course of the Orion shot for both the left and right eye. Building up the scene with layers provided the ability to render specific elements using different techniques and allowed for layer adjustments and proper depth occlusion. The scene layers included the Milky Way and galactic background, Hipparcos star catalog, Horsehead and Flame nebula, Trapezium stars, proplyds, bowshocks, and several layers of the Orion nebula and its wispy veil.

"The scenes we created for Hubble 3D leveraged our experience in astronomical visualization along with a host of new advancements to achieve the final ultra-high-resolution renderings," says Patterson.

After viewing the movie, Daniel Terdiman, blogging on CNET, wrote about the Orion nebula scene, "This moment alone makes seeing Hubble 3D worth the price of admission."

Voyage to Virgo

To create the voyage from the Milky Way to the Virgo cluster of galaxies, AVL used a model of the Milky Way they'd previously created and added new 3D particle models for the Andromeda and M87 galaxies. These galaxy models were properly positioned in a larger galaxy catalog, obtained from Brent Tully at the University of Hawaii, that contains approximately 30,000 galaxy positions and types. Levy worked on the rendering of this larger collection of galaxies, using imagery from Hubble and from ground-based telescopes.

After the voyage from the Milky Way to the Virgo cluster, the scene transitions to the Hubble Ultra Deep Field (HUDF) collection of galaxies, prepared by Summers and his team at STScI. Christensen, Levy, and Cox created the camera move for the pullback from the HUDF and choreographed a seamless transition to a visualization of a supercomputer simulation of the large-scale structure of the universe: the cosmic web, developed by Jeremiah Ostriker and Paul Bode of Princeton University. The visualization of the cosmic web provides the context for a dramatic ending to the film.

The premiere

Throughout the scene-creation process, the AVL team shared test animations with the IMAX team and STScI for review, and IMAX's Murray provided important feedback on the 3D stereo treatment.

By the project's end, the AVL team had created over 2.4 terabytes of final composited images at 5,616 x 4,096 pixel resolution; they were shipped to David Keighley Productions in Los Angeles for IMAX film recording.

They attended a premiere in Washington, D.C., in March, that included the NASA astronauts featured in the film, the production team, and key people from politics and science. A University of Illinois kick-off event at Chicago's Navy Pier in April not only allowed guests to see presentations by Vice Chancellor for Institutional Advancement Jim Schroeder, Frank Summers, and Donna Cox, but the AVL team had a chance to tell about making their scenes.

What's next

"This work was a challenging computational and cinematic feat. One that stretched our intellectual and technical skills as well as our creative problem-solving. The satisfaction and sense of accomplishment when it all came together was rewarding and a tremendous payoff. We achieved some truly remarkable digital scenes for the Hubble 3D IMAX film," says Cox.

But there has been little time to rest on their laurels. In the months since Hubble 3D hit IMAX theaters the AVL team created a new scene for a digital full dome planetarium show, "Life: A Cosmic Story," at the California Academy of Sciences in San Francisco that premieres Nov. 6. At the same time, AVL is immersed in collaborating with scientists and producers for a digital full dome project for the Denver Museum of Nature and Science. And AVL is in the initial phases of planning and creating scenes for a new show for the Adler Planetarium in Chicago. In the midst of these large-scale outreach projects, AVL enthusiastically provides their expertise and advice to organizations in their own community and on the University of Illinois campus.

Just a few weeks ago, Hubble 3D won three top awards at this year's Giant Screen Cinema Association: Best Picture, Best Cinematography, and Best Life-Long Learning Film. The film is currently playing in 64 theaters around the world, with another 120 still to launch. It has been seen by nearly three million viewers and grossed over $16.3 million. The narration has been translated into Czech, French, German, Latin American Spanish, Mandarin, Polish, Russian, Ukrainian, Arabic, Taiwanese Mandarin, Danish, Swedish, and Dutch.

"It doesn't get any better than this," says Cox.

How'd they do that?

NCSA's Advanced Visualization Laboratory used their dedicated visualization cluster and the center's Abe supercomputer to develop and render the two scenes for the Hubble 3D project. The AVL cluster and the Abe supercomputer both use 64-bit Linux machines, with each machine having eight cores and 16 gigabytes of memory, comparable to a high-end PC or Mac such as a graphics designer might use. Given this environment, how much computation did the visualization project require?

The project required 28,000 PC machine-hours total, which is equal to one fast personal computer running for three years around the clock or 40 fast personal computers working for four weeks around the clock.

Computing just the final version of the two scenes, not counting pre-processing data and tests: 6,500 PC machine-hours (5,000 PC machine-hours for the voyage to the Orion Nebula, 1,500 PC machine-hours for the journey from the Milky Way to the cosmic web) is equal to one fast personal computer running nine months around the clock or 40 fast personal computers running for one week around the clock.

Facts on final image data
  • The two scenes of astrophysical visualizations created by the team at NCSA make up more than 20 percent of Hubble 3D (9 min. 18 sec. total)
  • Journey to Orion Nebula: 6,417 frames, 4 min. 27 sec.
  • Flight from Milky Way to cosmic web: 6,987 frames, 4 min. 51 sec.
  • Total 13,404 stereo image pairs at 5,616 x 4,096 resolution
  • 6.28 terabytes of high dynamic range (HDR) rendered image sequence and layers (135,030 stereo pairs), assembled into 2.24 terabytes of final images
  • 64 stereo computer graphics layers were composited for the two scenes (12 for Milky Way to cosmic web and 52 for journey to the Orion Nebula)