DIGITAL MOSAIC ENHANCEMENTS & SCENE EXTENSIONS
The main camera on NASA’s Mars Exploration Rovers, Spirit and Opportunity, is the panoramic camera, or Pancam. It sits atop a long mast that can move up or down and rotate through 360 degrees. The Mars rover cameras gather hundreds of individual images over an extended period of time, sometimes even days, which are later “stitched” together into one larger view, or “mosaic.” Over the period of time that it takes to create these images, the lighting changes on Mars, and dust levels change in the atmosphere. The end result is a patchwork of countless levels of data that affect light, shadow, and image contrast. The precise alignment of the image-to-image seams is also extremely critical when the goal is to make the combined
images appear as a single, seamless photograph. One of my assignments on “ROVING MARS” was to digitally enhance all these properties that made up the mosaic images in order to simulate the view that a person would see if all of the images in the mosaic were taken on the same day, and at the very same moment while still composing an aesthetically pleasing image for the giant IMAX® screen. Up until now, the rover images had mostly been seen by the folks at NASA’s JPL and to those who frequented their MER mission website, and certainly no one had ever seen them up on an 8-story high movie screen before.
When the high-resolution, panoramic images (up to 24,000 pixels wide) from NASA’s Jet Propulsion Laboratory arrived they had already been, to a certain extent, processed and loosely stitched together. NASA’s Pancam team, headed up by Dr. Jim Bell, had executed the elaborate colorization and photographic stitching, but upon closer inspection of these images, I felt they weren’t ready for the visual scrutiny they’d be subjected to up on the giant IMAX® screen. Since NASA’s job is extremely science-based there was no reason or consideration taken for maintaining any visual acuity for an IMAX® sized presentation. To give people a sense of being on Mars scientists have to combine views through telescopes, data from past Mars missions, and new information from the current mission to create a “true color” representation for each photo.
While the Pancam team took extreme care in attempting to normalize the imagery, and while being as accurate as possible with their image data manipulation, I was challenged in another arena by putting the finishing touches on each of the mosaics. One of my most formidable challenges on this project was to digitally extend the various images so they not only filled an IMAX® frame (1.33 aspect ratio), but to also compose and produce an aesthetically pleasing image. There are many considerations when composing for the IMAX® flat & dome screens, one of which is to compose for the “sweet-spot” for both venues.
Many of the panoramic landscapes contained extremely low horizons with virtually no sky at all. Once recomposed for the IMAX® screen, the image required a very prominent sky that made up at least 75% of the entire frame. The skies were void of any detail since there are no clouds on Mars, but for the most part they were also very vibrant. Comparable to what is frequently required on most live-action exterior day shots where the sky portion of a landscape makes up almost 30% of the upper frame, I applied a neutral density gradient to tone down and diminish any potential light refraction when projected onto an IMAX® silver screen, or on the IMAX® dome.
The majority of the landscape mosaics contained black voids in areas of the frame, either as a result of original composition, or mosaic seaming. In order for these images to work in an IMAX®-sized aspect ratio, these areas would require additional picture information to be seamlessly integrated into the original image. By digitally manipulating existing components of the “original” picture data, I was able to paint in the black voids to produce an extended, detailed landscape. I matched the perspectives, shadow angles, colors, and textures careful not to replicate exact picture information. This was extremely critical to the success of each enhancement.
Correcting the displaced seams on each of the mosaics, introducing a gradient in the sky, as well as balancing the colors and densities were the prescribed finishing touches applied to each of the labor-intensive scene extensions. All of this was achieved with the understanding that animated camera moves were to be added to virtually all of the still images, therefore taking careful consideration to the extended composition of each shot as well.
Of the nine cameras that make up each of the rover’s eyes, seven of them transmitted either black & white, or monochromatic images back to Earth. In addition to the enhancements and scene extensions, many of the black & white images required colorization. Once an overall color tone had been established for the look of the Martian landscape, the B/W still photo images needed to take on similar characteristics with specific consideration to some of the narrative content. For example, there is a series of images that represents a location where the rover’s microscopic imaging camera picks up detailed surfaces of a rock that contained tiny spheres that resembled blueberries in a muffin. These professed “blueberries” are mentioned in the narration, and are visually depicted as having a bluish color value to them in direct contrast with the brownish-red tones of the rocks and surrounding Martian landscape. Although there are clearly no distinct blueberry colors on the planet Mars, the Director took artistic license in expressing the visual attributes of this otherworldly surface.
I always achieve a great sense of satisfaction in revitalizing old and distraught photographic images. The hours spent staring at ones likeness, or even images of a distant location as far away as another planet, as in the case of the “ROVING MARS” project, provided me with a sense of reverence and balance. During the many hours I spent looking at Spirit and Opportunity’s images, I couldn’t help but wonder if one day we may get the opportunity to actually see these images, first hand, with our very own eyes, on Mars.
