+++ATH0

« Privacy Legislation | Main | OS X and ext2 »

April 12, 2003

State of Graphics

This lecture happened a short while after (read 30 minutes) listening to Ari Schwartz speak, but my writeup has been unfortunately been delayed in posting due to numerous other events going on. Dr. Daniel Aliaga presented a talk on "Capturing and Rendering Real-World Environments" (white papers here and here), where he went into detail on the steps his team and he have taken to accurately rendering a real world environment with a minimal amount manual labor (i.e. taking measurements, etc).

You can read over many of the mathematical details in the white papers used to explain the research so I will keep this down to a minimum on details. The fascinating part of all of this has been the end resulting rendering. Dr. Aliaga worked to impress upon us many of the minor details that were able to be reproduced, like specular highlighting on the wall plaques.

As I understood it conceptually, the project starts out by taking an extremely high resolution data rich sample set on a set plane. Meaning an omnidirectional camera was set on a parallel plane to the floor collecting a series of high resolutions photos (1024x1024) within a densely covered area. If you look at his research page, the image of a floor plan with a series of red scribbles, that are actually dots not lines, represents the camera path taken and points of data collection. The robot was controlled manually through a handheld RC device, but there is hope for future research to allow the robot to be self controlling.

Once this data set has been collected, a method of associating and identifying features is executed. While the team used some obvious fiducials for tracking, they were also able to identify common themes through images via some computer vision algorithms. Each image was taken at about a distance of 3" apart, so a means of stitching the image together was created, called Sea of Images.

Interesting detail pointed out was the resolution capability to be generated for a single image through this stitching process was visibly higher than that of an MPEG2 stream (their initial data compression method). While it was still not as detailed as an original source image, many more details of small text and complex shapes could be identified in the SoI method.

The final demonstration provided a virtual walk through of a series of locations, like the Bell Labs library. Oddly enough in the walk through not one of the images seen was actually ever taken. Each image used on the walk through was dynamically generated from the data collected earlier. While the walkthrough only allowed a single perpendicular view to the floor (no angles up or down) but the detail and clarity was outstanding. This wasn't being run on any special hardware either. A 1.7Ghz Pentium4 processor with a "recent off the shelf video card around $200" was used.

There were still some problems. Occasionally the walk through would burp for a noticeable few frames, and when on a still frame occasional image ghosting artifacts could be found from the stitching.

Before seeing this presentation, I read an article written by Steve Silberman for Wired News on the state of the art graphics being employed by the new Matrix movies. Essentially the author was blown away with the quality of the 3D animation being used in real time, but provided nothing but fluff about it. I was a slight bit skeptical on the level of realism that could be provided, but after seeing this presentation I am now very excited to see the outcome. If Dr. Aliaga can provide essentially photo-realistic quality on a real-time basis, I can't wait to see the level of detail brought in a completely rendered world.

Posted by Dan at April 12, 2003 07:50 AM

Comments

Posted by: Steve Silberman at April 17, 2003 04:18 PM