Windows Live Photo Gallery shipped this week.  You can download it for free from http://get.live.com  The panoramic stitching component of the product was developed by the Interactive Visual Media group in Microsoft Research.  This isn’t the first time that Microsoft has shipped image stitching.  So, some of the core technology is the same as what we’ve previously delivered, but, we have added some powerful new capabilities to the latest release.  This stitcher has a very simple user interface, multi-select images and choose an output file name.  It works very reliably if the photos are well shot (sufficient overlap between images and minor exposure differences).  This product should make image stitching accessible to a very large audience of users.

A lot of people have asked what is the relationship between this and Photosynth.  Both technologies were developed by the Interactive Visual Media Group.  Several other blogs have already covered the details of how these technologies relate – see here and here.

Fully Automatic Image Matching

To use image stitching in WLPG you simply select a group of photos that have some overlap, then use the right mouse button to get the menu pictured to the right, and select "Create panoramic photo…".  In this example I selected 13 input images and generated a 360 panorama, which you can view above as a flat jpg, or here as an HD View. 

Over the past few years, we have refined the algorithm that performs automatic image matching.  A lot of the recent work in automatically matching images was inspired by the Recognising Panoramas paper that Matthew Brown, a grad student at University of British Columbia, published in 2003.  We were certainly inspired by his work and subsequently hired Matthew.  His efforts combined with some technology that our group had previously developed (Szeliski & Shum, Siggraph ’97) allowed us to build the automatic photo stitching feature that appeared in 2004, in the Digital Image Suite family of products.  Subsequently, several other automatic stitching applications have also used Matthew Brown’s work as the basis for their image matching, these include Autopano Pro, Autostitch, and Autopano-SIFT.

Intelligent Seam Selection

Matching images is only part of the problem in creating a good looking image stitch.  Another important component is how the images are composed together.  In WLPG we added intelligent seam selection technology.  For this problem we have built on another research effort, Interactive Digital Photomontage, that was jointly developed by Microsoft Research and University of Washington.  This paper outlines a general framework for doing many different kinds of image composition.  In our application, panoramic stitching, this technology provides two things.  First, it finds the ‘most invisible’ place to cut between two images.  For example if a moving car is in the scene, the transition between images would not occur through the car, rather it would occur in the static parts of the images.  The second thing is that even after finding the ideal transition between images there may be some remaining differences still visible at the seam line.  These remaining differences are intelligently diffused through the composite image so that the seam line effectively disappears.

This isn’t the first application of Interactive Digital Photomontage that we have delivered.  Back in 2005 we posted Group Shot on our Research downloads page.  This is a fun and useful app that you can use to select your favorite parts in each shot of a series and it will automatically build a composite image.  This technology has also shown up in other products.  Aseem Agarwala, the UW grad student who did this work, was hired by Adobe.  Among other things he is working on, he helped ship this as the Auto-Blend Layers feature in Adobe Photoshop CS3.   

HD View Compatibility

We haven’t yet gotten seamless HD View creation in the Photo Gallery (then again HD View is still a research prototype).  However, there is a very handy feature of WLPG that the HD View tools now leverage.  When creating very wide-field-of-view panoramas, some distortion is necessarily introduced in order to get the panorama to print as a flat image.  This is akin to the distortion you see when the globe is unwrapped as a flat map or the distortion you see in a fisheye lens.  One noticeable artifact in both these cases is that straight lines in the world become curved in the flat image.  Ideally when viewing these images in an interactive viewer, like HD View, this distortion would be removed as you zoom into the image.  In order to do this the stitching application has to communicate with the viewer a few bits of information about how the projection was done.  We have added this information as metadata in the stitched panoramas that WLPG generates.  The details are outlined here.  This details page also describes how the WLPG metadata can be mapped to the tags defined in the PhotoOverlay section of the recent 2.2 update of KML.

I generated two different panoramic images to show off the photo stitching feature.  I then used the hdmake tool to read the WLPG metadata and create an HD View web page.  First are some old hand (not tripod) shot images I had of the climbing pinnacle at REI Seattle.  I selected the 8Megapixel source images (12 of them) in WLPG and did ‘Create panoramic photo…’.  I then saved the result as a jpg file when prompted to save (note in the current WLPG the result must be saved as jpeg in order to have the pano metadata).  Next to convert the jpg result to an HD View web page, I used the hdmake command line utility as follows:

hdmake -src stitch.jpg -dst pinnacle 0.8 -xmlwlpg pinnacle.xml -html pinnacle.htm

Notice the -xmlwlpg argument.  This tells hdmake to use the WLPG metadata in order to create the HD View xml manifest.  I uploaded the generated files to our server and you can view them here.  I also created a version that doesn’t use the metadata, so HD View just treats it as a flat image, you can see that version here.  Notice the difference?  The first link feels much more immersive.

The big news around Microsoft Research Redmond today is that we are moving into a great new building.  So, for my second result, I took 13 images of the lobby this morning with a 16MPix Canon 1DS MarkII.  These were automatically stitched using WLPG and I’ve posted the HD View result here.  Notice that WLPG appropriately communicated to HD View that this was a 360 environment.

I encourage everyone to download and try out the new Windows Live Photo Gallery.  In addition to the panoramic stitching feature it has a host of other capabilities.  Some of my favorites are the import wizard, the improved Fix functionality, and the upload to web services.

As always we appreciate feedback on any of this.

-Matt