A few quick links to some great tutorials created by Image Composite Editor (ICE) users …

• Over at www.panoramashots.com a very useful tutorial for those of you shooting panoramas with an indexed tripod head.  This tutorial shows how to fix alignment issues by reusing alignment information from one project to the next.
• Thomas from lightroom-tutorial.de shows how to launch ICE directly from Adobe Lightroom.
• A good basic intro to shooting panoramas and stitching them with ICE from flickr member Victor W.
• Aleks Clarks shows how to use ICE to create HDR panoramas.

This is the advice I see many people give in their comments when they post ICE panoramas on flickr, for example (1, 2, 3, 4, 5, etc).   I agree, these images are best viewed large, but web browsers do a poor job letting you explore large images.  This, after all, is why we created HD View.  But creating an HD View web page is cumbersome compared to the simplicity of uploading a jpeg to your favorite photo-sharing Web site.  ICE, Windows Live Photo Gallery (WLPG), and HDView have a cool feature that can help.  When you create a panorama in ICE or WLPG, those programs store metadata in the jpeg that describes the result.  For example, if the output is a 360-degree image, this is noted in the metadata.  On the viewing end, HDView interprets this metadata and can then provide a surround-video experience. 

Here’s an example from Aleks Clark, one of my favorite ICE-using flickr photogs:

Notice that HDView reads the metadata out of this image and understands that it is actually a 360 full wraparound.  And to view it large you can just hit the full screen button on the top right.  All I had to do to embed this in the blog was to insert the following small embed code:

<iframe style='width: 500px; height: 300px; margin: 0' frameborder='0' src='http://silverlight.services.live.com/invoke/63450/HDView/iframe.html?FileName=http://farm4.static.flickr.com/3394/3497207244_74d632775a_o_d.jpg'></iframe>

To use this, simply replace the URL of the jpg (underlined and in blue above) with your own.  This works with any Web photo service that preserves the original, like SkyDrive, SmugMug, or pro accounts on flickr.  A few caveats.  The first is that, in this case, unlike regular HDView pages, which show something almost immediately (thanks to the multiple resolutions stored on the server), the full jpg has to download before anything is shown.  On my home connection, Aleks’s image took about 17 seconds of buffering.  The second caveat is that unfortunately this feature doesn’t work in Silverlight yet, so it is HDView only for now.  Finally, the image can’t have been edited by another tool after ICE or WLPG, because these are likely to strip the relevant metadata.

So, next time you want to share your uploaded image LARGE, take a few extra minutes and provide an HDView embed on your blog!  If you try this out I’d love to see links in the comments.

-Matt

Today we are releasing an update to HD View, HD View SL, and the associated creation tools (hdmake, and Photoshop plugin).  There are plenty of great new features in this update.

The Deep Zoom Format

HD View content had a slightly different format than Silverlight’s Deep Zoom.  As of this update that is no longer the case.  We have standardized on the Deep Zoom format.  This has a bunch of advantages.  The first is that in almost all cases⁽¹⁾ content that you generate for HDView will now also be viewable in the cross platform Silverlight and on your iPhone.  We have also started porting HD View completely to Silverlight, the name of that project is, surprisingly, HD View SL.  The SL version doesn’t have all of the HD View features, but we plan to close this gap in the future.  For now the default web pages that are generated by the updated hdmake and Photoshop plugin will use HD View on Windows and HD View SL on Mac.  This will allow everyone to see your content.  As before you can also use hdmake and the Photoshop plugin to create web pages that uniquely use Silverlight regardless of client platform. 

Eric Stollnitz, the developer working on HD View SL, has created a nice side-by-side comparison page of the two projects as they stand today.  Notice as you pan and zoom in one pane the view is synchronized in the other pane.  This is a good way to see the benefits of the advanced HD View features like tone-mapping and color management (we hope to see these features in Silverlight in the future).  It’s also a good example of how to use the HD View (SL) javascript API.

Embedding

We’ve updated our embed codes so that you can now embed HD View in spaces.live.com, blogger.com, or any other blogging service that accepts <iframe> tags from the live.com domain.  When embedded in a blog your content shows up as described above – using HD View on Windows and SL on Mac.  Here’s an example embed …

For more info on how to use embedding  please see our instructions page.  Included there are detailed instructions for a variety of ways to use HD View on your blog, including how to show KML photo-overlays, Zoomify content, and even Windows Live Photo Gallery panoramas stored on Flickr.

New Toolbar including Full Screen Mode

We’ve added a new vertical toolbar in the top-right corner that is common between HD View and HD View SL.  Included in this toolbar are buttons for full-screen mode, reset view, zoom in and zoom out.  The old HD View toolbar is also still present and as always has buttons to control panning mode, tone-mapping mode and lens mode. 

.zip tile archives

A big pain when generating HD View or Deep Zoom content is the many thousands on tiny image files that get generated.  These are particularly cumbersome when they need to be copied one-by-one up to a web server.  We found that many people were creating zip archives of these tile sets in order to facilitate transfer to their servers.  In the latest tools we’ve added the feature to directly create a zip file for the tiles.  This eliminates the step of having to generate a zip file yourself, and eliminates those thousands of individual files on your disk.  In addition, for local preview⁽²⁾, HD View can read the tiles directly out of the zip archive.

--Matt Uyttendaele

the fine print:

(1) HDR and 16bit images are still HDView only.  Also wide-gamut images will be converted to sRGB by our utilities for cases where HD View SL is selected. 

(2) HD View SL cannot do local preview when the tiles are stored in a zip file.

Last week I was chatting with Jonathan Fay, the architect of Microsoft’s World Wide Telescope (WWT). Over the years Jonathan has been a great friend to the HD View project.  He provided software early on that helped bootstrap HD View.  He also introduced us to the good folks at Meade Telescope, who provided our camera control platform (see pic1, pic2, video).  HD View has certainly benefited from and taken inspiration from work done in the astronomy community.  So I always pay close attention when Jonathan shows me the latest goings-on in astronomy.  Last week he demoed for me the astrometry flickr group.

At first glance this seems like any other group on flickr.  However, behind the scenes something pretty amazing is happening.  When a photographer posts an image to this group, a program called the blind astrometry server is put to work.  What this program does is automatically determine what part of the sky the image represents.  It works by extracting features, in this case quadruples of stars, from the image and comparing these against similar features extracted from a whole sky survey.  According to a technical description, this comparison runs in less than a second thanks to sophisticated indexing techniques.

Positioning the image enables some cool new features unique to the astrometry group.  The first is that objects in the picture are automatically tagged.  See these examples of the Jellyfish Nebula and the International Space Station flying over Seattle.  Notice that the notes in these pictures indicating star names were all placed there automatically by the blind astrometry server.  The second is that a link is added to allow the image to be browsed within the World Wide Telescope.  This lets users see the image in the context of the entire night sky.  For further reading, I recommend an interview with Christopher Stumm on the code.flickr blog.

So how close are we to a similar service for terrestrial photos?  Even though the universe is a vast place, surprisingly I think that auto-locating earth-based photos is a harder problem.  First there is far more data.  The portion of the Sloan Digital Survey used for flickr astrometry contains about 1 terabyte of imagery.  Last year the VE team added 36 terabytes of data in just one release.  I’m sure that Google Street View similarly contains many terabytes of data.  The second challenge is that there are more degrees of freedom for positioning an image on earth.  For space there are 4 variables to solve when placing an image, 3 for rotation and 1 for scale.  On earth there are at least 7 variables – 3 for position, 3 for rotation and 1 for focal length, and this ignores other terrestrial factors like variation over time and lighting changes.

These difficult problems are slowly being overcome though.  You already see Microsoft and Google systematically surveying the whole earth in a manner similar to the whole sky survey mentioned above.  The computer vision community has developed fast image recognition techniques for non-sky pictures.  There are tools for manually (not yet automatically) placing images within Google Earth or Virtual Earth 3D.  And, the Photo Tourism project (which became Photosynth) demonstrated auto-annotation of flickr images.  Over the coming years I think that we’ll see these technologies packaged up in an easy-to-use web service.

-Matt

I saw a new “I’m a PC” commercial running during American Idol this week.  I’m pretty biased, but I really like this commercial.  Having been very involved in the development of this technology, I know that under the hood it is quite complex.  What this commercial brings home for me is that through the power of software, we have managed to hide all this complexity from the user and boil it down to, in Alexa’s words, “squishing all my little pictures into one big one.”

    Video: The Rookies: Alexa, Age 7

And here are some real-world examples of novices creating panoramic photos using Windows Live Photo Gallery:

• A panoramic photo competition on a Holland America cruise ship
• A retiree, whose online name is gadgetgran, teaching her friends to make panoramas

All of this reminds me of why I work at Microsoft.  While it’s been great to get positive peer reviews on the underlying technology that we develop (for example, see the photography-related publications on Rick Szeliski's page), what really excites me is putting this technology to use in Microsoft products and enabling millions of people to do something new with their PCs.

-Matt

On Tuesday I blogged about some of the great Photosynth images available through the CNN’s “The Moment” project.  The goal of the project was to have people attending the inauguration send pictures from the crowd and have Photosynth stitch them together.  When you crowd source pictures some cool unexpected things happen.  In this case a lot of people submitted pictures of themselves watching the event on TV.  The result is quite moving.  This isn’t the type of Synth that you explore, rather you should just hit the play button in the top right of the window, sit back, and watch.

http://photosynth.net/silverlight/photosynth.aspx?cid=44fbe543-600c-404f-8ceb-8af763ca9b00

-Matt

Great use of Photosynth and HD View today

• CNN features a crowd sourced Photosynth called “The Moment”.  Spectators at Barrack Obama’s inauguration submitted photos via email to CNN and Photosynth was used to assemble them.  A video of CNN anchor John King describing the technology is also available.
• MSNBC features several Photosynths created by their staff photographers.
• MSNBC is using HD View to display a very high resolution, zoomable satellite photo (taken by GeoEye-1) of the mall during the ceremony.

I’ve noticed a few examples of people using ICE to stitch content from video.  One of my favorites came out right after the ICE launch.  The blogger Cinema Squid used ICE to generate panoramic images from panning sequences he grabbed out of the movie Starship Troopers.  Over on flickr, user jasolo has created a group called Panoramas from movies.  So far most of his examples are from old black & white videos that are available from the Library of Congress as part of the Prelinger Archives.  To the left is an example jasolo did with ICE.  This was made from frames of "Amateur Film European Trip 1930s".  I really like the idea of applying stitching to video footage, especially travel video.  More often than not when I see people taking video of famous landmarks what they are doing is panning over a large area to make sure that they capture everything.  When viewing this content it seems more natural to me to experience it in a panorama viewer instead of as raw video.

I also like the idea of unlocking archive content like jasolo did and making it more accessible.  There is an opportunity here to use tools like ICE and Photosynth to help organize this content.  Photosynth is a natural for this task as it allows for more general camera motions (beyond panning) that is often found in video.  I did a quick search of Photosynth for synths made from video content.    I did find one example of an old “Bigfoot” film made into a synth.

Seeing these examples makes me think that a great feature would be to allow ICE and Photosynth to analyze video directly. Microsoft Research has studied this problem a bit and back in 2005 present a paper titled “Efficiently Registering Video into Panoramic Mosaics”.  It may be time to productize this technology.

Update 12/22/2008: I just did an Internet search and found an additional 14 panoramic composites done by Cinema Squid from the film Baraka.

-Matt

p.s. this blog was composed using the latest update of the excellent Windows Live Writer.  Get it here. 

The Deep Zoom team at Microsoft has just announced Seadragon, their new application for the iPhone.  Deep Zoom is the technology built into Silverlight, that enables gigapixel images to be browsed over the web.  That same Deep Zoom technology is now available for the iPhone.  This is great news for readers of this blog that have used the Deep Zoom output option in Microsoft ICE or in the HD View Photoshop plugin.  All of the content that you've created with those tools can now be browsed on the iPhone.  For example in the picture below I'm showing Gerard Maynard's 17 gigapixel image of Yosemite.  On the left is the Silverlight viewer on his site.  On the right is the same content being displayed on my iPod.   Seadragon for the iPhone delivers the same smooth pan and zoom experience.  It's amazing to be able to browse a 17 gigapixel image on a hand-held device.

To get the app, search for Seadragon in the App Store.  When you first load it, it comes with some great default content to browse.  One of my favorites is "Running the Numbers" by Seattle photographer Chris Jordan.  You can also then add your own content.  This comes in three flavors.

(1) Links to your or any other Photosynth imagery (note this isn't the full Photosynth experience - rather just the source pictures streamed to your phone).  

(2) Deep Zoom Content.  When you generate Deep Zoom content in Deep Zoom Composer, Microsoft ICE, or the Photoshop Plugin, an xml file is generated.  After you've uploaded this xml file plus the imagery to your web server, you can then point the Seadragon App to the url of the xml file. 

(3) An RSS feed.  This is a way for you to generate a list of frequently updated Deep Zoom imagery that iPhone users can browse.  I just started one for my favorite gigapixel photographs.  You can see my feed by tapping the '+' at the bottom right of the Seadragon App, then select RSS Feed from the options and type the following URL: http://research.microsoft.com/ivm/HDViewsl/dzfavs.xml

I'll be updating this RSS feed often so stay tuned at Seadragon on the iPhone.

-Matt Uyttendaele

Last week I saw that Flash 10 is now color managed (with some caveats).  It's great to see Adobe taking this step.  This reminded me that I had never properly discussed the color support that we added to HD View 3.  Our philosophy in developing HD View was that we would enable the delivery of photos over the web with no limitations in dimensions, dynamic range, or color gamut.  I've previously written about HD View's support for gigapixel images and high dynamic range.  In this post, I want to focus on the last point of gamut.

I won't try to give a detailed background of the state of color management on the web.  For that I refer you to an excellent summary written by Jeffrey Friedl.  Jeffrey makes many good points in his article.  He says that it is unfortunate that Internet Explorer is not color managed.  I agree.   I'm confident that this will be fixed, hopefully sooner rather than later.  Since Windows and the web have evolved to this point without much color management, I will concede that it is difficult for IE to simply just turn it on now.  Bobby Holley gives a good rundown of the issues (performance, bad monitor profiles and plugins) that he encountered during the development of color management in Firefox.  I applaud the FF team for tackling this feature.  They seem to be addressing the blocking issues by gradually introducing color management to their users.  FF 3.0 had color support, but it was off by default (I would guess that most users don't turn it on).  It sounds like Firefox 3.1 will take the step of turning on color management but not for the entire web page, rather only for embedded photos that are tagged with profiles.  When preparing photos for this new WWW of color managed browsers, it is important that your personal photo editing software also be color managed.

I'm happy to say that pretty much all Microsoft products in the photography space are color managed, including the Vista Photo Gallery, Windows Live Photo Gallery, and the Expression Suite.  This will certainly help rout out the lingering bad display profiles that Holley talks about.  Likewise, all of Adobe's photo applications are, of course, color managed.  Picasa currently does no color management.  For users of that product, I encourage you to provide feedback to Google that this feature is important to you.

Perhaps one reason that we haven't seen color management become more prevalent is that most computer monitors are very close to the sRGB color space. Not coincidentally, almost all JPEG images are also stored in this same sRGB space.  This means that doing no color adjustment on photos (what Friedl calls "color stupid") looks approximately correct most of the time.  I feel like this is about to change.  Every major display vendor is unveiling wider gamut monitors--a few examples: Samsung's MagicSpectrum product line, HP's DreamColor line, and TrueLife RGB LED displays in Dell laptops.  While these monitors are now fairly high-end, I believe that they will become mainstream within a few years.

To understand what a wide gamut monitor does, I need to introduce the requisite chromaticity diagram present in all articles about color. The blob of colors in the diagram is meant to show the range of colors that humans can see.  The inner triangular region is representative of the range of colors of the sRGB standard.  The outer triangle is the range of colors that another standard known as AdobeRGB contains.  Note that the colors in this diagram are not accurate (no display can show the colors at the very fringe of the blob); rather, this is just meant to convey that AdobeRGB can represent more colors than sRGB, but neither represents all visible colors.  All of the new generation monitors mentioned above claim to display the full AdobeRGB gamut.  What that means is that "color stupid" applications (IE, Picasa, default settings of Firefox 3.0) will display photos, well, stupidly.  With these applications, even if your photo is correctly tagged as sRGB, that won't help.  The new monitors are reporting to the operating system that they expect AdobeRGB, but the application is ignoring this and instead providing sRGB.  What you end up with are images that look too colorful, because what was a saturated color (#FF) in sRGB is even more so in AdobeRGB.

So, why don't these color standards just allow for all colors to be represented?  The main reason is due to the bit depth limitations of image formats.  JPEG is an 8-bit per component standard.  This means that a color can have 256 levels of red, green, or blue.  When sRGB was designed it was wisely determined that it was more important to spread those 256 levels over the most common colors rather attempting to stretch to also include the very saturated colors.  In other words, the trade-off is that you can represent fine changes of color in a picture, but you can't represent the extreme colors.  There is quite a bit of debate among photographers and color experts about this point, for example see this article on sRGB vs. AdobeRGB.

Luckily, we won't have to live with the 8-bit restriction of JPEG forever--HD Photo to the rescue.  As I've written before, HD Photo supports 16-bit or 32-bit per color, while keeping the same efficient file size of JPEG.  This means that we can now deliver 16-bit images over the web ,and a 16-bit image has 65,536 distinct levels of R,G, and B.  We no longer need to trade off fine color steps versus extreme colors; we can have both!  So, to best display a photo, you need an application that is both 16-bit-aware and performs color management.  This is what HD View delivers.

HD View respects the color profiles on both the source imagery and the monitor.  When using our Photoshop plugin, we encode the working color space from Photoshop and send that along with the image pixels (encoded using HD Photo) to HD View.  We also allow you to export from Photoshop to HD View in 8-, 16-, or 32-bit mode.  What all of this means is that the image you see in Photoshop will be preserved across the web.  To see this in action take a look at the color examples on the HD View web page.  These examples are meant to show that using HD View, you can see extreme colors while preserving the fine changes within those colors. (Note: your mileage may vary on the examples, as you need a calibrated monitor that goes beyond the sRGB gamut to see the full effect.)  Lest you be concerned that all of this color manipulation comes at a penalty in application performance, remember that HD View uses your GPU to process the imagery.  This allows HD View to pan and zoom on HDR, gigapixel images and perform color management all at 60 frames per second.  Preserving color over the web this way is a marked change from the typical "export to web" process of today which usually involves converting to sRGB.

This brings me back to the Flash 10 announcement that I mentioned at the top.  Flash 10 does indeed respect display profiles, however they have decided to only support sRGB source images.  This may be the right design for today's world where most monitors are still limited to sRGB.  But it unfortunately limits what can be seen on the new generation of monitors.  Effectively, this turns a spiffy new AdobeRGB-capable monitor back into an sRGB monitor.  The authors of the Flash announcement made the statement that "[p]resenting your images through Flash is now the best way to preserve the fidelity of their color online."  I may be misinterpreting what is meant by "color fidelity" here, but perhaps it is more accurate to say: Flash is now the best RIA for displaying the correct colors in sRGB images over the web.  Due to the support of images with embedded profiles in Safari and Firefox 3.1, I think that those two applications do a better job preserving color fidelity.  And thanks to the HD Photo and color management support in HD View, I will claim that HD View is the best way to preserve color fidelity when presenting your photos over the web to a Windows PC.

-Matt

Last year I blogged about a 13 gigapixel image of Harlem done by NYC based artist Gerard Maynard.  The image was pretty well referenced on the web, lots of diggs, well watched on youtube, and over a million views on break.com.  This past spring I was fortunate to meet Gerard as part of the Yosemite Extreme Pano project.  While I was only spending a few short days in Yosemite for the project, I remember being jealous that Gerard had given himself a whole week to shoot the valley.  He put that time to good use.  Today Gerard unveiled his latest project called yosemite-17-gigapixels.  The name probably gave it away - yep this image is even more detailed than the Harlem picture.  Gerard says it is the largest known stitched panorama.  Below is a screen shot of the site.  The Glacier Point image shown here is the 17 gigapixel monster.  But, be sure to also click on the other names along the bottom, these links lead to other Yosemite views that are all pretty impressive.

The HD View team helped in a small way with the production of this.  Gerard used our Photoshop plugin to generate both the Silverlight and HD View options on the site.  The "View in HD View SL" link on the site is a Silverlight viewer that uses Deep Zoom technology.  I encourage you to compare the SL experience with the site's "View in Zoomify" option.  I think most will agree that SL with Deep Zoom is a much more immersive and fluid experience than Zoomify.

Congrats Gerard.

-Matt

Hello.  Last week Silverlight 2.0 shipped.  Today, we are doing a corresponding release of all of our tools.  To get these updates, please visit the ICE download page and/or the Photoshop Plugin and hdmake page.  In addition to getting the latest Silverlight 2.0 goodness, this ICE release addresses several issues reported by our users:

• we now offer http download instead of ftp
• the crash on second use bug has been fixed
• the random drop of one image from the result is fixed
• the metadata issue that prevented geotiffs from loading is fixed

A big thank you to all users who took time to try the product and give feedback on the forums.  We welcome the feedback and are working hard to address issues in a timely fashion.

Also due to popular demand we are releasing the source code for HD View SL, the Silverlight based viewer that is output by ICE and by the Photoshop Plugin.  As this name indicates we are gradually migrating HD View, a Windows only browser plugin, to HD View SL, a cross-platform Silverlight Rich Internet Application.  There are still some significant differences between HD View and HD View SL but we are confident that these differences will gradually disappear.  For a detailed comparison of the two technologies please see the HD View SL CodePlex page.

If you are interested in Silverlight programming I encourage you to download the source and start making customizations to the viewer.  Some customizations that I would love to see are different navigation UI, auto-scroll of the view, and keyboard short-cuts.  If you create a custom version and would like to share it with ICE users, then check out these instructions* for adding new choices to the ICE export options.  Even if you aren't specifically interested in the HD View application, this project is a great example of how to use the new Deep Zoom feature (also known as MultiScaleImage) in Silverlight 2.0.

-Matt

* Instructions are in the Creating Templates.rtf document located in the Program Files\Microsoft Research\Image Composite Editor\Templates ICE installation folder.

We spent the first 3 weeks after the ICE launch flying without a forum.  We've done our best trying to answer questions that have come via email, but some of your emails may have fallen through the cracks.  Well we've finally gotten a proper forum running.  Please use this for future questions, bug reports, suggestions, discussions, etc ...

http://community.research.microsoft.com/forums/112.aspx

-Matt

Today we are very pleased to announce the Microsoft Image Composite Editor.  It is a free download available today from here.  This application is the best image stitching tool that we have shipped to date.  The stitching technology from the Interactive Visual Media Group has shipped previously in Windows Live Photo Gallery and more recently in Deep Zoom Composer.  The core stitching engine in the Image Composite Editor uses the same advanced matching and blending techniques that I described in my Deep Zoom Composer blog entry.  Beyond that this application adds the following features:

• A GPU accelerated orientation adjustment tool.  Sometimes the automatic stitching software doesn't quite get the viewing direction correct, or perhaps you want to use a rectilinear projection instead of a cylindrical projection.  This new tool allows you to interactively make these adjustments. 
• 360 blending support.  Our fast poisson blend technique now creates a seamless 360 blend.
• Output to Photoshop layers.
• Create an HD View web page.
• Create a Silverlight Deep Zoom web page, including new 360 support in our Silverlight application.
• Integration with both the Windows Shell and the next version of Windows Live Photo Gallery, so that you can quickly launch a new stitching project.

Please try it out today, and let us know what you think.

-Matt Uyttendaele

P.S. We will be giving demos at the Photokina show all week.  If you happen to be attending, please stop by the Microsoft booth to see this and a lot of other great technology.

Since releasing HD View 3, I've started getting links to some great new 360 HDR content.  The combination of 360 and HDR really shines in HD View.  Click here for a full page of examples from Bernhard Vogl, and here for an example from Greg Downing.

Traugott Emrich has put together a large gallery of HD View content on his site.  Click here for some great images of the Czech Republic.  The imagery is hosted on gigapan.org.  Traugott is using our new kml support to link his page to gigapan's servers.  Also, even though the name is very similar, he is not using a gigapan unit.  Rather, he has built his own custom pan/tilt head that he calls the GigaPanBot.  So far, the results look really good.

And finally, MSNBC has posted a large HD View panorama taken from the floor of Invesco Field during the Democratic National Convention.

-Matt

The Beta 3 version of HD View can be embedded in a PowerPoint slide.  Below are a few easy steps to do that in PowerPoint 2007.

1) Make sure that you have HD View for Internet Explorer (as opposed to the Firefox plugin) on your machine.  If you don't yet have it, go here: http://research.microsoft.com/research/downloads/Details/63888522-375b-49b5-a8a0-5cb308deb5c5/Details.aspx

2) Enable the developer tab in the ribbon.  Instructions for that are here: http://research.microsoft.com/research/downloads/Details/63888522-375b-49b5-a8a0-5cb308deb5c5/Details.aspx

3) Go to the slide that you want to embed HD View in.  Click the Developer tab and click on the "More Controls" button as show here:

4) Select "HD View Control" in the "More Controls" list that pops up

5) This will give you a cross cursor.  Drag the cursor on the slide to indicate the desired size and position of the HD View region on your slide.  Then right click on selected region and select "Properties" from the menu as below

6) This will bring up the Properties list for HD View (the properties are defined here).  At a minimum you need to set the FileName as in the screen shot below:

That's it.  Note the control will not load until you go into slide show mode.

There are two main tools for converting a large image into the HD View format.  They are hdmake, a command line application, and the HD View Photoshop plugin.  The HD View format is very similar to the Deep Zoom format.  So, we are pleased to announce that we have added Silverlight Deep Zoom capabilities to our latest release of these two tools.  In either tool you can now indicate Deep Zoom output and it will generate the image tiles, Silverlight files, and an example web page.  That means that our Photoshop plug-in gives you capabilities comparable to the Zoomify export built into CS3.  See this blog post for a take on Deep Zoom's advantages over Zoomify.

The download link is: http://research.microsoft.com/research/downloads/details/345a52c3-fe44-4045-94b4-4b26a93a907c/details.aspx 

-Matt

Deep Zoom is a terrific new feature that is available in Silverlight 2.0.  The best way to describe Deep Zoom is that it allows Silverlight to natively load arbitrarily large images.  To get an idea of what Deep Zoom can do, please take a look at the initial results of the Yosemite Extreme Panoramic Imaging Project.

But, what I really want to talk about today is the Deep Zoom authoring tool - Deep Zoom Composer (DZC).  The latest beta of DZC offers a feature that my group developed - automatic image stitching.  While we've shipped this in the past in other products like Windows Live Photo Gallery, this version of the stitcher has some significant improvements.  The first improvement is a better technique for automatically finding matching control points between images.  The second is better blending between images, especially when there are significant exposure differences.

Control points are very familiar to anyone who has used manually operated stitching software.  For those who haven't used such software, the basic work flow is that you open one image, click on a point in that image that you think will likely appear in another overlapping image, then open that other image and click on the corresponding point.  You do this over and over again until all images that comprise your panorama have points matching into other images.  Recently computer vision algorithms have been invented that can automatically perform this function.  The most well known of these algorithms is the Scale-invariant feature transform (SIFT).  SIFT works so well that it has largely replaced manual control point editing and there are several automatic stitchers available, such as Autostitch, that take advantage of SIFT.

SIFT however isn't perfect at matching images and sometimes misses matches or generates erroneous matches.  Last year, Simon Winder and Matthew Brown, two researchers in our group, tackled the problem of designing a better matching technique and published their work in a paper titled Learning Local Image Descriptors.  This paper is meant for a technical audience, so I will try to summarize the findings here.  SIFT was designed by smart people testing different ideas on a large number of test images.  The new technique instead uses a much much larger number of test images (produced by the Photosynth pipeline), and uses mathematical optimization techniques to automatically 'learn' a new control point matcher.  The net result is that the new technique produces 1/3 the number of false matches when compared to SIFT on our test data.  And ultimately for users of DZC this will lead to a higher success rate for your stitching projects.

It is personally very exciting for me to see Microsoft Research (MSR) results like these moving from academic journals to actual usable products so quickly.  The MSR computer vision researchers have long been active in academic circles, but now are finding more and more outlets in Microsoft products as well.  There are other examples such as Photosynth and HD View, where preview software was available very near to technical publication^{(1, 2)}.  I look forward to this trend continuing.

That said, please give the new Deep Zoom Composer a spin.  You can learn more about it and download it from:

http://blogs.msdn.com/expression/archive/2008/08/01/download-the-new-deep-zoom-composer-preview.aspx 

Note DZC is still Beta software and there are a few rough edges such as limits on output size.

-Matt Uyttendaele

⁽¹⁾ Photosynth was published at Siggraph 2006 as "Photo tourism: Exploring photo collections in 3D" in August and made available publicly in November.

⁽²⁾ HD View was published at Siggraph 2007 as "Capturing and Viewing Gigapixel Images" and made available in conjunction with the conference.

National Geographic's web site has a great gallery of 360 panoramas from Petra, Jordan.  They have a slide show of 6 different panoramas from around the ancient city.  For each of these you can click on "View in HD" to get the full-screen HD View experience.  Check it out here: http://www.nationalgeographic.com/history/ancient/petra-360.html

-Matt

This post is all about high dynamic range imaging (HDRI).  As I mentioned in the Beta 3 announcement, HD View is the best way, that we are aware of, to interact with HDR images over the web.  We take advantage of three recent advances to make this happen.  They are HD Photo compression technology, the PC's graphic processing unit (GPU), and high contrast displays.  I will describe each of these things, but first, a quick intro to high dynamic range imaging (HDRI). 

What is HDRI?

image courtesy of Mark Fairchild

Light in the world has a very large range of intensities.  In the figure above, you can see a 100 million to 1 range (alternatively 26 stops) in intensity from starlight to direct sunlight.  An HDR image is one that contains a measure of the intensity of light in a scene.  At each pixel, it records one of the numbers on the scale in the figure above.  This means that an HDR image can represent a scene that has a very large intensity difference between the shadows and the highlights.  Cameras today can't directly record scenes that have such an extreme range.  So to create these images, photographers stack multiple exposures together, one correctly exposed for the shadows, one for the mid-tones, one for the highlights, and so on.  These separate exposures are then merged by software^{(1, 2)} to produce an HDR image.  This is quite a different representation than a JPEG image.  In terms of range, a JPEG can represent only a small portion of the  possible range of intensities in a scene, which makes JPEG a low dynamic range (LDR) format.  This is why in a backlit situation your camera can produce a good JPEG of the foreground object or one of the bright background, but not both at once.  It turns out that most computer displays are also LDR, so even if a JPEG could store a greater range, you wouldn't be able to see it on a standard monitor.  JPEGs also differ from HDR in the way they store color information, but that is a topic for my next post.

So why are there so many HDR images stored as JPEGs on flickr?  That is thanks to a technique known as tone-mapping.  In order to make an HDR image display on an LDR monitor, tone-mapping algorithms compress the wide range of intensities in the HDR image to fit in the narrow range of the monitor.  There have been terrific advances in tone-mapping, but it is a very hard problem to make tone-mapped images look natural.  In order to compress the range, dark parts of the scene need to be brightened to the point that they have similar intensity to bright parts.  This can lead to an unnatural look, because as a human observer of this image, you expect bright objects to look a certain way when compared to dark objects, but the tone-mapping process has changed the tonal relationship between these objects.  I've heard many people say that they "don't like the HDR effect," but what I think they object to is the effect of the tone-mapping process.  It is important to separate these two things.  HDR images represent the intensities of light in a scene.  Tone-mappers make HDR images presentable on the computer monitors available today.

HD Photo and JPEG XR

Now onto how we are supporting HDR in HD View.  On the High Dynamic Range page of the HD View web site we include the following diagram that describes our HDR pipeline.

The pipeline starts with a camera capturing one or more RAW images.  A single RAW image by itself contains a greater dynamic range than JPEG can store, and as described above, if multiple bracketed RAW images are captured, these can be merged into an HDR image.  If we want to deliver this over the web, today that would involve the tone-mapping process described above followed by storing the result as a JPEG.  In our pipeline, we instead store the extended dynamic range image directly in a new image format called HD Photo.   There are other file formats available today that could also be used such as, .HDR, OpenEXR, and TIFF.  But, none of these offer the sophisticated compression algorithms that HD Photo does.  This compression allows us to effectively transfer HDR images over the web.  The file sizes of these other HDR file formats are simply too large for interactive web access.  As an example, the images on our HDR Survey page were downloaded from Mark Fairchild's site in OpenEXR format and converted to the HD Photo format.  The OpenEXR files range in size from 35MB to 46MB, by comparison after conversion to HD Photo the images are about 1/10th the size (3.6MB to 6MB). 

HD Photo started as a Microsoft Research project.  It was then refined and further developed by the Microsoft Codec Group.  Microsoft subsequently presented HD Photo to the JPEG standards body, and that organization has decided to make HD Photo a standard that will be known as JPEG XR.  I am very much looking forward to the time when JPEG XR emerges from the standardization process and is in ubiquitous use.  For now though, you can start experimenting with the new technology in the form of HD Photo.  The HD View team has provided a simple command line tool and a Photoshop plugin that you can use to convert your RAW and HDR images to the HD Photo format that HD View requires for streaming over the web.  HD Photo has lots of other great features beyond HDR, please see Bill Crow's blog for many more details.

GPUs

The graphics processing unit (GPU) of almost every modern PC now has HDR capabilities.  For example, a new $300 PC from Dell uses the Intel 3100 GPU.  This chip can load and process floating point images which is the same representation that HD Photo uses and is a great representation for HDRI⁽³⁾.  Of course higher end GPUs from NVidia and ATI have these capabilities as well, but the point is that even entry level PCs can now process HDR imagery in real time.  GPUs have this capability because in order to generate realistic looking 3D graphics, game designers need to simulate real world lighting.  Thus, today's most realistic looking games are in fact doing HDR.  GPUs also have the ability to run 'shader' programs.  These shaders, as they are known, can run fairly sophisticated image processing algorithms at interactive rates.  We use these capabilities to perform real time tone-adjustment on HDR images.

Displaying HDR

HDR displays are coming but they aren't available just yet.  When they become available our HD View pipeline won't need to change and any content that you've created for HD View will look great on them.  But for the foreseeable future we will also have LDR displays.  So now that we've delivered an HDR image all the way to an LDR monitor what do we do with it?  Remember that HD View is an interactive image viewer.  So, we can create the ideal tone adjustment given the end-users display, the portion of the image they are currently viewing and any intents we let them express.  This is quite different from the tone-mapped images you see today.  Those images are processed assuming that they will be viewed full-frame on an LDR monitor or on paper.  This can lead to the distorted relationship between tonal values I described above.  It can also lead to a flattening or loss of contrast in the image.  In HD View we instead do tone adjustment on the fly as the user views the image.  For example, when zoomed all the way out on an image we might see the shadows be truly dark, but as we zoom into those dark areas HD View will auto-expose to show previously indiscernible details (the ability to retain those details in the shadows is thanks to HD Photo).  In our pipeline, we've retained all of the dynamic range of the original photographs and we have the ability to run shaders, so we offer users a variety of different ways to do tone-adjustment on the fly.  These are accessed via the toolbar in the upper right of the window and are summarized in the table below.  For some screen shots of these modes in effect see our help page.

Camera for the Web

To wrap it up, I have described a pipeline that lets you maintain all of the light information that was captured from the original scene.  A common term used to describe this is a 'scene-referred' workflow.  In the past scene-referred workflows have been advocated for the image editing process but not for delivery over the web.  For web delivery conversion to JPEG is the standard.  Here we are allowing scene-referred imagery to travel all the way to the end user's monitor.  One way that we like to think of this is that HD View is a camera for the web.  It is performing a very similar task to camera hardware today which is reading scene-referred rays of light and generating the best possible JPEG image that it can.  We instead are enabling that scene-referred data to travel over the web.  The HD View "camera" can then generate the best possible photograph using additional information that is only available at the end of this pipeline.  This includes taking into account the monitor being used, the user's current pan/zoom interaction, and any additional settings that we want to expose to the user.  Currently these additional settings are the three tone-adjustments listed above.  However, we can see expanding this to any of the settings that you find in hardware cameras today.  Beyond that, we are running on powerful PC hardware, so our software camera can perform image processing that is just not feasible in your point and shoot camera.

Try it out and we always appreciate feedback.

-Matt Uyttendaele

⁽²⁾ Wide field-of-view panoramic images are also typically HDR.  In the Capturing and Viewing Gigapixel Images paper, a pipeline to produce HDR gigapixel images is described,  PT Gui Pro and Autopano Pro both have the ability to create HDR panoramas.

⁽³⁾ See figure 3. in Greg Ward's HDR encodings paper.