HD View

A few quick links to some great tutorials created by Image Composite Editor (ICE) users …

• Over at www.panoramashots.com a very useful tutorial for those of you shooting panoramas with an indexed tripod head.  This tutorial shows how to fix alignment issues by reusing alignment information from one project to the next.
• Thomas from lightroom-tutorial.de shows how to launch ICE directly from Adobe Lightroom.
• A good basic intro to shooting panoramas and stitching them with ICE from flickr member Victor W.
• Aleks Clarks shows how to use ICE to create HDR panoramas.

This is the advice I see many people give in their comments when they post ICE panoramas on flickr, for example (1, 2, 3, 4, 5, etc).   I agree, these images are best viewed large, but web browsers do a poor job letting you explore large images.  This, after all, is why we created HD View.  But creating an HD View web page is cumbersome compared to the simplicity of uploading a jpeg to your favorite photo-sharing Web site.  ICE, Windows Live Photo Gallery (WLPG), and HDView have a cool feature that can help.  When you create a panorama in ICE or WLPG, those programs store metadata in the jpeg that describes the result.  For example, if the output is a 360-degree image, this is noted in the metadata.  On the viewing end, HDView interprets this metadata and can then provide a surround-video experience. 

Here’s an example from Aleks Clark, one of my favorite ICE-using flickr photogs:

Notice that HDView reads the metadata out of this image and understands that it is actually a 360 full wraparound.  And to view it large you can just hit the full screen button on the top right.  All I had to do to embed this in the blog was to insert the following small embed code:

<iframe style='width: 500px; height: 300px; margin: 0' frameborder='0' src='http://silverlight.services.live.com/invoke/63450/HDView/iframe.html?FileName=http://farm4.static.flickr.com/3394/3497207244_74d632775a_o_d.jpg'></iframe>

To use this, simply replace the URL of the jpg (underlined and in blue above) with your own.  This works with any Web photo service that preserves the original, like SkyDrive, SmugMug, or pro accounts on flickr.  A few caveats.  The first is that, in this case, unlike regular HDView pages, which show something almost immediately (thanks to the multiple resolutions stored on the server), the full jpg has to download before anything is shown.  On my home connection, Aleks’s image took about 17 seconds of buffering.  The second caveat is that unfortunately this feature doesn’t work in Silverlight yet, so it is HDView only for now.  Finally, the image can’t have been edited by another tool after ICE or WLPG, because these are likely to strip the relevant metadata.

So, next time you want to share your uploaded image LARGE, take a few extra minutes and provide an HDView embed on your blog!  If you try this out I’d love to see links in the comments.

-Matt

Today we are releasing an update to HD View, HD View SL, and the associated creation tools (hdmake, and Photoshop plugin).  There are plenty of great new features in this update.

The Deep Zoom Format

HD View content had a slightly different format than Silverlight’s Deep Zoom.  As of this update that is no longer the case.  We have standardized on the Deep Zoom format.  This has a bunch of advantages.  The first is that in almost all cases⁽¹⁾ content that you generate for HDView will now also be viewable in the cross platform Silverlight and on your iPhone.  We have also started porting HD View completely to Silverlight, the name of that project is, surprisingly, HD View SL.  The SL version doesn’t have all of the HD View features, but we plan to close this gap in the future.  For now the default web pages that are generated by the updated hdmake and Photoshop plugin will use HD View on Windows and HD View SL on Mac.  This will allow everyone to see your content.  As before you can also use hdmake and the Photoshop plugin to create web pages that uniquely use Silverlight regardless of client platform. 

Eric Stollnitz, the developer working on HD View SL, has created a nice side-by-side comparison page of the two projects as they stand today.  Notice as you pan and zoom in one pane the view is synchronized in the other pane.  This is a good way to see the benefits of the advanced HD View features like tone-mapping and color management (we hope to see these features in Silverlight in the future).  It’s also a good example of how to use the HD View (SL) javascript API.

Embedding

We’ve updated our embed codes so that you can now embed HD View in spaces.live.com, blogger.com, or any other blogging service that accepts <iframe> tags from the live.com domain.  When embedded in a blog your content shows up as described above – using HD View on Windows and SL on Mac.  Here’s an example embed …

For more info on how to use embedding  please see our instructions page.  Included there are detailed instructions for a variety of ways to use HD View on your blog, including how to show KML photo-overlays, Zoomify content, and even Windows Live Photo Gallery panoramas stored on Flickr.

New Toolbar including Full Screen Mode

We’ve added a new vertical toolbar in the top-right corner that is common between HD View and HD View SL.  Included in this toolbar are buttons for full-screen mode, reset view, zoom in and zoom out.  The old HD View toolbar is also still present and as always has buttons to control panning mode, tone-mapping mode and lens mode. 

.zip tile archives

A big pain when generating HD View or Deep Zoom content is the many thousands on tiny image files that get generated.  These are particularly cumbersome when they need to be copied one-by-one up to a web server.  We found that many people were creating zip archives of these tile sets in order to facilitate transfer to their servers.  In the latest tools we’ve added the feature to directly create a zip file for the tiles.  This eliminates the step of having to generate a zip file yourself, and eliminates those thousands of individual files on your disk.  In addition, for local preview⁽²⁾, HD View can read the tiles directly out of the zip archive.

--Matt Uyttendaele

the fine print:

(1) HDR and 16bit images are still HDView only.  Also wide-gamut images will be converted to sRGB by our utilities for cases where HD View SL is selected. 

(2) HD View SL cannot do local preview when the tiles are stored in a zip file.

Last week I was chatting with Jonathan Fay, the architect of Microsoft’s World Wide Telescope (WWT). Over the years Jonathan has been a great friend to the HD View project.  He provided software early on that helped bootstrap HD View.  He also introduced us to the good folks at Meade Telescope, who provided our camera control platform (see pic1, pic2, video).  HD View has certainly benefited from and taken inspiration from work done in the astronomy community.  So I always pay close attention when Jonathan shows me the latest goings-on in astronomy.  Last week he demoed for me the astrometry flickr group.

At first glance this seems like any other group on flickr.  However, behind the scenes something pretty amazing is happening.  When a photographer posts an image to this group, a program called the blind astrometry server is put to work.  What this program does is automatically determine what part of the sky the image represents.  It works by extracting features, in this case quadruples of stars, from the image and comparing these against similar features extracted from a whole sky survey.  According to a technical description, this comparison runs in less than a second thanks to sophisticated indexing techniques.

Positioning the image enables some cool new features unique to the astrometry group.  The first is that objects in the picture are automatically tagged.  See these examples of the Jellyfish Nebula and the International Space Station flying over Seattle.  Notice that the notes in these pictures indicating star names were all placed there automatically by the blind astrometry server.  The second is that a link is added to allow the image to be browsed within the World Wide Telescope.  This lets users see the image in the context of the entire night sky.  For further reading, I recommend an interview with Christopher Stumm on the code.flickr blog.

So how close are we to a similar service for terrestrial photos?  Even though the universe is a vast place, surprisingly I think that auto-locating earth-based photos is a harder problem.  First there is far more data.  The portion of the Sloan Digital Survey used for flickr astrometry contains about 1 terabyte of imagery.  Last year the VE team added 36 terabytes of data in just one release.  I’m sure that Google Street View similarly contains many terabytes of data.  The second challenge is that there are more degrees of freedom for positioning an image on earth.  For space there are 4 variables to solve when placing an image, 3 for rotation and 1 for scale.  On earth there are at least 7 variables – 3 for position, 3 for rotation and 1 for focal length, and this ignores other terrestrial factors like variation over time and lighting changes.

These difficult problems are slowly being overcome though.  You already see Microsoft and Google systematically surveying the whole earth in a manner similar to the whole sky survey mentioned above.  The computer vision community has developed fast image recognition techniques for non-sky pictures.  There are tools for manually (not yet automatically) placing images within Google Earth or Virtual Earth 3D.  And, the Photo Tourism project (which became Photosynth) demonstrated auto-annotation of flickr images.  Over the coming years I think that we’ll see these technologies packaged up in an easy-to-use web service.

-Matt

I saw a new “I’m a PC” commercial running during American Idol this week.  I’m pretty biased, but I really like this commercial.  Having been very involved in the development of this technology, I know that under the hood it is quite complex.  What this commercial brings home for me is that through the power of software, we have managed to hide all this complexity from the user and boil it down to, in Alexa’s words, “squishing all my little pictures into one big one.”

    Video: The Rookies: Alexa, Age 7

And here are some real-world examples of novices creating panoramic photos using Windows Live Photo Gallery:

• A panoramic photo competition on a Holland America cruise ship
• A retiree, whose online name is gadgetgran, teaching her friends to make panoramas

All of this reminds me of why I work at Microsoft.  While it’s been great to get positive peer reviews on the underlying technology that we develop (for example, see the photography-related publications on Rick Szeliski's page), what really excites me is putting this technology to use in Microsoft products and enabling millions of people to do something new with their PCs.

-Matt