Discussion about Image Search
Thomas Hawk (Thomas Hawk's Digital Connection: Webshots, Google, Yahoo!/Flickr, MSN and the Landscape of Image Search) and Webshots founder Narendra Rocherolle (No Soap, Radio!) have started an interesting thread on image search.
Indexing images is one of the hardest "search" challenges out there because basically all you have is a bag of bits in the form of an image file to start with. With most web content you have text and context. Even with most commercial video you have the closed captioning feed (which Google is using for indexing their new video search service). Some photos, such as those from a news wire service, have a substantial amount of metadata usually defined in something called the IPTC fields. IPTC is a standard originally developed by the Associated Press and others for news desk applications at newspapers but has become the defacto standard for annotation of images by professional photographers and is supported by the major stock houses like Corbis and Getty. Obviously a photojournalist on assignment or professional photographer adding their images to a stock agency database are much more motivated to take the time to accurately annotate their images though even then, time is an issue. If you ever take time to look at wire service images, those capturing a major event or with a famous person are well annotated, the others are often tagged with a general keyword like "atmosphere".
Searching photos by trying to recognize their content has been the dream of many a lab/university researcher and has all kinds of interesting technical challenges that have resulted in several approaches - some that work better than others. See the paper Content-based Image Search Survery for some great examples. One of those university thesis papers that has actually made it into commercial use is technology developed at MIT in the late '90's and commercialized by LTU Technologies of France. Their ImageSeeker platform is able to extract a "DNA" fingerprint for an image that is used for both searching and recognition.
Folksonomies and tagging. There certaily is a lot of buzz and excitement about "cooperative classification" or tagging a la Del.icio.us and Flickr. This will be an important piece of the overall solution though it is too early to tell if this type of tagging alone will be sufficient to create a robust enough index.
AutoTagging: Over the next few years there will be more and more metadata that will be automatically captured and associated with images as they are captured or processed. New services like GeoSnapper.com let you geotag your images after they have been taken and the GeoSnapper Mobile application works on the Motorola i860 from Nextel to automatically capture the GPS coordinates from the handset. Location means more that just lat/lon, in order for it to be a useful search attribute it needs to be translated to address and a place name. Just knowing the photographer can also a big help in using an inference approach to tagging.
So the bottom line is that image search will get better with better tagging and metadata. Some of that better metadata will come from "cooperative classification" by a community of viewers, some of it will come from an increase in the automatic capture of things like location and photographer and the rest will come from smart tagging methods that may combine inference from the known data combined with other available data sources (like a calendar for example) as well as more intelligent image recognition (like LTU's image DNA) to improve the quality and quantity of what is known about a photo.
With billions of images being captured in digital format each year this will be an important area of innovation and opportunity.
Great resource site with lots of information on how images have been managed and annotated in the past:
ControlledVocabulary.com
Some interesting papers on image search:
Content-based Image Search Survey
A Flexible Image Search Engine - Washington University
W3C Paper on RDF and inferencing
Indexing images is one of the hardest "search" challenges out there because basically all you have is a bag of bits in the form of an image file to start with. With most web content you have text and context. Even with most commercial video you have the closed captioning feed (which Google is using for indexing their new video search service). Some photos, such as those from a news wire service, have a substantial amount of metadata usually defined in something called the IPTC fields. IPTC is a standard originally developed by the Associated Press and others for news desk applications at newspapers but has become the defacto standard for annotation of images by professional photographers and is supported by the major stock houses like Corbis and Getty. Obviously a photojournalist on assignment or professional photographer adding their images to a stock agency database are much more motivated to take the time to accurately annotate their images though even then, time is an issue. If you ever take time to look at wire service images, those capturing a major event or with a famous person are well annotated, the others are often tagged with a general keyword like "atmosphere".
Searching photos by trying to recognize their content has been the dream of many a lab/university researcher and has all kinds of interesting technical challenges that have resulted in several approaches - some that work better than others. See the paper Content-based Image Search Survery for some great examples. One of those university thesis papers that has actually made it into commercial use is technology developed at MIT in the late '90's and commercialized by LTU Technologies of France. Their ImageSeeker platform is able to extract a "DNA" fingerprint for an image that is used for both searching and recognition.
Folksonomies and tagging. There certaily is a lot of buzz and excitement about "cooperative classification" or tagging a la Del.icio.us and Flickr. This will be an important piece of the overall solution though it is too early to tell if this type of tagging alone will be sufficient to create a robust enough index.
AutoTagging: Over the next few years there will be more and more metadata that will be automatically captured and associated with images as they are captured or processed. New services like GeoSnapper.com let you geotag your images after they have been taken and the GeoSnapper Mobile application works on the Motorola i860 from Nextel to automatically capture the GPS coordinates from the handset. Location means more that just lat/lon, in order for it to be a useful search attribute it needs to be translated to address and a place name. Just knowing the photographer can also a big help in using an inference approach to tagging.
So the bottom line is that image search will get better with better tagging and metadata. Some of that better metadata will come from "cooperative classification" by a community of viewers, some of it will come from an increase in the automatic capture of things like location and photographer and the rest will come from smart tagging methods that may combine inference from the known data combined with other available data sources (like a calendar for example) as well as more intelligent image recognition (like LTU's image DNA) to improve the quality and quantity of what is known about a photo.
With billions of images being captured in digital format each year this will be an important area of innovation and opportunity.
Great resource site with lots of information on how images have been managed and annotated in the past:
ControlledVocabulary.com
Some interesting papers on image search:
Content-based Image Search Survey
A Flexible Image Search Engine - Washington University
W3C Paper on RDF and inferencing
posted by MyronK @ 1:41 PM
0 Comments:
Post a Comment
<< Home