No matter how good search engines are these days, people keep trying to do one better. Double that when you're talking about photos or videos. Granted there are "real" image search engines out there (sorry, Google Images doesn't count; it searches the text that's affixed to an image), but they're compute intensive and require extensive "training" to distinguish individual objects store in a huge database.

"If you're looking for images of bicycles, for instance, current algorithms have to be shown pictures of hundreds, if not thousands, of bicycles in order to be able to recognize a bicycle," explains Peyman Milanfar, a professor of electrical engineering at the University of California-Santa Cruz. To address this problem, Milanfar and grad student Hae Jong Seo developed a Detection of Human Actions from a Single Example algorithm that enables automated recognition of both objects in images and actions in videos. The software analyzes an image or video and characterizes the most important constituents of the object or action represented. It then searches for those constituents in image and video databases.

The end result is that, to Milanfar, a single photo of a bicycle at night can be used as a template to locate pictures of bicycles in full sunlight, in the foreground or the background. A bicycle is a simply a bicycle, or more precisely, a bunch of pixels. Milanfar's software examines these pixels and their relation to one another. To find actions within videos, like a man riding a bicycle, Milanfar's software completes the same procedures but incorporates the manner in which those pixel relationships move over time. The software analyzes the map of pixel relationships and determines the salient geometric features of the object or action. These components remain perceptually constant within an object regardless of image quality.

Search engines can use this algorithm to identify patterns of pixel relationships by calculating the statistical likelihood that a candidate image contains the queried object. If the template is a bicycle, the outcome consists of a series of photographs containing bicycles of all shapes and sizes, ranked in order of similarity.

"The geometry of the bicycle is recognizable by the shape of the wheels and the way they are connected to the body, for example," Milanfar said. "We compute features from an image that are very stable. They are there even if we make the object bigger or smaller, change the background, or add noise."

-- Jonathan Erickson
jerickson@ddj.com