Providing Short Videos with Dynamic Looping—Automatically

“We have developed a technique to automatically create an infinitely looping video from a short input video sequence of five to 10 seconds,” explains Hoppe, a principal researcher and manager of the Computer Graphics Group. “One can shoot a five-second video of a pretty landscape and generate an animated version of the scene to play on a PC desktop or in a slide show.”

This is the latest manifestation of an effort envisioned in 2006 by Microsoft Research Redmond’s Michael Cohen and Richard Szeliski to do a better job of “capturing the moment.” It comes on the heels of related work on Cliplets and BLINK.

“The goal,” says Hoppe, a fellow of the Association for Computing Machinery (ACM), “is to bring static photographs back to life. Last year’s work on Cliplets shares this same goal. Cliplets reanimates a portion of a scene based on user-specified regions and time intervals. The emphasis is on creative control—creating contrasting juxtapositions of frozen and dynamic content.

“In contrast, ‘automated video looping’ requires no user input. It tries to incorporate as much dynamic content as possible, so it is able to capture more scene activity, including subtle motions like water ripples and swaying grass.”

The problem is challenging. The automated solution offered by Liao, Joshi, and Hoppe enables each pixel to determine its own looping period.

“What is unique over prior work is that in the looping video, regions of pixels can have different looping periods, and these periods are found automatically as part of an optimization algorithm,” says Hoppe, a member of the SIGGRAPH 2013 Technical Papers Committee, as is Microsoft Research Redmond colleague Johannes Kopf. “It would be very tedious for a user to manually identify all these small regions and their natural loops.”

That segmentation gives users the interactive ability to adjust the dynamism of the scene. The level of dynamic activity in the scene can depend on the user’s personal taste or mood.

“The second contribution of the paper,” says Hoppe, recipient of the ACM SIGGRAPH Computer Graphics Achievement Award in 2004, “is to show that once we have created the ‘most dynamic video loop,’ we can use information from the optimization to provide the user with simple controls over dynamism in the resulting scene.

“One such control is a slider that lets the user adjust the overall level of dynamism. In essence, we are traversing a sequence of video loops, from completely static to most dynamic. The clever part is that we don’t actually have to store all these various loops. We just store the most dynamic loop, together with an ‘activation’ value per pixel that indicates when that pixel should ‘turn on’—become dynamic—as the slider moves.”

The researchers also have ensured that all regions of a scene loop well together.

“As dynamism, varies, it is necessary that small regions of pixels start or stop looping together,” Hoppe says. “For example, if only half of a branch was moving, we would see an obvious spatial discontinuity. We identify these ‘independent looping regions,’ again automatically, and let the user spatially adjust dynamism at the granularity of these regions.”

The resulting nested segmentation of static and dynamic regions are extremely compact. Such work could be applied to subtly animated desktop backgrounds and replacements for still images in slide shows or webpages. Understandably, Hoppe and his colleagues are proud of what they have achieved.

“It was an ambitious goal to find the most natural looping period of each pixel in a video using a global optimization,” Hoppe smiles. “This resulted in a new formulation that initially looked daunting, but we were able to make the problem tractable using some clever simplifications. We’re looking forward to sharing the work during SIGGRAPH to see what the community has to say.”