Generating Photo Manipulation Tutorials by Demonstration, SIGGRAPH 2009

Floraine Grabler, Maneesh Agrawala, Wilmot Li, Mira Dontcheva, Takeo Igarashi

Abstract

We present a demonstration-based system for automatically generating succinct step-by-step visual tutorials of photo manipulations. An author first demonstrates the manipulation using an instrumented version of GIMP that records all changes in interface and application state. From the example recording, our system automatically generates tutorials that illustrate the manipulation using images, text, and annotations. It leverages automated image labeling (recognition of facial features and outdoor scene structures in our implementation) to generate more precise text descriptions of many of the steps in the tutorials. A user study comparing our automatically generated tutorials to hand-designed tutorials and screen-capture video recordings finds that users are 20–44% faster and make 60–95% fewer errors using our tutorials. While our system focuses on tutorial generation, we also present some initial work on generating content-dependent macros that use image recognition to automatically transfer selection operations from the example image used in the demonstration to new target images. While our macros are limited to transferring selection operations we demonstrate automatic transfer of several common retouching techniques including eye recoloring, whitening teeth and sunset enhancement.

As a user demonstrates an eye recoloring portrait manipulation technique, our system records his actions in the interface and the corresponding image processing operations in the application. It then combines information from the recordings with image recognition to automatically generate a succinct, step-by-step visual tutorial that others can use to learn the technique (left). Our system can also generate content-dependent macros that automatically transfer selection operations from the example to new target images. In this case our system uses face recognition to identify the eyes and then transfers the recoloring procedure to the new target images (right).
 
 

Research Paper: [PDF]

Video

Supplemental Materials

Automatically Generated Tutorials
Macro Transfer Results

Related

Adobe recently launched Photoshop Tutorial Builder, a product that allows users to generate step-by-step tutorials from a sequence of actions recorded in the application. The resulting tutorials include text description and tool icons for the recorded actions, and can be played back either within Photoshop or via a connected iPad. Photoshop Tutorial Builder is based on this work on generating photo manipulation tutorials. Check out the video which features our collaborators, Wilmot Li and Mira Dontcheva.

Tutorial Player has been featured in the Los Altos History Museum.

We’ve also recently published a follow-on paper describing content adaptive photo manipulation macros.

This entry was posted on Tuesday, April 21st, 2009 at 4:46 am and is filed under Research and tagged with . You can follow any responses to this entry through the RSS 2.0 feed.