Magic Brush

Click to download video (opens in new tab)

We develop the MagicBrush system, a novel painting-based image search engine. This system enables users to draw a color sketch as a query to find images. Different from existing works on sketch-based image retrieval, most of which focus on matching the shape structure without carefully considering other important visual modalities, MagicBrush takes into account the indispensable value of “color” related to “shape”, and explores to make use of both the shape and color expectations that users usually have when they’re imaging or searching for an image. To achieve this, we 1) develop a user-friendly interface to allow users to easily “paint out”their colorful visual expectations; 2) design a compact feature “color-edge word” to encode both shape and color information in a organic way; and 3) develop a novel matching and index structure to support a real-time response in 6.4 million images. By taking into account both shape and color information, the MagicBrush system helps users to vividly present what they are imagining, and retrieve images in a more natural way.

Date:

July 29, 2016