study

1. Image Vectorization using Gradient Meshes Jian Sun, Lin Liang, Fang Wen, Heung-Yeung Shum Microsoft Research Asia ACM SIGGRAPH 2007

2. Cutout tool Initial mesh Input image Optimized gradient mesh Reconstruction

3. Outline Introduction Background Gradient Mesh Optimized Gradient Mesh Result Conclusions

4. Introduction

5. Image Vectorization Goal : convert a raster image into a vector graphics Compact Scalable Easy to animate Requirements Vector-based contents (eg. Flash or SVG) on the Internet Vector-based GUIs used in Windows Vista

6. Gradient mesh Gradient mesh, adrawing tool of commercial vector graphics editors Tracing photograph Start adding mesh points Selecting mid value skin tone Sampling colors from face mesh to hide seam Sampling colors from photo Sampling colors from within the mesh Finished eye/eye socket http://www.creativebush.com/tutorials/mesh_tutorial.php

7. Image represented by gradient mesh gradient mesh http://www.creativebush.com/tutorials/mesh_tutorial.php

8. Image vectorization tools Adobe Illustrator, “Live Trace” Corel CoreDraw, “CorelTrace” AutoTrace, “AutoTrace” Input image Adobe, Live Trace

9. Optimized gradient mesh Blend surface colors according to the control points color as constructing surface by the control points Optimize the gradient mesh as an energy minimization problem Advantages Efficiency of use Easy to edit – modify, animation Scalability Compact representation JPEG, 37.5 KB Optimized, 7.7KB

10. Background

11. Object-based vectorization Object-based vectorization [Price and Barrett 06] Hierarchically segmentation of object and sub-objects by a recursive graph cut algorithm Subdivide meshes until the reconstruct error is below a threshold Input image Subdivision mesh

12. RaveGrid [Swaminarayan and Prasad 06] Constrained Delaunay triangulation of the edge contour set

13. cont.

14. Ardeco Automatic Region Detection and Conversion algorithm [Lecot and Levy 06] Cubic splines Each region filled with a constant color, or a linear or circular gradient

15. cont.

16. Gradient Mesh

19. Ferguson patch TA TB [Ferguson 1964] P(0)=A Basic Curve Segmentation P(1)=B

20. Ferguson patch Basic Surface Segmentation 0

21. Ferguson patch Basic Surface Segmentation 0

23. Ferguson patches are lack of Cv and Cu ! Color interpolation

24. [Wolberg and Alfy 99] Determine the smoothest possible curve that passes through its control points and satisfy monotonic constraint The seven data points are monotonically increasing in f(xi) for 0 ≦i ≦ 6, the cubic spline is not monotonic Monotonic cubic spline

25. Rendering Ferguson patches Sample color of control points Estimate Cu, Cv by Monotoic Cubic Spline algorithm Render Ferguson patches

27. Minimize E(M) min arg.

28. Solve NULL problem using LM algorithm Minimizing E(M) is a non-linear least squares (NULL) problem Energy function z: vector form of unknowns in M Levenberg-Marquardt (LM) algorithm is the most successful solver for NULL [Levenberg 44], [Nocedal and Wright 99]

31. Vector line guided optimized gradient mesh User guided vector, V Initial mesh Opt. gradient mesh with user guided vector (err. 0.5/pixel) Directly optimized gradient mesh (err. 2.5/pixel)

32. Vector line guided optimized gradient mesh w = 1/5 L Initial mesh Opt. gradient mesh with V

33. Boundary constraint The boundary of a gradient – one or more cubic Bezier spline The control points on the boundary only move along the spline Ex: control point q on the spline S in u direction

34. Results

35. Red pepper Optimized the highlight and shadow regions are reconstructed Initial gradient meshes Gradient meshes by an artist (354 patches)

36. Sculpture Optimized gradient mesh Input image Reconstruction

37. Face Optimized gradient mesh Input image Reconstruction

38. Conclusions Input image Introduce the gradient mesh as an image representation tool first Present optimized gradient mesh Limitations A fine image details and highly textured image Boundaries or topologies are too complicated Reconstructed image Optimized gradient meshes

39. END