Color Harmonization Techniques

Universidade Federal do Rio de Janeiro - UFRJ - Campus Cidade Universitária - Rio de Janeiro - Ilha do Fundão, CEP: 21941-972 - COPPE/PESC/LCG
Color Harmonization Survey :: A Gentle Introduction to Color Harmonization Techniques :: Seminários de Computação Gráﬁca II :: September 12, 2013
Color Harmonization Survey
A Gentle Introduction to Color Harmonization Techniques
Michel Alves dos Santos
Pós-Graduação em Engenharia de Sistemas e Computação
Universidade Federal do Rio de Janeiro - UFRJ - COPPE
Cidade Universitária - Rio de Janeiro - CEP: 21941-972
Docentes Responsáveis: Prof. Dsc. Ricardo Marroquim & Prof. PhD. Cláudio Esperança
{michel.mas, michel.santos.al}@gmail.com
September 12, 2013
Michel Alves dos Santos: Laboratório de Computação Gráﬁca - LCG Pós-Graduação em Engenharia de Sistemas e Computação - PESC

Introduction
Yellow-
orange
Yellow-
green
Red-
orange
Red-
violet
Blue-
violet
Blue-
green
Red
Green
Violet
Blue
Orange
Yellow
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
x
y
D65
500
490
480
470
460 380
700
620
600
580
560
540
520

Introduction
Yellow-
orange
Yellow-
green
Red-
orange
Red-
violet
Blue-
violet
Blue-
green
Red
Green
Violet
Blue
Orange
Yellow
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
x
y
D65
500
490
480
470
460 380
700
620
600
580
560
540
520
In this presentation we
will talk about ...

Introduction
Color
Harmonization
Yellow-
orange
Yellow-
green
Red-
orange
Red-
violet
Blue-
violet
Blue-
green
Red
Green
Violet
Blue
Orange
Yellow
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
x
y
D65
500
490
480
470
460 380
700
620
600
580
560
540
520
will talk about ...

Introduction
Color
Harmonization
Yellow-
orange
Yellow-
green
Red-
orange
Red-
violet
Blue-
violet
Blue-
green
Red
Green
Violet
Blue
Orange
Yellow
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
x
y
D65
500
490
480
470
460 380
700
620
600
580
560
540
520
will talk about ...
But ﬁrst, what is the concept of Harmony? And Color Harmony?

What’s Harmony?
The Concept of Harmony
[Form + Content] :: [Syntax + Semantics]
Harmony can be deﬁned as a pleasing arrangement of parts,
whether it be music, poetry, color, gastronomy, etc.
Appropriate structural relation to sensory perception!

What’s Color Harmony?
The Concept of Color Harmony
Itten [1960]: ‘Color Harmony means relationships on the hue wheel’

What´s the Importance of Color Harmony?
Importance of Colors and Color Harmony
Figure: The Birth of Venus (Sandro Botticelli) and Composition VII (Wassily Kandinsky). Works
that enchant with their mastery of combining tones, reﬁned aesthetic and unique harmonic sense.
◮ Highlight information and attract attention;
◮ Increase cognitive ability;
◮ Associate syntax to semantics.

Color Harmony and Color Harmonization
Harmony .VS. Harmonization
Harmony =⇒ Property :: Harmonization =⇒ Proccess
Harmonic colors are color sets which have special internal
relationships that are aesthetically pleasing to the human eye.
Color Harmonization is the process to ﬁnd the best sets of colors
which will make the image more comfortable to human visions.
Why use Color Harmonization?

Why use Color Harmonization?
Reasons for Using the Color Harmonization
◮ Because harmonic colors are pleasing to the eye;
◮ And because harmonic sets involve the human observer and provide a sense of
order and balance in the visual experience.
How can we ﬁnd the set of colors more harmonious?

Harmonic Sets
Harmonic Sets and the Colorization Process
Obtaining these harmonic sets can be boring due to the tedious work of colorization.
At this point it is necessary to use smarter approaches!

Harmonic Sets
Harmonic Sets and the Colorization Process
Obtaining these harmonic sets can be boring due to the tedious work of colorization.
Such as the Color Harmonization Techniques!

Color Harmonization Techniques
Papers on Color Harmonization
Before talking about the selected works we will perform a short
explanation of the ﬁrst contributions.

The First Contributions
Background of Color Harmonization
◮ First considerations were made by Pythagoras [≈ 500 BC], Aristotle
[≈ 320 BC], Plato [≈ 340 BC];
◮ System suitable for the mixing of colours: Leon Battista Alberti [≈
1430], Leonardo da Vinci [≈ 1490];
◮ Discoveries in the ﬁeld of the theory of harmonization: Newton,
Goethe, Young and Maxwell [≈ 1703 to 1860];
◮ Introduction of a quantitative representation of harmony: Moon &
Spencer [1944], Granville & Jacobson [1944];
◮ Modern Theory of Colours: Munsell [1969], Ostwald & Birren [1969]
and Itten [1960];
◮ Introducing a new color circle where harmony is emphasized by the
hue component: Itten [1960];
◮ Introduction of 80 harmonic schemes based on studies of Itten:
Matsuda [1995];
◮ Harmonic schemes of Tokumaru [2002] to the present day...

Chosen Works ::
1 Geometric Formulation of Classical Color Harmony, 1944
2 Color Design Support System Considering Color Harmony, 2002
3 Computational Color Harmony Based on Coloroid System, 2005
4 Color Harmonization, 2006⋆
5 Color Conceptualization, 2007
6 Color Harmonization for Videos, 2008
7 Color Design for Illustrative Visualization, 2008
8 Harmonic Colormaps for Volume Visualization, 2008
9 An Improved Method for Color Harmonization, 2009
10 Image Appearance Exploration by Model-Based Navigation, 2009
11 Multi-scale Image Harmonization, 2010
12 Color Harmonization for Augmented Reality, 2010
13 Data-Driven Image Color Theme Enhancement, 2010
14 Optimizing Photo Composition, 2010
15 Cost-eﬀective Feature Enhancement for Volume Datasets, 2010
16 Online Video Stream Abstraction and Stylization, 2011
17 ColourVis: Exploring Colour Usage in Paintings Over Time, 2011
18 Example-Based Image Color and Tone Style Enhancement, 2011
19 Color Compatibility from Large Datasets, 2012
20 Improving Photo Composition Elegantly, 2012
21 Image Composition With Color Harmonization, 2012
22 Aﬀective Image Colorization, 2012
23 Understanding and Improving the Realism of Image Composites, 2012
24 Color Harmonization Enhancement, 2012
25 Saliency-Guided Consistent Color Harmonization, 2013
1944 20132002 2006 20122009 20112005 2007 20102008

Chosen Works ::
1944 20132002 2006 20122009 20112005 2007 20102008
We will do a quick
analysis of these
works!

Chosen Works ::
1944 20132002 2006 20122009 20112005 2007 20102008
We will do a quick
analysis of these
works!
Considering the
main contributions!

Chosen Works ::
1944 20132002 2006 20122009 20112005 2007 20102008
We will do a quick
analysis of these
works!
Considering the
main contributions!
Go get it!

Chosen Works :: #1 :: Moon & Spencer, 1944
•••••••••••••••••••••••••••••••••••••

Chosen Works :: #1 :: Moon & Spencer, 1944
There is no reason why the balance point should
be limited to any particular class of colors.
Birren12
mentions applications in which a chro-
matic balance point is chosen to obtain a par-
ticular psychological effect.
The further development of colorharmony re-
N'
'4
a '-CYLINDEROFCONSTANTCHROMA
FIG. 1. The metric colorspace, showing the artesian
coordinate system (, 2
, w
3
) and the cylindrical system
(r, 0, z).
IGeorge Field, Chromatics (London, 1845).
8 A. H. Munsell, A Color Notation (Munsell Color Com-
pany, Baltimore, 1941).
DW. v. Bezold, Te Theory of Color (American edition,
Boston, 1876).
10E. BrUcke, Die Physiologie der Farben (Leipzig, 1866);
Les Couleurs (Paris, 1866).
" W. Ostwald, Die Harnionieder Farben (Leipzig, 1922);
Farbkunde (Leipzig, 1923); Color Science, translated into
English by J. S. Taylor (Vol. 1, London, 1931; Vol. II,
London, 1933).
12 F. Birren, Color Dimensions (Crimson Press, Chicago,
that one must educate himself to appreciate the
more complicated color arrangements. The Ost-
wald ideas of harmony can be applied equally
well to the Munsell colorsolid,3 though this fact
does not seem to be generally realized. Munsell
himself gave very little specific information on
harmony, though he did mention the balance of
colors about a neutral or other point and the
orderly arrangement in the color solid.
Thus technical developmentsin the production
of colorants and progress in color theory have
both helped in the advancement of color theory.
There has been a real advance in the theory of
color harmony, but this advance has been almost
entirely qualitative. The Ostwald color solid still
rests on a very insecure foundation, and it is only
during the past five years that the Munsell sys-
tem has been placed on a satisfactory scientific
basis2 with the aid of the C.I.E. system. So it is
not strange that the vague ideasof the artist have
not been translated into more scientific terms.
3. THE METRIC COLORSPACE
To make the theory of colorharmony a branch
of geometry, one must have first a metric color-
space. Such a space was developed for this pur-
pose, as noted in a previous paper.' The C.I.E.
specification of color gives an affine colorspace,
where angles in general do not have any meaning
and wheredistances in differentdirections cannot
be compared. Such a space cannot be used for the
geometrical formulation of color harmony until
a metric is introduced. A Euclidean metric was
13 M. E. Bond and Dorothy Nickerson, J. Opt. Soc. Am.
PARRY MOON AND D.
CHOSEN
COLOR
12
28I
FIG. 2. Regions of similarity and contrast in a plane
z = const. (constant Munsell value).
worked out in Munsell notation without direct
use of the C.I.E. specification.
4. POSTULATES
Before attempting any new work in this field,
one may ask himself, "What is color harmony?"
The assumption, which is nowhere stated but
which seems to be taken for granted throughout
find ve
likes. T
provide
tions t
aesthet
called
though
not con
The
and su
two po
Pleas
(1) t
unambig
(2) c
represen
simple
These
fundam
observe
there s
Applied
means
togethe
were in
50
Metric Colorspace and Regions of Similarity
Roots of Mathematical Foundation of Color Harmony

Chosen Works :: #2 :: Tokumaru et al., 2002
•••••••••••••••••••••••••••••••••••••
Abstract - Color design is very important for a product design.
In this paper, we propose a system which aims to support such a
color design. Proposed system is composed of 5 parts, such as the
part which evaluates the harmony of colors, the color combining
part, color scheme image judging part, image word output part
and lastly image comparison part. First, the system requires the
user to input a color and his preferring image of color scheme
including his inputting color with image keyword. Next, the system
selects colors from the Munsell color database, which are in
harmony with the color inputted into the system . Then, the system
builds color schemes to combine the color inputted by the user
with the colors selected from the database by the system. Finally,
images of the color schemes are evaluated and outputted the color
combinations whose images accord with the image keyword which
the user inputs into the system. Experimental result shows that
effective judgments of color harmony and color image are executed
and we can get some good color schemes by the system.
Color Design Support System Considering Color Harmony
Masataka Tokumaru
Faculty of Engineering
Kansai University
toku@ipcku.kansai-u.ac.jp
Noriaki Muranaka
Kansai University
muranaka@ipcku.kansai-u.ac.jp
Shigeru Imanishi
Kansai University
imanishi@k3ki.densi.kansai-u.ac.jp
words) by neural networks[2][3]. This method is convenient
because the system does not require any rules for the relationship
between the input and the output, but it is difficult to correct
parts of the system and to introduce technical knowledge into
the system because it is difficult to grasp the internal state of
the system.
On the other hand, many researchers who study color science
proposed methods and logics to analyze color schemes and color
images. However a lot of them are not a computer system and
they require estimation and interpolation by user about the part
where investigation isn't accomplished. Then we compose a
computer system which automatically designs color scheme
whose harmony is well and whose image corresponds with
user’s preferring color image[6]~[8].
This paper is comprised of 6 chapters. We show the outline
of the system in the next chapter. The system adopts following

Chosen Works :: #2 :: Tokumaru et al., 2002
with fuzzy membership functions. Our system judges a color
Fig. 1 Flowchart of the proposed system.
Harmonized Color
Combinations User wants
Fuzzy Rules
and
Membership Functions
Harmonized Color
Combinations
Color Image
Scale
select the image of
the color combination
in the Color Image Scale
KeyWord
Image Scale
Matchingselect the image word of
the color combination in
the KeyWord Image Scale
Image
Munsell Color Database
evaluate harmony of
the color combination
Fuzzy Rules
and
Membership Functions
Hue Components Tone Components
Color
User
compare the images
Image Word
Fig. 2 Transform The Munsell Color Space
into The Hue and Tone Distribution.
Value
Chroma
Hue
Hue
Value
Chroma
Type i Type V Type L Type I
Type T Type Y Type X Type N
Deﬁnition of Tokumaru Templates

Chosen Works :: #3 :: Neumann et al., 2005
•••••••••••••••••••••••••••••••••••••
Computational Color Harmony based on Coloroid System
László Neumann†
, Antal Nemcsics‡
, and Attila Neumann§
†Grup de Gràﬁcs de Girona, Universitat de Girona, and Institució Catalana de Recerca i Estudis Avançats, ICREA, Barcelona, Spain
‡Budapest University of Technology and Economics, Hungary
§Institute of Computer Graphics and Algorithms, Vienna University of Technology, Austria
†lneumann@ima.udg.es, ‡nemcsics.antal@axelero.hu, §aneumann@cg.tuwien.ac.at
(a) (b)
Figure 1: (a) visualization of the overall appearance of a dichromatic color set with ‘caleidoscope’ option of the Color Plan
Designer software and (b) interactive color selection of a dichromatic color set in multi-layer mode, applying rotated regular

Figure 3: A cylindrical projection of the continuous limit-
color curve of the Coloroid
Deﬁnition of Coloroid System
Circle of 48 Limit-colors

Figure 14: A dichromatic scene, where colors of diffuse
parts build a harmonic set. It was used in a BRDF study

Chosen Works :: #4 :: Cohen-Or et al., 2006
•••••••••••••••••••••••••••••••••••••
Color Harmonization
Daniel Cohen-Or Olga Sorkine Ran Gal Tommer Leyvand Ying-Qing Xu
Tel Aviv University∗ Microsoft Research Asia†
original image harmonized image
Figure 1: Harmonization in action. Our algorithm changes the colors of the background image to harmonize them with the foreground.
Abstract
Harmonic colors are sets of colors that are aesthetically pleasing
in terms of human visual perception. In this paper, we present a
method that enhances the harmony among the colors of a given
photograph or of a general image, while remaining faithful, as much
as possible, to the original colors. Given a color image, our method
finds the best harmonic scheme for the image colors. It then allows
a graceful shifting of hue values so as to fit the harmonic scheme
while considering spatial coherence among colors of neighboring
colors are sets of colors that hold some special internal relation-
ship that provides a pleasant visual perception. Harmony among
colors is not determined by specific colors, but rather by their rel-
ative position in color space. Generating harmonic colors has been
an open problem among artists and scientists [Holtzschue 2002].
Munsell [1969] and Goethe [1971] have defined color harmony as
balance, in an effort to transfer the concept of color harmony from
a subjective perspective to an objective one. Although currently
there is no formulation that defines a harmonic set, there is a con-
sensus among artists that defines when a set is harmonic, and there

i type V type L type I type
T type Y type X type N type
Harmonic Templates on the Hue Wheel
The templates may be rotated by an arbitrary angle

(a) (b) (c) (d) (e)
Figure 3: Overview of the color harmonization process. (a) The original image. (b) The hue histogram of the image before and after
harmonization. The top histogram refers to the original image, with best-ﬁtting I-type template superimposed. The bottom histogram shows
the hues shifted to match the template sectors. (c) The resulting harmonized image. Note that the harmonization tried to preserve the original
colors as much as possible. (d) The user manually rotates the template (top), and the hues are shifted accordingly (bottom). (e) The result of
the manual choice of template orientation.
Figure 4: Manual choice of harmonic schemes. The original image and its hue histogram are displayed in the left column. Harmonic templates
with various orientations result in different palettes.

1
2
(a) input (b) nearest sector (c) optimized sector (d) hue histogram
Figure 5: A naive implementation associates colors with their nearest sector, yielding the artifacts in the middle image (b). Using optimized
graph-cut labeling alleviates the problem, producing a more coherent result (c). The hue histogram and the harmonic scheme are shown in (d),
to visualize the source of the problem: in the naive implementation, hues which are nearly equally close to both sectors of the template may
“choose” their sector arbitrarily, and this causes color discontinuities in the resulting image. Note that when the optimization (c) is applied,
two pixels with exactly the same color are not necessarily shifted to the same sectors, since we take into account the spatial relation among
pixels.
F(X,(m,α)) = ∑
p∈X
H(p)−ETm(α)(p) ·S(p),
H′
(p) = C(p)+
w
2
(1−Gσ ( H(p)−C(p) )) ,

original image harmonization (mirror-L template) harmonization (V template) harmonization (Y template)
(a) original (b) harmonization result (X template) (c) harmonization result (mirror-L template)
Some Applications of Color Harmonization Technique
Useful tool for designing posters, presentations, web sites and other kinds of combined imagery

Chosen Works :: #5 :: Hou & Zhang, 2007
•••••••••••••••••••••••••••••••••••••
Color Conceptualization
Xiaodi Hou
Department of Computer Science and
Engineering, Shanghai Jiao Tong University
No.800, Dongchuan Road
Shanghai, China, 200240
http://bcmi.sjtu.edu.cn/~houxiaodi
Liqing Zhang
Department of Computer Science and
Engineering, Shanghai Jiao Tong University
No.800, Dongchuan Road
Shanghai, China, 200240
zhang-lq@cs.sjtu.edu.cn
ABSTRACT
In this paper, we propose a method to manipulate colors of
an image. Based on a library of natural color images, our
system evolves several prototypes of color distribution of the
library, which we call “color concepts”. By applying these
color concepts on an input image, a user can easily change
the mood of image colors in a global manner. Our results
of photographs and paintings indicate that this method is
capable of high-quality color manipulations.
Categories and Subject Descriptors
I.4.8 [IMAGE PROCESSING AND COMPUTER VI-
SION]: Scene Analysis - Color
General Terms
Algorithms, Design, Experimentation
Keywords
Color Concept, Color Transfer, Scene Analysis
“Forest” conceptualized image“Autumn” conceptualized image
The input images are applied to “spring” and “autumn” concepts respecƟvely.
The leŌ part of each image displays the original input, while the right part is
the output of color conceptualizaƟon.
Figure 1: Examples of color conceptualization.
enhancement. By ﬁtting the color histogram of a harmonic
scheme, incongruent colors can be replaced by colors that
satisfy established harmonic rules. However, color harmo-
nization is a full automatic approach. A user cannot use it
to edit colors based on his/her subjective ideas.

“Forest” conceptualized image“Autumn” conceptualized image
The input images are applied to “spring” and “autumn” concepts respecƟvely.
The leŌ part of each image displays the original input, while the right part is
the output of color conceptualizaƟon.
Examples of Color Conceptualization

Coast
Warm
Cold
(a) input image (b) “Forest” conceptualized image

Chosen Works :: #6 :: Sawant & Mitra, 2008
•••••••••••••••••••••••••••••••••••••
Color Harmonization for Videos
Nikhil Sawant Niloy J. Mitra
Dept. of CSE, Indian Institute of Technology, Delhi
{nikhilus85,niloym}@gmail.com
Abstract
Color harmonization is an artistic technique to adjust
the colors of a given image in order to enhance their visual
harmony. In this paper, we present a method to automati-
cally improve the color harmony of images. Harmonization
is performed using a carefully designed optimization in the
hue space, while keeping the saturation and intensity com-
ponents unchanged. Finally, for videos, we pose the prob-
lem as an efficient joint optimization in space and time, thus
minimizing flickering or visual artifacts in the harmonized
output video. We report the performance of our algorithm
on a variety of test images and video sequences.
1. Introduction
tury, Newton gave us the first color wheel. Subsequently,
Maxwell came up with important contributions in the field
of colors [3, 4]. Itten [2] was the first to introduce the color
wheel based on hue information. He proposed a scheme
of color harmony based on relative positions of colors on a
color wheel. He introduced two (complementary colors),
three (equilateral triangle), four (square), six (hexagon)
color harmony schemes. Tokumaru [5] extended this work
for harmony evaluation. He also introduced template-based
harmonization schemes. Such templates attempt to quantify
our perception and understanding of matching colors, al-
lowing us to solve the color harmonization problem, whose
goal is to improve the visual appeal of an image, in an opti-
mization framework.
Our work is inspired by the recent work on image color
harmonization by Cohen-Or and colleagues [1]. They used
the templates proposed by Tokumaru et al. [5] to harmonize
the images along with a graph cut method to ensure contigu-

Figure 1: (Left) Input image. (Right) The color harmonized
image is visually more pleasing. The output depends on the
type of hue-template [5] used, template X in this case.

template i template V template L template I
template T template Y template X template N
Figure 2: Color harmonization can be seen as ﬁtting or ap-
proximating hue information using harmonizing templates,
which were originally proposed by Tokumaru et al. [5].
{i, V , L, I, T, Y , X, N}

Figure 6: A video sequence, when color harmonized by simply processing each frame individually, results in flickering.
Figure 8: Our joint space-time optimization approach results in a flicker-free color harmonized video. A naive approach on
the same sequence results in artifacts as seen in Figure 6.
The method used in this work avoids flicker and other artifacts

Chosen Works :: #7 :: Wang et al., 2008
•••••••••••••••••••••••••••••••••••••
Color Design for Illustrative Visualization
Lujin Wang, Joachim Giesen, Kevin T. McDonnell, Member, IEEE, Peter Zolliker, Member, IEEE,
and Klaus Mueller, Senior Member, IEEE
Abstract—Professional designers and artists are quite cognizant of the rules that guide the design of effective color palettes,
from both aesthetic and attention-guiding points of view. In the field of visualization, however, the use of systematic rules
embracing these aspects has received less attention. The situation is further complicated by the fact that visualization often uses
semi-transparencies to reveal occluded objects, in which case the resulting color mixing effects add additional constraints to the
choice of the color palette. Color design forms a crucial part in visual aesthetics. Thus, the consideration of these issues can be of
great value in the emerging field of illustrative visualization. We describe a knowledge-based system that captures established
color design rules into a comprehensive interactive framework, aimed to aid users in the selection of colors for scene objects and
incorporating individual preferences, importance functions, and overall scene composition. Our framework also offers new
knowledge and solutions for the mixing, ordering and choice of colors in the rendering of semi-transparent layers and surfaces. All
design rules are evaluated via user studies, for which we extend the method of conjoint analysis to task-based testing scenarios.
Our framework’s use of principles rooted in color design with application for the illustration of features in pre-classified data
distinguishes it from existing systems which target the exploration of continuous-range density data via perceptual color maps.
Index Terms—Color design, volume rendering, transparency, user study evaluation, conjoint analysis, illustrative visualization
1 INTRODUCTION
Recent years have seen multifarious efforts to better integrate and
exploit properties of human visual perception into visualization de-
sign. Illustrative rendering techniques have been developed that ren-
der the scene at different levels of abstractions [30] and detail [32] or
in different rendering styles [5], with applications ranging from in-
formation visualization [20] to full-scale volume rendering. In these
approaches, the levels of abstraction are most often controlled by a
task- or object-dependent importance parameter [31]. Another per-
ception-motivated strategy is to guide viewer attention to salient
features [16]. Color can play a major role in these particular efforts.
However, there is no illustrative rendering system so far that incor-
porates rules from color design directly into the visualization engine.
Yet, the working scenario of graphics designers is quite similar to
Professional designers and artists are quite cognizant of rules that
guide the design of color palettes, not only from an aesthetic point of
view but also from an attention-guiding, salient one. Likewise, visu-
alization is not only concerned with providing a pleasing image – it
also has a mission, that is, to help users gain quick and accurate in-
sight into the visualized data [34]. Our framework captures these
known color design rules into a knowledge-based system, which then
combines them with scene analysis, user preferences, and importance
functions to derive appropriate colorizations. These latter considera-
tions are typically not embodied in currently existing frameworks.
The scope of our system is image-centric as well as volume visu-
alization. Both often use semi-transparencies to reveal occluded ob-
jects. Here, color mixing effects and the perceived depth-ordering of

Fig. 12: Illustrative visualizations of a six-dimensional dataset using illustrative parallel coordinates. (a) Ideal visualization with appropriate
weightings and color choices, and the use of the local model in overlapping areas. (b) Improper weightings are employed. The blue cluster
no longer seems to be in front. (c) The use of improper weightings and the disabling of the local model results in a confusing visualization.
(a) (b) (c)
α
α
in the same compositing order, but with opacities assigned in the
Fig. 8. Color mixing in a volume rendered body. (a) Cyan and
e two
r
opacity
tes
am-
es
ont
color
heavily
po-
ted
no false
a
prevent
Fig. 6. Colorizations of Transmission Electron Microscopy (TEM)
data. From left to right, classes A, B, or C (the small oblong,
elliptical cells) were most important.
stem suggests hues
cyan
e same
provides guidelines on
of each class is
r the former,
importance, 0.7 for
age size)
e area to
meter can
portant classes
portance
a higher
portance)
ations
varying

Chosen Works :: #8 :: Wang & Mueller, 2008
•••••••••••••••••••••••••••••••••••••
Harmonic Colormaps for Volume Visualization
Lujin Wang and Klaus Mueller
Center for Visual Computing, Computer Science, Stony Brook University, NY
Abstract
Color design forms a crucial part in visual aesthetics, and it has been shown that a visually aesthetic visuali-
zation will be looked at more carefully. An important role plays here the choice of a colormap that is com-
posed of harmonic colors. This paper presents an interface that allows users to choose harmonic colors in
volume visualization applications. In addition, it describes mechanisms by which non-harmonic colormaps
can be converted to harmonic ones, but keeping lightness constant to preserve the original contrast relation-
ships. Finally, we also show how harmonic colors can be used for the highlighting of important volume fea-
tures.
Categories and Subject Descriptors (according to ACM CSS): I.3.3 [Computer Graphics]: Display Algorithms.
1 Introduction
In visualization and volume graphics, image and volume
datasets typically come in form of 2D and 3D arrays of
scalar densities, which are mostly obtained via simula-
accomplishing perceptually uniform color scales (color-
maps). The research presented in [9] specifically ad-
dressed the lightness component of these colormaps,
devising a simple method to specify such colormaps on
commodity non-calibrated displays. Other work [5] de-
IEEE/ EG Symposium on Volume and Point-Based Graphics (2008)
H.-C. Hege, D. Laidlaw, R. Pajarola, O. Staadt (Editors)

Figure 2: Color harmonization without lightness-preservation (Buckyball dataset). (a) Original rendering with
corresponding hue wheel; (b) Rendering with colormap harmonization using just the hue shifting (hue wheel on the
right); (c) Rendering with hue harmonization, but with our lightness-preservation scheme applied after hue shifting
(same hue wheel than (b)).
(a) (b) (c)
scheme to preserve lightness after hue shifting. The hue Then, when the user employs the hue wheel for a harmo-
Figure 5: Volume rendered segmented frog dataset. (a)
(a) (b)
(a) (b)

Figure 6: Composition of shockwave and jet dataset rendering.(a) Non-harmonic trans-
fer function; (b) Harmonic transfer function (T type); (c) Harmonic transfer function (V
type).
(a) (b) (c)
the back-
e mixture.
all assume
control the
e of color
ransparent
trict our
achieving
har-
variety of
the capa-
thod for
me visuali-
we show
ered frog
-harmonic
(see the

Figure 6: Features are highlighted one by one: (a) All features are rendered in neutral colors, no feature
is highlighted, (b)-(d) The outside feature is highlighted by increasing the vividness of its color gradually,
while preserving the lightness, (e) The vividness of the outside feature decreases, (f)-(h) The inside feature
is highlighted gradually, (i)-(j) The vividness of the inside feature decreases.

Chosen Works :: #9 :: Huo & Tan, 2009
•••••••••••••••••••••••••••••••••••••
An Improved Method for Color
Harmonization
Xing Huo
Hefei university of Technology, Math Department
Hefei university of Technology, Computer and
Information institute
Hefei,China
Jieqing Tan
Hefei university of Technology, Math Department
Hefei university of Technology, Computer and
Information institute
Hefei,China
Abstract—Harmonic colors are color sets which have special
internal relationships that are aesthetically pleasing to the
human eye. Color harmonization is to find the best sets of
colors which will make the image more comfortable to human
visions. An improved technique of best scheme searching for
automatic color harmonization is proposed in this paper. By
considering the predominant color in image and its
contribution to the harmonic image, this paper formulates a
ratio method to find the best scheme. Moreover this paper
suggests some change to the strategy of color shifting, thus
makes the algorithm more efficient. In addition, this paper
utilizes the conventional technique but produces better results
through a set of simple processing. It is shown that the
efficiency and accuracy of the new technique is significantly
better than the traditional technique.
Keywords-color harmonization;hue;graph-cut;color space
I. INTRODUCTION
Color harmony originated from the theory of white light
al.[5]
presented a system for designing colors based on
several color rules, and applied them to a graphical user
inter-face (GUI) building tool, Daniel Cohen[6]
proposed a
new method which could automatically harmonizes a given
color palette through an optimization process, and provides a
means to automatically recolor an arbitrary image. Color
harmony also gives some nudges in color transfer[7,10,11]
.
Our method is based on the work of Cohen, it can cope
with the image with rich colors ,find the most suitable
scheme automatically and keep some regions intact. The
method proposed in this paper improved the optimization
and recoloring method of Cohen’s work. It takes the
predominant hue of the image into account as well as a slight
change in the recoloring function which minimizes the time
cost.
II. HARMONIC SCHEMES
The harmonic schemes in this paper are brought forth by
Matsuda[4]
.There are eight schemes in Figure 1, each scheme

(a) Original Image (b) Result Image(α = 54) (c) Result Image(α = 60)
(d) Hue Histogram of
Original Image(α = 54,60)
(e) Hue Histogram of
Harmonized Image (α =54)
(f) Hue Histogram of
Harmonized Image (α =60)
Figure 2 Result Images and Hue Histograms by Cohen and Improved algorithm
Figure 3. Harmonized background image in respect to foreground

TABLE I. COMPARE COHEN’S ALGORITHM WITH IMPROVED ALGORITHM
Cohen’s
method
Template i V L I T Y X
F-template 2224714 1072405 493496 422310 189772 220565 40446
Orientation
angle
54
Time cost 33984 ms
Improved
method
Template i V L I T Y X
F-template 2514050 1749780 564215 555936 247651 396070 46311
Orientation
angle
60
Time cost 79 ms
hhh snF •=α
work :
)))()((1(
2
)()('
pCpHE
w
pCpH −−+= σ
(3)
σ
)),(,( αmXF
∈
•−=
Xp
T pSpEpHmXF m
)()()()),(,( )(αα
Where X is the given image, p stands for any pixel in th
2
2
2
)( σ
σ
t
etE
−
=
σ

Chosen Works :: #10 :: Shapira et al., 2009
•••••••••••••••••••••••••••••••••••••
Image Appearance Exploration by Model-Based Navigation
L. Shapira1,
and A. Shamir2
and D. Cohen-Or1
1Tel-Aviv University, Israel
2Interdisciplinary Center Herzliya, Israel
Abstract
Changing the appearance of an image can be a complex and non-intuitive task. Many times the target image colors
and look are only known vaguely and many trials are needed to reach the desired results. Moreover, the effect of
a specific change on an image is difficult to envision, since one must take into account spatial image considera-
tions along with the color constraints. Tools provided today by image processing applications can become highly
technical and non-intuitive including various gauges and knobs.
In this paper we introduce a method for changing image appearance by navigation, focusing on recoloring im-
ages. The user visually navigates a high dimensional space of possible color manipulations of an image. He can
either explore in it for inspiration or refine his choices by navigating into sub regions of this space to a specific
goal. This navigation is enabled by modeling the chroma channels of an image’s colors using a Gaussian Mixture
Model (GMM). The Gaussians model both color and spatial image coordinates, and provide a high dimensional
parameterization space of a rich variety of color manipulations. The user’s actions are translated into transfor-
mations of the parameters of the model, which recolor the image. This approach provides both inspiration and
intuitive navigation in the complex space of image color manipulations.
Categories and Subject Descriptors (according to ACM CCS): Computer Graphics [I.3.6]: Interaction Techniques—
Image Processing [I.4.3]: Enhancement—Image Processing [I.4.9]: Applications—Image Processing [I.4.10]: Im-
age Representation - Multidimensional—

Chosen Works :: #10 :: Shapira et al., 2009
L. Shapira & A. Shamir & D. Cohen-Or / Image Appearance Exploration by Model-Based Navigation

Chosen Works :: #11 :: Sunkavalli et al., 2010
•••••••••••••••••••••••••••••••••••••
Multi-scale Image Harmonization
Kalyan Sunkavalli∗
Harvard University
Micah K. Johnson†
MIT
Wojciech Matusik‡
Disney Research, Zurich
Hanspeter Pﬁster§
Harvard University
(a) Source / Target (b) Seamless Cloning (c) Harmonization (d) Close-ups
Figure 1: In traditional image compositing (a) a user applies geometric transformations to a source image (top) and inserts it into a target
image (bottom). Tools such as the Photoshop Healing Brush use gradient domain compositing to ensure that the composite is seamless (b) but
the inconsistencies between the two images, make the result look unrealistic: the inserted face is much smoother than the rest of the image.
Our method “harmonizes” the images before blending them, producing a composite that is seamless and realistic (c). The close-up images
(d) compare traditional gradient-domain blending (top) to the harmonized result (bottom).
Abstract
Traditional image compositing techniques, such as alpha matting
and gradient domain compositing, are used to create composites
that have plausible boundaries. But when applied to images taken
1 Introduction
Combining regions of multiple photographs or videos into a seam-
less composite is a fundamental problem in many vision and graph-
ics applications, such as image compositing, mosaicing, scene com-

Chosen Works :: #11 :: Gruber et al., 2010
(a) Source / Target (b) Inserting source into target (c) Close-ups (d) Inserting target into source (e) Close-ups
Figure 10: In this example, the source ((a) top) is smooth while the target ((a) bottom) is noisy. When inserting the source into the target,
harmonization adds noise to produce a realistic composite (b). Conversely, when the target image is inserted into the source, harmonization
removes most of the noise to match the images (d).
(a) Source / Target (b) Seamless Cloning (c) Harmonization (d) Close-ups

Smooth
Histogram
Matching
Harmonized
Pyramid
Final Composite
Alpha, Seamless
BoundariesNoise Pyramid
Target Pyramid
Source Pyramid
Smooth
Histogram
Matching
Noise Image
Target Image
Source Image
Pyramid
Compositing
Multi-scale
Harmonization
Noise
Matching
Figure 2: An overview of the Multi-scale Image Harmonization
framework. The input source and target images, and a uniform ran-

•••••••••••••••••••••••••••••••••••••
Color Harmonization for Augmented Reality
Lukas Gruber1
, Denis Kalkofen2
and Dieter Schmalstieg3
Graz University of Technology
ABSTRACT 2 REAL-TIME COLOR SHIFTING
In this paper we discuss color harmonization for Augmented
Reality. Color harmonization is a technique used to adjust the
combination of colors in order to follow aesthetic guidelines.
We implemented a system which is able to harmonize the
combination of the colors in video based AR systems. The
presented approach is able to re-color virtual and real-world
items, achieving overall more visually pleasant results. In order
to allow preservation of certain colors in an AR composition, we
furthermore introduce the concept of constraint color
harmonization.
To achieve real-time performance, we implemented the color
shifting on the GPU using pre-computed look-up tables. We
identify and execute the necessary color shifts on a per pixel
basis. Our algorithm consists of the following steps:
1. Pre-compute color shift for all templates and orientations
and store in a look-up table (off-line)
2. Select harmonization template T and its orientation θ,
according to [1]
3. Select split border between template sectors, according to
[3]
4. Shift the hue value of every pixel, using a pixel shader
1 INTRODUCTION
For each color template T, we compute a single look-up table
before the application is started. This color template look-up
table (CT-LUT) holds the results of each possible color shift, for
each hue value. We set the granularity to 1 degree of arc (0 to
359), resulting in a texture dimension of 360 by 360 texel. Each
texel encodes the mapping result for a single hue in a single
orientation of the template. The different mapping directions for
a single hue have been assigned to different color channels of
the texture. At runtime, a template and an orientation and its
mapping direction is chosen for an image according to [1].
Finally, a simple texture lookup defines the new color per pixel.
In Augmented Reality (AR) oftentimes virtual objects must be
generated without any knowledge about the real world
environment in which they are going to be deployed. This is
particularly relevant for outdoor AR applications (e. g., Layar4
),
where neither the application developer nor the content creator
have reasonable control over the real world background, which
will be blended with the virtual content. Even if the knowledge
about the visual properties of the real world surroundings is
available, the dynamics of the real world environment in all but
the most constrained laboratory conditions make it at least very
difficult to ensure that virtual content will always fit into its real
world environment. It is therefore desirable to investigate online
composition algorithms that can adjust the virtual and real image
components to yield a better visual match.
3 FRAME COHERENT COLOR SHIFTING
Frame coherent color harmonization for AR differs strongly
from offline frame coherent color shifting as discussed by
Sawant et al. [3]. Firstly, in an interactive system it is impossible
to process upcoming frames and secondly, AR systems have to
Among the known properties of appealing images, the
combination of colors is accepted as one of the most influential

Figure 1. Histogram scaling versus template search restriction. (A) Restricting the template search to only constrained elements may heavily
change the appearance of the entire visualization. The environment in this example consists of a single constrained object (the red stop
sign) which appears in a single color. By restricting the template search to only the hue values of the stop sign, the hue values of the
entire environment will shift towards red. (B) By scaling the histogram at hue values of constrained pixels we are able to compute the
template and its orientation taking into account the remaining part of the imagery. This technique is able to harmonize the environment
using fewer modifications. Notice the slight changes of hue values on the formerly blue arm of the LEGO figure riding the motor-bike.
This will lead to an increased cognitive load, making the AR sign (Figure 1A).
To achieve real-time performance, this technique implements the color
shifting on the GPU using pre-computed look-up tables.

•••••••••••••••••••••••••••••••••••••
Data-Driven Image Color Theme Enhancement
Baoyuan Wang ∗∗
Yizhou Yu†∗
Tien-Tsin Wong‡
Chun Chen∗
Ying-Qing Xu§
∗
Zhejiang University †
University of Illinois at Urbana-Champaign
‡
The Chinese University of Hong Kong §
Microsoft Research Asia
Figure 1: An example of image color theme enhancement using our method. Left: Original image; middle: recolored result with the
“nostalgic” color theme; and right: recolored result with the “lively” color theme.
Abstract
It is often important for designers and photographers to convey or
enhance desired color themes in their work. A color theme is typ-
ically defined as a template of colors and an associated verbal de-
scription. This paper presents a data-driven method for enhancing a
desired color theme in an image. We formulate our goal as a unified
optimization that simultaneously considers a desired color theme,
texture-color relationships as well as automatic or user-specified
Psychological studies confirm the strong associations between col-
ors and semantic themes. For instance, “graceful” is often associ-
ated with a combination of pinkish and purple colors while “sad”
is typically defined using a combination of blue and grey colors.
In general, a color theme is a template of colors and an associated
verbal description. Given a color theme, our goal is to perform
color theme enhancement of an image, which seeks a global trans-
formation of the colors in the original image so that it maintains
the realism of natural images while the new color composition is

(a) input image (b) soft segmentation
(h) recolored images(g) color histograms for textures
O
ptim
ization
solver
TextureClustering
Build
H
istogram
s
(e) training examples
.
.
.
Theme
Database
(c) color themes
from communities
Texture
Library
(d)
(f)
...
...
Figure 3: The overall pipeline of our framework. Each sub-image in (b) is an influence map of a soft segment. Each soft segment finds the most
relevant texture class in (f) and adopt the corresponding color histogram in (g). The histograms in (g) are further converted into continuous
probability density distributions using Gaussian Mixture Model(GMM). The user needs to select a desired color theme from (d), then our
optimization solver takes the prior knowledge (color probabilities), the desired color theme and the image segments into consideration to
generate final recolored images in (h).

Figure 6: Left: an input image and a color theme. Middle: image generated by the greedy initial assignment in section 5.1, without further
optimization. Right: ﬁnal image with further optimization.
Figure 7: Left: an input image and a target color theme. Middle: result image without E3. Right: result image with E3.
Examples of Harmonization Using Color Themes

Figure 12: A variety of images and their recolored ones with different color theme enhancements. The ﬁrst column shows the original images.

Chosen Works :: #14 :: Liu et al., 2010
•••••••••••••••••••••••••••••••••••••
Optimizing Photo Composition
Ligang Liu1†
Renjie Chen1
Lior Wolf2
Daniel Cohen-Or2
1Zhejiang University, China
2Tel-Aviv University, Israel
(a) (b) (c) (d)
0
RT DA VB SZ
1
0.2
0.4
0.6
0.8
Sum
(a)
(b)
(c)
Figure 1: Optimizing the aesthetics of the original photograph in (a) by our approach leads to the new image composition
shown in (c). (b) shows the cropping result of the approach of [Santella et al. 2006]. The aesthetic scores are shown in (d).
Our result in (c) obtains higher aesthetic score than (a). RT(rule of thirds), DA(diagonal), VB(visual balance), and SZ(region
size) are components of the objective function.
Abstract
Aesthetic images evoke an emotional response that transcends mere visual appreciation. In this work we develop
a novel computational means for evaluating the composition aesthetics of a given image based on measuring
several well-grounded composition guidelines. A compound operator of crop-and-retarget is employed to change
the relative position of salient regions in the image and thus to modify the composition aesthetics of the image. We
propose an optimization method for automatically producing a maximally-aesthetic version of the input image. We
validate the performance of the method and show its effectiveness in a variety of experiments.
Categories and Subject Descriptors (according to ACM CCS): I.3.3 [Computer Graphics]: Picture/Image

Chosen Works :: #14 :: Liu et al., 2010
(a) (b) (c) (d)
0
RT DA VB SZ
1
0.2
0.4
0.6
0.8
Sum
(a)
(b)
(c)
Figure 1: Optimizing the aesthetics of the original photograph in (a) by our approach leads to the new image composition
shown in (c). (b) shows the cropping result of the approach of [Santella et al. 2006]. The aesthetic scores are shown in (d).
Our result in (c) obtains higher aesthetic score than (a). RT(rule of thirds), DA(diagonal), VB(visual balance), and SZ(region
size) are components of the objective function.
(a) (b) (c) (d)
0
RT DA VB SZ
1
0.2
0.4
0.6
0.8
Sum
(a)
(b)
(c)
(a)
(b)
(c)
RT DA VB SZSum
0
1
0.2
0.4
0.6
0.8
Figure 7: Results of aesthetic composition. (a) The original images; (b) an arbitrary cropping frame of (a); (c) the aesthetic
composition result by our approach; (d) the aesthetic scores of (a),(b),and (c).

Chosen Works :: #15 :: Díaz et al., 2010
•••••••••••••••••••••••••••••••••••••
Cost-effective Feature Enhancement for Volume Datasets
J. Díaz1
, J. Marco2
and P. Vázquez1
1MOVING Group, Universitat Politècnica de Catalunya
2GESSI Group, Universitat Politècnica de Catalunya
Abstract
Volume models often show high complexity. Local details and overall shape may sometimes be difﬁcult to perceive.
Unsharp masking techniques improve the perception of those small features by increasing the local contrast. In
this paper we present a simple and fast method for feature enhancement based on 3D mipmaps. In contrast to other
approaches, in addition to increasing luminance on the feature details, we also darken the valleys of the volume
thus increasing local contrast and making neighboring details more visible. Our approach is fast and simple, with
small memory requirements thanks to the use of 3D mipmaps. We also propose a color selection strategy, based
on harmonic colors, that further enhances the salient features without abrupt or uncomfortable color changes.
Categories and Subject Descriptors (according to ACM CCS): I.3.3 [Computer Graphics]: Picture/Image
Generation–Display Algorithms
1. Introduction
Contrast enhancement is a popular 2D image processing tool
that improves the image appearance. Concretely, it helps
users to understand complex models by emphasizing small
features. Several experiments have proven that users pre-
• The method is simple and fast, thus maintaining high
framerates for complex models.
• The memory consumption is limited: Auxiliary data struc-
tures consist of a set of 3D mipmaps of the volume
dataset, leading to small memory requirements as com-
pared with the original model.

Chosen Works :: #15 :: Díaz et al., 2010
(a) Ray Casting (b) Protrusion enhancement (c) Sinkage darkening (d) Enhanced view
Mipmap 2, γ = 1.0 Mipmap 2, γ = 1.0
Figure 1: Feature enhancement is achieved by combining a luminance increase for the protruding zones (image c) and darken-
ing for the sinking regions (image d). The ﬁnal combination is shown in Figure b.
(a) Ray-Casting (b) Unsharped
MM8, γ = 0.4
Figure 4: Feature enhancement of a volumetric model.
Ray-Casting Feature emphasis
with harmonic color
Figure 10: Harmonic color-based feature emphasis for the
pollen grain model.

Chosen Works :: #16 :: Zhang et al., 2011
•••••••••••••••••••••••••••••••••••••
Online Video Stream Stylization
Figure 1: Stylization example. 1: Original video frame; 2 - 4: Three frames of the stylized video with color scheme replacement.
Abstract1
This paper gives an automatic method for online video stream styl-2
ization, producing a temporally coherent output video stream. Our3
system transforms video into an abstract style with large regions of4
constant color and highlighted bold edges. Our system includes two5
novel components. Firstly, to provide coherent and simplified out-6
put, we segment frames, and use optical flow to propagate segmen-7
tation information from frame to frame; an error control strategy8
is used to help ensure that the propagated information is reliable.9
Secondly, to achieve coherent and attractive coloring of the output,10
we use a color scheme replacement algorithm specifically designed11
for an online video stream. We demonstrate real-time performance,12
allowing our approach to be used for live communication, video13
games, and related applications.14
1 Introduction15
Video stream processing has many applications to areas such as live
with user-guided colors, and high contrast. In comparison, Win-43
nemöller [2006] produces a different artistic style based on simpli-44
fied but smoothly shaded contents. Our cartoon-like style means45
that temporal coherence requirements are particularly strict.46
Current stylization methods fall into three categories, each hav-47
ing limitations. Some methods focus on image processing and do48
not readily generalize to video. Others use simple image filters to49
achieve real-time performance, producing simplified and smoothly50
shaded contents, but the output may lack temporal coherence if in-51
put video streams are of low-quality. Yet others require significant52
user interaction to produce high-quality artistic results, and have a53
high computational cost.54
We present a real-time system for a particular style of video styliza-55
tion while providing good coherence. Our approach benefits from56
two novel aspects:57
• a segmentation strategy which uses optical flow to propagate58
segmentation information from frame to frame, with an error59
control strategy to help ensure that the propagated information

Input Video
Edge Detection
Optical Flow
Computation
Coherent Image
Segmentation
Color Scheme
Replacement
Output Video
Importance Map
Boundary Smoothing
& Edge Overlaying
Figure 2: Pipeline. Frame segmentation uses optical ﬂow and edges as inputs, tuned by an importance map, to produce coherent regions.
The color scheme replacement step transforms region colors according to a reference video.
Stages of the Method: Harmonization is implemented on the penultimate
stage to preserve the coherence between regions

Figure 7: Video stylization results. Columns 1, 3: original video frames; columns 2, 4: corresponding stylization results.
Figure 9: Comparison of stylization effects. From left to right: source image, results using Winnem¨oller’s method, results of color quantiza-
tion, results using our method.

Chosen Works :: #17 :: Haber et al., 2011
•••••••••••••••••••••••••••••••••••••
Computational Aesthetics in Graphics, Visualization, and Imaging (2011)
D. Cunningham and T. Isenberg (Editors)
ColourVis:
Exploring Colour Usage in Paintings Over Time
Jonathan Haber, Sean Lynch, Sheelagh Carpendale
{jmhaber, sglynch, sheelagh} @ucalgary.ca
Department of Computer Science, University of Calgary, Calgary, Alberta, Canada
Figure 1: ColourVis representation of paintings by Vincent van Gogh.
Abstract
The colour palette of painters over history has been of interest to many, including: art historians, archeologists,

Chosen Works :: #17 :: Haber et al., 2011
Figure 11: Picasso paintings visualized using ColourVis.
Figure 12: Representation of 30 years of paintings from the
cubism movement.
Figure 13: ColourVis representation of Realism and Surre-
alism painting movements.
Figure 14: Comparison of the Rembrandt Harmenszoon van
Rijn / Vincent van Gogh / Pablo Picasso painting collections.

•••••••••••••••••••••••••••••••••••••
Example-Based Image Color and Tone Style Enhancement
Baoyuan Wang ∗∗
Yizhou Yu†‡
Ying-Qing Xu§
∗
Zhejiang University †
University of Illinois at Urbana-Champaign
‡
The University of Hong Kong §
Microsoft Research Asia
(a) (b) (c) (d)
Figure 1: Style enhancement results. (a) Original photo taken by iPhone 3G, (b) enhanced photo that mimics the color and tone style of
Canon EOS 5D Mark Π; (c) Original photo, (d) enhanced photo with a style learned from a photographer.
Abstract
Color and tone adjustments are among the most frequent image en-
hancement operations. We define a color and tone style as a set
of explicit or implicit rules governing color and tone adjustments.
Our goal in this paper is to learn implicit color and tone adjust-
ment rules from examples. That is, given a set of examples, each
of which is a pair of corresponding images before and after adjust-
ments, we would like to discover the underlying mathematical rela-
tionships optimally connecting the color and tone of corresponding
pixels in all image pairs. We formally define tone and color adjust-
ment rules as mappings, and propose to approximate complicated
spatially varying nonlinear mappings in a piecewise manner. The
reason behind this is that a very complicated mapping can still be
ferent photographs. As another example, it is well-known that pho-
tographs taken by different digital cameras have varying degrees of
tone and color discrepancies. This is because each type of camera
has its own built-in radiance and color response curves. We define
a tone and color style as a set of explicit or implicit rules or curves
governing tonal and color adjustments.
Manually adjusting the tone and color of a photograph to achieve
a desired style is often tedious and labor-intensive. However, if
tone and color styles can be formulated mathematically in a digital
form, they can be automatically and easily applied to novel input
images to make them look more appealing. Unfortunately, the rules
governing a tone and color style are most often not explicitly avail-
able. For instance, it is typically very hard for a photographer to
mathematically summarize the rules he uses to achieve a certain

(a) (b) (c) (d)
Figure 1: Style enhancement results. (a) Original photo taken by iPhone 3G, (b) enhanced photo that mimics the color and tone style of
Canon EOS 5D Mark Π; (c) Original photo, (d) enhanced photo with a style learned from a photographer.
Figure 4: (A) is an original photo taken by iPhone 3G while (D) is the reference photo taken by Canon 5D Mark Π. (B) is the result from an
afﬁne color mapping while (C) is the result from a quadratic color mapping. Between (B) and (C), (C) is overall visually closer to (D).
The technique has two stages: Training and Style Enhancement Stage
Uses a SVM in the ﬁrst stage for the Color and Gradient Mappings

Image
Registration
Feature
Extraction
Image Pairs
Hierarchical Feature
Sub-space Division
Train Gradient
Mappings
Train Color Mappings
Features along
edges
Hierarchical Feature
Sub-space Division
Features away
from edges
Train Binary ClassifierNon-leaf
Nodes
Leaf Nodes
Leaf Nodes
...
......
A Gradient Mapping
Tree: Ttree
A Color Mapping
Tree: Ctree
Figure 2: The training stage. Leaf nodes are colored in green while intermediate nodes are colored in yellow.
Input
Image
Gradient
Mappings
Locate
Subspaces
In Ttree
Soft
Segmentation
Color
Mappings
Output
Image
Color
Soft Blending
Tone
Optimization
Locate
Subspaces
In Ctree
Edge Pixels
All Pixels
Figure 3: Image enhancement pipeline.
Two Stages of the Technique: Training and Style Enhancement

Color Harmonization Techniques

Color Harmonization Techniques

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie Color Harmonization Techniques

Ähnlich wie Color Harmonization Techniques (16)

Mehr von Michel Alves

Mehr von Michel Alves (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Color Harmonization Techniques