SlideShare ist ein Scribd-Unternehmen logo
1 von 7
Downloaden Sie, um offline zu lesen
“GrowCut” - Interactive Multi-Label N-D Image Segmentation By Cellular
                                Automata
                                           Vladimir Vezhnevets ∗             Vadim Konouchine
                                                      Graphics and Media Laboratory †
                                          Faculty of Computational Mathematics and Cybernetics
                                                         Moscow State University,
                                                             Moscow, Russia.


Abstract                                                                 4. Performs multi-label image segmentation (the computation
                                                                            time does not depend on the number of labels);
In this paper we describe a novel algorithm for interactive multi-
label segmentation of N-dimensional images. Given a small num-           5. Is extensible, allows constructing new families of segmenta-
ber of user-labelled pixels, the rest of the image is segmented au-         tion algorithms with specific properties;
tomatically by a Cellular Automaton. The process is iterative, as
the automaton labels the image, user can observe the segmentation        6. Is truly interactive - the user observes the process of comput-
evolution and guide the algorithm with human input where the seg-           ing the segmentation and is able to make modifications and
mentation is difficult to compute. In the areas, where the segmen-           corrections at any time;
tation is reliably computed automatically no additional user effort
is required. Results of segmenting generic photos and medical im-         Second, and most important, we would like to attract attention to
ages are presented. Our experiments show that modest user effort       another possible view on segmentation problem, apart from graph
is required for segmentation of moderately hard images.                theory. Cellular automata family is very rich and diverse, and our
                                                                       hope is that by applying experience and knowledge accumulated in
Keywords: interactive image segmentation, graph cut, cellular          this field to interactive image segmentation task, research commu-
automata, image editing, foreground extraction                         nity will be able to come up with some novel and ingenious solu-
                                                                       tions.
1      Introduction
                                                                       1.1    Related work
Image segmentation is an integral part of image processing appli-
cations like medical images analysis and photo editing. A wide         In this section we briefly outline main features of current state-of-
range of computational vision algorithms can also benefit from ex-      the-art interactive segmentation techniques. We informally divide
istence of reliable and efficient image segmentation technique. For     these methods into two families - proposed for generic image edit-
instance, intermediate-level vision problems such as shape from sil-   ing, and developed especially for medical images. In reality, most
houette, shape from stereo and object tracking could make use of       of the methods can be successfully applied in both domains.
reliable segmentation of the object of interest from the rest of the
scene. Higher-level problems such as recognition and image index-      Magic Wand - is a common selection tool for almost any image
ing can also make use of segmentation results in matching.                 editor nowadays. It gathers color statistics from the user-
   Fully automated segmentation techniques are being constantly            specified image point (or region) and segments (connected)
improved, however, no automated image analysis technique can               image region with pixels, which color properties fall within
be applied fully autonomously with guaranteed results in general           some given tolerance of the gathered statistics.
case. That is why semi-automatic segmentation techniques that al-
low solving moderate and hard segmentation tasks by modest effort      Intelligent paint - is a region-based interactive segmentation tech-
on the part of the user are becoming more and more popular.                  nique, based on hierarchical image segmentation by tobog-
   Several powerful techniques for interactive image segmentation            ganing [Reese 1999]. It uses a connect-and-collect strategy to
have been proposed recently based on graph cuts [Boykov and Jolly            define an objects region. This strategy uses a hierarchical to-
2001], [Rother et al. 2004] and random walker [Grady and Funka-              bogganing algorithm to automatically connect image regions
Lea 2004]. They seem to significantly outperform earlier methods              that naturally flow together, and a user-guided, cumulative
both by resulting segmentation quality and required user effort.             cost-ordered expansion interface to interactively collect those
   This paper’s contribution is twofold. First, we propose a new             regions which constitute the object of interest. This strategy
interactive segmentation scheme, based on cellular automata, that            coordinates human-computer interaction to extract regions of
has several favorable properties (see section 2 for details):                interest from complex backgrounds using paint strokes with a
                                                                             mouse.
    1. Capable of solving moderately hard segmentation tasks (see
       examples in this paper);                                        Intelligent scissors - is a boundary-based method, that computes
    2. Is easy in implementing and allows efficient parallel imple-           minimum-cost path between user-specified boundary points
       mentation;                                                            [Mortensen and Barrett 1998]. It treats each pixel as a graph
                                                                             node and uses shortest-path graph algorithms for boundary
    3. Works with images of any dimension N ≥ 1;                             calculation. A faster variant of region-based intelligent scis-
                                                                             sors uses tobogganing for image oversegmentation and then
    ∗ e-mail: vvp@graphics.cmc.msu.ru                                        treats homogenous regions as a graph nodes [Mortensen and
    † www:   http://graphics.cmc.msu.ru                                      Barrett 1999].
Graph Cut is a combinatorial optimization technique, which was           achieving better segmentation results with less user effort required,
    applied by [Boykov and Jolly 2001] to the task of image seg-         compared with other methods. One of the few drawbacks of the
    mentation. The image is treated as a graph - each pixel is a         graph-based methods is that they are not easily extended to multi-
    graph node. For the case of two labels (i.e. object and back-        label task and the other is that they are not very flexible - the only
    ground) the globally optimal pixel labelling (with respect to        tunable parameters are the graph weighting and cost function coef-
    defined cost function) can be efficiently computed by max-             ficients. For example, additional restrictions on the object boundary
    flow/min-cut algorithms. This technique can be applied to             smoothness or soft user-specified segmentation constraints cannot
    N-dimensional images. Given user-specified object and back-           be added readily. As for the intelligent paint, judging by the ex-
    ground seed pixels, the rest of the pixels are labelled automat-     amples supplied by the authors, the advantage of their method over
    ically.                                                              the traditional ‘magic wand’ is in speed and number of user in-
                                                                         teractions. As it appears from the algorithm description and pre-
GrabCut [Rother et al. 2004] extends graph-cut by introducing            sented results, it is unlikely that intelligent paint would be capable
    iterative segmentation scheme, that uses graph-cut for inter-        of solving hard segmentation problems like in [Rother et al. 2004],
    mediate steps. The user draws rectangle around the object            [Boykov and Jolly 2001]. Precise object boundary estimation is
    of interest - this gives the first approximation of the final ob-      also questionable, because the finest segmentation level is obtained
    ject/background labelling. Then, each iteration step gathers         by initial tobogganing oversegmentation, which may not coincide
    color statistics according to current segmentation, re-weights       with actual object borders.
    the image graph and applies graph-cut to compute new re-                Speaking about medical images, the best performing method is
    fined segmentation. After the iterations stop the segmentation        random walker (judging by the provided examples). It leaves be-
    results can be refined by specifying additional seeds, similar        hind both watershed segmentation and region growing behind in
    to original graph-cut.                                               quality and robustness of segmentation. The quality of segmenta-
                                                                         tion comparable to is graph cuts, but random walker is capable of
Medical images have their own unique properties. In many cases           finding the solution for number of labels > 2. However, it is rather
they are graylevel and objects that should be segmented are very         slow and its implementation is not an easy task. Also, method
different in their structure and appearance from the objects that are    extension to achieve some special algorithm properties (i.e. con-
common in photo editing. Probably that is why much research ef-          trollable boundary smoothness) is not straightforward. It should
fort was applied for developing specific and efficient segmentation        be mentioned, that multi-labelling tasks can be solved by min-cut
methods for the medical images domain. Nevertheless, some seg-           graph algorithms [Boykov et al. 2001], but no attempt to apply this
mentation techniques like Graph Cuts can successfully be applied         multi-labelling method to interactive image segmentation is known
to medical images also. Some specific ‘medical’ segmentation tech-        to us.
niques are reviewed further.

Marker-based watershed transformation uses watershed trans-              1.2    Proposed method
    form, supported by user-specified markers (seeds) for seg-            We take an intuitive user interaction scheme - user specifies certain
    menting graylevel images [Moga and Gabbouj 1996]. Wa-                image pixels (we will call them seed pixels) that belong to objects,
    tershed transform treats the image as a surface with the relief      that should be segmented from each other. The task is to assign
    specified by the pixel brightness, or by absolute value of the        labels to all other image pixels automatically, preferably achiev-
    image gradient. The valleys of the resulting ‘landscape’ are         ing the segmentation result the user is expecting to get. The task
    filled with water, until it reaches the ‘mountains’. Markers          statement and input data is similar to [Boykov and Jolly 2001] and
    placed in the image specify the initial labels that should be        [Grady and Funka-Lea 2004], however the segmentation instrument
    segmented from each other.                                           differs.
                                                                            Our method uses cellular automaton for solving pixel labelling
Random walker [Grady and Funka-Lea 2004] given a small num-              task. The method is iterative, giving feedback to the user while
    ber of pixels with user-defined seed labels (labels number can        the segmentation is computed. Proposed method allows (but not
    be grater than 2), analytically determines the probability that      requires) human input during labelling process, to provide dynamic
    a random walker starting at each unlabelled pixel will first          interaction and feedback between the user and the algorithm. This
    reach one of the pre-labelled pixels. By assigning each pixel        allows to correcting and guidance of the algorithm with user input
    to the label for which the greatest probability is calculated, im-   in the areas where the segmentation is difficult to compute, yet does
    age segmentation is obtained. This method provides unique            not require additional user effort where the segmentation is reliably
    segmentation solution into connected segments, with some             computed automatically.
    robustness against ‘weak’ boundaries. A confidence rating
                                                                            Important properties of our method, that we would like to outline
    of each pixel’s membership in the segmentation is also esti-
                                                                         are:
    mated.
                                                                           1. Capable of solving moderately hard segmentation tasks (see
Interactive region growing is a descendant of one of the classic
                                                                              examples in this paper);
     image segmentation techniques. Initially, the seed pixel in-
     side the object of interest is specified, and then neighboring         2. Works with images of any dimension N ≥ 1;
     pixels are iteratively added to the growing region, while they
     conform to some region homogeneity criterium. The main                3. Performs multi-label image segmentation (the computation
     problem is ‘leaking’ of the growing region through ‘weak’                time is not directly affected by the number of labels);
     boundaries, however some treatment of this problem was pro-
     posed [Heimann et al. 2004]. This method works with two               4. Is extensible, allowing construction of new families of seg-
     labels only - object and background.                                     mentation algorithms with specific properties;

   The performance of described photo editing methods was eval-            5. Interactivity - as the segmentation is refined with each itera-
uated in [Rother et al. 2004] (except for the intelligent paint). The         tion, user can observe the evolution and refine the segmenta-
authors have clearly shown, that methods based on graph cuts allow            tion “on the fly”;
6. The algorithm is simple in both understanding and implemen-
       tation;

    7. Using cellular automata allows fast parallel implementation;

   The rest of the paper is organized follows. Section 2 describes
the proposed method in detail. Section 3 gives the results, section                           (a)                  (b)                      (c)
4 provides discussion and comparison with other methods. Finally
the conclusive remark is given in section 5.                                    Figure 1: Segmentation of a color image. (a) Source image, (b)
                                                                                user-specified seeds, (c) segmentation results

2      Method details
                                                                                   At iteration t + 1 cell labels l t+1 and strengths θ p are updated
                                                                                                                    p
                                                                                                                                        t+1
2.1     Basic method                                                            as follows:
Cellular automata (CA) were introduced by Ulam and von Neu-
                                                                                Code 1 Automata evolution rule
mann [von Neumann 1966]. Since then they’ve been used to
model wide variety of dynamical systems in various application do-                 // For each cell...
mains, including image denoising and edge detection [Popovici and                  for ∀p ∈ P
Popovici 2002], [Hernandez and Herrmann 1996]. A cellular au-                           // Copy previous state
tomaton is generally an algorithm discrete in both space and time,                      l t+1 = l tp ;
                                                                                          p
that operates on a lattice of sites p ∈ P ⊆ Z n (pixels or voxels in                    θp = θp;
                                                                                           t+1       t

image processing).                                                                      // neighbors try to attack current cell
   A (bi-directional, deterministic) cellular automaton is a triplet                    for ∀q ∈ N(p)
A = (S, N, δ ), where S is an non-empty state set, N is the neighbor-                          if g( C p − Cq 2 ) · θq > θ p
                                                                                                                     t     t
hood system, and δ : SN → S is the local transition function (rule).                                 l t+1 = lq
                                                                                                       p
                                                                                                              t
This function defines the rule of calculating the cell’s state at t + 1                               θ p = g( C p − Cq 2 ) · θq
                                                                                                        t+1                   t
time step, given the states of the neighborhood cells at previous
                                                                                               end if
time step t.
                                                                                        end for
   Commonly used neighborhood systems N are the von Neumann
                                                                                   end for
and Moore neighborhoods:
    • von Neumann neighborhood
                                                    n                             Where g is a monotonous decreasing function bounded to [0, 1],
             N(p) = {q ∈ Z n : p − q        1 :=   ∑ |pi − qi | = 1};    (1)    we use a simple one:
                                                   i=1                                                                   x
                                                                                                        g(x) = 1 −                ;                     (4)
                                                                                                                     max C    2
    • Moore neighborhood
                                                                                   To supply an intuitive explanation to the pseudocode above we
           N(p) = {q ∈ Z n : p − q      ∞   := max |pi − qi | = 1};      (2)    can use biological metaphor. We can treat pixel labelling process as
                                               i=1,n
                                                                                growth and struggle for domination of K types of bacteria. The bac-
                                                                                teria start to spread (grow) from the seed pixels and try to occupy
The cell state S p in our case is actually a triplet (l p , θ p , C p ) - the   all the image. That is why we called the method ‘GrowCut’. The
label l p of the current cell, ‘strength’ of the current cell θ p , and cell    rules of bacteria growth and competition are obvious - at each dis-
feature vector C p , defined by the image. Without loss of generality            crete time step, each cell tries to ‘attack’ its neighbors. The attack
we will assume θ p ∈ [0, 1].                                                    force is defined by the attacker cell’s strength θq , and the distance
   A digital image is a two-dimensional array of k × m pixels. An               between attacker’s and defender’s feature vectors Cq and C p . If the
unlabelled image may be then considered as a particular configura-               attack force is greater than defender’s strength - the defending cell
tion state of a cellular automaton, where cellular space P is defined            is ‘conquered’ and its label and strength are changed. The result
by the k × m array set by the image, and initial states for ∀p ∈ P are          of these local competitions is that the strongest bacteria occupy the
set to:                                                                         neighboring sites and gradually spread over the image.
                      l p = 0, θ p = 0, C p = RGB p ;                     (3)      The calculation continues until automaton converges to stable
                                                                                configuration, where cell states seize to change. The example of
where RGB p is the three dimensional vector of pixel’s p color in               image segmentation is shown in figures 1 and 2. The method is
RGB space. The final goal of the segmentation is to assign each                  guaranteed to converge as the strength of each cell is a increasing
pixel one of the K possible labels.                                             and bounded.
   When user starts the segmentation by specifying the segmenta-
tion seeds, the seeded cells labels are set accordingly, while their
strength is set to the seed strength value (more about it in section
2.3). This sets the initial state of the cellular automaton.

                                                                                        (a)          (b)           (c)                (d)         (e)

                                                                                Figure 2: Bacteria evolution steps. White - ‘object’ labelled bacte-
                                                                                ria, black - ‘background’ labelled bacteria, gray - neutral territory.
                                                                                (a) - step 1, (b) - step 10, (c) - step 25, (d) - step 40, (e) - step 65.
2.2    Adding controllable boundary smoothness                               One important difference from the methods based on graph cuts
                                                                          is that seeds do not necessarily specify hard segmentation con-
The basic scheme is very simple and yet is able to achieve quality        straints. In other words - user brush strokes need not to specify
segmentation (see figures 1, 4, 5). But in some images, the result-        only the areas of firm foreground or firm background, but instead
ing segments boundary can be ragged, see figure 3 (a). It can be           can adjust the pixels state continuously, making them ‘more fore-
acceptable, or even necessary when the task is to capture the small-      ground’ or ‘a little more background’ for example. This gives more
est detail of the boundary (i.e. in medical applications), but this       versatile control of the segmentation from the user part and makes
can be an unwanted artifact when editing a generic high-resolution        the process tolerable to inaccurate paint strokes. This is especially
photo. To achieve smoother boundary, we propose an extension to           helpful for segmenting accurate boundaries of objects like flowers
the automata in section 2.1. Local transition rule is modified with        or plant leaves, like in figure 7.
two additional conditions. First: the cell, that has too many enemies        The seed ‘strength’ is controlled by the initial strength θ that is
around - enemiest (p) ≥ T1 is prohibited to attack its neighbors. Sec-    set by the user’s brush strokes. Hard constraints specify the pix-
ond: the cell that has enemiest (p) ≥ T2 is forced to be occupied by      els that should not change during the evolution, which is easily
the weakest of it’s enemies, no matter the cell’s strength. The ene-      achieved by setting seed cells strength to 1. Soft constraints set
mies number is defined by:                                                 their initial strength values < 1 or just increase or decrease current
                                                                          cell strength by some value.
                enemiest (p) = max (
                                 l=1,K
                                              ∑            1)      (5)       Unlike graph cuts, adding seeds in already correctly segmented
                                         q∈N(p),lq =l tp
                                                 t
                                                                          area (i.e. near the border) can help to achieve better segmentation
                                                                          in troubled areas. This is especially true when accurate border in
   The thresholds values T1 , T2 control the boundary smoothness.         blurry or camouflaged area is estimated.
Reasonable values for 2D images and Moore neighborhood range
from 9 (no smoothing) to 6. The examples of segmented border for
identical initial seeds and different threshold values are presented
in figure 3.                                                               3     Results
                                                                          We demonstrate our segmentation method in several examples in-
                                                                          cluding photo editing and medical image segmentation. We show
                                                                          original data and resulting segments generated by our technique, for
                                                                          a given set of seeds.


                                                                          3.1    Medical images
                                                                          Figure 4 shows segmentations that have been obtained on several
                         (a)                 (b)
                                                                          medical images from the DICOM image samples [Barre n. d.] -
                                                                          user-specified seeds and resulting segmentation.
Figure 3: Different boundary smoothness restrictions. (a) T1 , T2 =
9, (b) T1 , T2 = 6.



2.3    User interaction
For the discussion on user interaction we will consider the case of
two labels - object and background. To our opinion, this is the most
often case of segmentation task in photo editing. However, this is
only for the sake of clarity, everything said here is true for the case
of labels K > 2 also.
    The segmentation is started by specifying the initial seeds. This
is done by user’s strokes with ‘object’ and ‘background’ brushes
(see figure 1.b - red pixels correspond to ‘object’ brush strokes, blue
- to ‘background’ strokes). Each paint stroke of a defined brush sets
the initial labels and strengths of seed pixels.
    After the initial seeds are set, the automata evolution starts. The             Figure 4: Medical images segmentation results.
initial, incomplete user-labelling is often sufficient to allow the en-
tire segmentation to be completed automatically, but by no means
always. While the cell labels are being computed, the user can
observe the progress and interactively correct and guide the la-          3.2    Photo editing
belling process if necessary. User editing takes the form of brushing
pixels for adding new segmentation constraints. Each new paint            In Figures 5 and 7 we show results of segmentation of several pho-
stroke changes the states of the underlying pixels and affects the        tographs.
automaton evolution. This strategy allows to extract regions of in-          Naturally, the hope is that the method can quickly identify the
terest from complex backgrounds using simple paint strokes with a         right object, with little user interaction required. If the user needs
mouse.                                                                    to specify too many seeds, this is not much better than a manual
    Just as in [Rother et al. 2004] it is usually sufficient to brush,     segmentation. The performance of an algorithm can be assessed by
roughly, just part of a wrongly labelled area. Note that correction       the amount of efforts required from the user. Thus, the results of
can be done not only after the segmentation process is finished, but       our segmentation are shown along with seeds that we entered by
in ‘the middle’ of the segmentation computation.                          the user.
method by ourselves, but the results presented in the paper [Grady
                                                                         and Funka-Lea 2004] and in the presentation poster are very close
                                                                         to what we’ve been able to achieve on similar images.
                                                                            Generally, methods based on graph cuts tend to produce
                                                                         smoother (actually, shorter) boundary than our basic method. How-
                                                                         ever our modified algorithm with controllable boundary smooth-
                                                                         ness (see section 2.2) makes this difference less perceptible.
                                                                            Several authors mention that segmentation methods, based on
                                                                         minimum graph cuts tend to cut away small isolated image seg-
                                                                         ments [Grady and Funka-Lea 2004], [Veksler 2000]. We performed
                                                                         some investigation on how often this problem actually arises. In our
                                                                         experience, this situation occurs only in cases when regional term
                                                                         R(a) in eq. (1) in [Boykov and Jolly 2001] becomes insignificant
                                                                         compared to boundary term B(A). One such example is shown in
                                                                         figure 6, one can identify both smoother boundary of the graph cuts
                                                                         segmentation, and isolated background segments on the right part
                                                                         of the image. This can usually be avoided by tuning the relative
                                                                         importance coefficient λ .




                                                                                     (a)                 (b)                 (c)
          Figure 5: Generic photos segmentation results.                 Figure 6: Graph cuts and GrowCut segmentation difference for
                                                                         same seeds. The Graph cuts produce smoother boundary, but has
                                                                         problems with the background on the right image part, where two
4     Discussion                                                         isolated clusters instead of the whole region were segmented

As we have already emphasized in the introduction, our hope is              It also should not be forgotten that both Random Walker and
to stir up the research community, motivating to search new ideas        GrowCut are capable of solving multi-labelling tasks.
in the field of cellular automata and evolutionary computation and           Random Walker boasts of creating segments connected to the
applying them to interactive image segmentation.                         initial seeds always. So as GrowCut does. Graph Cut potentially
   We expect that results exceeding our current can be obtained.         can (and sometimes does) create segments not connected to ini-
However, our current method can already compete with elegant             tial seeds, if the regional term R(A) is stronger than boundary term
achievements of graph theory. In this section we will try to compare     B(A). Same applies GrabCut. However, whether this an advantage
current top performing methods with ours and point out advantages        or a drawback is probably the question of exact segmentation task.
and disadvantages of our scheme.
   We take four methods - Graph Cuts, GrabCut, Random Walker
and GrowCut and compare them by several criteria: segmentation           4.2    Speed
quality, speed and convenience for the user. Accurately speaking,
                                                                         The method speed is surely important. The reaction time to the
the methods differ seriously by the amount of information that they
                                                                         user input should be small enough not to produce discomfort and
extract from the image. GrabCut uses most information - it com-
                                                                         irritation, otherwise the method cannot be called interactive. An
putes the evolving color statistics of foreground and background
                                                                         important measure is not only how fast the initial segmentation is
and takes into account color difference between neighboring pix-
                                                                         achieved, but also how fast can the segmentation be recomputed,
els. Graph Cuts differs in using color statistics collected from the
                                                                         when user adds more seeds to refine the initial segmentation.
user-specified seeds only, computed before the segmentation start.
                                                                             The fastest are algorithms based on graph cuts. The initial seg-
Random Walker uses only intensity difference between neighboring
                                                                         mentation computation time for Graph Cuts reported by the authors
pixels. Our current GrowCut variant also does not take advantage of
                                                                         is ‘less than a second’ for ‘most 2D images (up to 512x512)’ on
object color statistics, however it can be easily extended to maintain
                                                                         ‘333MHz Pentium III’. This looks terrific, but judging from the
regions color statistics and use them in automaton evolution.
                                                                         Intel processor manufacturer site, the frequency for Pentium III
                                                                         processor family ranges from 450 MHz to 1.33 GHz. This sug-
4.1    Segmentation quality                                              gests that actually running times are given for 1.33 GHz processor,
                                                                         which is still a very good result. GrabCut processing time was given
Judging by our own experiments, Graph Cuts, GrabCut (we used             for 450x300 image and equals to 0.9sec on 2.5GHz CPU with 512
our own implementation of both methods) and GrowCut are all able         Mb RAM. Recomputing segmentation after new user-added seeds
to produce high quality segmentation in natural and medical im-          is even faster than that. However, our own implementation of these
ages, so in this case mostly the amount user effort required should      methods, based on Vladimir Kolmogorov’s publicly available code
be the measure of performance. However, there are some inter-            [Boykov and Kolmogorov 2004] was not able to reach this speed.
esting features in these methods results, mentioned in next para-            Random walker cannot boast of such speed, for the computation
graph. Concerning Random Walker, we’ve not been able to test the         time for 256x256 image on dual Intel Xeon 2.4GHz processor with
1GB RAM is almost 5 (4.831) seconds. No indications that adding
more seeds will need less time to compute a refined segmentation
were found in the method description. But again, Random Walker
solves a harder task of multi-labelling.
   Our current implementation goes somewhere in between. It
takes about 4 seconds on 256x256 image on 2.5GHz processor.
However, this is the total automaton evolution time (till complete
stop). Usually the desired segmentation result is obtained much
earlier (about 1.5-2 seconds) and the later automaton evolution does
not change the segmentation and can be stopped by the user, when
he is satisfied with the result. Another important issue is that our
methods allows user intervention at any iteration step, so adding
additional seeds throughout the computation does not increase the
whole working time. Again - our method solves the task of multi-
labelling.


4.3    User convenience
In our opinion, most important performance measure is the con-
venience of the image segmentation instrument, based on the algo-
rithm. Or, in other words, how much user effort is needed to achieve
satisfying segmentation result.
   The results reported by Boykov et al. [Boykov and Jolly 2001]
say that it takes from 1 to 3-4 minutes to segment an image with
Graph Cuts, depending on the scene. Our own experiments have
shown that usually it takes about 1-2 minutes for the user, armed
with GrowCut to segment an image like shown in figures 4, 5, 7.


5     Conclusion and future work
In this paper, we have proposed a novel algorithm for interactive im-
age segmentation, based in cellular automaton. Experiments have
shown, that user effort required for segmenting generic photos and
medical images is rather modest, no harder than in state-of-the-
art graph-based methods like Graph Cuts, GrabCut and Random
Walker.
   Proposed method combines the advantages, distributed between
the mentioned methods (multi-label segmentation, N-dimensional
images processing, speed high enough for interactive segmenta-
tion), and offers more - algorithm extensibility by varying the au-
tomaton evolution rule, more interactivity and user control of the
segmentation process.
   Future work will concentrate on exploring theoretical properties
of the algorithm, research into exploiting color statistics during seg-                  Figure 7: More photo segmentation results.
mentation, and practical issues like faster implementation.


6     Acknowledgements                                                      B OYKOV, Y., V EKSLER , O., AND Z ABIH , R. 2001. Fast approximate
                                                                               energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach.
We gratefully acknowledge valuable comments from the Graphicon                 Intell. 23, 11, 1222–1239.
reviewing committee members, Yuri Boykov and Olga Veksler. We
                                                                            G RADY, L., AND F UNKA -L EA , G. 2004. Multi-label image segmentation
also would like to thank our lab’s head Yuri Bayakovsky for attract-           for medical applications based on graph-theoretic electrical potentials.
ing our attention to the field of cellular automata some years ago.             In ECCV Workshops CVAMIA and MMBIA, 230–245.

                                                                            H EIMANN , T., T HORN , M., K UNERT, T., AND M EINZER , H.-P. 2004.
References                                                                     New methods for leak detection and contour correction in seeded region
                                                                               growing segmentation. In 20th ISPRS Congress, Istanbul 2004, Inter-
BARRE ,       S.                    Medical         image       samples.       national Archives of Photogrammetry and Remote Sensing, vol. XXXV,
  http://www.barre.nom.fr/medical/samples/.                                    317–322.
B OYKOV, Y., AND J OLLY, M.-P. 2001. Interactive graph cuts for optimal
   boundary and region segmentation of objects in n-d images. In Proc. of   H ERNANDEZ , G., AND H ERRMANN , H. J. 1996. Cellular automata for
   the International Conference on Computer Vision, vol. 1, 105–112.           elementary image enhancement. CVGIP: Graphical Model and Image
                                                                               Processing 58, 1, 82–89.
B OYKOV, Y., AND KOLMOGOROV, V. 2004. An experimental comparison
   of min-cut/max-flow algorithms for energy minimization in vision. IEEE    M OGA , A., AND G ABBOUJ , M. 1996. A parallel marker based watershed
   Trans. Pattern Anal. Mach. Intell. 26, 9, 1124–1137.                       transformation. In ICIP96, II: 137–140.
M ORTENSEN , E. N., AND BARRETT, W. A. 1998. Interactive segmenta-
  tion with intelligent scissors. Graphical Models and Image Processing
  60, 5, 349–384.
M ORTENSEN , E. N., AND BARRETT, W. A. 1999. Toboggan-based intel-
  ligent scissors with a four-parameter edge model. In CVPR, 2452–2458.
P OPOVICI , A., AND P OPOVICI , D. 2002. Cellular automata in image pro-
   cessing. In Fifteenth International Symposium on Mathematical Theory
   of Networks and Systems.
R EESE , L. 1999. Intelligent Paint: Region-Based Interactive Image Seg-
   mentation. Master’s thesis, Department of Computer Science, Brigham
   Young University, Provo, UT.
ROTHER , C., KOLMOGOROV, V., AND B LAKE , A. 2004. Grabcut - in-
  teractive foreground extraction using iterated graph cuts. Proc. ACM
  Siggraph.
V EKSLER , O. 2000. Image segmentation by nested cuts. In Proc. IEEE
   Conf. Computer Vision and Pattern Recognition, vol. 1, 339–344.
VON    N EUMANN , J. 1966. Theory of Self-Reproducing Automata. Univer-
    sity of Illinois Press. Ed. and Completed by A. Burks.


7     About the authors
Vladimir Veznevets is a research fellow at Graphics and Media
Laboratory of Moscow State Lomonosov University. He graduated
with honors Faculty of Computational Mathematics and Cyber-
netics of Moscow State University in 1999. He received his PhD
in 2002 in Moscow State University also. His research interests
include image processing, pattern recognition, computer vision and
adjacent fields. His email address is vvp@graphics.cs.msu.ru.

Vadim Konouchine is a third year student of Moscow State
Lomonosov University, Faculty of Computational Mathematics and
Cybernetics. His research interests lie in the fields of statistics and
image processing. His email address is kvad@inbox.ru.

Weitere ähnliche Inhalte

Was ist angesagt?

Comparative Study and Analysis of Image Inpainting Techniques
Comparative Study and Analysis of Image Inpainting TechniquesComparative Study and Analysis of Image Inpainting Techniques
Comparative Study and Analysis of Image Inpainting TechniquesIOSR Journals
 
Texture Segmentation Based on Multifractal Dimension
Texture Segmentation Based on Multifractal DimensionTexture Segmentation Based on Multifractal Dimension
Texture Segmentation Based on Multifractal Dimensionijsc
 
COLOUR BASED IMAGE SEGMENTATION USING HYBRID KMEANS WITH WATERSHED SEGMENTATION
COLOUR BASED IMAGE SEGMENTATION USING HYBRID KMEANS WITH WATERSHED SEGMENTATIONCOLOUR BASED IMAGE SEGMENTATION USING HYBRID KMEANS WITH WATERSHED SEGMENTATION
COLOUR BASED IMAGE SEGMENTATION USING HYBRID KMEANS WITH WATERSHED SEGMENTATIONIAEME Publication
 
Image Enhancement and Restoration by Image Inpainting
Image Enhancement and Restoration by Image InpaintingImage Enhancement and Restoration by Image Inpainting
Image Enhancement and Restoration by Image InpaintingIJERA Editor
 
Importance of Mean Shift in Remote Sensing Segmentation
Importance of Mean Shift in Remote Sensing SegmentationImportance of Mean Shift in Remote Sensing Segmentation
Importance of Mean Shift in Remote Sensing SegmentationIOSR Journals
 
SEGMENTATION USING ‘NEW’ TEXTURE FEATURE
SEGMENTATION USING ‘NEW’ TEXTURE FEATURESEGMENTATION USING ‘NEW’ TEXTURE FEATURE
SEGMENTATION USING ‘NEW’ TEXTURE FEATUREacijjournal
 
Comparative analysis and implementation of structured edge active contour
Comparative analysis and implementation of structured edge active contour Comparative analysis and implementation of structured edge active contour
Comparative analysis and implementation of structured edge active contour IJECEIAES
 
Paper id 252014130
Paper id 252014130Paper id 252014130
Paper id 252014130IJRAT
 
Image Segmentation Using Pairwise Correlation Clustering
Image Segmentation Using Pairwise Correlation ClusteringImage Segmentation Using Pairwise Correlation Clustering
Image Segmentation Using Pairwise Correlation ClusteringIJERA Editor
 
REMOVING OCCLUSION IN IMAGES USING SPARSE PROCESSING AND TEXTURE SYNTHESIS
REMOVING OCCLUSION IN IMAGES USING SPARSE PROCESSING AND TEXTURE SYNTHESISREMOVING OCCLUSION IN IMAGES USING SPARSE PROCESSING AND TEXTURE SYNTHESIS
REMOVING OCCLUSION IN IMAGES USING SPARSE PROCESSING AND TEXTURE SYNTHESISIJCSEA Journal
 
Fpga implementation of image segmentation by using edge detection based on so...
Fpga implementation of image segmentation by using edge detection based on so...Fpga implementation of image segmentation by using edge detection based on so...
Fpga implementation of image segmentation by using edge detection based on so...eSAT Publishing House
 
Fpga implementation of image segmentation by using edge detection based on so...
Fpga implementation of image segmentation by using edge detection based on so...Fpga implementation of image segmentation by using edge detection based on so...
Fpga implementation of image segmentation by using edge detection based on so...eSAT Journals
 
Texture Unit based Approach to Discriminate Manmade Scenes from Natural Scenes
Texture Unit based Approach to Discriminate Manmade Scenes from Natural ScenesTexture Unit based Approach to Discriminate Manmade Scenes from Natural Scenes
Texture Unit based Approach to Discriminate Manmade Scenes from Natural Scenesidescitation
 
OBJECT SEGMENTATION USING MULTISCALE MORPHOLOGICAL OPERATIONS
OBJECT SEGMENTATION USING MULTISCALE MORPHOLOGICAL OPERATIONSOBJECT SEGMENTATION USING MULTISCALE MORPHOLOGICAL OPERATIONS
OBJECT SEGMENTATION USING MULTISCALE MORPHOLOGICAL OPERATIONSijcseit
 

Was ist angesagt? (17)

Comparative Study and Analysis of Image Inpainting Techniques
Comparative Study and Analysis of Image Inpainting TechniquesComparative Study and Analysis of Image Inpainting Techniques
Comparative Study and Analysis of Image Inpainting Techniques
 
Texture Segmentation Based on Multifractal Dimension
Texture Segmentation Based on Multifractal DimensionTexture Segmentation Based on Multifractal Dimension
Texture Segmentation Based on Multifractal Dimension
 
A010210106
A010210106A010210106
A010210106
 
COLOUR BASED IMAGE SEGMENTATION USING HYBRID KMEANS WITH WATERSHED SEGMENTATION
COLOUR BASED IMAGE SEGMENTATION USING HYBRID KMEANS WITH WATERSHED SEGMENTATIONCOLOUR BASED IMAGE SEGMENTATION USING HYBRID KMEANS WITH WATERSHED SEGMENTATION
COLOUR BASED IMAGE SEGMENTATION USING HYBRID KMEANS WITH WATERSHED SEGMENTATION
 
Image Enhancement and Restoration by Image Inpainting
Image Enhancement and Restoration by Image InpaintingImage Enhancement and Restoration by Image Inpainting
Image Enhancement and Restoration by Image Inpainting
 
Importance of Mean Shift in Remote Sensing Segmentation
Importance of Mean Shift in Remote Sensing SegmentationImportance of Mean Shift in Remote Sensing Segmentation
Importance of Mean Shift in Remote Sensing Segmentation
 
SEGMENTATION USING ‘NEW’ TEXTURE FEATURE
SEGMENTATION USING ‘NEW’ TEXTURE FEATURESEGMENTATION USING ‘NEW’ TEXTURE FEATURE
SEGMENTATION USING ‘NEW’ TEXTURE FEATURE
 
Comparative analysis and implementation of structured edge active contour
Comparative analysis and implementation of structured edge active contour Comparative analysis and implementation of structured edge active contour
Comparative analysis and implementation of structured edge active contour
 
Paper id 252014130
Paper id 252014130Paper id 252014130
Paper id 252014130
 
Image Segmentation Using Pairwise Correlation Clustering
Image Segmentation Using Pairwise Correlation ClusteringImage Segmentation Using Pairwise Correlation Clustering
Image Segmentation Using Pairwise Correlation Clustering
 
Id105
Id105Id105
Id105
 
154 158
154 158154 158
154 158
 
REMOVING OCCLUSION IN IMAGES USING SPARSE PROCESSING AND TEXTURE SYNTHESIS
REMOVING OCCLUSION IN IMAGES USING SPARSE PROCESSING AND TEXTURE SYNTHESISREMOVING OCCLUSION IN IMAGES USING SPARSE PROCESSING AND TEXTURE SYNTHESIS
REMOVING OCCLUSION IN IMAGES USING SPARSE PROCESSING AND TEXTURE SYNTHESIS
 
Fpga implementation of image segmentation by using edge detection based on so...
Fpga implementation of image segmentation by using edge detection based on so...Fpga implementation of image segmentation by using edge detection based on so...
Fpga implementation of image segmentation by using edge detection based on so...
 
Fpga implementation of image segmentation by using edge detection based on so...
Fpga implementation of image segmentation by using edge detection based on so...Fpga implementation of image segmentation by using edge detection based on so...
Fpga implementation of image segmentation by using edge detection based on so...
 
Texture Unit based Approach to Discriminate Manmade Scenes from Natural Scenes
Texture Unit based Approach to Discriminate Manmade Scenes from Natural ScenesTexture Unit based Approach to Discriminate Manmade Scenes from Natural Scenes
Texture Unit based Approach to Discriminate Manmade Scenes from Natural Scenes
 
OBJECT SEGMENTATION USING MULTISCALE MORPHOLOGICAL OPERATIONS
OBJECT SEGMENTATION USING MULTISCALE MORPHOLOGICAL OPERATIONSOBJECT SEGMENTATION USING MULTISCALE MORPHOLOGICAL OPERATIONS
OBJECT SEGMENTATION USING MULTISCALE MORPHOLOGICAL OPERATIONS
 

Andere mochten auch

实验二 用Mathmatica软件求极限
实验二  用Mathmatica软件求极限实验二  用Mathmatica软件求极限
实验二 用Mathmatica软件求极限guestfe33f0e
 
实验五 用Mathematica软件计算一元函数的积分
实验五  用Mathematica软件计算一元函数的积分实验五  用Mathematica软件计算一元函数的积分
实验五 用Mathematica软件计算一元函数的积分guestfe33f0e
 
презентация Apartments Alexandre Iii сжат
презентация Apartments Alexandre Iii сжатпрезентация Apartments Alexandre Iii сжат
презентация Apartments Alexandre Iii сжатguest234f15
 
实验四 用Mathematica软件作导数应用
实验四  用Mathematica软件作导数应用实验四  用Mathematica软件作导数应用
实验四 用Mathematica软件作导数应用guestfe33f0e
 
Chapter 1
Chapter 1Chapter 1
Chapter 1Ghostly
 
Laberinto[1] Solucionado
Laberinto[1] SolucionadoLaberinto[1] Solucionado
Laberinto[1] Solucionadoguest790743a9
 
实验三 用Mathematica软件计算导数与微分
实验三  用Mathematica软件计算导数与微分实验三  用Mathematica软件计算导数与微分
实验三 用Mathematica软件计算导数与微分guestfe33f0e
 
Philosophy For Old Age George Carlin1
Philosophy For Old Age  George Carlin1Philosophy For Old Age  George Carlin1
Philosophy For Old Age George Carlin1Samisoni Tukana
 
Inoke [ James] Waqavesi Jnr
Inoke [ James] Waqavesi JnrInoke [ James] Waqavesi Jnr
Inoke [ James] Waqavesi JnrSamisoni Tukana
 
实验八 用Mathematica进行向量运算及曲面绘制
实验八  用Mathematica进行向量运算及曲面绘制实验八  用Mathematica进行向量运算及曲面绘制
实验八 用Mathematica进行向量运算及曲面绘制guestfe33f0e
 
实验一 Mathematica软件简介
实验一   Mathematica软件简介实验一   Mathematica软件简介
实验一 Mathematica软件简介guestfe33f0e
 
Meggin McIntosh, PhD Curriculum Vitae (updated 10-24-16)
Meggin McIntosh, PhD Curriculum Vitae (updated 10-24-16)Meggin McIntosh, PhD Curriculum Vitae (updated 10-24-16)
Meggin McIntosh, PhD Curriculum Vitae (updated 10-24-16)Meggin McIntosh
 
Edgar Dales Cone Of Experience1
Edgar Dales Cone Of Experience1Edgar Dales Cone Of Experience1
Edgar Dales Cone Of Experience1jacobsonsea
 

Andere mochten auch (17)

实验二 用Mathmatica软件求极限
实验二  用Mathmatica软件求极限实验二  用Mathmatica软件求极限
实验二 用Mathmatica软件求极限
 
实验五 用Mathematica软件计算一元函数的积分
实验五  用Mathematica软件计算一元函数的积分实验五  用Mathematica软件计算一元函数的积分
实验五 用Mathematica软件计算一元函数的积分
 
AIDS
AIDSAIDS
AIDS
 
презентация Apartments Alexandre Iii сжат
презентация Apartments Alexandre Iii сжатпрезентация Apartments Alexandre Iii сжат
презентация Apartments Alexandre Iii сжат
 
实验四 用Mathematica软件作导数应用
实验四  用Mathematica软件作导数应用实验四  用Mathematica软件作导数应用
实验四 用Mathematica软件作导数应用
 
Chapter 1
Chapter 1Chapter 1
Chapter 1
 
Laberinto[1] Solucionado
Laberinto[1] SolucionadoLaberinto[1] Solucionado
Laberinto[1] Solucionado
 
实验三 用Mathematica软件计算导数与微分
实验三  用Mathematica软件计算导数与微分实验三  用Mathematica软件计算导数与微分
实验三 用Mathematica软件计算导数与微分
 
Philosophy For Old Age George Carlin1
Philosophy For Old Age  George Carlin1Philosophy For Old Age  George Carlin1
Philosophy For Old Age George Carlin1
 
Inoke [ James] Waqavesi Jnr
Inoke [ James] Waqavesi JnrInoke [ James] Waqavesi Jnr
Inoke [ James] Waqavesi Jnr
 
实验八 用Mathematica进行向量运算及曲面绘制
实验八  用Mathematica进行向量运算及曲面绘制实验八  用Mathematica进行向量运算及曲面绘制
实验八 用Mathematica进行向量运算及曲面绘制
 
实验一 Mathematica软件简介
实验一   Mathematica软件简介实验一   Mathematica软件简介
实验一 Mathematica软件简介
 
Meggin McIntosh, PhD Curriculum Vitae (updated 10-24-16)
Meggin McIntosh, PhD Curriculum Vitae (updated 10-24-16)Meggin McIntosh, PhD Curriculum Vitae (updated 10-24-16)
Meggin McIntosh, PhD Curriculum Vitae (updated 10-24-16)
 
Sunu1
Sunu1Sunu1
Sunu1
 
Snoopy friends
Snoopy friendsSnoopy friends
Snoopy friends
 
Tumbukan
TumbukanTumbukan
Tumbukan
 
Edgar Dales Cone Of Experience1
Edgar Dales Cone Of Experience1Edgar Dales Cone Of Experience1
Edgar Dales Cone Of Experience1
 

Ähnlich wie GrowCut

A novel predicate for active region merging in automatic image segmentation
A novel predicate for active region merging in automatic image segmentationA novel predicate for active region merging in automatic image segmentation
A novel predicate for active region merging in automatic image segmentationeSAT Journals
 
Image Segmentation from RGBD Images by 3D Point Cloud Attributes and High-Lev...
Image Segmentation from RGBD Images by 3D Point Cloud Attributes and High-Lev...Image Segmentation from RGBD Images by 3D Point Cloud Attributes and High-Lev...
Image Segmentation from RGBD Images by 3D Point Cloud Attributes and High-Lev...CSCJournals
 
Analysis and Implementation Image Segmentation Through k-mean Algorithm with ...
Analysis and Implementation Image Segmentation Through k-mean Algorithm with ...Analysis and Implementation Image Segmentation Through k-mean Algorithm with ...
Analysis and Implementation Image Segmentation Through k-mean Algorithm with ...Editor Jacotech
 
An interactive image segmentation using multiple user input’s
An interactive image segmentation using multiple user input’sAn interactive image segmentation using multiple user input’s
An interactive image segmentation using multiple user input’seSAT Publishing House
 
An interactive image segmentation using multiple user inputªs
An interactive image segmentation using multiple user inputªsAn interactive image segmentation using multiple user inputªs
An interactive image segmentation using multiple user inputªseSAT Journals
 
Garbage Classification Using Deep Learning Techniques
Garbage Classification Using Deep Learning TechniquesGarbage Classification Using Deep Learning Techniques
Garbage Classification Using Deep Learning TechniquesIRJET Journal
 
A Survey on Image Segmentation and its Applications in Image Processing
A Survey on Image Segmentation and its Applications in Image Processing A Survey on Image Segmentation and its Applications in Image Processing
A Survey on Image Segmentation and its Applications in Image Processing IJEEE
 
Semi-Supervised Method of Multiple Object Segmentation with a Region Labeling...
Semi-Supervised Method of Multiple Object Segmentation with a Region Labeling...Semi-Supervised Method of Multiple Object Segmentation with a Region Labeling...
Semi-Supervised Method of Multiple Object Segmentation with a Region Labeling...sipij
 
Review paper on segmentation methods for multiobject feature extraction
Review paper on segmentation methods for multiobject feature extractionReview paper on segmentation methods for multiobject feature extraction
Review paper on segmentation methods for multiobject feature extractioneSAT Journals
 
International Journal of Engineering Research and Development (IJERD)
International Journal of Engineering Research and Development (IJERD)International Journal of Engineering Research and Development (IJERD)
International Journal of Engineering Research and Development (IJERD)IJERD Editor
 
Review of Image Segmentation Techniques based on Region Merging Approach
Review of Image Segmentation Techniques based on Region Merging ApproachReview of Image Segmentation Techniques based on Region Merging Approach
Review of Image Segmentation Techniques based on Region Merging ApproachEditor IJMTER
 
An effective RGB color selection for complex 3D object structure in scene gra...
An effective RGB color selection for complex 3D object structure in scene gra...An effective RGB color selection for complex 3D object structure in scene gra...
An effective RGB color selection for complex 3D object structure in scene gra...IJECEIAES
 
A deep learning based stereo matching model for autonomous vehicle
A deep learning based stereo matching model for autonomous vehicleA deep learning based stereo matching model for autonomous vehicle
A deep learning based stereo matching model for autonomous vehicleIAESIJAI
 
IRJET - Deep Learning Approach to Inpainting and Outpainting System
IRJET -  	  Deep Learning Approach to Inpainting and Outpainting SystemIRJET -  	  Deep Learning Approach to Inpainting and Outpainting System
IRJET - Deep Learning Approach to Inpainting and Outpainting SystemIRJET Journal
 

Ähnlich wie GrowCut (20)

A novel predicate for active region merging in automatic image segmentation
A novel predicate for active region merging in automatic image segmentationA novel predicate for active region merging in automatic image segmentation
A novel predicate for active region merging in automatic image segmentation
 
Image Segmentation from RGBD Images by 3D Point Cloud Attributes and High-Lev...
Image Segmentation from RGBD Images by 3D Point Cloud Attributes and High-Lev...Image Segmentation from RGBD Images by 3D Point Cloud Attributes and High-Lev...
Image Segmentation from RGBD Images by 3D Point Cloud Attributes and High-Lev...
 
1388586134 10545195
1388586134  105451951388586134  10545195
1388586134 10545195
 
Analysis and Implementation Image Segmentation Through k-mean Algorithm with ...
Analysis and Implementation Image Segmentation Through k-mean Algorithm with ...Analysis and Implementation Image Segmentation Through k-mean Algorithm with ...
Analysis and Implementation Image Segmentation Through k-mean Algorithm with ...
 
An interactive image segmentation using multiple user input’s
An interactive image segmentation using multiple user input’sAn interactive image segmentation using multiple user input’s
An interactive image segmentation using multiple user input’s
 
An interactive image segmentation using multiple user inputªs
An interactive image segmentation using multiple user inputªsAn interactive image segmentation using multiple user inputªs
An interactive image segmentation using multiple user inputªs
 
Garbage Classification Using Deep Learning Techniques
Garbage Classification Using Deep Learning TechniquesGarbage Classification Using Deep Learning Techniques
Garbage Classification Using Deep Learning Techniques
 
Bx31498502
Bx31498502Bx31498502
Bx31498502
 
Bx31498502
Bx31498502Bx31498502
Bx31498502
 
A Survey on Image Segmentation and its Applications in Image Processing
A Survey on Image Segmentation and its Applications in Image Processing A Survey on Image Segmentation and its Applications in Image Processing
A Survey on Image Segmentation and its Applications in Image Processing
 
Semi-Supervised Method of Multiple Object Segmentation with a Region Labeling...
Semi-Supervised Method of Multiple Object Segmentation with a Region Labeling...Semi-Supervised Method of Multiple Object Segmentation with a Region Labeling...
Semi-Supervised Method of Multiple Object Segmentation with a Region Labeling...
 
Review paper on segmentation methods for multiobject feature extraction
Review paper on segmentation methods for multiobject feature extractionReview paper on segmentation methods for multiobject feature extraction
Review paper on segmentation methods for multiobject feature extraction
 
International Journal of Engineering Research and Development (IJERD)
International Journal of Engineering Research and Development (IJERD)International Journal of Engineering Research and Development (IJERD)
International Journal of Engineering Research and Development (IJERD)
 
D04402024029
D04402024029D04402024029
D04402024029
 
Review of Image Segmentation Techniques based on Region Merging Approach
Review of Image Segmentation Techniques based on Region Merging ApproachReview of Image Segmentation Techniques based on Region Merging Approach
Review of Image Segmentation Techniques based on Region Merging Approach
 
An effective RGB color selection for complex 3D object structure in scene gra...
An effective RGB color selection for complex 3D object structure in scene gra...An effective RGB color selection for complex 3D object structure in scene gra...
An effective RGB color selection for complex 3D object structure in scene gra...
 
I017265357
I017265357I017265357
I017265357
 
A deep learning based stereo matching model for autonomous vehicle
A deep learning based stereo matching model for autonomous vehicleA deep learning based stereo matching model for autonomous vehicle
A deep learning based stereo matching model for autonomous vehicle
 
IRJET - Deep Learning Approach to Inpainting and Outpainting System
IRJET -  	  Deep Learning Approach to Inpainting and Outpainting SystemIRJET -  	  Deep Learning Approach to Inpainting and Outpainting System
IRJET - Deep Learning Approach to Inpainting and Outpainting System
 
Dk34681688
Dk34681688Dk34681688
Dk34681688
 

Kürzlich hochgeladen

The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Allon Mureinik
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Neo4j
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxKatpro Technologies
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024The Digital Insurer
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfEnterprise Knowledge
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 

Kürzlich hochgeladen (20)

The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 

GrowCut

  • 1. “GrowCut” - Interactive Multi-Label N-D Image Segmentation By Cellular Automata Vladimir Vezhnevets ∗ Vadim Konouchine Graphics and Media Laboratory † Faculty of Computational Mathematics and Cybernetics Moscow State University, Moscow, Russia. Abstract 4. Performs multi-label image segmentation (the computation time does not depend on the number of labels); In this paper we describe a novel algorithm for interactive multi- label segmentation of N-dimensional images. Given a small num- 5. Is extensible, allows constructing new families of segmenta- ber of user-labelled pixels, the rest of the image is segmented au- tion algorithms with specific properties; tomatically by a Cellular Automaton. The process is iterative, as the automaton labels the image, user can observe the segmentation 6. Is truly interactive - the user observes the process of comput- evolution and guide the algorithm with human input where the seg- ing the segmentation and is able to make modifications and mentation is difficult to compute. In the areas, where the segmen- corrections at any time; tation is reliably computed automatically no additional user effort is required. Results of segmenting generic photos and medical im- Second, and most important, we would like to attract attention to ages are presented. Our experiments show that modest user effort another possible view on segmentation problem, apart from graph is required for segmentation of moderately hard images. theory. Cellular automata family is very rich and diverse, and our hope is that by applying experience and knowledge accumulated in Keywords: interactive image segmentation, graph cut, cellular this field to interactive image segmentation task, research commu- automata, image editing, foreground extraction nity will be able to come up with some novel and ingenious solu- tions. 1 Introduction 1.1 Related work Image segmentation is an integral part of image processing appli- cations like medical images analysis and photo editing. A wide In this section we briefly outline main features of current state-of- range of computational vision algorithms can also benefit from ex- the-art interactive segmentation techniques. We informally divide istence of reliable and efficient image segmentation technique. For these methods into two families - proposed for generic image edit- instance, intermediate-level vision problems such as shape from sil- ing, and developed especially for medical images. In reality, most houette, shape from stereo and object tracking could make use of of the methods can be successfully applied in both domains. reliable segmentation of the object of interest from the rest of the scene. Higher-level problems such as recognition and image index- Magic Wand - is a common selection tool for almost any image ing can also make use of segmentation results in matching. editor nowadays. It gathers color statistics from the user- Fully automated segmentation techniques are being constantly specified image point (or region) and segments (connected) improved, however, no automated image analysis technique can image region with pixels, which color properties fall within be applied fully autonomously with guaranteed results in general some given tolerance of the gathered statistics. case. That is why semi-automatic segmentation techniques that al- low solving moderate and hard segmentation tasks by modest effort Intelligent paint - is a region-based interactive segmentation tech- on the part of the user are becoming more and more popular. nique, based on hierarchical image segmentation by tobog- Several powerful techniques for interactive image segmentation ganing [Reese 1999]. It uses a connect-and-collect strategy to have been proposed recently based on graph cuts [Boykov and Jolly define an objects region. This strategy uses a hierarchical to- 2001], [Rother et al. 2004] and random walker [Grady and Funka- bogganing algorithm to automatically connect image regions Lea 2004]. They seem to significantly outperform earlier methods that naturally flow together, and a user-guided, cumulative both by resulting segmentation quality and required user effort. cost-ordered expansion interface to interactively collect those This paper’s contribution is twofold. First, we propose a new regions which constitute the object of interest. This strategy interactive segmentation scheme, based on cellular automata, that coordinates human-computer interaction to extract regions of has several favorable properties (see section 2 for details): interest from complex backgrounds using paint strokes with a mouse. 1. Capable of solving moderately hard segmentation tasks (see examples in this paper); Intelligent scissors - is a boundary-based method, that computes 2. Is easy in implementing and allows efficient parallel imple- minimum-cost path between user-specified boundary points mentation; [Mortensen and Barrett 1998]. It treats each pixel as a graph node and uses shortest-path graph algorithms for boundary 3. Works with images of any dimension N ≥ 1; calculation. A faster variant of region-based intelligent scis- sors uses tobogganing for image oversegmentation and then ∗ e-mail: vvp@graphics.cmc.msu.ru treats homogenous regions as a graph nodes [Mortensen and † www: http://graphics.cmc.msu.ru Barrett 1999].
  • 2. Graph Cut is a combinatorial optimization technique, which was achieving better segmentation results with less user effort required, applied by [Boykov and Jolly 2001] to the task of image seg- compared with other methods. One of the few drawbacks of the mentation. The image is treated as a graph - each pixel is a graph-based methods is that they are not easily extended to multi- graph node. For the case of two labels (i.e. object and back- label task and the other is that they are not very flexible - the only ground) the globally optimal pixel labelling (with respect to tunable parameters are the graph weighting and cost function coef- defined cost function) can be efficiently computed by max- ficients. For example, additional restrictions on the object boundary flow/min-cut algorithms. This technique can be applied to smoothness or soft user-specified segmentation constraints cannot N-dimensional images. Given user-specified object and back- be added readily. As for the intelligent paint, judging by the ex- ground seed pixels, the rest of the pixels are labelled automat- amples supplied by the authors, the advantage of their method over ically. the traditional ‘magic wand’ is in speed and number of user in- teractions. As it appears from the algorithm description and pre- GrabCut [Rother et al. 2004] extends graph-cut by introducing sented results, it is unlikely that intelligent paint would be capable iterative segmentation scheme, that uses graph-cut for inter- of solving hard segmentation problems like in [Rother et al. 2004], mediate steps. The user draws rectangle around the object [Boykov and Jolly 2001]. Precise object boundary estimation is of interest - this gives the first approximation of the final ob- also questionable, because the finest segmentation level is obtained ject/background labelling. Then, each iteration step gathers by initial tobogganing oversegmentation, which may not coincide color statistics according to current segmentation, re-weights with actual object borders. the image graph and applies graph-cut to compute new re- Speaking about medical images, the best performing method is fined segmentation. After the iterations stop the segmentation random walker (judging by the provided examples). It leaves be- results can be refined by specifying additional seeds, similar hind both watershed segmentation and region growing behind in to original graph-cut. quality and robustness of segmentation. The quality of segmenta- tion comparable to is graph cuts, but random walker is capable of Medical images have their own unique properties. In many cases finding the solution for number of labels > 2. However, it is rather they are graylevel and objects that should be segmented are very slow and its implementation is not an easy task. Also, method different in their structure and appearance from the objects that are extension to achieve some special algorithm properties (i.e. con- common in photo editing. Probably that is why much research ef- trollable boundary smoothness) is not straightforward. It should fort was applied for developing specific and efficient segmentation be mentioned, that multi-labelling tasks can be solved by min-cut methods for the medical images domain. Nevertheless, some seg- graph algorithms [Boykov et al. 2001], but no attempt to apply this mentation techniques like Graph Cuts can successfully be applied multi-labelling method to interactive image segmentation is known to medical images also. Some specific ‘medical’ segmentation tech- to us. niques are reviewed further. Marker-based watershed transformation uses watershed trans- 1.2 Proposed method form, supported by user-specified markers (seeds) for seg- We take an intuitive user interaction scheme - user specifies certain menting graylevel images [Moga and Gabbouj 1996]. Wa- image pixels (we will call them seed pixels) that belong to objects, tershed transform treats the image as a surface with the relief that should be segmented from each other. The task is to assign specified by the pixel brightness, or by absolute value of the labels to all other image pixels automatically, preferably achiev- image gradient. The valleys of the resulting ‘landscape’ are ing the segmentation result the user is expecting to get. The task filled with water, until it reaches the ‘mountains’. Markers statement and input data is similar to [Boykov and Jolly 2001] and placed in the image specify the initial labels that should be [Grady and Funka-Lea 2004], however the segmentation instrument segmented from each other. differs. Our method uses cellular automaton for solving pixel labelling Random walker [Grady and Funka-Lea 2004] given a small num- task. The method is iterative, giving feedback to the user while ber of pixels with user-defined seed labels (labels number can the segmentation is computed. Proposed method allows (but not be grater than 2), analytically determines the probability that requires) human input during labelling process, to provide dynamic a random walker starting at each unlabelled pixel will first interaction and feedback between the user and the algorithm. This reach one of the pre-labelled pixels. By assigning each pixel allows to correcting and guidance of the algorithm with user input to the label for which the greatest probability is calculated, im- in the areas where the segmentation is difficult to compute, yet does age segmentation is obtained. This method provides unique not require additional user effort where the segmentation is reliably segmentation solution into connected segments, with some computed automatically. robustness against ‘weak’ boundaries. A confidence rating Important properties of our method, that we would like to outline of each pixel’s membership in the segmentation is also esti- are: mated. 1. Capable of solving moderately hard segmentation tasks (see Interactive region growing is a descendant of one of the classic examples in this paper); image segmentation techniques. Initially, the seed pixel in- side the object of interest is specified, and then neighboring 2. Works with images of any dimension N ≥ 1; pixels are iteratively added to the growing region, while they conform to some region homogeneity criterium. The main 3. Performs multi-label image segmentation (the computation problem is ‘leaking’ of the growing region through ‘weak’ time is not directly affected by the number of labels); boundaries, however some treatment of this problem was pro- posed [Heimann et al. 2004]. This method works with two 4. Is extensible, allowing construction of new families of seg- labels only - object and background. mentation algorithms with specific properties; The performance of described photo editing methods was eval- 5. Interactivity - as the segmentation is refined with each itera- uated in [Rother et al. 2004] (except for the intelligent paint). The tion, user can observe the evolution and refine the segmenta- authors have clearly shown, that methods based on graph cuts allow tion “on the fly”;
  • 3. 6. The algorithm is simple in both understanding and implemen- tation; 7. Using cellular automata allows fast parallel implementation; The rest of the paper is organized follows. Section 2 describes the proposed method in detail. Section 3 gives the results, section (a) (b) (c) 4 provides discussion and comparison with other methods. Finally the conclusive remark is given in section 5. Figure 1: Segmentation of a color image. (a) Source image, (b) user-specified seeds, (c) segmentation results 2 Method details At iteration t + 1 cell labels l t+1 and strengths θ p are updated p t+1 2.1 Basic method as follows: Cellular automata (CA) were introduced by Ulam and von Neu- Code 1 Automata evolution rule mann [von Neumann 1966]. Since then they’ve been used to model wide variety of dynamical systems in various application do- // For each cell... mains, including image denoising and edge detection [Popovici and for ∀p ∈ P Popovici 2002], [Hernandez and Herrmann 1996]. A cellular au- // Copy previous state tomaton is generally an algorithm discrete in both space and time, l t+1 = l tp ; p that operates on a lattice of sites p ∈ P ⊆ Z n (pixels or voxels in θp = θp; t+1 t image processing). // neighbors try to attack current cell A (bi-directional, deterministic) cellular automaton is a triplet for ∀q ∈ N(p) A = (S, N, δ ), where S is an non-empty state set, N is the neighbor- if g( C p − Cq 2 ) · θq > θ p t t hood system, and δ : SN → S is the local transition function (rule). l t+1 = lq p t This function defines the rule of calculating the cell’s state at t + 1 θ p = g( C p − Cq 2 ) · θq t+1 t time step, given the states of the neighborhood cells at previous end if time step t. end for Commonly used neighborhood systems N are the von Neumann end for and Moore neighborhoods: • von Neumann neighborhood n Where g is a monotonous decreasing function bounded to [0, 1], N(p) = {q ∈ Z n : p − q 1 := ∑ |pi − qi | = 1}; (1) we use a simple one: i=1 x g(x) = 1 − ; (4) max C 2 • Moore neighborhood To supply an intuitive explanation to the pseudocode above we N(p) = {q ∈ Z n : p − q ∞ := max |pi − qi | = 1}; (2) can use biological metaphor. We can treat pixel labelling process as i=1,n growth and struggle for domination of K types of bacteria. The bac- teria start to spread (grow) from the seed pixels and try to occupy The cell state S p in our case is actually a triplet (l p , θ p , C p ) - the all the image. That is why we called the method ‘GrowCut’. The label l p of the current cell, ‘strength’ of the current cell θ p , and cell rules of bacteria growth and competition are obvious - at each dis- feature vector C p , defined by the image. Without loss of generality crete time step, each cell tries to ‘attack’ its neighbors. The attack we will assume θ p ∈ [0, 1]. force is defined by the attacker cell’s strength θq , and the distance A digital image is a two-dimensional array of k × m pixels. An between attacker’s and defender’s feature vectors Cq and C p . If the unlabelled image may be then considered as a particular configura- attack force is greater than defender’s strength - the defending cell tion state of a cellular automaton, where cellular space P is defined is ‘conquered’ and its label and strength are changed. The result by the k × m array set by the image, and initial states for ∀p ∈ P are of these local competitions is that the strongest bacteria occupy the set to: neighboring sites and gradually spread over the image. l p = 0, θ p = 0, C p = RGB p ; (3) The calculation continues until automaton converges to stable configuration, where cell states seize to change. The example of where RGB p is the three dimensional vector of pixel’s p color in image segmentation is shown in figures 1 and 2. The method is RGB space. The final goal of the segmentation is to assign each guaranteed to converge as the strength of each cell is a increasing pixel one of the K possible labels. and bounded. When user starts the segmentation by specifying the segmenta- tion seeds, the seeded cells labels are set accordingly, while their strength is set to the seed strength value (more about it in section 2.3). This sets the initial state of the cellular automaton. (a) (b) (c) (d) (e) Figure 2: Bacteria evolution steps. White - ‘object’ labelled bacte- ria, black - ‘background’ labelled bacteria, gray - neutral territory. (a) - step 1, (b) - step 10, (c) - step 25, (d) - step 40, (e) - step 65.
  • 4. 2.2 Adding controllable boundary smoothness One important difference from the methods based on graph cuts is that seeds do not necessarily specify hard segmentation con- The basic scheme is very simple and yet is able to achieve quality straints. In other words - user brush strokes need not to specify segmentation (see figures 1, 4, 5). But in some images, the result- only the areas of firm foreground or firm background, but instead ing segments boundary can be ragged, see figure 3 (a). It can be can adjust the pixels state continuously, making them ‘more fore- acceptable, or even necessary when the task is to capture the small- ground’ or ‘a little more background’ for example. This gives more est detail of the boundary (i.e. in medical applications), but this versatile control of the segmentation from the user part and makes can be an unwanted artifact when editing a generic high-resolution the process tolerable to inaccurate paint strokes. This is especially photo. To achieve smoother boundary, we propose an extension to helpful for segmenting accurate boundaries of objects like flowers the automata in section 2.1. Local transition rule is modified with or plant leaves, like in figure 7. two additional conditions. First: the cell, that has too many enemies The seed ‘strength’ is controlled by the initial strength θ that is around - enemiest (p) ≥ T1 is prohibited to attack its neighbors. Sec- set by the user’s brush strokes. Hard constraints specify the pix- ond: the cell that has enemiest (p) ≥ T2 is forced to be occupied by els that should not change during the evolution, which is easily the weakest of it’s enemies, no matter the cell’s strength. The ene- achieved by setting seed cells strength to 1. Soft constraints set mies number is defined by: their initial strength values < 1 or just increase or decrease current cell strength by some value. enemiest (p) = max ( l=1,K ∑ 1) (5) Unlike graph cuts, adding seeds in already correctly segmented q∈N(p),lq =l tp t area (i.e. near the border) can help to achieve better segmentation in troubled areas. This is especially true when accurate border in The thresholds values T1 , T2 control the boundary smoothness. blurry or camouflaged area is estimated. Reasonable values for 2D images and Moore neighborhood range from 9 (no smoothing) to 6. The examples of segmented border for identical initial seeds and different threshold values are presented in figure 3. 3 Results We demonstrate our segmentation method in several examples in- cluding photo editing and medical image segmentation. We show original data and resulting segments generated by our technique, for a given set of seeds. 3.1 Medical images Figure 4 shows segmentations that have been obtained on several (a) (b) medical images from the DICOM image samples [Barre n. d.] - user-specified seeds and resulting segmentation. Figure 3: Different boundary smoothness restrictions. (a) T1 , T2 = 9, (b) T1 , T2 = 6. 2.3 User interaction For the discussion on user interaction we will consider the case of two labels - object and background. To our opinion, this is the most often case of segmentation task in photo editing. However, this is only for the sake of clarity, everything said here is true for the case of labels K > 2 also. The segmentation is started by specifying the initial seeds. This is done by user’s strokes with ‘object’ and ‘background’ brushes (see figure 1.b - red pixels correspond to ‘object’ brush strokes, blue - to ‘background’ strokes). Each paint stroke of a defined brush sets the initial labels and strengths of seed pixels. After the initial seeds are set, the automata evolution starts. The Figure 4: Medical images segmentation results. initial, incomplete user-labelling is often sufficient to allow the en- tire segmentation to be completed automatically, but by no means always. While the cell labels are being computed, the user can observe the progress and interactively correct and guide the la- 3.2 Photo editing belling process if necessary. User editing takes the form of brushing pixels for adding new segmentation constraints. Each new paint In Figures 5 and 7 we show results of segmentation of several pho- stroke changes the states of the underlying pixels and affects the tographs. automaton evolution. This strategy allows to extract regions of in- Naturally, the hope is that the method can quickly identify the terest from complex backgrounds using simple paint strokes with a right object, with little user interaction required. If the user needs mouse. to specify too many seeds, this is not much better than a manual Just as in [Rother et al. 2004] it is usually sufficient to brush, segmentation. The performance of an algorithm can be assessed by roughly, just part of a wrongly labelled area. Note that correction the amount of efforts required from the user. Thus, the results of can be done not only after the segmentation process is finished, but our segmentation are shown along with seeds that we entered by in ‘the middle’ of the segmentation computation. the user.
  • 5. method by ourselves, but the results presented in the paper [Grady and Funka-Lea 2004] and in the presentation poster are very close to what we’ve been able to achieve on similar images. Generally, methods based on graph cuts tend to produce smoother (actually, shorter) boundary than our basic method. How- ever our modified algorithm with controllable boundary smooth- ness (see section 2.2) makes this difference less perceptible. Several authors mention that segmentation methods, based on minimum graph cuts tend to cut away small isolated image seg- ments [Grady and Funka-Lea 2004], [Veksler 2000]. We performed some investigation on how often this problem actually arises. In our experience, this situation occurs only in cases when regional term R(a) in eq. (1) in [Boykov and Jolly 2001] becomes insignificant compared to boundary term B(A). One such example is shown in figure 6, one can identify both smoother boundary of the graph cuts segmentation, and isolated background segments on the right part of the image. This can usually be avoided by tuning the relative importance coefficient λ . (a) (b) (c) Figure 5: Generic photos segmentation results. Figure 6: Graph cuts and GrowCut segmentation difference for same seeds. The Graph cuts produce smoother boundary, but has problems with the background on the right image part, where two 4 Discussion isolated clusters instead of the whole region were segmented As we have already emphasized in the introduction, our hope is It also should not be forgotten that both Random Walker and to stir up the research community, motivating to search new ideas GrowCut are capable of solving multi-labelling tasks. in the field of cellular automata and evolutionary computation and Random Walker boasts of creating segments connected to the applying them to interactive image segmentation. initial seeds always. So as GrowCut does. Graph Cut potentially We expect that results exceeding our current can be obtained. can (and sometimes does) create segments not connected to ini- However, our current method can already compete with elegant tial seeds, if the regional term R(A) is stronger than boundary term achievements of graph theory. In this section we will try to compare B(A). Same applies GrabCut. However, whether this an advantage current top performing methods with ours and point out advantages or a drawback is probably the question of exact segmentation task. and disadvantages of our scheme. We take four methods - Graph Cuts, GrabCut, Random Walker and GrowCut and compare them by several criteria: segmentation 4.2 Speed quality, speed and convenience for the user. Accurately speaking, The method speed is surely important. The reaction time to the the methods differ seriously by the amount of information that they user input should be small enough not to produce discomfort and extract from the image. GrabCut uses most information - it com- irritation, otherwise the method cannot be called interactive. An putes the evolving color statistics of foreground and background important measure is not only how fast the initial segmentation is and takes into account color difference between neighboring pix- achieved, but also how fast can the segmentation be recomputed, els. Graph Cuts differs in using color statistics collected from the when user adds more seeds to refine the initial segmentation. user-specified seeds only, computed before the segmentation start. The fastest are algorithms based on graph cuts. The initial seg- Random Walker uses only intensity difference between neighboring mentation computation time for Graph Cuts reported by the authors pixels. Our current GrowCut variant also does not take advantage of is ‘less than a second’ for ‘most 2D images (up to 512x512)’ on object color statistics, however it can be easily extended to maintain ‘333MHz Pentium III’. This looks terrific, but judging from the regions color statistics and use them in automaton evolution. Intel processor manufacturer site, the frequency for Pentium III processor family ranges from 450 MHz to 1.33 GHz. This sug- 4.1 Segmentation quality gests that actually running times are given for 1.33 GHz processor, which is still a very good result. GrabCut processing time was given Judging by our own experiments, Graph Cuts, GrabCut (we used for 450x300 image and equals to 0.9sec on 2.5GHz CPU with 512 our own implementation of both methods) and GrowCut are all able Mb RAM. Recomputing segmentation after new user-added seeds to produce high quality segmentation in natural and medical im- is even faster than that. However, our own implementation of these ages, so in this case mostly the amount user effort required should methods, based on Vladimir Kolmogorov’s publicly available code be the measure of performance. However, there are some inter- [Boykov and Kolmogorov 2004] was not able to reach this speed. esting features in these methods results, mentioned in next para- Random walker cannot boast of such speed, for the computation graph. Concerning Random Walker, we’ve not been able to test the time for 256x256 image on dual Intel Xeon 2.4GHz processor with
  • 6. 1GB RAM is almost 5 (4.831) seconds. No indications that adding more seeds will need less time to compute a refined segmentation were found in the method description. But again, Random Walker solves a harder task of multi-labelling. Our current implementation goes somewhere in between. It takes about 4 seconds on 256x256 image on 2.5GHz processor. However, this is the total automaton evolution time (till complete stop). Usually the desired segmentation result is obtained much earlier (about 1.5-2 seconds) and the later automaton evolution does not change the segmentation and can be stopped by the user, when he is satisfied with the result. Another important issue is that our methods allows user intervention at any iteration step, so adding additional seeds throughout the computation does not increase the whole working time. Again - our method solves the task of multi- labelling. 4.3 User convenience In our opinion, most important performance measure is the con- venience of the image segmentation instrument, based on the algo- rithm. Or, in other words, how much user effort is needed to achieve satisfying segmentation result. The results reported by Boykov et al. [Boykov and Jolly 2001] say that it takes from 1 to 3-4 minutes to segment an image with Graph Cuts, depending on the scene. Our own experiments have shown that usually it takes about 1-2 minutes for the user, armed with GrowCut to segment an image like shown in figures 4, 5, 7. 5 Conclusion and future work In this paper, we have proposed a novel algorithm for interactive im- age segmentation, based in cellular automaton. Experiments have shown, that user effort required for segmenting generic photos and medical images is rather modest, no harder than in state-of-the- art graph-based methods like Graph Cuts, GrabCut and Random Walker. Proposed method combines the advantages, distributed between the mentioned methods (multi-label segmentation, N-dimensional images processing, speed high enough for interactive segmenta- tion), and offers more - algorithm extensibility by varying the au- tomaton evolution rule, more interactivity and user control of the segmentation process. Future work will concentrate on exploring theoretical properties of the algorithm, research into exploiting color statistics during seg- Figure 7: More photo segmentation results. mentation, and practical issues like faster implementation. 6 Acknowledgements B OYKOV, Y., V EKSLER , O., AND Z ABIH , R. 2001. Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. We gratefully acknowledge valuable comments from the Graphicon Intell. 23, 11, 1222–1239. reviewing committee members, Yuri Boykov and Olga Veksler. We G RADY, L., AND F UNKA -L EA , G. 2004. Multi-label image segmentation also would like to thank our lab’s head Yuri Bayakovsky for attract- for medical applications based on graph-theoretic electrical potentials. ing our attention to the field of cellular automata some years ago. In ECCV Workshops CVAMIA and MMBIA, 230–245. H EIMANN , T., T HORN , M., K UNERT, T., AND M EINZER , H.-P. 2004. References New methods for leak detection and contour correction in seeded region growing segmentation. In 20th ISPRS Congress, Istanbul 2004, Inter- BARRE , S. Medical image samples. national Archives of Photogrammetry and Remote Sensing, vol. XXXV, http://www.barre.nom.fr/medical/samples/. 317–322. B OYKOV, Y., AND J OLLY, M.-P. 2001. Interactive graph cuts for optimal boundary and region segmentation of objects in n-d images. In Proc. of H ERNANDEZ , G., AND H ERRMANN , H. J. 1996. Cellular automata for the International Conference on Computer Vision, vol. 1, 105–112. elementary image enhancement. CVGIP: Graphical Model and Image Processing 58, 1, 82–89. B OYKOV, Y., AND KOLMOGOROV, V. 2004. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE M OGA , A., AND G ABBOUJ , M. 1996. A parallel marker based watershed Trans. Pattern Anal. Mach. Intell. 26, 9, 1124–1137. transformation. In ICIP96, II: 137–140.
  • 7. M ORTENSEN , E. N., AND BARRETT, W. A. 1998. Interactive segmenta- tion with intelligent scissors. Graphical Models and Image Processing 60, 5, 349–384. M ORTENSEN , E. N., AND BARRETT, W. A. 1999. Toboggan-based intel- ligent scissors with a four-parameter edge model. In CVPR, 2452–2458. P OPOVICI , A., AND P OPOVICI , D. 2002. Cellular automata in image pro- cessing. In Fifteenth International Symposium on Mathematical Theory of Networks and Systems. R EESE , L. 1999. Intelligent Paint: Region-Based Interactive Image Seg- mentation. Master’s thesis, Department of Computer Science, Brigham Young University, Provo, UT. ROTHER , C., KOLMOGOROV, V., AND B LAKE , A. 2004. Grabcut - in- teractive foreground extraction using iterated graph cuts. Proc. ACM Siggraph. V EKSLER , O. 2000. Image segmentation by nested cuts. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, 339–344. VON N EUMANN , J. 1966. Theory of Self-Reproducing Automata. Univer- sity of Illinois Press. Ed. and Completed by A. Burks. 7 About the authors Vladimir Veznevets is a research fellow at Graphics and Media Laboratory of Moscow State Lomonosov University. He graduated with honors Faculty of Computational Mathematics and Cyber- netics of Moscow State University in 1999. He received his PhD in 2002 in Moscow State University also. His research interests include image processing, pattern recognition, computer vision and adjacent fields. His email address is vvp@graphics.cs.msu.ru. Vadim Konouchine is a third year student of Moscow State Lomonosov University, Faculty of Computational Mathematics and Cybernetics. His research interests lie in the fields of statistics and image processing. His email address is kvad@inbox.ru.