1. International Journal of Computer Engineering and Technology (IJCET), ISSN 0976-
6367(Print), ISSN 0976 – 6375(Online) Volume 4, Issue 3, May – June (2013), © IAEME
86
T.V. COMMERCIAL DETECTION IN SERIAL VIDEOS
Swati D. Bendale1
, Bijal.J.Talati2
1
PG Student of Computer Engineering, GTU, Gujrat, India.
2
Department of Computer Engineering, SVIT, Vasad, Gujarat, India.
ABSTRACT
In recent years Commercial detection has become a widely research area of digital
video processing because of its huge importance. The task of detection of commercial in TV
videos is extremely difficult because of diversities involved. Diversities are involved as exact
timings of start and end of commercial break is not fixed. The number of commercials
appearing in a break is not fixed. The length of each individual commercial is different. The
number of commercial breaks and their duration are normally dependent on the duration of
program to be aired, the time at which program is aired and genre of program. Here we have
discussed some existing commercial detection works and proposed a new scheme for
detection of commercial break in TV serial videos. The proposed work is based on the
concept of frequency of number of cuts and appearance of caption text.
Keywords: Histogram based cut detection, Scene change rate, Template matching.
I. INTRODUCTION
Advertise is a way to publicize a product, it’s features and it’s working to people.
Analysis of advertisement helps the marketing personnel and sponsor’s to decide their
strategy to publicize product as compared to their competitors. T.V. commercials are a big
source of revenue for broadcasters. They must keep track of the statistics of this source.
Automatic detection of T.V. commercials minimize the human involvement, error occurred
due to human involvement and also reduces the cost of tracking of commercials. The work
presented here is carried out with intention of fulfilling above discussed objectives.
A Albiol et al. detect T.V. commercial based on assumption that T.V. or network logo
disappear during advertisements [1]. J H Yeh et al deal with the problem of commercial
detection by two-level hard cut based commercial detection scheme for TV news programs
[2]. Pinar Duygulu et al in his study proposed a method that combines two scheme of
INTERNATIONAL JOURNAL OF COMPUTER ENGINEERING
& TECHNOLOGY (IJCET)
ISSN 0976 – 6367(Print)
ISSN 0976 – 6375(Online)
Volume 4, Issue 3, May-June (2013), pp. 86-92
© IAEME: www.iaeme.com/ijcet.asp
Journal Impact Factor (2013): 6.1302 (Calculated by GISI)
www.jifactor.com
IJCET
© I A E M E

2. International Journal of Computer Engineering and Technology (IJCET), ISSN 0976-
6367(Print), ISSN 0976 – 6375(Online) Volume 4, Issue 3, May – June (2013), © IAEME
87
commercial detection one of which is based on repeated sequence detection where author
assumes that commercial repeats itself and uses this fact for commercial detection and
another method is based on FLD classifier that uses distinctive color and audio features for
commercial detection [3]. Z Cao proposed commercial boundary detection method based on
reference commercial database which is made of commercial samples [4]. M Mizutani
suggested a method that differentiate between commercial and program segment using
discriminative classifier that is based on audio/visual features of video segment [5].
The proposed approach for commercial break detection is based on frequency of cuts
or scene change rate in a group of frames and appearance of caption text. We have
concentrated our work on SONY TV serial videos. SONY TV is India’s leading
entertainment channel. The same work can be extended to serials of other channels as well
with little modification.
The general analysis of SONY TV serial video generates some interesting observation
that length of each commercial normally ranges from 15sec -2 minutes, normally 2
commercial blocks appears in a serial of 30 and 60 minutes duration and Length of each
commercial block is in the range of 3 – 5 minutes in a serial of 30 minutes duration.
The paper is further organized as follows. Section II gives the overview of proposed
commercial detection algorithms. Section III discusses how the proposed scheme identifies
the commercial segment or commercial break and section IV and V discus the technique used
to refines the boundary of identified commercial segment. Section VI gives the experimental
results and Section VII concludes the paper with related future work.
II. PROPOSED SCHEME : AT A GLANCE
Below is the block diagram that explains the proposed commercial detection scheme.
The proposed scheme identifies commercial break detection as two-step process.
Figure 1. The block diagram of the proposed commercial detection system
In step-1 approximate commercial break, commercial group of frames who’s exact
starting and ending frame is not known, is identified and fed as input in Step-2. Step-2 helps
in refining the boundary of commercial break. That is in step-2 exact starting and ending
commercial frames are identified.
Step-1 uses the concept of frequency of cuts that is scene change rate in each block
for finding approximate commercial break. The frequency of cuts present in commercial is
high as compared to serials as commercial ad producer has to show more information in short
span of time. Ad producer has to attract the viewers as viewers are normally not as much
interested in ads as they are in TV serials and tend to change the channels during ad duration
or take break to do some work. Therefore ad producer uses more number of cuts and use
only specific colors for ad to emphasize product and attract the viewer’s [2]. Hence the area
where frequency of cut identified by putting high threshold is more, can be identified as
commercial break.

3. International Journal of Computer Engineering and Technology (IJCET), ISSN 0976-
6367(Print), ISSN 0976 – 6375(Online) Volume 4, Issue 3, May – June (2013), © IAEME
88
Step-2 identifies exact start of commercial segment by the appearance of caption text
“AAGE DEKHIYE” which gives brief story upcoming for the serial after break. Step-2
identifies exact end of commercial segment as the last cut identified in commercial segment.
III. IDENTIFING THE COMMERCIAL BREAK
Cut or shot change is transitions between successive shots. A “cut” can also be
defined as is the editing between two individual continuous camera shots [9]. The proposed
scheme process the video in a bunch of 1000 frames each. Each bunch of 1000 frames is
termed as block. The scheme identifies number of cuts in each block. Equation (1) given
below can be used to compute the cut which is based on histogram based shot boundary
detection technique in RGB space. If number of cuts in a block is greater than thresh-value
then this group is identified as candidate commercial block. If 6 or more consecutive blocks
contains candidate commercial block then these consecutive blocks are identified as
commercial segment or commercial break. This fact is based on fact that commercial break in
SONY TV serials normally ranges in 3-5 minutes duration.
( , 1) | ( ) ( 1) | | ( ) ( 1) | | ( ) ( 1) |
IR IG IB
Diff i i h i h i h i h i h i h i+ = − + + − + + − +∑ ∑ ∑ ……………. (1)
Here h(i) is the histogram of ith
frame and h(i+1) is the histogram of (i+1)th
frame.
The absolute sum of histogram difference between all the three red, green and blue
component gives the Diff( i , i+1 ) that is the difference between frame i and (i+1). Likewise
find out the histogram difference between all the frames of a video and all the frames
between which this difference Diff( i , i+1 ) exceed some threshold value is identified as
location of shot change.
IV. REFINING THE BOUNDARIES OF IDENTIFIED COMMERCIAL
SEGMENT
Each identified commercial segment has 2 boundaries, start and end boundary. For
identifying exact starting commercial frame, identify the presence of “AAGE DEKHIYE”
caption text in each frame of the identified first candidate commercial block and in each
frame of the surrounding blocks. The observation of SONY TV serials leads us to conclusion
that “AAGE DEKHEYE” caption text is appearing in bottom-left corner of screen and the
font and style of appearance of this text always follow same pattern. Hence the presence of
caption text is identified using template detection technique discussed in [8].
A) Algorithm for detecting exact start frame of commercial break
1. Let 1st
candidate commercial block of identified commercial break be known as F1.
2. Detect the presence of the “AAGE DEKHIYE” caption in each frame of the F1 block,
and blocks surrounded by F1 block, using template detection algorithm. The template
detection algorithm will return the correlation coefficient.
3. Put the flag = true if correlation coefficient is greater than 0.8. This indicates presence
of caption text “AAGE DEKHIYE” in frame.
4. If flag = true and difference between correlation coefficient of consecutive frames is
greater than thresh-value then the frame where difference is greater than thresh-value is
declared as first frame of commercial break.

4. International Journal of Computer Engineering and Technology (IJCET), ISSN 0976-
6367(Print), ISSN 0976 – 6375(Online) Volume 4, Issue 3, May – June (2013), © IAEME
89
5. If flag = false then first frame of the FIRST block is declared as first frame of
commercial break. As it gives the first position where maximum color difference is
appearing as might be the case in serial and commercial. Therefore it is the highest
probability where first commercial is starting.
Below is the graph that shows the calculation of correlation coefficient in one of the
sample video. Horizontal axis represents frame number and vertical axis is correlation
coefficient. The frame for which correlation coefficient raises above 0.8 value indicates that
recap of serial which will be coming ahead has been started and sudden drop in correlation
value indicates that recap has finished and commercial has started.
Figure 2 : Corelation graph for identifying caption text “Aage dekhiye” in F1 block
Frame 782 in above block Frame 782 in above block
Figure 3 : Frames identified as before and after start of commercial break
B) Algorithm for detecting exact end of commercial break.
1. Let the last block in identified candidate commercial break be known as L1 and the
block following L1 block as N1.
Identified start frame
of commercial is :
frame no 782
last frame of serial start of commercial block

5. International Journal of Computer Engineering and Technology (IJCET), ISSN 0976-
6367(Print), ISSN 0976 – 6375(Online) Volume 4, Issue 3, May – June (2013), © IAEME
90
2. Check the number of cuts in N1 block. If numbers of cuts in N1 block are equal to
(thresh_value-1) or more then the first cut frame in the N1 block is identified as exact
end of commercial block.
3. Else last cut frame in L1 block is considered as the exact end of commercial break.
0 100 200 300 400 500 600 700 800 900 1000
0
2
4
6
8
10
12
14
x 10
5
Figure 4: Cut detector result for L1 block
Frame 327 in above block Frame 340 in above block
Figure 5: Frames identified as after and before end of commercial break
Frames 328-339 are blank frames in above block. Frame 339 is identified as last
frame of commercial break by proposed algorithm.
V. OUTLIER REMOVAL
Some Serials may have many shot changes or high motion. In such serials some
program frames may be falsely identified as commercial break. For any such serial videos
more than 2 commercial breaks are identified and some extra processing is needed to identify
true commercial breaks and reject false breaks. The fact that SONY TV serials contain
exactly 2 commercial breaks is utilized over here. To identify true commercial break search
for SONY TV logo is carried in bottom area of each frame of identified commercial breaks.
The break which contains at least one SONY TV logo is considered as true commercial break
and the breaks in which no logo is detected is identified as false break.
Last cut
identified at
frame : 340 of
this block
Threshold
value

6. International Journal of Computer Engineering and Technology (IJCET), ISSN 0976-
6367(Print), ISSN 0976 – 6375(Online) Volume 4, Issue 3, May – June (2013), © IAEME
91
Figure 6. (a) Definition of bottom area of each frame, (b) and (c) are commercial
frames containing SONY TV logos
VI. EXPERIMENTAL RESULTS
Experiments are carried in Matlab R2010b on the intel core 2 duo 2.40 GHz processor
with 3 GB RAM . Our experiments are carried out on almost all SONY TV serial videos, where
all videos are captured with the help of tv tuner card. Almost 22 serial videos are checked where
13 of them are of almost 30 minutes serials 3 are almost 60 minutes serials and one is 40 minutes
serial shoot. Out of 22 videos we have to carry out outlier removal for 2 videos. Precision and
Recall for 8 videos are displayed in Table 1 below.
The metrics used to evaluate the proposed algorithm are recall and precision [12]. The formulae
for the two are:
1) Recall Rate = TP/(TP+FN)
2) Precision = TP/(TP+FP)
Where,
The number of frames of commercials correctly identified (true: positive TP)
The number of frames of commercials missed (false negative: FN) And
The number of frames of programs recognized as commercials (false positive: FP)
Table1 : Performance Analysis of Commercial Break Detection
Sr No. serial name Approx duration
in minutes
Recall Precision
1 comedy circus ke ajube 9 min 16 sec 100% 100%
2 parvarish kuch khati kuch mithi 30 min 28 sec 100% 100%
3 Bade aache lagte he 30 min 38 sec 99.99% 100%
4 dil ki nazar se khubsurat 27 min 59 sec 100% 99.85%
5 kya hua tera vada 30 min 19 sec 100% 89.96%
6 kuch tooh log kahenge 29 min 15 sec 85.68% 96.70%
7 anamika 30 min 46 sec 100% 95.73%
8 comedy circus ke ajube 55 min 30 sec 95.05% 100%
VII. CONCLUSION AND FUTURE ENHANCEMENT
By video scene change rate and caption detection, an effective commercial break
detection scheme is proposed for SONY TV serial videos. This work can be easily extended to
serials of other broadcast networks. However as the technology changes, the TV commercial
broadcasting strategies also changes. Hence this work may need to be enhanced ahead to suit
changing requirements. The result can be fastened by implementing the proposed scheme in some
other simulating software such as open CV.

7. International Journal of Computer Engineering and Technology (IJCET), ISSN 0976-
6367(Print), ISSN 0976 – 6375(Online) Volume 4, Issue 3, May – June (2013), © IAEME
92
REFERENCES
[1] A Albiol, M J Ch, Fulla, A Albiol, L Torres “DETECTION OF TV COMMERCIALS,”
Acoustic Speech and Signal Processing,2004.
[2] Jen-Hao Yeh , Jun-Cheng Chen , Jin-Hau Kuo ,Ja-Ling Wu, “TV Commercial
Detection in News Program Videos”, IEEE 2005
[3] Pinar Duygulu , Ming-yu Chen and Alexander Hauptmann , “Comparison and
Combination of Two Novel Commercial Detection Methods” The 2004 International
Conference on multimedia and Expo(ICME’04), Taipei, June 27-30,2004.
[4] Zheng Cao, Ming Zhu, “An Efficient Method of TV Commercial Detection and
Retrieval.” URL: www.tech-ex.com/article-images/448644/1-14.pdf.
[5] Masami Mizutani, Shahram Ebadollahi, Shih-Fu Chang, “ COMMERCIAL
DETECTION IN HETEROGENEOUS VIDEO STREAMS USING FUSED
MULTIMODAL AND TEMPORAL FEATURES”, ADVENT Technical Report, NO.
204-2004-4, http://www.ee.columbia.edu/dvmm/newPublication.htm
[6] Mr. Sandip T. Dhagdi, Dr. P.R. Deshmukh, “ Keyframe Based Video Summarization
Using Automatic Threshold & Edge Matching Rate”, International Journal of Scientific
and Research Publications, Volume 2, Issue 7, July 2012 , ISSN 2250-3153.
[7] R. Kasturi, S.H. Strayer, U. Gargi, S. Antani, “An Evaluation of Color Histogram Based
Methods in Video Indexing”, Department of Computer Science and Engineering
Technical Report CSE-96-053, September 15, 1996.
[8] Radu ARSINTE,“Aspects of algorithm development for systems with online detection
and recognition of TV commercials”,ACTA TECHNICA NAPOCENSIS,Electronics
and telecommunication, Volume 50,Number 4, 2009.
[9] Weiming Hu, Nianhua Xie, Li Li, Xianglin Zeng and Stephen MayBank, “A survey on
Visual Content Based video indexing and retrieval”, IEEE TRANSACTIONS ON
SYSTEMS, MAN, AND CYBERNETICS- PART : APPLICATIONS AND
REVIEWS, VOL. 41,NO. 6, NOVEMBER 2011.
[10] Ronald Glasberg, Cengiz Tas and Thomas Sikora, “RECOGNIZING COMMERCIALS
IN REAL-TIME USING THREE VISUAL DESCRIPTORS AND A DECISION-
TREE”, IEEE,2006.
[11] Chen, Jun-Cheng Chen, et al. “Improvement of Commercial Boundary Detection Using
Audiovisual Features”, Advances in Multimedia Processing-PCM(2005): 776-786
[12] Ganesh Ramesh, Amit Bagga, “A Text-based Method for Detection and Filtering of
Commercial Segments in Broadcast News.”
[13] Khushboo Khurana and Dr. M. B. Chandak, “Key Frame Extraction Methodology for
Video Annotation”, International Journal of Computer Engineering & Technology
(IJCET), Volume 4, Issue 2, 2013, pp. 221 - 228, ISSN Print: 0976 – 6367,
ISSN Online: 0976 – 6375.
[14] Gagandeep Kaur and Sumeet Kaur, “A Novel Approach for Real Time Moving Object
Detection”, International Journal of Computer Engineering & Technology (IJCET),
Volume 3, Issue 2, 2012, pp. 344 - 353, ISSN Print: 0976 – 6367, ISSN Online: 0976 –
6375.