2. Compression
p
• Compression – Representing information by less
bit than the original information
• Lossless Compression – Original information
and compressed information are identical
identical.
example LZ, TAR and other compression
techniques.
techniques
• Lossy Compression – Compressed info is not the
same as uncompressed info E ample MP3
ncompressed info. Example: MP3,
JPEG etc
•LLossy compression is often MODEL B
i i ft Based d
Compression
Fast Forward Your Development www.dsp-ip.com
3. Compression terms
p
• Encoder – Module which compress the
information
• Decoder – Module which decompress the
information
• CODEC – (en)CODer / DEcoder
• Channel – the medium which the information is
passed through for example ADSL line or disk
Channel Decoder
Encoder
Disk
Fast Forward Your Development www.dsp-ip.com
4. Sampling
p g
• Digitization of analog signal
• Voice - Sampling in time only The number of
only.
samples per second is the Frequency – F. in
telephony,
telephony voice sampling frequency is 8KHz to
transfer 4KHz of sound.
•IImage - SSampling in 2D space - F F O thi
li i Fx, Fy. On this
screen the images are sampled 1024x768
• Vid - S
Video Sampling i 2D space and ti
li in D d time – F F
Fx, Fy,
Fn. On TV Fn is 25 or 30 fps (frames per second)
Fast Forward Your Development www.dsp-ip.com
5. Quantization
• Lowering the number of bits needed to describe
an object
object.
• For example I scored 66 in the test an object with
100 steps need 7 description bits
bits.
• How is she looking ? 10 ! Needs only 5 bits for
description.
d i ti
• Could we lower the number of bits more? What
is th
i the minimum?
i i ?
• Quantization is not always linear - ADPCM
Fast Forward Your Development www.dsp-ip.com
7. Model Base Transform - example
• Transforming the information from a
representation of similar importance
to all variables to a representation of
varying importance
• Th “poor sheriff” problem
The “ h iff” bl
▫ The sheriff needs to telegram the
dangers criminal profile t
d i i l fil to
headquarters.
▫ A telegram costs a cent for each
bit.
▫ Sending it bit by bit would finish
the poor sheriff salary.
Fast Forward Your Development www.dsp-ip.com
8. Solution
• The sheriff and headquarters
agree on a b
base that would
h ld
describe images according to a
base known to both sides.
• The transformation base is:
▫ Mouth
▫ Nose
▫ Eyes
▫ Ears
• The values are the sizes of each
item quantized to millimeters
• Mad Max will be transmitted as:
8,14,12,17
8 14 12 17
Fast Forward Your Development www.dsp-ip.com
9. Model Based Compression (Multimedia)
Pre- Model Based Quantize / Entropy
Processing Transform Prioritize Reorder Codingg
Fast Forward Your Development www.dsp-ip.com
10. Human Visual System
• The human eye has two basic light receptors:
▫ Rods – Light Intensity receptors
▫ Cons – Colored light receptors
•
Fast Forward Your Development www.dsp-ip.com
11. The Human Eye
• Rods Concentration >> Cons Concentration
• Green Discrimination << Red, Blue
Red
Discrimination
•LLow Frequency > High Frequency
F Hi h F
Fast Forward Your Development www.dsp-ip.com
12. Model Based transformations
• RGB (3 equally quantized colors) ->
>
• YUV (Light Intensity + two color channels)
• Pi l based d
Pixel b d domain -> F
i > Frequency d domain
i
Fast Forward Your Development www.dsp-ip.com
13. Speech coding
• In speech coding, the vocal tract is used as a
model:
Fast Forward Your Development www.dsp-ip.com
14. Audio / Music Coding
• In general Audio Coding, the ear is used as a
model:
• Frequencies -> Frequency bands
• Masking and Temporal Masking are used
Fast Forward Your Development www.dsp-ip.com
16. What is a digital image?
• Audio PCM
▫ One 1-D array of
sample
• BMP Image
▫ Three 2-D arrays of
2D
numbers
representing Red,
Green and Blue
values
Fast Forward Your Development www.dsp-ip.com
17. Image Compression? Why?
• Image size = 720*580
720 580
• 3 Image Layers RGB =720*580*3
• 8 Bits per pixel 720*580*3*8
720 580 3 8
= 10022400 bits
• L t of bit f one L
Lots f bits for Lena
Fast Forward Your Development www.dsp-ip.com
18. Color matching
• How can we compare
colors so that the
content creators and
consumers know what
kno hat
they are seeing?
•MMany diff
different ways
t
including CIE
chromacity di
h it diagram
Fast Forward Your Development www.dsp-ip.com
19. Video Color Transforms
• Largely derived from older analog methods of coding
color for TV. Luminance is separated from color
TV
information.
• YIQ is used to transmit TV signals in North America and
Japan.This coding also makes its way into VHS video
tape coding in these countries since video tape
technologies also use YIQ.
• In Europe, video tape uses the PAL or SECAM codings,
which are based on TV that uses a matrix transform
hi h b d h i f
called YUV.
• Finally digital video mostly uses a matrix transform
Finally,
called YCbCr that is closely related to YUV
Fast Forward Your Development www.dsp-ip.com
20. YUV Separation
Fast Forward Your Development www.dsp-ip.com
21. YUV Color Model
•YUV codes a luminance signal (for gamma-corrected
signals) equal to Y , the “luma".
•Chrominance refers to the difference between a color
and a reference white at the same luminance. (U and V)
The transform is:
Fast Forward Your Development www.dsp-ip.com
23. YIQ Color Model
YIQ is used in NTSC color TV broadcasting.
broadcasting
Again, gray pixels generate zero (I;Q)
chrominance signal.
g
I and Q are a rotated version of U and V .
The transform is:
Th t f i
Fast Forward Your Development www.dsp-ip.com
24. YCbCr Color Model
1. The Rec. 601 standard for digital video uses
another color space YCbCr which closely
related to the YUV transform.
2. The YCbCr transform is used in JPEG image
compression and MPEG video compression.
For 8-bit coding:
Fast Forward Your Development www.dsp-ip.com
25. Sampling
p g
• Digitization of analog signal
• Voice - Sampling in time only The number of
only.
samples per second is the Frequency – F. in
telephony,
telephony voice sampling frequency is 8KHz to
transfer 4KHz of sound.
•IImage - SSampling in 2D space - F F O thi
li i Fx, Fy. On this
screen the images are sampled 1024x768
• Vid - S
Video Sampling i 2D space and ti
li in D d time – F F
Fx, Fy,
Fn. On TV Fn is 25 or 30 fps (frames per second)
Fast Forward Your Development www.dsp-ip.com
26. Quantization
• Lowering the number of bits needed to describe
an object
object.
• For example I scored 66 in the test an object with
100 steps need 7 description bits
bits.
• How is she looking ? 10 ! Needs only 5 bits for
description.
d i ti
• Could we lower the number of bits more? What
is th
i the minimum?
i i ?
• How to describe an object with one bit?
Fast Forward Your Development www.dsp-ip.com
27. Color based decimation
• Our eyes have better resolution and scaling for
luminance then for color
color.
• Compress color by using 4:2:0 method
Fast Forward Your Development www.dsp-ip.com
28. Counting the bits
• How much can we save by color compression?
▫ 3*Image size in RGB 24 bit color representation
representation.
▫ 1 + 2*1/4 Image size in 4:2:0 YUV representation.
▫ Compression ratio is 2 !!
• Actual saving is bigger due to different Y and UV
quantization.
quantization
Fast Forward Your Development www.dsp-ip.com
29. Linear Transform
• If the signal is formatted as a
g • Energy compaction p p y
gy p property:
vector, a linear transform can The transformed signal vector
be formulated as a matrix- has few, large coefficients and
vector product that transform many nearly zero small
the signal into a different coefficients. These few large
domain. coefficients can be encoded
• Examples: efficiently with few bits while
▫ K-L Transform retaining the majority of
▫ Discrete Fourier Transform energy of the original signal.
▫ Discrete cosine transform
▫ Discrete wavelet transform
Fast Forward Your Development www.dsp-ip.com
30. Block-based Image Coding
• Block-based image
Block based • Advantages:
coding scheme: ▫ Parallel processing can
partitions the entire be applied to p
pp process
image into 8 by 8 or individual blocks in
parallel.
16 by 16 (or other
▫ Redundant information
size) blocks. in close proximity (like
• The coding algorithm cache)
is applied to
individual blocks
independently.
Fast Forward Your Development www.dsp-ip.com
31. Transform - DCT
• The DCT transform the data from pixel intensity
to frequency intensity
intensity.
• Low frequency are important high frequency less
1 7 7 (2m 1)u (2n 1)v
• (You’ll F (u, v) cos even if you don’t remember the
4 get launch
u 0 v 0 16
cos
16
m n 0;
f (m, n) 7 7
IDCT1f
formula above) 1)u (2n 1)v
l b (2m)
8
F (u , v) cos cos 0 m, n 7; m n 0.
u 0 v 0 16 16
Fast Forward Your Development www.dsp-ip.com
33. AC Coefficients
• AC coefficients are first
weighted with a quantization
i h d ih i i 1 2 6 7 15 16 28 29
matrix: 3 5 8 14 17 27 30 43
C(i,j)/q(i,j)
C(i j)/q(i j) = Cq(i j)
(i,j) 4 9 13 18 26 31 42 44
Then quantized. 10 12 19 25 32 41 45 54
• Then they are scanned in a 11 20 24 33 40 46 53 55
zig-zag order into a 1D 21 23 34 39 47 52 56 61
sequence to be subject to AC 22 35 38 48 51 57 60 62
Huffman encoding.
ff di
36 37 49 50 58 59 63 64
• Question: Given a 8 by 8
array,
array how to convert it into a
Zig-Zag scan order
vector according to the zig-
zag scan order? What is the
g
algorithm?
Fast Forward Your Development www.dsp-ip.com
39. JPEG Image Coding Algorithms
Q
Quantization DC
8x8 Matrix DC DPCM Huffman
block
DCT Q
Zig Zag AC
AC Scan Huffman
Code books
JPEG Encoding Process
Fast Forward Your Development www.dsp-ip.com
40. Generalization of JPEG Coding
Transform Entropy
Color,
Color Frequency Quantize Reorder Coding
JPEG Encoding Process
Fast Forward Your Development www.dsp-ip.com
42. Video Coding
• Video coding is often implemented as encoding
a sequence of i
f images.Motion compensation
i i
is used to exploit temporal redundancy between
successive f
i frames.
• Examples: MPEG-I, MPEG-II, MPEG-IV,
H.263, H.263+, H264
• Existing video coding standards are based on
JPEG image compression as well as motion
compensation.
Fast Forward Your Development www.dsp-ip.com
43. Video Coding Standardization
Scope
• Only restrictions on the Bitstream, Syntax, and
Decoder are standardized:
▫ Permits the optimization of encoding
▫ Permits complexity reduction
p y
▫ Provides no guarantees on quality
Fast Forward Your Development www.dsp-ip.com
44. Video Encoding
Buffer control
Current
frame x(t) r Bit stream
+ DCT Q VLC Buffer
Q-1 This is a simplified block
diagram where the
encoding of intra coded
IDCT frames is not shown.
Xp(t): predicted ^ reconstructed residue
r(t):
frame
+
^(t)
x(t): reconstructed
t t d
Motion ^ current frame
x(t) x(t-1) Frame
Estimation &
Compensation Buffer
Motion vectors
Fast Forward Your Development www.dsp-ip.com
45. Video Encoding
Color Frequency
Transform Buffer control
Transform
+ Q Reorder Entropy
Q-1 This is a simplified block
diagram where the
encoding of intra coded
Tf-1 frames is not shown.
Xp(t): predicted ^ reconstructed residue
r(t):
frame
+
^(t)
x(t): reconstructed
t t d
Motion ^ current frame
x(t) x(t-1) Frame
Estimation &
Compensation Buffer
Motion vectors
Fast Forward Your Development www.dsp-ip.com
46. Forward Motion Estimation
1 2 3 4 1 2 4
3
5 6 7 8 5 7 8
6
9 10 11 12 9 11 12
10
0
13 15 16
13 14 15 16 14
Current frame constructed From
different parts of reference frame Reference frame
Fast Forward Your Development www.dsp-ip.com
47. Video sequence : Tennis frame 0, 1
previous frame current frame
50 50
100 100
150 150
200 200
50 100 150 200 250 300 350 50 100 150 200 250 300 350
Fast Forward Your Development www.dsp-ip.com
48. Frame Difference
Frame Difference :frame 0 and 1
Fast Forward Your Development www.dsp-ip.com
49. What is motion estimation?
Motion Vector Field of frame 1
50
0
-50
-100
-150
-200
-250
0 50 100 150 200 250 300 350 400
Fast Forward Your Development www.dsp-ip.com
50. What is motion compensation ?
Motion compensated frame
50
100
150
200
50 100 150 200 250 300 350
Fast Forward Your Development www.dsp-ip.com
51. Motion Compensated Frame Difference
p
Motion Compensated Frame Difference :frame 0 and 1
Frame Difference :frame 0 and 1
Fast Forward Your Development www.dsp-ip.com
52. Frame Types
• Three types of frames:
yp
▫ Intra (I): the frame is coded as if it is an image
▫ Predicted (P): predicted from an I or P frame
▫ Bi-directional (B): forward and backward predicted
from a pair of I or P frames.
• A typical f
i l frame arrangement i is:
I1 B1 B2 P1 B3 B4 P2 B5 B6 I2
• P1, P2 are b h f
both forward-predicted f
d di d from I1. B1, B2 are
interpolated from I1 and P1, B3, B4 are interpolated
from P1, P2, and B5, B6 are interpolated from P2, I2.
• New Coding standards added other frame types: SP,
SI, D
Fast Forward Your Development www.dsp-ip.com
53. Macro-blocks and Blocks
Y(16x16) Cr (8x8)
RGB
Cb (8x8)
16x16x3
Fast Forward Your Development www.dsp-ip.com