SlideShare ist ein Scribd-Unternehmen logo
1 von 108
Downloaden Sie, um offline zu lesen
INFORMATION THEORY AND
CODING
5th SEM E&C
JAYANTHDWIJESH H P M.tech (DECS)
Assistant Professor – Dept of E&C
B.G.S INSTITUTE OF TECHNOLOGY (B.G.S.I.T)
B.G Nagara, Nagamangala Tq, Mandya District- 571448
FORMULAS FOR REFERENCE
MODULE –3 ( INFORMATION CHANNELS)
 The entropy of input symbol is given by
H (A) = 𝑷(𝒂𝒊)𝐥𝐨𝐠(
𝟏
𝑷 (𝒂 𝒊)
𝒓
𝒊=𝟏 ) bits/ message-symbol
 The entropy of output symbol is given by
H (B) = 𝑷(𝒃𝒋)𝐥𝐨𝐠(
𝟏
𝑷(𝒃 𝒋)
𝒔
𝒋=𝟏 ) bits/ message-symbol
 The joint entropy is given by
H (A, B) = 𝑷( 𝒂𝒊, 𝒃𝒋)𝐥𝐨𝐠(
𝟏
𝑷( 𝒂 𝒊,𝒃 𝒋)
)𝒔
𝒋=𝟏
𝒓
𝒊=𝟏 bits/ message-symbol
 The equivocation H(A/B) is given by
H (A / B) = 𝑷( 𝒂𝒊, 𝒃𝒋)𝐥𝐨𝐠(
𝟏
𝑷 ( 𝒂 𝒊/𝒃 𝒋)
)𝒔
𝒋=𝟏
𝒓
𝒊=𝟏 bits/ message-symbol
Since P( 𝒂𝒊/𝒃𝒋) =
𝑷( 𝒂 𝒊,𝒃 𝒋)
𝑷 (𝒃 𝒋
)
And another formula
H (A/B) = H (A, B) – H (B) bits/ message-symbol
 The equivocation H(B/A) is given by
H (B / A) = 𝑷( 𝒂𝒊, 𝒃𝒋)𝐥𝐨𝐠(
𝟏
𝑷(𝒃 𝒋/ 𝒂 𝒊)
)𝒔
𝒋=𝟏
𝒓
𝒊=𝟏 bits/ message-symbol
Since P(𝒃𝒋/ 𝒂𝒊) =
𝑷( 𝒂 𝒊,𝒃 𝒋)
𝑷 (𝒂 𝒊)
And another formula
H (B/A) = H (A, B) – H (A) bits/ message-symbol
 The mutual information is given by
I (A, B) = H (A) – H (A/B) bits/ message-symbol
 The channel capacity is given by
C = (log s – h) 𝒓 𝒔 bits/ sec
Since h= 𝐇
𝐁
𝐀
= 𝑷𝒋 𝐥𝐨𝐠(
𝟏
𝑷 𝒋
𝒔
𝒋=𝟏 ) bits/ message-symbol
 The capacity of the channel is given by
C=B 𝒍𝒐𝒈 𝟐 [1 + S/N]
 Source efficiency 𝜼 𝑺 is given by
𝜼 𝑺 =
𝑯(𝑿)
𝑯 𝑿 𝒎𝒂𝒙
x 100%
 Source efficiency 𝑹 𝜼 𝑺
is given by
𝑹 𝜼 𝑺
= 𝟏 − 𝜼 𝑺
 Channel efficiency is given by
𝜼 𝒄𝒉 =
𝐈(𝐗,𝐘)
𝐂
x 100%
 Channel redundancy is given by
Channel redundancy = (1- 𝜼 𝒄𝒉) x 100%
 Estimation of channel capacity by MUROGA’S method
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).
VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).

Weitere ähnliche Inhalte

Was ist angesagt?

MATLAB IMPLEMENTATION OF SELF-ORGANIZING MAPS FOR CLUSTERING OF REMOTE SENSIN...
MATLAB IMPLEMENTATION OF SELF-ORGANIZING MAPS FOR CLUSTERING OF REMOTE SENSIN...MATLAB IMPLEMENTATION OF SELF-ORGANIZING MAPS FOR CLUSTERING OF REMOTE SENSIN...
MATLAB IMPLEMENTATION OF SELF-ORGANIZING MAPS FOR CLUSTERING OF REMOTE SENSIN...Daksh Raj Chopra
 
Bloom Filters: An Introduction
Bloom Filters: An IntroductionBloom Filters: An Introduction
Bloom Filters: An IntroductionIRJET Journal
 
Genetic programming
Genetic programmingGenetic programming
Genetic programmingYun-Yan Chi
 
Distributed Computing Seminar - Lecture 2: MapReduce Theory and Implementation
Distributed Computing Seminar - Lecture 2: MapReduce Theory and ImplementationDistributed Computing Seminar - Lecture 2: MapReduce Theory and Implementation
Distributed Computing Seminar - Lecture 2: MapReduce Theory and Implementationtugrulh
 
Lec5 pagerank
Lec5 pagerankLec5 pagerank
Lec5 pagerankCarlos
 
Pagerank (from Google)
Pagerank (from Google)Pagerank (from Google)
Pagerank (from Google)Sri Prasanna
 
Self-Organising Maps for Customer Segmentation using R - Shane Lynn - Dublin R
Self-Organising Maps for Customer Segmentation using R - Shane Lynn - Dublin RSelf-Organising Maps for Customer Segmentation using R - Shane Lynn - Dublin R
Self-Organising Maps for Customer Segmentation using R - Shane Lynn - Dublin Rshanelynn
 

Was ist angesagt? (7)

MATLAB IMPLEMENTATION OF SELF-ORGANIZING MAPS FOR CLUSTERING OF REMOTE SENSIN...
MATLAB IMPLEMENTATION OF SELF-ORGANIZING MAPS FOR CLUSTERING OF REMOTE SENSIN...MATLAB IMPLEMENTATION OF SELF-ORGANIZING MAPS FOR CLUSTERING OF REMOTE SENSIN...
MATLAB IMPLEMENTATION OF SELF-ORGANIZING MAPS FOR CLUSTERING OF REMOTE SENSIN...
 
Bloom Filters: An Introduction
Bloom Filters: An IntroductionBloom Filters: An Introduction
Bloom Filters: An Introduction
 
Genetic programming
Genetic programmingGenetic programming
Genetic programming
 
Distributed Computing Seminar - Lecture 2: MapReduce Theory and Implementation
Distributed Computing Seminar - Lecture 2: MapReduce Theory and ImplementationDistributed Computing Seminar - Lecture 2: MapReduce Theory and Implementation
Distributed Computing Seminar - Lecture 2: MapReduce Theory and Implementation
 
Lec5 pagerank
Lec5 pagerankLec5 pagerank
Lec5 pagerank
 
Pagerank (from Google)
Pagerank (from Google)Pagerank (from Google)
Pagerank (from Google)
 
Self-Organising Maps for Customer Segmentation using R - Shane Lynn - Dublin R
Self-Organising Maps for Customer Segmentation using R - Shane Lynn - Dublin RSelf-Organising Maps for Customer Segmentation using R - Shane Lynn - Dublin R
Self-Organising Maps for Customer Segmentation using R - Shane Lynn - Dublin R
 

Ähnlich wie VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).

Information Theory and Coding - Lecture 5
Information Theory and Coding - Lecture 5Information Theory and Coding - Lecture 5
Information Theory and Coding - Lecture 5Aref35
 
MIMO Capacity and Duality Between MAC and BC
MIMO Capacity and Duality Between MAC and BCMIMO Capacity and Duality Between MAC and BC
MIMO Capacity and Duality Between MAC and BCShristi Pradhan
 
Analog Communication fast mode material.pdf
Analog Communication fast mode material.pdfAnalog Communication fast mode material.pdf
Analog Communication fast mode material.pdfEngineering Funda
 
Physical security with power beacon assisted in half-duplex relaying networks...
Physical security with power beacon assisted in half-duplex relaying networks...Physical security with power beacon assisted in half-duplex relaying networks...
Physical security with power beacon assisted in half-duplex relaying networks...TELKOMNIKA JOURNAL
 
Art%3 a10.1155%2fs1110865704401036
Art%3 a10.1155%2fs1110865704401036Art%3 a10.1155%2fs1110865704401036
Art%3 a10.1155%2fs1110865704401036SANDRA BALSECA
 
Direction of arrival estimation using music algorithm
Direction of arrival estimation using music algorithmDirection of arrival estimation using music algorithm
Direction of arrival estimation using music algorithmeSAT Journals
 
Direction of arrival estimation using music algorithm
Direction of arrival estimation using music algorithmDirection of arrival estimation using music algorithm
Direction of arrival estimation using music algorithmeSAT Publishing House
 
Pilot based channel estimation improvement in orthogonal frequency-division m...
Pilot based channel estimation improvement in orthogonal frequency-division m...Pilot based channel estimation improvement in orthogonal frequency-division m...
Pilot based channel estimation improvement in orthogonal frequency-division m...IJECEIAES
 
129966864405916304[1]
129966864405916304[1]129966864405916304[1]
129966864405916304[1]威華 王
 
Channel and clipping level estimation for ofdm in io t –based networks a review
Channel and clipping level estimation for ofdm in io t –based networks a reviewChannel and clipping level estimation for ofdm in io t –based networks a review
Channel and clipping level estimation for ofdm in io t –based networks a reviewIJARIIT
 

Ähnlich wie VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme). (11)

Information Theory and Coding - Lecture 5
Information Theory and Coding - Lecture 5Information Theory and Coding - Lecture 5
Information Theory and Coding - Lecture 5
 
MIMO Capacity and Duality Between MAC and BC
MIMO Capacity and Duality Between MAC and BCMIMO Capacity and Duality Between MAC and BC
MIMO Capacity and Duality Between MAC and BC
 
Analog Communication fast mode material.pdf
Analog Communication fast mode material.pdfAnalog Communication fast mode material.pdf
Analog Communication fast mode material.pdf
 
Physical security with power beacon assisted in half-duplex relaying networks...
Physical security with power beacon assisted in half-duplex relaying networks...Physical security with power beacon assisted in half-duplex relaying networks...
Physical security with power beacon assisted in half-duplex relaying networks...
 
Art%3 a10.1155%2fs1110865704401036
Art%3 a10.1155%2fs1110865704401036Art%3 a10.1155%2fs1110865704401036
Art%3 a10.1155%2fs1110865704401036
 
Direction of arrival estimation using music algorithm
Direction of arrival estimation using music algorithmDirection of arrival estimation using music algorithm
Direction of arrival estimation using music algorithm
 
Direction of arrival estimation using music algorithm
Direction of arrival estimation using music algorithmDirection of arrival estimation using music algorithm
Direction of arrival estimation using music algorithm
 
Pilot based channel estimation improvement in orthogonal frequency-division m...
Pilot based channel estimation improvement in orthogonal frequency-division m...Pilot based channel estimation improvement in orthogonal frequency-division m...
Pilot based channel estimation improvement in orthogonal frequency-division m...
 
129966864405916304[1]
129966864405916304[1]129966864405916304[1]
129966864405916304[1]
 
Channel and clipping level estimation for ofdm in io t –based networks a review
Channel and clipping level estimation for ofdm in io t –based networks a reviewChannel and clipping level estimation for ofdm in io t –based networks a review
Channel and clipping level estimation for ofdm in io t –based networks a review
 
40220130405004
4022013040500440220130405004
40220130405004
 

Mehr von Jayanth Dwijesh H P

Network and cyber security module(15ec835, 17ec835)
Network and cyber security module(15ec835, 17ec835)Network and cyber security module(15ec835, 17ec835)
Network and cyber security module(15ec835, 17ec835)Jayanth Dwijesh H P
 
VTU Network & cyber security (1-5 Module) Full notes
VTU Network & cyber security (1-5 Module) Full notesVTU Network & cyber security (1-5 Module) Full notes
VTU Network & cyber security (1-5 Module) Full notesJayanth Dwijesh H P
 
Vtu network security(10 ec832) unit 3 notes.
Vtu network security(10 ec832) unit 3 notes.Vtu network security(10 ec832) unit 3 notes.
Vtu network security(10 ec832) unit 3 notes.Jayanth Dwijesh H P
 
VTU E&C,TCE CBCS[NEW] 5th Sem Information Theory and Coding Module-5 notes(15...
VTU E&C,TCE CBCS[NEW] 5th Sem Information Theory and Coding Module-5 notes(15...VTU E&C,TCE CBCS[NEW] 5th Sem Information Theory and Coding Module-5 notes(15...
VTU E&C,TCE CBCS[NEW] 5th Sem Information Theory and Coding Module-5 notes(15...Jayanth Dwijesh H P
 
VTU E&C,TCE CBCS[NEW] 5th Sem Information Theory and Coding Module-4 notes(15...
VTU E&C,TCE CBCS[NEW] 5th Sem Information Theory and Coding Module-4 notes(15...VTU E&C,TCE CBCS[NEW] 5th Sem Information Theory and Coding Module-4 notes(15...
VTU E&C,TCE CBCS[NEW] 5th Sem Information Theory and Coding Module-4 notes(15...Jayanth Dwijesh H P
 
Network security 10EC832 vtu notes
Network security 10EC832 vtu notesNetwork security 10EC832 vtu notes
Network security 10EC832 vtu notesJayanth Dwijesh H P
 
VTU network security(10 ec832) unit 6 notes
VTU network security(10 ec832) unit 6 notesVTU network security(10 ec832) unit 6 notes
VTU network security(10 ec832) unit 6 notesJayanth Dwijesh H P
 
Vtu network security(10 ec832) unit 5 notes.
Vtu network security(10 ec832) unit 5 notes.Vtu network security(10 ec832) unit 5 notes.
Vtu network security(10 ec832) unit 5 notes.Jayanth Dwijesh H P
 
Vtu network security(10 ec832) unit 2 notes..
Vtu network security(10 ec832) unit 2 notes..Vtu network security(10 ec832) unit 2 notes..
Vtu network security(10 ec832) unit 2 notes..Jayanth Dwijesh H P
 
Vtu network security(10 ec832) unit 1 notes
Vtu network security(10 ec832) unit 1 notesVtu network security(10 ec832) unit 1 notes
Vtu network security(10 ec832) unit 1 notesJayanth Dwijesh H P
 
VTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -5 notes
VTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -5 notesVTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -5 notes
VTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -5 notesJayanth Dwijesh H P
 
VTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -4 notes
VTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -4 notesVTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -4 notes
VTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -4 notesJayanth Dwijesh H P
 

Mehr von Jayanth Dwijesh H P (12)

Network and cyber security module(15ec835, 17ec835)
Network and cyber security module(15ec835, 17ec835)Network and cyber security module(15ec835, 17ec835)
Network and cyber security module(15ec835, 17ec835)
 
VTU Network & cyber security (1-5 Module) Full notes
VTU Network & cyber security (1-5 Module) Full notesVTU Network & cyber security (1-5 Module) Full notes
VTU Network & cyber security (1-5 Module) Full notes
 
Vtu network security(10 ec832) unit 3 notes.
Vtu network security(10 ec832) unit 3 notes.Vtu network security(10 ec832) unit 3 notes.
Vtu network security(10 ec832) unit 3 notes.
 
VTU E&C,TCE CBCS[NEW] 5th Sem Information Theory and Coding Module-5 notes(15...
VTU E&C,TCE CBCS[NEW] 5th Sem Information Theory and Coding Module-5 notes(15...VTU E&C,TCE CBCS[NEW] 5th Sem Information Theory and Coding Module-5 notes(15...
VTU E&C,TCE CBCS[NEW] 5th Sem Information Theory and Coding Module-5 notes(15...
 
VTU E&C,TCE CBCS[NEW] 5th Sem Information Theory and Coding Module-4 notes(15...
VTU E&C,TCE CBCS[NEW] 5th Sem Information Theory and Coding Module-4 notes(15...VTU E&C,TCE CBCS[NEW] 5th Sem Information Theory and Coding Module-4 notes(15...
VTU E&C,TCE CBCS[NEW] 5th Sem Information Theory and Coding Module-4 notes(15...
 
Network security 10EC832 vtu notes
Network security 10EC832 vtu notesNetwork security 10EC832 vtu notes
Network security 10EC832 vtu notes
 
VTU network security(10 ec832) unit 6 notes
VTU network security(10 ec832) unit 6 notesVTU network security(10 ec832) unit 6 notes
VTU network security(10 ec832) unit 6 notes
 
Vtu network security(10 ec832) unit 5 notes.
Vtu network security(10 ec832) unit 5 notes.Vtu network security(10 ec832) unit 5 notes.
Vtu network security(10 ec832) unit 5 notes.
 
Vtu network security(10 ec832) unit 2 notes..
Vtu network security(10 ec832) unit 2 notes..Vtu network security(10 ec832) unit 2 notes..
Vtu network security(10 ec832) unit 2 notes..
 
Vtu network security(10 ec832) unit 1 notes
Vtu network security(10 ec832) unit 1 notesVtu network security(10 ec832) unit 1 notes
Vtu network security(10 ec832) unit 1 notes
 
VTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -5 notes
VTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -5 notesVTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -5 notes
VTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -5 notes
 
VTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -4 notes
VTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -4 notesVTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -4 notes
VTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -4 notes
 

Kürzlich hochgeladen

دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratoryدليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide LaboratoryBahzad5
 
Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...
Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...
Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...amrabdallah9
 
UNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptxUNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptxrealme6igamerr
 
Dev.bg DevOps March 2024 Monitoring & Logging
Dev.bg DevOps March 2024 Monitoring & LoggingDev.bg DevOps March 2024 Monitoring & Logging
Dev.bg DevOps March 2024 Monitoring & LoggingMarian Marinov
 
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS Bahzad5
 
Gender Bias in Engineer, Honors 203 Project
Gender Bias in Engineer, Honors 203 ProjectGender Bias in Engineer, Honors 203 Project
Gender Bias in Engineer, Honors 203 Projectreemakb03
 
Graphics Primitives and CG Display Devices
Graphics Primitives and CG Display DevicesGraphics Primitives and CG Display Devices
Graphics Primitives and CG Display DevicesDIPIKA83
 
Basic Principle of Electrochemical Sensor
Basic Principle of  Electrochemical SensorBasic Principle of  Electrochemical Sensor
Basic Principle of Electrochemical SensorTanvir Moin
 
Landsman converter for power factor improvement
Landsman converter for power factor improvementLandsman converter for power factor improvement
Landsman converter for power factor improvementVijayMuni2
 
Phase noise transfer functions.pptx
Phase noise transfer      functions.pptxPhase noise transfer      functions.pptx
Phase noise transfer functions.pptxSaiGouthamSunkara
 
The relationship between iot and communication technology
The relationship between iot and communication technologyThe relationship between iot and communication technology
The relationship between iot and communication technologyabdulkadirmukarram03
 
Engineering Mechanics Chapter 5 Equilibrium of a Rigid Body
Engineering Mechanics  Chapter 5  Equilibrium of a Rigid BodyEngineering Mechanics  Chapter 5  Equilibrium of a Rigid Body
Engineering Mechanics Chapter 5 Equilibrium of a Rigid BodyAhmadHajasad2
 
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docx
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docxSUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docx
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docxNaveenVerma126
 
Guardians and Glitches: Navigating the Duality of Gen AI in AppSec
Guardians and Glitches: Navigating the Duality of Gen AI in AppSecGuardians and Glitches: Navigating the Duality of Gen AI in AppSec
Guardians and Glitches: Navigating the Duality of Gen AI in AppSecTrupti Shiralkar, CISSP
 
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...Sean Meyn
 
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdf
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdfRenewable Energy & Entrepreneurship Workshop_21Feb2024.pdf
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdfodunowoeminence2019
 
EPE3163_Hydro power stations_Unit2_Lect2.pptx
EPE3163_Hydro power stations_Unit2_Lect2.pptxEPE3163_Hydro power stations_Unit2_Lect2.pptx
EPE3163_Hydro power stations_Unit2_Lect2.pptxJoseeMusabyimana
 
Mohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptxMohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptxKISHAN KUMAR
 

Kürzlich hochgeladen (20)

دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratoryدليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
 
Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...
Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...
Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...
 
UNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptxUNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptx
 
Lecture 2 .pdf
Lecture 2                           .pdfLecture 2                           .pdf
Lecture 2 .pdf
 
Dev.bg DevOps March 2024 Monitoring & Logging
Dev.bg DevOps March 2024 Monitoring & LoggingDev.bg DevOps March 2024 Monitoring & Logging
Dev.bg DevOps March 2024 Monitoring & Logging
 
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS
 
Gender Bias in Engineer, Honors 203 Project
Gender Bias in Engineer, Honors 203 ProjectGender Bias in Engineer, Honors 203 Project
Gender Bias in Engineer, Honors 203 Project
 
Graphics Primitives and CG Display Devices
Graphics Primitives and CG Display DevicesGraphics Primitives and CG Display Devices
Graphics Primitives and CG Display Devices
 
Basic Principle of Electrochemical Sensor
Basic Principle of  Electrochemical SensorBasic Principle of  Electrochemical Sensor
Basic Principle of Electrochemical Sensor
 
Landsman converter for power factor improvement
Landsman converter for power factor improvementLandsman converter for power factor improvement
Landsman converter for power factor improvement
 
Phase noise transfer functions.pptx
Phase noise transfer      functions.pptxPhase noise transfer      functions.pptx
Phase noise transfer functions.pptx
 
The relationship between iot and communication technology
The relationship between iot and communication technologyThe relationship between iot and communication technology
The relationship between iot and communication technology
 
Engineering Mechanics Chapter 5 Equilibrium of a Rigid Body
Engineering Mechanics  Chapter 5  Equilibrium of a Rigid BodyEngineering Mechanics  Chapter 5  Equilibrium of a Rigid Body
Engineering Mechanics Chapter 5 Equilibrium of a Rigid Body
 
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docx
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docxSUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docx
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docx
 
Présentation IIRB 2024 Marine Cordonnier.pdf
Présentation IIRB 2024 Marine Cordonnier.pdfPrésentation IIRB 2024 Marine Cordonnier.pdf
Présentation IIRB 2024 Marine Cordonnier.pdf
 
Guardians and Glitches: Navigating the Duality of Gen AI in AppSec
Guardians and Glitches: Navigating the Duality of Gen AI in AppSecGuardians and Glitches: Navigating the Duality of Gen AI in AppSec
Guardians and Glitches: Navigating the Duality of Gen AI in AppSec
 
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
 
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdf
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdfRenewable Energy & Entrepreneurship Workshop_21Feb2024.pdf
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdf
 
EPE3163_Hydro power stations_Unit2_Lect2.pptx
EPE3163_Hydro power stations_Unit2_Lect2.pptxEPE3163_Hydro power stations_Unit2_Lect2.pptx
EPE3163_Hydro power stations_Unit2_Lect2.pptx
 
Mohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptxMohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptx
 

VTU E&C,TCE CBCS[NEW]5th Sem Information Theory and Coding Module-3 notes(15&17scheme).

  • 1. INFORMATION THEORY AND CODING 5th SEM E&C JAYANTHDWIJESH H P M.tech (DECS) Assistant Professor – Dept of E&C B.G.S INSTITUTE OF TECHNOLOGY (B.G.S.I.T) B.G Nagara, Nagamangala Tq, Mandya District- 571448
  • 2. FORMULAS FOR REFERENCE MODULE –3 ( INFORMATION CHANNELS)  The entropy of input symbol is given by H (A) = 𝑷(𝒂𝒊)𝐥𝐨𝐠( 𝟏 𝑷 (𝒂 𝒊) 𝒓 𝒊=𝟏 ) bits/ message-symbol  The entropy of output symbol is given by H (B) = 𝑷(𝒃𝒋)𝐥𝐨𝐠( 𝟏 𝑷(𝒃 𝒋) 𝒔 𝒋=𝟏 ) bits/ message-symbol  The joint entropy is given by H (A, B) = 𝑷( 𝒂𝒊, 𝒃𝒋)𝐥𝐨𝐠( 𝟏 𝑷( 𝒂 𝒊,𝒃 𝒋) )𝒔 𝒋=𝟏 𝒓 𝒊=𝟏 bits/ message-symbol  The equivocation H(A/B) is given by H (A / B) = 𝑷( 𝒂𝒊, 𝒃𝒋)𝐥𝐨𝐠( 𝟏 𝑷 ( 𝒂 𝒊/𝒃 𝒋) )𝒔 𝒋=𝟏 𝒓 𝒊=𝟏 bits/ message-symbol Since P( 𝒂𝒊/𝒃𝒋) = 𝑷( 𝒂 𝒊,𝒃 𝒋) 𝑷 (𝒃 𝒋 ) And another formula H (A/B) = H (A, B) – H (B) bits/ message-symbol  The equivocation H(B/A) is given by H (B / A) = 𝑷( 𝒂𝒊, 𝒃𝒋)𝐥𝐨𝐠( 𝟏 𝑷(𝒃 𝒋/ 𝒂 𝒊) )𝒔 𝒋=𝟏 𝒓 𝒊=𝟏 bits/ message-symbol Since P(𝒃𝒋/ 𝒂𝒊) = 𝑷( 𝒂 𝒊,𝒃 𝒋) 𝑷 (𝒂 𝒊) And another formula H (B/A) = H (A, B) – H (A) bits/ message-symbol  The mutual information is given by I (A, B) = H (A) – H (A/B) bits/ message-symbol  The channel capacity is given by C = (log s – h) 𝒓 𝒔 bits/ sec Since h= 𝐇 𝐁 𝐀 = 𝑷𝒋 𝐥𝐨𝐠( 𝟏 𝑷 𝒋 𝒔 𝒋=𝟏 ) bits/ message-symbol  The capacity of the channel is given by C=B 𝒍𝒐𝒈 𝟐 [1 + S/N]  Source efficiency 𝜼 𝑺 is given by 𝜼 𝑺 = 𝑯(𝑿) 𝑯 𝑿 𝒎𝒂𝒙 x 100%
  • 3.  Source efficiency 𝑹 𝜼 𝑺 is given by 𝑹 𝜼 𝑺 = 𝟏 − 𝜼 𝑺  Channel efficiency is given by 𝜼 𝒄𝒉 = 𝐈(𝐗,𝐘) 𝐂 x 100%  Channel redundancy is given by Channel redundancy = (1- 𝜼 𝒄𝒉) x 100%  Estimation of channel capacity by MUROGA’S method