Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Nächste SlideShare
×

# A Quick Introduction to the Chow Liu Algorithm

A very simple and quick introduction to the Chow-Liu algorithm.

• Full Name
Comment goes here.

Are you sure you want to Yes No
• Als Erste(r) kommentieren

### A Quick Introduction to the Chow Liu Algorithm

1. 1. A Quick Introduction to the Chow-Liu Algorithm Jee Vang, Ph.D. [email_address]
2. 2. What is the Chow-Liu Algorithm? <ul><li>Reported by Chow and Liu (1968). </li></ul><ul><ul><li>Also known as maximum weight spanning tree (MWST) algorithm or Kruskal’s algorithm. </li></ul></ul><ul><ul><li>Running time complexity is O(n 2 log(n)), where n is number of variables. </li></ul></ul><ul><li>The probability distribution associated to the tree constructed by the Chow-Liu algorithm is the one that is closest to the probability distribution associated to the data as measured by the Kullback-Leibler divergence (Pearl and Dechter 1989). </li></ul><ul><li>One of many uses include constructing graphical relationships between variables. </li></ul><ul><li>Assumptions </li></ul><ul><ul><li>No missing data </li></ul></ul><ul><ul><li>Variables are discrete </li></ul></ul><ul><ul><li>Data is independently and identically distributed </li></ul></ul>Copyright 2009 by Jee Vang
3. 3. Pseudo-code of Chow-Liu Algorithm <ul><li>Denote the following </li></ul><ul><ul><li>A(X i ,X j ) : a measure of association between two variables, X i and X j </li></ul></ul><ul><ul><li>U = {X 1 , X 2 , …, X n } : a set of n variables </li></ul></ul><ul><ul><li>D : a database of cases of the variables in U </li></ul></ul><ul><ul><li>T : a tree whose nodes have a 1-to-1 correspondence to the variables in U </li></ul></ul><ul><li>Inputs to Chow-Liu algorithm </li></ul><ul><ul><li>A, U, D </li></ul></ul><ul><li>Output of Chow-Liu algorithm </li></ul><ul><ul><li>T </li></ul></ul>Copyright 2009 by Jee Vang
4. 4. Pseudo-code of Chow-Liu Algorithm, Procedure <ul><li>Begin </li></ul><ul><ul><li>i <- 1 //a counter, set to one </li></ul></ul><ul><ul><li>L = {L 1 , L 2 , …, L ((n+1)/2)-n } <- all pairwise associations //L is a list of all pairwise associations in descending order; no self-pairing </li></ul></ul><ul><ul><li>P <- L i //P is a pointer to the first pair of variables in L </li></ul></ul><ul><ul><li>construct T //create a disconnected tree with nodes corresponding to variables in U </li></ul></ul><ul><ul><li>Do until n-1 edges have been added </li></ul></ul><ul><ul><ul><li>If a path does not exists between the pair of variables in P, add an undirected arc between them </li></ul></ul></ul><ul><ul><ul><li>P <- L (i+1) //set P to the next pair of variables in L </li></ul></ul></ul><ul><li>End </li></ul>Copyright 2009 by Jee Vang
5. 5. Measure of Association <ul><li>Mutual information was originally used as the measure of association (Chow and Liu 1968) </li></ul><ul><li>However, any measure of association satisfying the following property may be used </li></ul><ul><ul><li>Give three variables, X i , X j , and X k , such that X i and X k are conditionally independent given X j , I(X i ,X j ,X k ), any measure of association, A(X i ,X j ), such that, min(A(X i ,X j ), A(X j ,X k )) > A(X i ,X k ), can be used for the Chow-Liu algorithm (Acid and de Campos 1994). </li></ul></ul>Copyright 2009 by Jee Vang
6. 6. Example—Inputs <ul><li>A(X i ,X j ) : mutual information between two variables, X i and X j </li></ul><ul><li>U = {X 1 , X 2 , X 3 , X 4 , X 5 } : 5 variables </li></ul><ul><ul><ul><li>Domains of X 1 through X 5 are {true, false} </li></ul></ul></ul><ul><li>D : database of cases of X 1 through X 5 </li></ul>Copyright 2009 by Jee Vang X 1 X 2 X 3 X 4 X 5 true false false false false false true true false true false true false false true … … … … …
7. 7. Example—Procedure, Pairwise Associations <ul><li>Compute pairwise associations </li></ul><ul><ul><li>L 1,2 = 0.55 </li></ul></ul><ul><ul><li>L 1,3 = 0.34 </li></ul></ul><ul><ul><li>L 1,4 = 0.11 </li></ul></ul><ul><ul><li>L 1,5 = 0.59 </li></ul></ul><ul><ul><li>L 2,3 = 0.68 </li></ul></ul><ul><ul><li>L 2,4 = 0.03 </li></ul></ul><ul><ul><li>L 2,5 = 0.25 </li></ul></ul><ul><ul><li>L 3,4 = 0.01 </li></ul></ul><ul><ul><li>L 3,5 = 0.22 </li></ul></ul><ul><ul><li>L 4,5 = 0.10 </li></ul></ul><ul><li>Sort pairwise associations descendingly into list L </li></ul><ul><ul><li>L = {L 2,3 , L 1,5 , L 1,2 , L 1,3 , L 2,5 , L 3,5 ,L 1,4 ,L 4,5 , L 2,4 , L 3,4 } </li></ul></ul>Copyright 2009 by Jee Vang
8. 8. Example—Procedure, Tree Construction, Empty Tree <ul><li>L = {L 2,3 , L 1,5 , L 1,2 , L 1,3 , L 2,5 , L 3,5 ,L 1,4 ,L 4,5 , L 2,4 , L 3,4 } </li></ul><ul><li>Construct disconnected tree, T, with 5 nodes corresponding to variables in U </li></ul>X 2 X 3 X 5 X 4 X 1 Copyright 2009 by Jee Vang
9. 9. Example—Procedure, Tree Construction (cont…) <ul><li>L = { L 2,3 , L 1,5 , L 1,2 , L 1,3 , L 2,5 , L 3,5 ,L 1,4 ,L 4,5 , L 2,4 , L 3,4 } </li></ul><ul><li>Add arc between X 2 and X 3 in T, X 2 —X 3 </li></ul>X 3 X 2 X 5 X 4 X 1 Copyright 2009 by Jee Vang
10. 10. Example—Procedure, Tree Construction (cont…) <ul><li>L = {L 2,3 , L 1,5 , L 1,2 , L 1,3 , L 2,5 , L 3,5 ,L 1,4 ,L 4,5 , L 2,4 , L 3,4 } </li></ul><ul><li>Add arc between X 1 and X 5 in T, X 1 —X 5 </li></ul>X 3 X 2 X 5 X 4 X 1 Copyright 2009 by Jee Vang
11. 11. Example—Procedure, Tree Construction (cont…) <ul><li>L = {L 2,3 , L 1,5 , L 1,2 , L 1,3 , L 2,5 , L 3,5 ,L 1,4 ,L 4,5 , L 2,4 , L 3,4 } </li></ul><ul><li>Add arc between X 1 and X 2 in T, X 1 —X 2 </li></ul>X 3 X 2 X 5 X 4 X 1 Copyright 2009 by Jee Vang
12. 12. Example—Procedure, Tree Construction (cont…) <ul><li>L = {L 2,3 , L 1,5 , L 1,2 , L 1,3 , L 2,5 , L 3,5 ,L 1,4 ,L 4,5 , L 2,4 , L 3,4 } </li></ul><ul><li>Skip adding arc between X 1 and X 3 //path exists </li></ul><ul><li>Skip adding arc between X 2 and X 5 //path exists </li></ul><ul><li>Skip adding arc between X 3 and X 5 //path exists </li></ul>X 3 X 2 X 5 X 4 X 1 Copyright 2009 by Jee Vang
13. 13. Example—Procedure, Tree Construction (cont…) <ul><li>L = {L 2,3 , L 1,5 , L 1,2 , L 1,3 , L 2,5 , L 3,5 , L 1,4 ,L 4,5 , L 2,4 , L 3,4 } </li></ul><ul><li>Add arc between X 1 and X 4 in T, X 1 —X 4 </li></ul>X 3 X 2 X 5 X 4 X 1 Copyright 2009 by Jee Vang
14. 14. Example—Procedure, Tree Construction (cont…) <ul><li>L = {L 2,3 , L 1,5 , L 1,2 , L 1,3 , L 2,5 , L 3,5 ,L 1,4 , L 4,5 , L 2,4 , L 3,4 } </li></ul><ul><li>Final tree constructed </li></ul>X 3 X 2 X 5 X 4 X 1 Copyright 2009 by Jee Vang
15. 15. References <ul><li>Chow, C.K., and Liu, C.N. Approximating discrete probability distributions with dependence trees . IEEE Transactions on Information Theory , 14(3):462-467, 1968. </li></ul><ul><li>Pearl, J., and Dechter, R. Learning Structure from Data: A Survey . UCLA Cognitive Systems Laboratory, Technical Report CSD-910048 (R-132), June 1989. </li></ul><ul><li>Acid, S., and de Campos, L.M. Approximation of causal networks by polytrees. Proceedings of Information Processing and Management of Uncertainty in Knowledge-Based Systems , 1994. </li></ul>Copyright 2009 by Jee Vang

### Loggen Sie sich ein, um Kommentare anzuzeigen.

• #### mmktakahashi

Feb. 24, 2015
• #### takashinakashima

Feb. 24, 2015
• #### shoichipincotaguchi

Feb. 24, 2015
• #### MasayukiNakazawa

Feb. 25, 2015
• #### RenatoKano

Aug. 28, 2015

A very simple and quick introduction to the Chow-Liu algorithm.

#### Aufrufe

Aufrufe insgesamt

1.749

Auf Slideshare

0

Aus Einbettungen

0

Anzahl der Einbettungen

5

39

Geteilt

0

Kommentare

0

Likes

5