AAAl 2014 Tutorial Latent tree models Part II: Definition and Properties Nevin L Zhang Dept. of computer Science Engineering The hong Kong Univ of Sci. Tech http://www.cse.ust.hk/lzhang
Latent Tree Models Part II: Definition and Properties Nevin L. Zhang Dept. of Computer Science & Engineering The Hong Kong Univ. of Sci. & Tech. http://www.cse.ust.hk/~lzhang AAAI 2014 Tutorial
Part l: Concept and Properties Latent tree Models Definition Relationship with finite mixture models Relationship with phylogenetictrees Basic Properties AAAl2014 Tutorial Nevin L Zhang HKUST
AAAI 2014 Tutorial Nevin L. Zhang HKUST 2 Part II: Concept and Properties Latent Tree Models Definition Relationship with finite mixture models Relationship with phylogenetic trees Basic Properties
Basic Latent Tree ModelS LTM) Bayesian network All variables are discrete Y1 Structure is a rooted tree Y2 Y3 Leaf nodes are observed(manifest X4 variables Internal nodes are not observed X1)(X2)(X3 X5)(X6)(X7 (latent variables) aso known as hierarchical Parameters latent class(HLc)models, HLC models(Zhang JMLR 2004) P(Y1),P(Y2Y1),P(X1Y2),P(2Y2), Semantics P(X1,…,Xn,Y1,…,Ym) I P(Z I parent(Z) parent z∈{x1+XnY1……,Ym}
AAAI 2014 Tutorial Nevin L. Zhang HKUST 3 Basic Latent Tree Models (LTM) Bayesian network All variables are discrete Structure is a rooted tree Leaf nodes are observed (manifest variables) Internal nodes are not observed (latent variables) Parameters: P(Y1), P(Y2|Y1),P(X1|Y2), P(X2|Y2), … Semantics: Also known as Hierarchical latent class (HLC) models, HLC models (Zhang. JMLR 2004)
Joint distribution over observed variables Marginalizing out the latent variables in P(X1, ...,Xn, Y1,..., Ym),we get a joint distribution over the observed variables P(X1, ..., Xn) In comparison with bayesian network without latent variables, LTM Is computationally very simple to work with Represent complex relationships among manifest variables What does the structure look like without the latent variables? Y1 X4 X1)(X2)(X3 X5(X6)X7 AAAl2014 Tutorial Nevin L Zhang HKUST
AAAI 2014 Tutorial Nevin L. Zhang HKUST 4 Marginalizing out the latent variables in , we get a joint distribution over the observed variables . In comparison with Bayesian network without latent variables, LTM: Is computationally very simple to work with. Represent complex relationships among manifest variables. What does the structure look like without the latent variables? Joint Distribution over Observed Variables
Pouch Latent Tree Models(PLTM) An extension of basic ltm (Poon et al. ICML 2010) Rooted tree Internal nodes represent discrete latent variables Each leaf node consists of one or more continuous observed variable called a pouch P(x1,x2y2)=N(x1,x2|2,∑y2) (-25,-25):y=51 (0,0 y2 (25,25) Y23) (Y3(3) Y4(3 10.5 y2 0.51 ∈{5}(2)(0)()(如)( AAAl2014 Tutorial Nevin L Zhang HKUST 5
AAAI 2014 Tutorial Nevin L. Zhang HKUST 5 Pouch Latent Tree Models (PLTM) An extension of basic LTM Rooted tree Internal nodes represent discrete latent variables Each leaf node consists of one or more continuous observed variable, called a pouch. (Poon et al. ICML 2010)
More general latent variable tree models Some internal nodes can be observed Internal nodes can be continuous Choi et al. JMLR 20I D) Forest @E回 2。 @e@@ @ BsisO G @@ Primary focus of this tutorial the basic ltm AAAl2014 Tutorial Nevin L Zhang HKUST
AAAI 2014 Tutorial Nevin L. Zhang HKUST 6 More General Latent Variable Tree Models Some internal nodes can be observed Internal nodes can be continuous Forest Primary focus of this tutorial: the basic LTM (Choi et al. JMLR 2011)
Part l: Concept and Properties Latent tree Models Definition Relationship with finite mixture models Relationship with phylogenetictrees Basic Properties AAAl2014 Tutorial Nevin L Zhang HKUST
AAAI 2014 Tutorial Nevin L. Zhang HKUST 7 Part II: Concept and Properties Latent Tree Models Definition Relationship with finite mixture models Relationship with phylogenetic trees Basic Properties
Finite Mixture Models( FMM) Gaussian Mixture Models(GMM) Continuous attributes p(x)=∑P(z=k)xz=k)=∑xp(x2=k) p(x2z=k)=N(xuk,∑k) Graphical model z X1,xX2,Xx3,X4,x5,X6,X7,X8×9 AAAl2014 Tutorial Nevin L Zhang HKUST
AAAI 2014 Tutorial Nevin L. Zhang HKUST 8 Finite Mixture Models (FMM) Gaussian Mixture Models (GMM): Continuous attributes Graphical model
Finite Mixture Models FMM) GMM with independence assumption Block diagonal co-variable matrix X1 2 X3 X1 X2 0 Y2(3) Graphical Model XI X3 X2 AAAl2014 Tutorial Nevin L Zhang HKUST
AAAI 2014 Tutorial Nevin L. Zhang HKUST 9 Finite Mixture Models (FMM) GMM with independence assumption Block diagonal co-variable matrix Graphical Model
Finite mixture models Latent class models (lcm): Discrete attributes Graphical Model P(x)=P(AA2…An)=∑P(2=k)ⅡP(4z=k) Distribution for cluster k: IIP(A)z Product multinomial distribution =1 All FMMs One latent variable Yielding one partition of data AAAl2014 Tutorial Nevin L Zhang HKUST 10
AAAI 2014 Tutorial Nevin L. Zhang HKUST 10 Finite Mixture Models Latent class models (LCM): Discrete attributes Distribution for cluster k: Product multinomial distribution: All FMMs One latent variable Yielding one partition of data Graphical Model