Overview www.wiley.com/wires/compstats is the diagonal matrix of singular values.Note that to have the largest possible variance(i.e.,inertia and A2 is equal to A which is the diagonal matrix of the therefore this component will 'explain'or 'extract' (nonzero)eigenvalues of XTX and XXT the largest part of the inertia of the data table). The inertia of a column is defined as the sum of The second component is computed under the the squared elements of this column and is computed constraint of being orthogonal to the first component as and to have the largest possible inertia.The other components are computed likewise (see Appendix A (2) for proof).The values of these new variables for the observations are called factor scores,and these factors scores can be interpreted geometrically as the The sum of all the y?is denoted I and it is called projections of the observations onto the principal components. the inertia of the data table or the total inertia.Note that the total inertia is also equal to the sum of the squared singular values of the data table (see Finding the Components Appendix B). The center of gravity of the rows [also called In PCA,the components are obtained from the SVD of the data table X.Specifically,with X=PAQT centroid or barycenter,see Ref 14],denoted g,is the (cf.Eq.1),the I x L matrix of factor scores,denoted vector of the means of each column of X.When X is F,is obtained as: centered,its center of gravity is equal to the 1 x row vector 0 F=PA (5) The(Euclidean)distance of the i-th observation to g is equal to The matrix Q gives the coefficients of the linear combinations used to compute the factors scores. (3) This matrix can also be interpreted as a projection matrix because multiplying X by Q gives the values of the projections of the observations on the principal When the data are centered Eq.3 reduces to components.This can be shown by combining Eqs.1 and 5 as: i.j (4) F=PA=P△QQ=XQ (6) The components can also be represented Note that the sum of all is equal to I which is the geometrically by the rotation of the original axes. inertia of the data table For example,if X represents two variables,the length of a word(Y)and the number of lines of its dictionary definition(W),such as the data shown in Table 1,then GOALS OF PCA PCA represents these data by two orthogonal factors. The geometric representation of PCA is shown in The goals of PCA are to Figure 1.In this figure,we see that the factor scores give the length (i.e.,distance to the origin)of the (1)extract the most important information from the projections of the observations on the components. data table; This procedure is further illustrated in Figure 2.In (2)compress the size of the data set by keeping only this context,the matrix Q is interpreted as a matrix this important information; of direction cosines(because Q is orthonormal).The matrix Q is also called a loading matrix.In this (3)simplify the description of the data set;and context,the matrix X can be interpreted as the (4)analyze the structure of the observations and the product of the factors score matrix by the loading variables. matrix as: In order to achieve these goals,PCA computes X=FOT with FF=A2 and Q'Q=I.(7) new variables called principal components which are obtained as linear combinations of the original This decomposition is often called the bilinear variables.The first principal component is required decomposition of X [see,e.g.,Ref 15]. 434 2010 John Wiley Sons,Inc. Volume 2,July/August 2010Overview www.wiley.com/wires/compstats is the diagonal matrix of singular values. Note that !2 is equal to " which is the diagonal matrix of the (nonzero) eigenvalues of XTX and XXT. The inertia of a column is defined as the sum of the squared elements of this column and is computed as γ 2 j = # I i x2 i,j . (2) The sum of all the γ 2 j is denoted I and it is called the inertia of the data table or the total inertia. Note that the total inertia is also equal to the sum of the squared singular values of the data table (see Appendix B). The center of gravity of the rows [also called centroid or barycenter, see Ref 14], denoted g, is the vector of the means of each column of X. When X is centered, its center of gravity is equal to the 1 × J row vector 0T. The (Euclidean) distance of the i-th observation to g is equal to d2 i,g = # J j $ xi,j − gj %2 . (3) When the data are centered Eq. 3 reduces to d2 i,g = # J j x2 i,j . (4) Note that the sum of all d2 i,g is equal to I which is the inertia of the data table . GOALS OF PCA The goals of PCA are to (1) extract the most important information from the data table; (2) compress the size of the data set by keeping only this important information; (3) simplify the description of the data set; and (4) analyze the structure of the observations and the variables. In order to achieve these goals, PCA computes new variables called principal components which are obtained as linear combinations of the original variables. The first principal component is required to have the largest possible variance (i.e., inertia and therefore this component will ‘explain’ or ‘extract’ the largest part of the inertia of the data table). The second component is computed under the constraint of being orthogonal to the first component and to have the largest possible inertia. The other components are computed likewise (see Appendix A for proof). The values of these new variables for the observations are called factor scores, and these factors scores can be interpreted geometrically as the projections of the observations onto the principal components. Finding the Components In PCA, the components are obtained from the SVD of the data table X. Specifically, with X = P!QT (cf. Eq. 1), the I × L matrix of factor scores, denoted F, is obtained as: F = P!. (5) The matrix Q gives the coefficients of the linear combinations used to compute the factors scores. This matrix can also be interpreted as a projection matrix because multiplying X by Q gives the values of the projections of the observations on the principal components. This can be shown by combining Eqs. 1 and 5 as: F = P! = P!QTQ = XQ. (6) The components can also be represented geometrically by the rotation of the original axes. For example, if X represents two variables, the length of a word (Y) and the number of lines of its dictionary definition (W), such as the data shown in Table 1, then PCA represents these data by two orthogonal factors. The geometric representation of PCA is shown in Figure 1. In this figure, we see that the factor scores give the length (i.e., distance to the origin) of the projections of the observations on the components. This procedure is further illustrated in Figure 2. In this context, the matrix Q is interpreted as a matrix of direction cosines (because Q is orthonormal). The matrix Q is also called a loading matrix. In this context, the matrix X can be interpreted as the product of the factors score matrix by the loading matrix as: X = FQT with FTF = !2 and QTQ = I. (7) This decomposition is often called the bilinear decomposition of X [see, e.g., Ref 15]. 434  2010 John Wiley & Son s, In c. Volume 2, July/Augu st 2010