Given a positive definite covariance matrix (Formula presented.) of dimension n, we approximate it with a covariance of the form (Formula presented.), where H has a prescribed number (Formula presented.) of columns and (Formula presented.) is diagonal. The quality of the approximation is gauged by the I-divergence between the zero mean normal laws with covariances (Formula presented.) and (Formula presented.), respectively. To determine a pair (H, D) that minimizes the I-divergence we construct, by lifting the minimization into a larger space, an iterative alternating minimization algorithm (AML) à la Csiszár-Tusnády. As it turns out, the proper choice of the enlarged space is crucial for optimization. The convergence of the algorithm is studied, with special attention given to the case where D is singular. The theoretical properties of the AML are compared to those of the popular EM algorithm for exploratory factor analysis. Inspired by the ECME (a Newton-Raphson variation on EM), we develop a similar variant of AML, called ACML, and in a few numerical experiments, we compare the performances of the four algorithms.
Factor analysis models via I-divergence optimization
Finesso L;
2015
Abstract
Given a positive definite covariance matrix (Formula presented.) of dimension n, we approximate it with a covariance of the form (Formula presented.), where H has a prescribed number (Formula presented.) of columns and (Formula presented.) is diagonal. The quality of the approximation is gauged by the I-divergence between the zero mean normal laws with covariances (Formula presented.) and (Formula presented.), respectively. To determine a pair (H, D) that minimizes the I-divergence we construct, by lifting the minimization into a larger space, an iterative alternating minimization algorithm (AML) à la Csiszár-Tusnády. As it turns out, the proper choice of the enlarged space is crucial for optimization. The convergence of the algorithm is studied, with special attention given to the case where D is singular. The theoretical properties of the AML are compared to those of the popular EM algorithm for exploratory factor analysis. Inspired by the ECME (a Newton-Raphson variation on EM), we develop a similar variant of AML, called ACML, and in a few numerical experiments, we compare the performances of the four algorithms.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.