Factor analysis

Factor analysis is a class of structural equation modeling algorithms aiming at discovering latent variables that explain observable variables in datasets.

It is often used in psychometrics, because many psychological constructs cannot be measured directly. There is no PCR test for autism, no radiography for anxiety. We need to characterize such traits based on how their manifestations cluster together in populations.

Factor models are often represented with observed variables as squares and latent variables are circles.

The value of each item $i$ is given by the factors $F_j$, the loadings ${\lambda }_{ij}$, the intercept ${\tau }_{ij}$ and the unique variance (error + specific variance) ${ϵ}_i$:

$$Y_i={\tau }_{ij}+\sum _j{\lambda }_{ij}{F}_j+{ϵ}_i$$

Factors are assumed to have mean 0 and variance 1, which can be achieved through normalization.

EFA

EFA (exploratory factor analysis) is the factor analysis algorithm to discover latent structure.

It is used with a chosen number of factors. Loadings are freely estimated.

Factors are then rotated, with two possible types of rotation:

• Orthogonal (varimax) when we want uncorrelated factors
• Oblique (promax, oblimin) when we accept correlated factors, often used in psychometrics

CFA

CFA (confirmatory factor analysis) to test the validity of a factor model, where factors can load onto limited item subsets.

Differences with PCA

PCA finds linear combinations of variables that maximize variance, whereas EFA operates under the hypothesis of underlying structure and finds latent variables.

Factor analysis also separates unique variance from factor variance, whereas PCA doesn’t, which makes the former more able to account for measurement error.