Given a set of variables, a Factor Analysis tries to form coherent subsets that are as independent from other subsets as possible, but where each variable in the same subset is as highly correlated as possible. Thus, variables that are correlated with one another and which are largely independent of other subsets of variables are combined into factors. Factors can then be thought of as the more fundamental features that are measured by the variables.

For example, if we administered a large battery of tests to high school students which included: addition, multiplication, analogies, reading comprehension, symbol matching, symbol rotation and so on, we might be able to extract 3 factors which might be labeled as: Mathematical, Verbal and Spatial processing abilities. Thus, tests of addition and multiplication are highly correlated, but are largely uncorrelated to tests of analogies or reading comprehension.

There are 2 features of Factor Analysis to be aware of. First of all, the extraction of factors depends on the measured variables fed to the analysis. Thus, if the measured variables do not represent the entire spectrum of variation, then the extracted factors will not cover the entire range of possible motivations. In other words, you cannot carve a pie so that you end up with more of it than you began with.

Secondly, a Factor Analysis tells you how well particular variables fit into specific factors, but the factors are not then labeled for you. In other words, one has to come up with the appropriate labels for each factor after understanding which variables load heavily into each factor. Oftentimes, it is hard to come up with a label that encapsulates all the underlying variables of a factor.