Factor analysis allows its user to input a large number of variables and yield an output of only a few variables (now called "factors"). In essence, factor analysis is similar to principal components analysis in that it reduces a large amount of data to a clear set of factors that can be used to describe certain aspects in the data. In addition, the user of factor analysis has a moderate degree of freedom in choosing the number of factors in a solution, allowing the user to plan how much to simplify the data.
Unlike principal components analysis, factor analysis has the ability to describe correlations hidden in the data. In fact, one of the main benefits of using factor analysis is that you can instantaneously know the correlations between a large set of variables; this would be difficult to do in a meaningful way using other statistical methods. Ultimately, factor analysis uses these correlations to determine which variables are similar enough to each other to be interpreted as a single factor.
One of the main reasons factor analysis is so popular in the sciences is its ability to take abstract data and convert it into interpretable, concrete factors. This characteristic of interpretability does not exist in principle components analysis. The interpretability of factor analysis is clear enough that even a novice user can directly use the output of a factor analysis to categorize the resulting factors.
Another interesting characteristic that factor analysis has is its multiplicity of solutions. If the user of factor analysis does not accept or cannot interpret the output of the analysis, the user can rotate the solution until an acceptable solution appears. This characteristic gives factor analysis versatility that is impossible with principal components analysis.