Factor analysis is extremely insistent regarding the sample size. Because it relies on multiple variables, factor analysis requires a larger sample size for each variable added to a study. "The Essentials of Factor Analysis" by Dennis Child recommends at least five data points per variable. So a study that includes 20 variables needs a sample size of at least 100, which is not a small sample by any interpretation.
After you arrive at a set of factors, the following step in factor analysis is to label these factors. Many researchers performing factor analysis do so with a hypothesis. Hence, researchers have a tendency to "see what is not really there." Q. McNemar, who wrote "Psychometrika; the Factors in Factoring Behavior," called this the "struggle syndrome," a phenomenon in which researchers struggle to force the factors to correspond with their hypotheses.
The variables themselves prove to be a problem. First, choosing the appropriate variables for the analysis is essential in acquiring a reliable result. Second, you need many variables for most factor analyses; at least four variables should be in each factor definition. Third, the variables should have a wide range and be close to normally distributed; the sheer number of variables you must include in a factor analysis study coupled with this fact leads to the requirement of a large number of time-consuming distribution tests.
Adding a factor often changes other factors. The implication of this is that you cannot change your hypothesis in a way that adds more variables or factors without repeating the entire factor analysis. In other words, the results of a single factor analysis are not malleable, and any modifications to your theory cannot correspond to a slight modification in the factor analysis results.