Beware of the potential for researcher bias -- for example, the researcher simply found what he or she wanted to find.
Evaluate the extent to which the researcher engaged in reflexivity or critical self reflection (self assessment for bias).
Determine whether or not the researcher engaged in negative case sampling -- for example, he searched for examples that did not meet the researcher's expectations.
Determine whether the researcher utilized the tool of triangulation or cross-checking.
Decide if the researcher utilized multiple sources of data.
Decide if the researcher utilized multiple research methods to collect the data.
Decide if the researcher involved multiple investigators or researchers.
Determine the degree to which the researcher established commonalities among multiple sources, methods and researchers. The degree to which those commonalities were established increases the descriptive validity of the research, or that the results of the research, as reported by the researcher, are factually accurate.
Ask whether the researcher engaged in participant feedback to clarify any miscommunications about the researcher's interpretation of their input.
Apply the feedback of the participants to the researcher's understanding of the data.
Determine the degree to which the researcher accurately understood the input of the participants. The higher the degree of understanding, the higher the validity of the research.
Evaluate the researcher's use of extended fieldwork to examine the research participants and the setting of the research.
Judge the degree to which the researcher applied the tool of theory triangulation.
Ask whether the researcher considered that the phenomenon could be explained by multiple explanations.
Question whether the researcher engaged in pattern matching -- for example, making several predictions at once and observing whether or not results occur as predicted.
Question whether the researcher engaged in peer review - for example, he discussed his research with colleagues.
Evaluate the internal validity of the research by asking whether the researcher ruled out rival explanations.
Question whether the researcher offers a causal explanation for certain phenomenon and determine the degree to which this explanation can be justified.
Evaluate the external validity of the research by utilizing the tool of replication logic. The more times the results of the research have been replicated increases the external validity of the research.
Ask whether the researcher considered that the phenomenon could be explained by multiple explanations.