Examine the data set to assess the data as a whole and what the ratings are, if any. Identify different categories of analysis, how they were determined, marked and populated. Aim to spot potential differences in severity, or inequality, standards among raters or benchmarks.
Earmark specific characteristics that appear to be out of sync with others or that may impact ratings and their weight. These may include categories, such as temperature or heat expulsion, in electronic studies, or benchmarks, such as total output in economy or production (such as GDP) or absolute population.
Assess the potential for unmentioned characteristics to skew ratings. For instance, if the ratings are assessing the wage gap between women and men in the paid labor market in 15 countries based on absolute wages, first determine what the absolute number of men and women are in each country. The wage gap may seem very small in a country where only 10 percent to 15 percent of the total population of women are participating in the labor market, since they likely occupy the status of elites and are not representative of the total population so, while the wage gap may be low between elites, this fact may shield the fact that a large number of women are not participating in the formal labor market.
Weigh data based on this to equalize countries to better gauge the wage gap using a statistical method, such as correction factor, in analysis. For larger sociological data sets, consider using a gini coefficient or the Theil-index to measure existing inequalities to accurately gauge and explain compounding variables.
Chronicle each step taken to minimize or neutralize inequalities in ratings. Social research is most legitimate according to the scientific method when the experiment and results are replicable.