When comparing two things in research, you always start with the assumption that there is no difference between them. For example, two medicines work equally well, two textbooks impact test scores similarly or the child you are testing is similar to other children his age. This is called the "null hypothesis." "Null," meaning "zero," reminds you that you must initially assume "no difference" between your comparison groups.
Comparative research projects typically involve two groups: an experimental group that is the focus of your research and a control group to compare it to. For example, if you are testing a new medication, your experimental group would receive the new treatment and the control group would either receive no medication, a placebo or a currently used treatment. The probability value, or p-score, describes how your experimental group or test subject actually compares to the control group. The p-score is the likelihood that the results you observed could have occurred by chance if there were, in fact, no difference between the two.
The significance level is the dividing line between accepting and rejecting the null hypothesis. This means that if you get a p-score less than the significance level you are targeting, your results are likely due to something other than chance. Significance levels are expressed as a decimal number corresponding to the percent chance that your observation represents normal variation instead of a definitive difference. Within the scientific community, significance levels of .05, or 5 percent, are considered to be marginal, .01 are considered strong and .005 are very strong.
Significance levels refer to the dividing line between a meaningful difference or relationship and one due to normal variation. The probability value, on the other hand, refers to your project's specific values. There are many different statistical formulas that can be used to calculate probability values, but the significance level always refers to a fixed dividing line. For example, if your study shows a 3 percent probability that your observed relationships between candy and a reduction in teen moodiness are due to chance, then your p-value is .03. Your probability number is then compared to the conventional significance levels. In this case, .03 falls between the marginally significant value of .05 and the strongly significant .01, suggesting a fairly week relationship between candy and calm teenagers.