How to Test the Significance Between Overlapping Samples

Simple statistical procedures are available for determining whether there are significant differences between sets of samples. Differences can be determined between samples with overlapping data points, as well as samples that discrete sets of data. The procedure most commonly used to determine if there are significant differences is called the student's t-test, or, simply, the t-test. This test can be applied to samples that have overlapping data points. There are several formal steps to doing the test, the first of which is developing what is called a null hypothesis. Then there are the calculations that can either be done manually or online.

Things You'll Need

  • An equal number of sample values from each sample (if you are doing the calculations manually)
  • 2 samples that have five or more data points
  • Calculator
  • Table containing t-scores, degrees of freedom and P values
Show More

Instructions

  1. Calculating the Significance Manually

    • 1

      Make your null hypothesis. The null hypothesis applicable in this case would be that there is no difference between the means (averages) of the two samples. If you reject the null hypothesis based on your test results, then there is a probability of significant differences between the samples.

    • 2
      All of the calculations can be done on a simple calculator. Scientific calculators can have statistical functions as well.

      Calculate the average (mean) of each sample set, which is the sum of all of the values within each set divided by the number of values. For example, if you have the numbers 1, 2, 3, 4 and 5, they will add up to 15. When you divide that number by 5, the average is 3.

    • 3

      Calculate the variance (s^2) and the standard error (SE) for the two sample sets. Variance is calculated by first calculation the difference between each of the values in each set from the means and then squaring the value. So, if you have a set of three numbers like 10, 15 and 20, the mean is 15 and the respective differences are -5, 0 and +5. Each of these values is squared, then added together and divided by the number of values minus 1 (n-1 = 2). So, in this example, the variance is 50/2 = 25. The standard error value, if there are three values for each of the two samples, would be SE = sq rt [ (s^2 / 3) + (s^2 / 3) ], where the variance for each of the samples is included in the formula.

    • 4

      Calculate the t-score. First determine the differences between the two means. Divide this value by the standard error you previously calculated.

    • 5

      Determine if there is a significant difference between the two samples. This is done by using a table that has t-scores listed. First, find the row that has the number of degrees of freedom, which is the number of samples you have minus one. If you have six numbers for a sample set, the degrees of freedom will be 5.

      Go to the column in the table that has a probability value of 0.05 and find the t-score that is given. If your t-score is greater than that value, then you would "fail to reject" the null hypothesis, meaning there are no significant differences in your overlapping samples. However, if your t-score is equal or less than that value, then you would reject that hypothesis and conclude that there is evidence for significant differences.

    Determining the Difference Using Online Resources

    • 6

      Find a website that has a "t-test calculator." These sites allow you to enter your data, select the appropriate variety of t-test and perform the calculations. Another advantage of online calculators is that they can perform different types of tests that fit you data.

    • 7

      Enter the data for each of the two samples. Depending on the online site you use, you can enter raw data or descriptive statistics that you have already completed, such as the means, standard deviations and number of data points for each sample.

    • 8

      Select the appropriate type of test for your data. If your data has an equal number of data points, select a paired t-test, but if it has a different number of data points, use a non-paired t-test. Last, if you know that your data does not follow a normal distribution, select a non-parametric test. If you are not sure about your data distribution, the safe path is to select a non-parametric test.

    • 9

      Press the icon that allows the program to calculate the results. The results page should show the P value or level of significance. If the value is less than 0.05, there would be a significant difference between the two sample sets. However, if the P value is greater than 0.05, you will conclude that the samples are overlapping so much that there is no significant difference.

Learnify Hub © www.0685.com All Rights Reserved