Test-retest reliability
Assess the stability of a survey outcome across time
Test-retest reliability is a form of reliability that assesses the stability and precision of a construct across time. There is a baseline or "pretest" administration of the survey and then a "post-test" administration of the same survey after a predetermined period of time or intervention. Essentially, test-retest reliability measures the stability of scores across time. If scores from both administrations are highly correlated with stable scores and error variances across time, then evidence of test-retest reliability is assumed. Pearson's r is used to establish evidence of test-retest reliability.
Phenomena that affect test-retest reliability
There are certain phenomena associated with test-retest reliability that may grossly affect the stability of survey scores across time:
1. The length of time between administrations cannot be too short, nor too long. Base decisions on the time difference within the context of your research question. What amount of time is feasible for purposes of testing stability in this context? Most researchers use between one and six weeks of time between administrations. Time lapses of six months are considered too excessive to obtain a stable measure of effect.
2. The sample of individuals should be relatively homogenous in regards to demographic, confounding, clinical, and prognostic factors. People change differently according to their physiological and psychological development.
3. Due to both systematic and unsystematic error in measurement and the general variance and diversity in populations, some measures and constructs will be more unstable across time than others.
1. The length of time between administrations cannot be too short, nor too long. Base decisions on the time difference within the context of your research question. What amount of time is feasible for purposes of testing stability in this context? Most researchers use between one and six weeks of time between administrations. Time lapses of six months are considered too excessive to obtain a stable measure of effect.
2. The sample of individuals should be relatively homogenous in regards to demographic, confounding, clinical, and prognostic factors. People change differently according to their physiological and psychological development.
3. Due to both systematic and unsystematic error in measurement and the general variance and diversity in populations, some measures and constructs will be more unstable across time than others.
The steps for conducting test-retest reliability in SPSS
1. The data is entered in a within-subjects fashion.
2. Click Analyze.
3. Drag the cursor over the Correlate drop-down menu.
4. Click on Bivariate.
5. Click on the baseline observation, pre-test administration, or survey score to highlight it.
6. Click on the arrow to move the variable into the Variables: box.
7. Click on the second observation, post-test administration, or survey score to highlight it.
8. Click on the arrow to move the variable into the Variables: box.
9. Click OK.
2. Click Analyze.
3. Drag the cursor over the Correlate drop-down menu.
4. Click on Bivariate.
5. Click on the baseline observation, pre-test administration, or survey score to highlight it.
6. Click on the arrow to move the variable into the Variables: box.
7. Click on the second observation, post-test administration, or survey score to highlight it.
8. Click on the arrow to move the variable into the Variables: box.
9. Click OK.
The steps for interpreting the SPSS output for test-retest reliability
1. In the Correlations table, match the row to the column between the two observations, administrations, or survey scores. The Pearson Correlation is the test-retest reliability coefficient, the Sig. (2-tailed) is the p-value that is interpreted, and the N is the number of observations that were correlated.
If the p-value is LESS THAN .05, and the Pearson correlation coefficient is above 0.7, then researchers have evidence of test-retest reliability.
If the p-value is MORE THAN .05, or the Pearson correlation coefficient is below 0.7, then researchers do not have evidence of test-retest reliability.
If the p-value is LESS THAN .05, and the Pearson correlation coefficient is above 0.7, then researchers have evidence of test-retest reliability.
If the p-value is MORE THAN .05, or the Pearson correlation coefficient is below 0.7, then researchers do not have evidence of test-retest reliability.
Click on the Principal Components Analysis or Validity button to continue.
Statistician For Hire
DO YOU NEED TO HIRE A STATISTICIAN?
Eric Heidel, Ph.D. will provide statistical consulting for your research study at $100/hour. Secure checkout is available with PayPal, Stripe, Venmo, and Zelle.
- Statistical Analysis
- Sample Size Calculations
- Diagnostic Testing and Epidemiological Calculations
- Psychometrics