Receiver operator characteristic
Assess and compare the sensitivity and specificity of diagnostic tests using ROC curves
Receiver Operator Characteristic (ROC) curves assess the sensitivity and specificity of diagnostic tests scored with a continuous value or as a categorical "positive" or "negative." Sensitivity and specificity of a diagnostic test with a continuous outcome depends upon what the cut-off value is for a "positive" test result. Increasing or decreasing the cut-off value will yield different levels of sensitivity and specificity along all points of the numerical continuum.
The ROC curve is a plot of the "true-positives" against the "false-positives." As the curve approaches the upper left hand corner of the graph, the test becomes more accurate. This is because the "true-positive" rate would be approaching 1 or 100% and the "false-positive" rate would be approaching 0 or 0%. The diagonal line represents the chance that a test will be positive or negative by chance.
There is a statistical measure yielded by the receiver operator characteristic curve that tests the diagnostic efficacy of a test. It is called the area under the curve (AUC) or c-statistic. Higher values of AUC or c-statistics means that the diagnostic test is performing well.
Receiver operator characteristic curves can also be used to test the sensitivity and specificity of different diagnostic tests against each other. The diagnostic test with the highest AUC or c-statistic is considered the best test.
The ROC curve is a plot of the "true-positives" against the "false-positives." As the curve approaches the upper left hand corner of the graph, the test becomes more accurate. This is because the "true-positive" rate would be approaching 1 or 100% and the "false-positive" rate would be approaching 0 or 0%. The diagonal line represents the chance that a test will be positive or negative by chance.
There is a statistical measure yielded by the receiver operator characteristic curve that tests the diagnostic efficacy of a test. It is called the area under the curve (AUC) or c-statistic. Higher values of AUC or c-statistics means that the diagnostic test is performing well.
Receiver operator characteristic curves can also be used to test the sensitivity and specificity of different diagnostic tests against each other. The diagnostic test with the highest AUC or c-statistic is considered the best test.
The steps for conducting Receiver Operator Characteristic analysis in SPSS
1. The data is entered in a between-subjects fashion.
2. Click Analyze.
3. Drag the cursor all the way to the bottom of the drop-down menu and click ROC Curve.
4. Click on the continuous outcome variable to highlight it.
5. Click on the arrow to move the variable into the Test Variable: box.
6. Click on the "gold standard" continuous outcome variable to highlight it.
7. Click on the arrow to move the variable into the State Variable: box.
8. In the Value of State Variable: box, enter the cut-off value for a positive test for the "gold standard."
9. In the Display table, click on the With diagonal reference line, Standard error and confidence interval, and Coordinate points of the ROC Curve boxes to select them.
10. Click OK.
2. Click Analyze.
3. Drag the cursor all the way to the bottom of the drop-down menu and click ROC Curve.
4. Click on the continuous outcome variable to highlight it.
5. Click on the arrow to move the variable into the Test Variable: box.
6. Click on the "gold standard" continuous outcome variable to highlight it.
7. Click on the arrow to move the variable into the State Variable: box.
8. In the Value of State Variable: box, enter the cut-off value for a positive test for the "gold standard."
9. In the Display table, click on the With diagonal reference line, Standard error and confidence interval, and Coordinate points of the ROC Curve boxes to select them.
10. Click OK.
The steps for interpreting the output for Receiver Operator Characteristic in SPSS
Here is how to interpret the SPSS output:
1. Look at the ROC curve. The curve should be entirely above the diagonal line. If it falls below the line, the test is not interpretable.
2. Look in the Area Under the Curve table, under the Aysmptotic Sig. column. This is the p-value that is interpreted.
If the p-value is LESS THAN .05, then the test diagnoses the disease state at a statistically significant level.
If the p-value is MORE THAN .05, then the test does not do a significant job diagnostic disease states.
If significant, look under the Area column for the total AUC.
3. Look in the Coordinates of the Curve table, under the Positive if Greater Than or Equal To column. Whichever value in this column that coincides with the two highest values in the Sensitivity and 1-Specificity columns is the best cut-off point for the diagnostic test. One can see how different values of the continuous scale will affect sensitivity and specificity.
1. Look at the ROC curve. The curve should be entirely above the diagonal line. If it falls below the line, the test is not interpretable.
2. Look in the Area Under the Curve table, under the Aysmptotic Sig. column. This is the p-value that is interpreted.
If the p-value is LESS THAN .05, then the test diagnoses the disease state at a statistically significant level.
If the p-value is MORE THAN .05, then the test does not do a significant job diagnostic disease states.
If significant, look under the Area column for the total AUC.
3. Look in the Coordinates of the Curve table, under the Positive if Greater Than or Equal To column. Whichever value in this column that coincides with the two highest values in the Sensitivity and 1-Specificity columns is the best cut-off point for the diagnostic test. One can see how different values of the continuous scale will affect sensitivity and specificity.
The steps for comparing the sensitivity and specificity of several diagnostic tests in SPSS
1. The data is entered in a between-subjects fashion.
2. Click Analyze.
3. Drag the cursor all the way to the bottom of the drop-down menu and click ROC Curve.
4. Click on the first continuous outcome variable to highlight it.
5. Click on the arrow to move the variable into the Test Variable: box.
6. Repeat Steps 4 and 5 until all of the diagnostic test variables are in the Test Variable: box.
7. Click on the "gold standard" continuous outcome variable to highlight it.
8. Click on the arrow to move the variable into the State Variable: box.
9. In the Value of State Variable: box, enter the cut-off value for a positive test for the "gold standard."
10. In the Display table, click on the With diagonal reference line, Standard error and confidence interval, and Coordinate points of the ROC Curve boxes to select them.
11. Click OK.
2. Click Analyze.
3. Drag the cursor all the way to the bottom of the drop-down menu and click ROC Curve.
4. Click on the first continuous outcome variable to highlight it.
5. Click on the arrow to move the variable into the Test Variable: box.
6. Repeat Steps 4 and 5 until all of the diagnostic test variables are in the Test Variable: box.
7. Click on the "gold standard" continuous outcome variable to highlight it.
8. Click on the arrow to move the variable into the State Variable: box.
9. In the Value of State Variable: box, enter the cut-off value for a positive test for the "gold standard."
10. In the Display table, click on the With diagonal reference line, Standard error and confidence interval, and Coordinate points of the ROC Curve boxes to select them.
11. Click OK.
The steps for interpreting the SPSS ouput of an ROC comparing diagnostic tests
1. Look at the ROC curve. The curves should be entirely above the diagonal line. If any curve falls below the line, then the test is not interpreted.
2. Look in the Area Under the Curve table, under the Aysmptotic Sig. column. These are the p-values that are interpreted.
If a p-value is LESS THAN .05, then the test does a significant job at diagnosing disease states.
If a p-value is MORE THAN .05, then the test does NOT do a significant job diagnosing disease states.
If significant, look under the Area column for the total AUC.
3. Look in the Coordinates of the Curve table, under the Positive if Greater Than or Equal To column. Whichever value in this column that coincides with the two highest values in the Sensitivity and 1-Specificity columns is the best cut-off point for each diagnostic test. One can see how different values of each continuous scale will affect their respective sensitivity and specificity.
2. Look in the Area Under the Curve table, under the Aysmptotic Sig. column. These are the p-values that are interpreted.
If a p-value is LESS THAN .05, then the test does a significant job at diagnosing disease states.
If a p-value is MORE THAN .05, then the test does NOT do a significant job diagnosing disease states.
If significant, look under the Area column for the total AUC.
3. Look in the Coordinates of the Curve table, under the Positive if Greater Than or Equal To column. Whichever value in this column that coincides with the two highest values in the Sensitivity and 1-Specificity columns is the best cut-off point for each diagnostic test. One can see how different values of each continuous scale will affect their respective sensitivity and specificity.
Click on the Positive Predictive Value button to continue.
Hire A Statistician - Statistical Consulting for Professionals
DO YOU NEED TO HIRE A STATISTICIAN?
Eric Heidel, Ph.D. will provide statistical consulting for researchers, professionals, and organizations at $100/hour. Secure checkout is available with Stripe or PayPal.
Eric Heidel, Ph.D. will provide statistical consulting for researchers, professionals, and organizations at $100/hour. Secure checkout is available with Stripe or PayPal.
- Statistical Analysis
- Research Design
- Sample Size Calculations
- Diagnostic Testing and Epidemiological Calculations
- Survey Design and Psychometrics