{"id":166,"date":"2023-12-22T14:22:38","date_gmt":"2023-12-22T14:22:38","guid":{"rendered":"https:\/\/simplesage.io\/blog\/?p=166"},"modified":"2023-12-22T14:25:42","modified_gmt":"2023-12-22T14:25:42","slug":"roc_curves_and_auc","status":"publish","type":"post","link":"https:\/\/simplesage.io\/blog\/roc_curves_and_auc\/","title":{"rendered":"Understanding Receiver Operating Curves and the AUC: A Guide for Clinicians"},"content":{"rendered":"\n

For clinicians, understanding the diagnostic accuracy of the tests we order is vital. Often, test results are continuous, necessitating a dichotomous categorization into ‘disease present’ or ‘disease absent’. This decision hinges on setting an appropriate cut-off value at which we call the test a \u2018positive\u2019 result. The Receiver Operating Characteristic (ROC) curve is a fundamental tool to visualize the trade off between sensitivity and specificity at different thresholds of test positivity.<\/p>\n\n\n\n

<\/p>\n\n\n\n

The Basics: Sensitivity, Specificity, and the Decision Matrix<\/strong><\/h2>\n\n\n\n

To grasp ROC curves, we first need to understand sensitivity and specificity. Sensitivity measures the proportion of actual disease cases correctly identified by the test while specificity refers to the correct identification of non-disease cases (see formulas below). A perfect test would have a sensitivity and specificity of 1.0 (or 100%) but this does not occur in the real world.<\/p>\n\n\n\n

<\/p>\n\n\n

\n
\"Sensitivity<\/figure>\n<\/div>\n\n
\n
\"Specificity<\/figure>\n<\/div>\n\n\n

<\/p>\n\n\n\n

What is an ROC Curve?<\/strong><\/h2>\n\n\n\n

The ROC curve, developed during World War II for signal detection, now serves as a key tool in medical diagnostics. It plots sensitivity (true positive rate) against 1 – specificity (false positive rate) for various cut-off values of a test, helping to identify the optimal balance between sensitivity and specificity<\/strong>.<\/p>\n\n\n\n

<\/p>\n\n\n\n

Types of ROC curves<\/strong><\/h3>\n\n\n\n

ROC curves are either nonparametric (empirical) or parametric. The nonparametric ROC curve, commonly used in medicine, does not assume a specific distribution of test results and appears as a jagged line. The parametric curve, smoother in appearance, assumes a normal distribution of test results.<\/p>\n\n\n

\n
\"Parametric<\/figure>\n<\/div>\n\n\n

<\/p>\n\n\n\n

Drawing a ROC curve<\/h3>\n\n\n\n

Consider a scenario with ten patients undergoing a cancer marker test. As the cut-off value to determine whether the patient has cancer is adjusted, the sensitivity and specificity vary, allowing the plotting of these points on a graph. For example, we could make the test very sensitive and have very few false negatives, but this might come at the expense of false positives. The resulting curve helps visualize the trade-off between sensitivity and specificity.<\/p>\n\n\n\n

<\/p>\n\n\n\n

The importance of AUC (area under the curve)<\/strong><\/h2>\n\n\n\n

AUC quantifies the overall accuracy of a diagnostic test. An AUC of 1.0 signifies a perfect test, while an AUC of 0.5 indicates no diagnostic utility. Generally, an AUC above 0.8 is deemed acceptable for a diagnostic test, but this varies depending on the role of the test in the clinical pathway.<\/p>\n\n\n\n

<\/p>\n\n\n\n

Area under the Curve (AUC)<\/th>Interpretation<\/th><\/tr><\/thead>
0.9 \u2264 AUC<\/td>Excellent<\/td><\/tr>
0.8 \u2264 AUC < 0.9<\/td>Good<\/td><\/tr>
0.7 \u2264 AUC < 0.8<\/td>Fair<\/td><\/tr>
0.5 \u2264 AUC < 0.7<\/td>Poor<\/td><\/tr>
AUC < 0.5<\/td>Unacceptable <\/td><\/tr><\/tbody><\/table>
AUC threshold clinical interpretations <\/a><\/figcaption><\/figure>\n\n\n\n

Determining the Optimal Cut-off Value<\/strong><\/h3>\n\n\n\n

The optimal cut-off value is not merely about maximizing sensitivity and specificity but finding a suitable balance based on the clinical context. Various methods, such as Youden\u2019s J statistic<\/a>, Euclidean distance<\/a>, and cost approaches, help in determining this value.<\/p>\n\n\n\n

<\/p>\n\n\n\n

Statistical Programs for ROC Analysis<\/strong><\/h3>\n\n\n\n

Various commercial programs (R, SPSS) exist to generate ROC curves. I recommend using the free software JASP<\/a> which is super simple to use \u2013 you can just put in the values and automatically generate curves. No programming required!  <\/p>\n\n\n\n

\n
\"JASP<\/figure>\n\n\n\n
\"ROC<\/figure>\n<\/figure>\n\n\n\n

<\/p>\n\n\n\n

Conclusion<\/strong><\/h2>\n\n\n\n

ROC curves are helpful for clinicians to understand the tradeoff between sensitivity and specificity for different test positivity thresholds. The AUC is a valuable test metric that encompasses the overall accuracy of the test. <\/p>\n\n\n\n

<\/p>\n\n\n\n

Wanting to learn more about research methods? Continue reading about what a P-value is (and what a P-value is not!)<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

For clinicians, understanding the diagnostic accuracy of the tests we order is vital. Often, test results are continuous, necessitating a dichotomous categorization into ‘disease present’ or ‘disease absent’. This decision hinges on setting an appropriate cut-off value at which we call the test a \u2018positive\u2019 result. The Receiver Operating Characteristic (ROC) curve is a fundamental […] →Read more<\/a><\/p>\n","protected":false},"author":2,"featured_media":173,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[7],"class_list":["post-166","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-methods","tag-statistical-testing"],"_links":{"self":[{"href":"https:\/\/simplesage.io\/blog\/wp-json\/wp\/v2\/posts\/166","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/simplesage.io\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/simplesage.io\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/simplesage.io\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/simplesage.io\/blog\/wp-json\/wp\/v2\/comments?post=166"}],"version-history":[{"count":6,"href":"https:\/\/simplesage.io\/blog\/wp-json\/wp\/v2\/posts\/166\/revisions"}],"predecessor-version":[{"id":183,"href":"https:\/\/simplesage.io\/blog\/wp-json\/wp\/v2\/posts\/166\/revisions\/183"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/simplesage.io\/blog\/wp-json\/wp\/v2\/media\/173"}],"wp:attachment":[{"href":"https:\/\/simplesage.io\/blog\/wp-json\/wp\/v2\/media?parent=166"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/simplesage.io\/blog\/wp-json\/wp\/v2\/categories?post=166"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/simplesage.io\/blog\/wp-json\/wp\/v2\/tags?post=166"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}