Decision Curve Analysis: Step by Step 👣
Uriah Finkel
Acknowledgement:
I’d like to thank Dr. Andrew Vickers for his help with this presentation.
You can visit his page on mskcc.org/profile/andrew-vickers.
\[ \newcommand{\green}[1]{\color{green}{#1}} \newcommand{\red}[1]{\color{red}{#1}} \]
Agenda
This lecture follows the article A simple, step-by-step guide to interpreting decision curve analysis.
Introduction: Motivation behind Decision Curve Analysis and how to draw a Decision Curve.
How to interpret a Decision Curve: Step by Step guide.
Code for Decision Curve Analysis 💻
Standard software is available to run decision curves in R, SAS and Stata on decisioncurveanalysis.org. Python should be with us soon!
All interactive plots in this presentation were created with rtichoke R package (I am the author 👋), you are also invited to explore rtichoke blog for reproducible examples and some theory.
For ggplot2 outputs dcurves R package is available on CRAN.
Introduction
Which Model is Better? 🤔
Select patients for biopsy amongst men with elevated PSA - 🖖 ROC
Which Model is Better? 🤔
Select patients for biopsy amongst men with elevated PSA - ⚖️ Calibration Curve
Which Model is Better? 🤔
Select patients for biopsy amongst men with elevated PSA - 👌 Decision Curve
Traditional Statistical Metric:
Discrimination 🖖: Model’s ability to separate between events and non-events (ROC Curve, AUROC, Sensitivity, Specificity, NPV, PPV, Lift etc).
Calibration ⚖️: Agreement between predicted probabilities and the observed outcomes (Calibration Curve, Calibration in the large, Calibration in the small etc).
Problem🛑 These metrics are not directly informative to clinical value, nor to full decision analytic approaches.
Solution👌 Decision Curve Analysis calculates a clinical “Net Benefit” prediction models or diagnostic tests in comparison to default strategies of treating all or no patients.
How to draw a Decision Curve?
\[{\text{X axis: } {p_{t}} \text{ (Probability Threshold)}}\]
\[\begin{aligned} {\text{Y axis: }\text{NB Model} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}}} \end{aligned}\]
\[\begin{aligned} \text{NB Treat None} = 0 \end{aligned}\]
\[\begin{aligned} \text{NB Treat All} = {\text{Prevalence}} - {\text{(1 - Prevalence)}} *{\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}\]
Step 1 👣:
Benefit is Good
Green
🙂
Red
🙁
Green
Red
True Positives
Infected and Predicted as Infected - Good
💊
🤢
False Positives
Not-Infected and Predicted as Infected - BAD
💊
🤨
False Negatives
Infected and Predicted as Not-Infected - BAD
🤢
True Negatives
Not-Infected and Predicted as Not-Infected - GOOD
🤨
Decision Tree
Decision Tree
Decision Tree
Probability Threshold is derived from Equivalence between
\[EV_{Treat} = p(Disease) * U_{TP} + p(No Disease) * U_{FP}\]
\[EV_{NoTreat} = p(Disease) * U_{FN} + p(No Disease) * U_{TN}\]
Decision Tree
Probability Threshold is derived from Equivalence between
\[EV_{Treat} = EV_{{NoTreat}}\]
\[p(Disease) * U_{TP} + p(No Disease) * U_{FP} =p(Disease) * U_{FN} + p(No Disease) * U_{TN}\]
Decision Tree
Probability Threshold is derived from Equivalence between
\[EV_{Treat} = EV_{{NoTreat}}\]
\[p(Disease) * U_{TP} + (1-p(Disease)) * U_{FP} =p(Disease) * U_{FN} + (1-p(Disease)) * U_{TN}\]
Decision Tree
Probability Threshold is derived from Equivalence between
\[EV_{Treat} = EV_{{NoTreat}}\]
\[p(Disease) * (U_{TP} - U_{FN}) =(1-p(Disease)) * (U_{TN} - U_{FP})\]
Decision Tree
Probability Threshold is derived from Equivalence between
\[EV_{Treat} = EV_{{NoTreat}}\]
\[\frac{p(Disease)}{(1-p(Disease))} = \frac{(U_{TN} - U_{FP})}{(U_{TP} - U_{FN})}\]
Decision Tree
Probability Threshold is derived from Equivalence between
\[EV_{💊} = EV_{{No💊}}\]
\[\frac{p(🤢)}{(1-p(🤢))} = \frac{(U_{🤨} - U_{💊+🤨})}{(U_{💊+🤢} - U_{🤢})}\]
Step 1 👣: Benefit is Good
\[\begin{aligned} \text{Sensitivity } = \frac{\green{\bf{TP}}}{\text{TP + FN}} \end{aligned}\]
\[\begin{aligned} \text{Specificity} = \frac{\green{\bf{TN}}}{\text{TN + FP}} \end{aligned}\]
\[\begin{aligned} {\text{Prevalence}} = \frac{\text{TP + FN}}{\text{N}} \end{aligned}\]
\[\begin{aligned} \text{Net Benefit} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}\]
Step 1 👣: Benefit is Good
\[\begin{aligned} \text{Sensitivity } = \frac{\green{\bf{TP}}}{\text{Real Positives}} \end{aligned}\]
\[\begin{aligned} \text{Specificity} = \frac{\green{\bf{TN}}}{\text{Real Negatives}} \end{aligned}\]
\[\begin{aligned} {\text{Prevalence}} = \frac{\text{Real Positive}}{\text{N}} \end{aligned}\]
\[\begin{aligned} \text{Net Benefit} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}\]
Step 1 👣: Benefit is Good
\[\begin{aligned} \text{Sensitivity } = \frac{\green{\bf{TP}}}{\text{Real Positives}} \end{aligned}\]
\[\begin{aligned} \text{Specificity} = \frac{\green{\bf{TN}}}{\text{Real Negatives}} \end{aligned}\]
\[\begin{aligned} {\text{Prevalence}} = \frac{\text{Real Positive}}{\text{N}} \end{aligned}\]
\[\begin{aligned} {\text{1 - Prevalence}} = \frac{\text{Real Negatives}}{\text{N}} \end{aligned}\]
\[\begin{aligned} \text{Net Benefit} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}\]
Step 1 👣: Benefit is Good
\[\begin{aligned} \text{Sensitivity } = \frac{\green{\bf{TP}}}{\text{Real Positives}} \end{aligned}\]
\[\begin{aligned} \text{Specificity} = \frac{\green{\bf{TN}}}{\text{Real Negatives}} \end{aligned}\]
\[\begin{aligned} {\text{Prevalence}} = \frac{\text{Real Positive}}{\text{N}} \end{aligned}\]
\[\begin{aligned} {\text{1 - Prevalence}} = \frac{\text{Real Negatives}}{\text{N}} \end{aligned}\]
\[\begin{aligned} \text{Net Benefit} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}\]
\[\tiny{\begin{aligned} \text{Net Benefit} = {\text{Sensitivity}} * {\text{Prevalence}} - {\text{(1 - Specificity)}} * {\text{(1 - Prevalence)}} *{\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}}\]
Step 1 👣: Benefit is Good
\[\begin{aligned} \green{\bf{Sensitivity}} = \frac{\green{\bf{TP}}}{\text{Real Positives}} \end{aligned}\]
\[\begin{aligned} \red{\bf{1 - Specificity}} = \frac{\bf{\red{FP}}}{\text{Real Negatives}} \end{aligned}\]
\[\begin{aligned} {\text{Prevalence}} = \frac{\text{Real Positives}}{\text{N}} \end{aligned}\]
\[\begin{aligned} {\text{1 - Prevalence}} = \frac{\text{Real Negatives}}{\text{N}} \end{aligned}\]
\[\begin{aligned} \green{\bf{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}\]
\[\tiny{\begin{aligned} \green{\bf{Net Benefit}} = {\green{\bf{Sensitivity}}} * {\text{Prevalence}} - {\red{\bf{(1 - Specificity)}}} * {\text{(1 - Prevalence)}} *{\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}}\]
Step 1 👣: Benefit is Good
\[\begin{aligned} \green{\bf{Sensitivity}} = \frac{\green{\bf{TP}}}{\text{Real Positives}} \end{aligned}\]
\[\begin{aligned} \red{\bf{1 - Specificity}} = \frac{\bf{\red{FP}}}{\text{Real Negatives}} \end{aligned}\]
\[\begin{aligned} {\text{Prevalence}} = \frac{\text{Real Positives}}{\text{N}} \end{aligned}\]
\[\begin{aligned} {\text{1 - Prevalence}} = \frac{\text{Real Negatives}}{\text{N}} \end{aligned}\]
\[\begin{aligned} \green{\bf{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}\]
\[\tiny{\begin{aligned} \green{\bf{Net Benefit}} = {\green{\bf{Sensitivity}}} * {\text{Prevalence}} - {\red{\bf{(1 - Specificity)}}} * {\text{(1 - Prevalence)}} *{\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}}\]
\[\tiny{\begin{aligned} \text{Net Benefit Treat All} = {\green{\bf{1}}} * {\text{Prevalence}} - {\red{\bf{(1 - 0)}}} * {\text{(1 - Prevalence)}} *{\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}}\]
Step 1 👣: Benefit is Good
\[\begin{aligned} \green{\bf{Sensitivity}} = \frac{\green{\bf{TP}}}{\text{Real Positives}} \end{aligned}\]
\[\begin{aligned} \red{\bf{1 - Specificity}} = \frac{\bf{\red{FP}}}{\text{Real Negatives}} \end{aligned}\]
\[\begin{aligned} {\text{Prevalence}} = \frac{\text{Real Positives}}{\text{N}} \end{aligned}\]
\[\begin{aligned} {\text{1 - Prevalence}} = \frac{\text{Real Negatives}}{\text{N}} \end{aligned}\]
\[\begin{aligned} \green{\bf{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}\]
\[\tiny{\begin{aligned} \green{\bf{Net Benefit}} = {\green{\bf{Sensitivity}}} * {\text{Prevalence}} - {\red{\bf{(1 - Specificity)}}} * {\text{(1 - Prevalence)}} *{\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}}\]
\[\tiny{\begin{aligned} \text{Net Benefit Treat All} = {\text{Prevalence}} - {\text{(1 - Prevalence)}} *{\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}}\]
Step 1 👣: Benefit is Good
\[\begin{aligned} \green{\bf{Sensitivity}} = \frac{\green{\bf{TP}}}{\text{Real Positives}} \end{aligned}\]
\[\begin{aligned} \red{\bf{1 - Specificity}} = \frac{\bf{\red{FP}}}{\text{Real Negatives}} \end{aligned}\]
\[\begin{aligned} {\text{Prevalence}} = \frac{\text{Real Positives}}{\text{N}} \end{aligned}\]
\[\begin{aligned} {\text{1 - Prevalence}} = \frac{\text{Real Negatives}}{\text{N}} \end{aligned}\]
\[\begin{aligned} \green{\bf{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}\]
\[\tiny{\begin{aligned} \green{\bf{Net Benefit}} = {\green{\bf{Sensitivity}}} * {\text{Prevalence}} - {\red{\bf{(1 - Specificity)}}} * {\text{(1 - Prevalence)}} *{\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}}\]
\[\tiny{\begin{aligned} \text{Net Benefit Treat All} = {\text{Prevalence}} - {\text{(1 - Prevalence)}} *{\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}}\]
\[\tiny{\begin{aligned} \text{Net Benefit Treat None} = {\green{\bf{0}}} * {\text{Prevalence}} - {\red{\bf{(1 - 1)}}} * {\text{(1 - Prevalence)}} *{\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}}\]
Step 1 👣: Benefit is Good
\[\begin{aligned} \green{\bf{Sensitivity}} = \frac{\green{\bf{TP}}}{\text{Real Positives}} \end{aligned}\]
\[\begin{aligned} \red{\bf{1 - Specificity}} = \frac{\bf{\red{FP}}}{\text{Real Negatives}} \end{aligned}\]
\[\begin{aligned} {\text{Prevalence}} = \frac{\text{Real Positives}}{\text{N}} \end{aligned}\]
\[\begin{aligned} {\text{1 - Prevalence}} = \frac{\text{Real Negatives}}{\text{N}} \end{aligned}\]
\[\begin{aligned} \green{\bf{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}\]
\[\tiny{\begin{aligned} \green{\bf{Net Benefit}} = {\green{\bf{Sensitivity}}} * {\text{Prevalence}} - {\red{\bf{(1 - Specificity)}}} * {\text{(1 - Prevalence)}} *{\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}}\]
\[\tiny{\begin{aligned} \text{Net Benefit Treat All} = {\text{Prevalence}} - {\text{(1 - Prevalence)}} *{\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}}\]
\[\tiny{\begin{aligned} \text{Net Benefit Treat None} = 0 \end{aligned}}\]
Step 2 👣:
Preference Differ
Step 2 👣: Preference Differ
Low Probability Threshold means that I’m worried about the outcome:
- I’m worried about Prostate Cancer 🦀
- I’m worried about Heart Disease 💔
- I’m worried about Infection 🤢
Step 2 👣: Preference Differ
High Probability Threshold means that I’m worried about the Intervention:
- I’m worried about Biopsy 💉
- I’m worried about Statins 💊
- I’m worried about Antibiotics 💊
Step 3 👣:
Unit of Preference = Threshold Probability
Step 3 👣: Unit of Preference = Threshold Probability
Remember: almost always (specially in Health Care) -
Sensitivity does not have the same importance as Specificity and having 1 TP does not have the same clinical utility as having 1 FP.
A good example for a bad practice in evaluating performance of prediction model is the Youden’s J statistics:
\[{\displaystyle J={\frac {\green{\bf{TP}}}{{\green{\bf{TP}}}+{\red{\bf{FN}}}}}+{\frac {\green{\bf{TN}}}{{\green{\bf{TN}}}+{\red{\bf{FP}}}}}-1}\] Maximizing Youden’s J statistic might lead to a Probability Threshold that will do more harm than good, even if the prediction model is accurate!
Step 3 👣: Unit of Preference = Threshold Probability
Remember: almost always (specially in Health Care) -
Sensitivity does not have the same importance as Specificity and having 1 TP does not have the same clinical utility as having 1 FP.
A good example for a bad practice in evaluating performance of prediction model is the Youden’s J statistics:
\[{\displaystyle \green{\bf{J}}={\green{\bf{Sens}}}-{\red{\bf{(1 - Spec)}}}}\]
\[{\displaystyle \green{\bf{NB}}={\green{\bf{Sens}}} * {\text{Prev}}-{\red{\bf{(1 - Spec)}}}* {\text{(1 - Prev)}}*{\frac{{p_{t}}}{{1 - p_{t}}}}}\]
Step 3 👣: Unit of Preference = Threshold Probability
I wouldn’t give more than 4 antibiotics in order to help 1 infected patient.
If a patient’s risk was above 20% I will give him antibiotics, otherwise I won’t.
The risk of 20% is an odds of 1:4, so in using a threshold probability of 20%, the doctor is telling us “missing an infected patiant is 4 times worse than giving antibiotics to a healthy patient.”
\[p_t = \frac{1}{1 + 4} = 0.2\] \[\frac{p_t}{1 - p_t} = \frac{0.2}{1 - 0.2} = \frac{1}{4}\]
Step 3 👣: Unit of Preference = Threshold Probability
I will be indifferent 😐
For having 1 TP for 4 FP \[p_t = \frac{1}{1 + 4} = 0.2\] \[\frac{p_t}{1 - p_t} = \frac{0.2}{1 - 0.2} = \frac{1}{4}\]
\[\begin{aligned}[t] {\text{Net Benefit}} &= {\frac{\text{1}}{\text{5}} - \frac{\text{4}}{\text{5}} * {\frac{1}{4}} = 0} \end{aligned}\]
Step 3 👣: Unit of Preference = Threshold Probability
I will be sad 🙁
For having 1 TP for 5 FP \[p_t = \frac{1}{1 + 4} = 0.2\] \[\frac{p_t}{1 - p_t} = \frac{0.2}{1 - 0.2} = \frac{1}{4}\] \[\begin{aligned}[t]
{\text{Net Benefit}} &= {\frac{\text{1}}{\text{6}} - \frac{\text{5}}{\text{6}} * {\frac{1}{4}} = -0.04166'}
\end{aligned}\]
Step 3 👣: Unit of Preference = Threshold Probability
I will be happy 🙂
For having 1 TP for 3 FP \[p_t = \frac{1}{1 + 4} = 0.2\] \[\frac{p_t}{1 - p_t} = \frac{0.2}{1 - 0.2} = \frac{1}{4}\]
\[\begin{aligned}[t] {\text{Net Benefit}} &= {\frac{\text{1}}{\text{4}} - \frac{\text{3}}{\text{4}} * {\frac{1}{4}} = 0.0625} \end{aligned}\]
Step 4 👣:
Benefit is actually Net Benefit
Step 4 👣: Benefit is actually Net Benefit
Let’s think in terms of money 💸
A wine importer buys €1m of wine from France and sells it in the USA for $1.5m 🍷
Net Benefit = Income - Expenditure
Net Benefit = 1.5m$ - 1m€ = ? 🤔
Let’s say that 1€ is worth 1.25$
Exchange-Rate = 1.25 ($ / €)
1m€ = 1m * 1.25$
Net Benefit = 1.5m$ - 1.25m$ = 250,000$
Which is the equivalent of being given 250,000$:
Net Benefit = 250,000$ - 0$ = 250,000$
Step 4 👣: Benefit is actually Net Benefit
💊💊
🤢🤢😷😷😷😷😷😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}}\]
💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}}\]
Step 4 👣: Benefit is actually Net Benefit
💊💊
🤢🤢😷😷😷😷😷😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{10}} - \frac{\red{\bf{FP}}}{\text{10}} * {\frac{{1}}{{4}}}\]
💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{10}} - \frac{\red{\bf{FP}}}{\text{10}} * {\frac{{1}}{{4}}}\]
Step 4 👣: Benefit is actually Net Benefit
💊💊
🤢🤢😷😷😷😷😷😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{2}}}{\text{10}} - \frac{\red{\bf{FP}}}{\text{10}} * {\frac{{1}}{{4}}}\]
💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{3}}}{\text{10}} - \frac{\red{\bf{FP}}}{\text{10}} * {\frac{{1}}{{4}}}\]
Step 4 👣: Benefit is actually Net Benefit
💊💊
🤢🤢😷😷😷😷😷😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{2}}}{\text{10}} - \frac{\red{\bf{0}}}{\text{10}} * {\frac{{1}}{{4}}}\]
💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{3}}}{\text{10}} - \frac{\red{\bf{4}}}{\text{10}} * {\frac{{1}}{{4}}}\]
Step 4 👣: Benefit is actually Net Benefit
💊💊
🤢🤢😷😷😷😷😷😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{2}}}{\text{10}} - \frac{\red{\bf{0}}}{\text{10}} * {\frac{{1}}{{4}}} = \frac{\green{\bf{2}}}{\text{10}}\]
💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{3}}}{\text{10}} - \frac{\red{\bf{4}}}{\text{10}} * {\frac{{1}}{{4}}} = \frac{\green{\bf{2}}}{\text{10}}\]
Step 5 👣: Net benefit can also be expressed as Interventions Avoided
Step 5 👣: Net benefit can also be expressed as Interventions Avoided
💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}}\]
💊💊💊💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨🤨🤨🤨
\[{\text{Net Benefit Treat All}} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}}\]
Step 5 👣: Net benefit can also be expressed as Interventions Avoided
💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{10}} - \frac{\red{\bf{FP}}}{\text{10}} * {\frac{{1}}{{4}}}\]
💊💊💊💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨🤨🤨🤨
\[{\text{Net Benefit Treat All}} = \frac{\green{\bf{TP}}}{\text{10}} - \frac{\red{\bf{FP}}}{\text{10}} * {\frac{{1}}{{4}}}\]
Step 5 👣: Net benefit can also be expressed as Interventions Avoided
💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{3}}}{\text{10}} - \frac{\red{\bf{FP}}}{\text{10}} * {\frac{{1}}{{4}}}\]
💊💊💊💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨🤨🤨🤨
\[{\text{Net Benefit Treat All}} = \frac{\green{\bf{3}}}{\text{10}} - \frac{\red{\bf{FP}}}{\text{10}} * {\frac{{1}}{{4}}}\]
Step 5 👣: Net benefit can also be expressed as Interventions Avoided
💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{3}}}{\text{10}} - \frac{\red{\bf{4}}}{\text{10}} * {\frac{{1}}{{4}}}\]
💊💊💊💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨🤨🤨🤨
\[{\text{Net Benefit Treat All}} = \frac{\green{\bf{3}}}{\text{10}} - \frac{\red{\bf{7}}}{\text{10}} * {\frac{{1}}{{4}}}\]
Step 5 👣: Net benefit can also be expressed as Interventions Avoided
💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨😷😷😷
\[{\text{Net Benefit}} = \frac{{\green{\bf{12}}}}{\text{40}} - \frac{{\red{\bf{4}}}}{\text{40}} = \frac{{\green{\bf{8}}}}{\text{40}}\]
💊💊💊💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨🤨🤨🤨
\[{\text{Net Benefit Treat All}} = \frac{{\green{\bf{12}}}}{\text{40}} - \frac{{\red{\bf{7}}}}{\text{40}} = \frac{{\green{\bf{5}}}}{\text{40}}\]
Step 5 👣: Net benefit can also be expressed as Interventions Avoided
💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨🤨🤨🤨
\[{\text{Net Benefit}} = \frac{{\green{\bf{8}}}}{\text{40}}\]
💊💊💊💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨🤨🤨🤨
\[{\text{Net Benefit Treat All}}= \frac{{\green{\bf{5}}}}{\text{40}}\]
\[\scriptsize{\text{(Net Benefit - Net Benefit All)} * \frac{1-p_t}{p_t} = (\frac{{\green{\bf{8}}}}{\text{40}} - \frac{{\green{\bf{5}}}}{\text{40}}) * 4 = \frac{{\green{\bf{3}}}}{\text{10}} }\]
\[\scriptsize{\text{Interventions Avoided (per 100 cases)} = \frac{\green{\bf{3}}}{10} * 100 = \green{\bf{30}}}\]
Step 5 👣: Net benefit can also be expressed as Interventions Avoided
💊💊💊💊💊💊💊
😷😷😷😷😷😷😷🤨🤨🤨
\[\scriptsize{\text{Interventions Avoided (per 100 cases)} = (\frac{\green{\bf{TN}}}{\text{N}} - \frac{\red{\bf{FN}}}{\text{N}} * {\frac{{1 - p_{t}}}{{p_{t}}}}) * 100}\]
Step 5 👣: Net benefit can also be expressed as Interventions Avoided
💊💊💊💊💊💊💊
😷😷😷😷😷😷😷🤨🤨🤨
\[\scriptsize{\text{Interventions Avoided (per 100 cases)} = (\frac{\green{\bf{\text{3}}}}{\text{10}} - \frac{\red{\bf{FN}}}{\text{10}} * 4) * 100}\]
Step 5 👣: Net benefit can also be expressed as Interventions Avoided
💊💊💊💊💊💊💊
😷😷😷😷😷😷😷🤨🤨🤨
\[\scriptsize{\text{Interventions Avoided (per 100 cases)} = (\frac{\green{\bf{\text{3}}}}{\text{10}} - \frac{\red{\bf{\text{0}}}}{\text{10}} * 4) * 100}\]
Step 5 👣: Net benefit can also be expressed as Interventions Avoided
💊💊💊💊💊💊💊
😷😷😷😷😷😷😷🤨🤨🤨
\[\scriptsize{\text{Interventions Avoided (per 100 cases)} = (\frac{\green{\bf{\text{3}}}}{\text{10}} - \frac{\red{\bf{\text{0}}}}{\text{10}} * 4 )* 100} = \green{\bf{30}} \]
Step 5 👣: Net benefit can also be expressed as Interventions Avoided
💊💊
🤢🤢😷😷😷😷😷😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}}\]
💊💊💊💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨🤨🤨🤨
\[{\text{Net Benefit Treat All}} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}}\]
Step 5 👣: Net benefit can also be expressed as Interventions Avoided
💊💊
🤢🤢😷😷😷😷😷😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{10}} - \frac{\red{\bf{FP}}}{\text{10}} * {\frac{{1}}{{4}}}\]
💊💊💊💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨🤨🤨🤨
\[{\text{Net Benefit Treat All}} = \frac{\green{\bf{TP}}}{\text{10}} - \frac{\red{\bf{FP}}}{\text{10}} * {\frac{{1}}{{4}}}\]
Step 5 👣: Net benefit can also be expressed as Interventions Avoided
💊💊
🤢🤢😷😷😷😷😷😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{2}}}{\text{10}} - \frac{\red{\bf{FP}}}{\text{10}} * {\frac{{1}}{{4}}}\]
💊💊💊💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨🤨🤨🤨
\[{\text{Net Benefit Treat All}} = \frac{\green{\bf{3}}}{\text{10}} - \frac{\red{\bf{FP}}}{\text{10}} * {\frac{{1}}{{4}}}\]
Step 5 👣: Net benefit can also be expressed as Interventions Avoided
💊💊
🤢🤢😷😷😷😷😷😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{2}}}{\text{10}} - \frac{\red{\bf{0}}}{\text{10}} * {\frac{{1}}{{4}}}\]
💊💊💊💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨🤨🤨🤨
\[{\text{Net Benefit Treat All}} = \frac{\green{\bf{3}}}{\text{10}} - \frac{\red{\bf{7}}}{\text{10}} * {\frac{{1}}{{4}}}\]
Step 5 👣: Net benefit can also be expressed as Interventions Avoided
💊💊
🤢🤢😷😷😷😷😷😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{8}}}{\text{40}} - \frac{\red{\bf{0}}}{\text{10}} * {\frac{{1}}{{4}}} = {\frac{{\green{\bf{8}}}}{{40}}}\]
💊💊💊💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨🤨🤨🤨
\[{\text{Net Benefit Treat All}} = \frac{{\green{\bf{12}}}}{\text{40}} - \frac{{\red{\bf{7}}}}{\text{40}} = \frac{{\green{\bf{5}}}}{\text{40}}\]
Step 5 👣: Net benefit can also be expressed as Interventions Avoided
💊💊
🤢🤢🤢🤨🤨🤨🤨🤨🤨🤨
\[{\text{Net Benefit}} = \frac{{\green{\bf{8}}}}{\text{40}}\]
💊💊💊💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨🤨🤨🤨
\[{\text{Net Benefit Treat All}}= \frac{{\green{\bf{5}}}}{\text{40}}\]
\[\scriptsize{\text{(Net Benefit - Net Benefit All)} * \frac{1-p_t}{p_t} = (\frac{{\green{\bf{8}}}}{\text{40}} - \frac{{\green{\bf{5}}}}{\text{40}}) * 4 = \frac{{\green{\bf{3}}}}{\text{10}} }\]
\[\scriptsize{\text{Interventions Avoided (per 100 cases)} = \frac{\green{\bf{3}}}{10} * 100 = \green{\bf{30}}}\]
Step 5 👣: Net benefit can also be expressed as Interventions Avoided
💊💊
😷😷🤢🤨🤨🤨🤨🤨🤨🤨
\[\scriptsize{\text{Interventions Avoided (per 100 cases)} = (\frac{\green{\bf{TN}}}{\text{N}} - \frac{\red{\bf{FN}}}{\text{N}} * {\frac{{1 - p_{t}}}{{p_{t}}}}) * 100}\]
Step 5 👣: Net benefit can also be expressed as Interventions Avoided
💊💊
😷😷🤢🤨🤨🤨🤨🤨🤨🤨
\[\scriptsize{\text{Interventions Avoided (per 100 cases)} = (\frac{\green{\bf{\text{7}}}}{\text{10}} - \frac{\red{\bf{FN}}}{\text{10}} * 4) * 100}\]
Step 5 👣: Net benefit can also be expressed as Interventions Avoided
💊💊
😷😷🤢🤨🤨🤨🤨🤨🤨🤨
\[\scriptsize{\text{Interventions Avoided (per 100 cases)} = (\frac{\green{\bf{\text{7}}}}{\text{10}} - \frac{\red{\bf{\text{1}}}}{\text{10}} * 4) * 100}\]
Step 5 👣: Net benefit can also be expressed as Interventions Avoided
💊💊
😷😷🤢🤨🤨🤨🤨🤨🤨🤨
\[\scriptsize{\text{Interventions Avoided (per 100 cases)} = (\frac{\green{\bf{\text{7}}}}{\text{10}} - \frac{\red{\bf{\text{1}}}}{\text{10}} * 4 )* 100} = \green{\bf{30}} \]
How much of a difference is enough?
Invasive Test might add information to the absolute risk with a possible Test Harm in forms of side affects or expenditure of time, effort or money.
I will be indifferent 😐 for having 1 TP for 10 Invasive Tests \[TestHarm = \frac{1}{10} = 0.1\]
\[\begin{aligned} \text{Net Benefit} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned} - TestHarm\]
How much of a difference is enough?
Invasive Test might add information to the absolute risk with a possible Test Harm in forms of side affects or expenditure of time, effort or money.
I will be indifferent 😐 for having 1 TP for 10 Invasive Tests \[TestHarm = \frac{1}{10} = 0.1\]
\[\begin{aligned} \text{Net Benefit} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned} - TestHarm\]
The same is true for Implementation of Prediction Models 🔮
How much of a difference in curves is enough?
🧪🧪🧪🧪🧪🧪🧪🧪🧪🧪
💊💊💊
🤢🤢🤢😷😷😷😷😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}} - TestHarm\]
💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{N}} - \frac{\red{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}}\]
How much of a difference in curves is enough?
🧪🧪🧪🧪🧪🧪🧪🧪🧪🧪
💊💊💊
🤢🤢🤢😷😷😷😷😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{10}} - \frac{\red{\bf{FP}}}{\text{10}} * {\frac{1}{{4}}} - TestHarm\]
💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{10}} - \frac{\red{\bf{FP}}}{\text{10}} * {\frac{1}{{4}}}\]
How much of a difference in curves is enough?
🧪🧪🧪🧪🧪🧪🧪🧪🧪🧪
💊💊💊
🤢🤢🤢😷😷😷😷😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{10}} - \frac{\red{\bf{FP}}}{\text{10}} * {\frac{1}{{4}}} - \frac{1}{10} * \frac{\red{\bf{10}}}{10}\]
💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{TP}}}{\text{10}} - \frac{\red{\bf{FP}}}{\text{10}} * {\frac{1}{{4}}}\]
How much of a difference in curves is enough?
🧪🧪🧪🧪🧪🧪🧪🧪🧪🧪
💊💊💊
🤢🤢🤢😷😷😷😷😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{3}}}{\text{10}} - \frac{\red{\bf{FP}}}{\text{10}} * {\frac{1}{{4}}} - \frac{1}{10} * \frac{\red{\bf{10}}}{10}\]
💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{3}}}{\text{10}} - \frac{\red{\bf{FP}}}{\text{10}} * {\frac{1}{{4}}}\]
How much of a difference in curves is enough?
🧪🧪🧪🧪🧪🧪🧪🧪🧪🧪
💊💊💊
🤢🤢🤢😷😷😷😷😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{3}}}{\text{10}} - \frac{\red{\bf{0}}}{\text{10}} * {\frac{1}{{4}}} - \frac{1}{10} * \frac{\red{\bf{10}}}{10}\]
💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{3}}}{\text{10}} - \frac{\red{\bf{4}}}{\text{10}} * {\frac{1}{{4}}}\]
How much of a difference in curves is enough?
🧪🧪🧪🧪🧪🧪🧪🧪🧪🧪
💊💊💊
🤢🤢🤢😷😷😷😷😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{2}}}{\text{10}}\]
💊💊💊💊💊💊💊
🤢🤢🤢🤨🤨🤨🤨😷😷😷
\[{\text{Net Benefit}} = \frac{\green{\bf{2}}}{\text{10}}\]
How is treatment effect taken into account?
How is treatment effect taken into account?
