How to manually calculate AUC?

The recent assignment about manually calculating the Naive Bayes and KNN was very helpful.

How can I manually calculate the AUC of my models?

The equation in your paper shows that AUC equals
the integral (from 0 to 1) of (TP/P)d(FP/N).

If the equation was AUC equals the integral (from 1 to 2) of xdx,
I would solve (2)^2 - (1)^2 to find the answer (AUC=3).

If TP=3, P=6, FP=2, and N=7,
what would be the calculation for the AUC integral (from 1 to 2)?

Answers and follow-up questions

Answer or follow-up question 1

Dear student,

This is how you compute the AUC:

You would create a plot with on the x-axis the false positive rate (FP/N=2/7)
and on the y-axis the true positive rate (TP/P=3/6). So you would have a plot of one point
with x-value 2/7 and y-value 3/6.

You have computed your TP and FP by cutting up the predicted probabilities
in two classes: [0,1] -> {0,1}.

If we do this for all possible cutoffs:

if (predicted_probability >= 0.99) predicted_label <- 1 else predicted_label <- 0
if (predicted_probability >= 0.98) predicted_label <- 1 else predicted_label <- 0
if (predicted_probability >= 0.01) predicted_label <- 1 else predicted_label <- 0

we can compute the TP and FP for all cutoffs and plot all these values in our plot.
The result would be a curve, called the ROC curve (receiver operating characteristic curve).
The area under that curve is the AUC and that area is computing with an integral.

Michel Ballings

Sign in to be able to add an answer or mark this question as resolved.