## Question

How to manually calculate AUC?

The recent assignment about manually calculating the Naive Bayes and KNN was very helpful.

How can I manually calculate the AUC of my models?

The equation in your paper shows that AUC equals
the integral (from 0 to 1) of (TP/P)d(FP/N).

If the equation was AUC equals the integral (from 1 to 2) of xdx,
I would solve (2)^2 - (1)^2 to find the answer (AUC=3).

If TP=3, P=6, FP=2, and N=7,
what would be the calculation for the AUC integral (from 1 to 2)?

Dear student,

This is how you compute the AUC:

You would create a plot with on the x-axis the false positive rate (FP/N=2/7)
and on the y-axis the true positive rate (TP/P=3/6). So you would have a plot of one point
with x-value 2/7 and y-value 3/6.

You have computed your TP and FP by cutting up the predicted probabilities
in two classes: [0,1] -> {0,1}.

If we do this for all possible cutoffs:

if (predicted_probability >= 0.99) predicted_label <- 1 else predicted_label <- 0
if (predicted_probability >= 0.98) predicted_label <- 1 else predicted_label <- 0
...
if (predicted_probability >= 0.01) predicted_label <- 1 else predicted_label <- 0

we can compute the TP and FP for all cutoffs and plot all these values in our plot.
The result would be a curve, called the ROC curve (receiver operating characteristic curve).
The area under that curve is the AUC and that area is computing with an integral.

Michel Ballings