Question



Why use AUC?

A portion of our final grade on the large assignment is based on the AUC of our model.

Why do we use AUC?

Why do we not use the following equation instead?

(Sum of correct predictions / Sum of all predictions)

This way, you get a 0 for all incorrect predictions and a 1 for all correct predictions.

In your paper, you say that
"The advantage of AUC over other performance measures
(such as Percent Correctly Classified (PCC)) is that AUC includes all cut-off values."

Doesn't the equation I wrote above use all cut-off values?

Simple models are helpful in business.

How do I persuade a future client that AUC -- a model that uses integral calculus to
calculate -- is better for evaluating the accuracy of predictions than a model that
uses simple addition and division?





Answers and follow-up questions





Answer or follow-up question 1

Dear student,

You said:

"Why do we not use the following equation instead?

(Sum of correct predictions / Sum of all predictions)

This way, you get a 0 for all incorrect predictions and a 1 for all correct predictions."

Answer:

You can only use this if you are predicting labels. However, we are predicting probabilities.
Therefore we are using AUC.

You said:

"Doesn't the equation I wrote above use all cut-off values?"

Answer:

No, it only uses one cut-off. You probably used the round function,
meaning that if the probability >= 0.5 then you get a 1, otherwise you get
a 0. As we will see in the course (classifier evaluation) this is entirely incorrect.

You can read all about it in the textbook (Section classifier evaluation), or
simply sit back and wait until I cover it in class in a couple of classes.

Michel Ballings










Sign in to be able to add an answer or mark this question as resolved.