Question



Not Enough Distinct Predictions to compute area under ROC.

Dr. Ballings,

Do you have any suggestions on how to move forward when receiving the error:

Error in roc(predKNNoptimal, yTEST) :
Not enough distinct predictions to compute area under the ROC curve.


I have tried modifying the K number to a variety of different numbers. I have tried normalizing each column between 0 and 1 using the min
and max rather than using the standard deviation/mean method. All my columns are numeric, no data is missing, no duplicated data, the data
is split properly. I have tried modifying which columns to use, but even with all these I still get this error.

Any suggestions would be appreciated.

Thanks,
Michael





Answers and follow-up questions





Answer or follow-up question 1

Dear Michael,

Have you looked at the objects that you are putting in the roc function?

Please post

table(predKNNoptimal)

str(yTEST)

and we'll go from there.

Thanks,
Michel Ballings


Answer or follow-up question 2

Dr. Ballings,

Here is my output of that, please note I only ran it for k in 1:20 on this computer b/c it's slow.

> table(predKNNoptimal)
predKNNoptimal
0 0.166666666666667 0.333333333333333 0.5 0.666666666666667
38212 39340 23307 10483 3136
0.833333333333333 1
1142 1
> str(yTEST)
Factor w/ 2 levels "0","1": 1 2 1 1 1 1 2 2 1 1 ...
>


Also here is:

> str(predKNNoptimal)
num [1:115621] 0.167 0.167 0.167 0.167 0.167 ...


Thanks,
Michael


Answer or follow-up question 3

Dear Michael,

You get this error when there are not enough unique prediction values. However, you have 7, which I think is enough to
create the ROC. I think you need to have at least 5 unique values.

Are you still getting the error?

If yes, please come to my office hours so I can diagnose the problem.

Michel Ballings


Answer or follow-up question 4

I am still getting the error, when can I come by?


Answer or follow-up question 5

My office hours are listed in the syllabus:
MW 11am-noon, 2pm-5pm


Answer or follow-up question 6

Dr. Ballings,

I figured it out and am now getting AUC. Simple mistake of having some NA's in one of the column after doing some data modifying.

Thanks,
Michael



Sign in to be able to add an answer or mark this question as resolved.