My dog show predictive tool is having some trouble with its neural net. Broadly, I start with a couple of factors--age, weight, height, breed (which is a set of dummy variables), a subjective cuteness score--and predict whether the animal will win best in a show (a binary variable). I've got a few hundred shows, so tens of thousands of rows. I'm stuck using the nnet package instead of neuralnet, but I can pretty easily create a decent neural net using
model <- nnet(Win ~ ., x=X_training, y=Y_training, size=30, maxit=500)
which works fine. I've tested it against the holdout test data and the AUC and confusion matrix looks good, and I can use it for predicting future winners. The problem is, how do I determine which variable is most important in predicting the outcome for each new dog show? When I run a prediction for future winners, I use the following code
winpred = predict(model, x, type='raw')
which provides a score between 0 and 1 of the likelihood of winning. What I'd also like is some way of identifying for each row which variable--age, weight, height, breed dummies, and cuteness--contributed the most and least to the score in my predictions. If I were doing this for a single dog, I would use LIME or something else based on this article, but unfortunately, I want to run this on a few hundred odd dogs at a time.
Basically, I want to be able to say to dog owners that their pup is doing well because of x, but what's most holding it back is y (not terribly helpful if it's breed, of course). It seems like there isn't a way to do this with neural networks. Are there other machine learning techniques now that would allow this? If not, any suggestions on other models that would work?
I was thinking of using OLS as a linear probability model or logit/probit, perhaps with LASSO or Ridge. I could use my machine learning model to score dogs, but in the background pull from the regression to identify which variables contribute most strongly, positively, and negatively, to the score. Logit/probit are better from a statistical perspective, but is there any easy way to interpret in R/RStudio, for a few hundred dogs at a time, which attribute contributed most strongly to their score?
