Questions tagged [scoring]
The scoring tag has no usage guidance.
0
votes
1answer
22 views
How to get mean test scores from GridSearchCV with multiple scorers - scikit-learn
I'm trying to get mean test scores from scikit-learn's GridSearchCV with multiple scorers.
grid.cv_results_ displays lots of ...
0
votes
0answers
13 views
how to set the positive and the negative class for sequential features selection?
I'm working and classification problem, so after choosing Random Forest as a model, I should the best features for it, for that I used "Sequential Feature Selector" from ...
2
votes
0answers
193 views
Credit scoring using scorecardpy with XGBoost
I used XGBoost for scoring creditworthiness. At first I thought I could use predict_proba for scoring but then I saw that there was a module scorecardpy based on WOE to claculate code scoring. I tried ...
3
votes
2answers
49 views
Is there a definitive and more conclusive way of interpreting the R^2 score from a linear regression model in terms of prediction accuracy?
I'm trying to find a definitive way to conclude the R^2 score from a prediction accuracy point of view rather than variance. How should I do it?
Conceptually, most blogs / articles explain R^2 as:
...
6
votes
1answer
140 views
Why is the F-measure preferred for classification tasks?
Why is the F-measure usually used for (supervised) classification tasks, whereas the G-measure (or Fowlkes–Mallows index) is generally used for (unsupervised) clustering tasks?
The F-measure is the ...
3
votes
2answers
41 views
How can I compare classes from clusterings performed on two different data sets?
I have two data sets defined by real valued vectors, and I have performed clustering on both of them. Now I want to compare the classes to see how they map to each other. If I put the data sets ...
1
vote
0answers
54 views
Predict class having only class proportions for every attribute (non labeled data)
I am working with a big data set (millions of observations) where for each observation I am trying to predict a probability (or score it) of being of a class. I haven't any labeled training data and ...
0
votes
1answer
18 views
How to approach model reporting task
I have been tasked to report on an ensemble model that was created in h2o which includes several model subtypes such as Random Forest, GBM, linear models etc. The end goal is to predict churn rates ...
1
vote
1answer
24 views
Is the maximum BDeu Bayesian Network always the empty network?
I'm recently reading a paper about Scoring Mechanisms for Bayesian Networks.
For the BDeu score, it appears that the maximum possible score of BDeu for Bayesian Network structure learning is zero.
...
1
vote
0answers
288 views
Ranking algorithm based on a handful of features
I am trying to determine the apt algorithm for a ranking problem that I am working on. I have social media metrics - engagement, sentiment, audience size etc for several brands and am looking for a ...
0
votes
0answers
50 views
What is an appropriate way to compare classifiers with different sets of classes?
I have three classifiers for language identification:
A: en, de, ru, fr, ij, kl
B: en, de, ru, fr, xy
C: en, de, ru, fr, no, pq, rs
and I have a balanced dataset ...
2
votes
0answers
24 views
Intuitive interpretation of ratios between training set scores and validation set scores
I'm training models with the usual setup where you hold back a portion (in my case, 20%) of the data just to see how your trained model generalizes to unseen data, to see if it's overfitting.
When ...
2
votes
3answers
81 views
How to learn to score new documents based on a existing set of scored documents?
I have a 50 000 documents of 1000 words or more ranked between 0 and 2000. They all deal with a similar topic. I'd like to create an algorithm that can learn to score new documents.
What approach do ...
2
votes
1answer
686 views
Selecting the right algorithm for match probability prediction
Looking for assistance kick-starting a new machine learning scenario. In this case I need to pair one entity (ex. person) with a group of entities (ex. other people) given a history of matching ...
4
votes
2answers
1k views
Converting non-numeric data values into equivalent rank scores
Consider a data-frame similar to the one shown (the actual data-frame is much larger)
...
2
votes
1answer
733 views
Prediction model for marketing to prospective customers (using pandas)
I'm currently working on a part-time project which involves predicting the likelihood of customers going to buy a product using data analytics. The company I'm interning with has given me a customer ...
6
votes
2answers
877 views
Why are precision and recall used in the F1 score, rather than precision and NPV?
In binary classification problems it seems the F1 score is often used as a performance measure. As far as I've understood the idea is to find the best tradeoff between precision and recall. The ...
1
vote
2answers
134 views
Creating validation data for model comparison
I am working on building a scoring algorithm for student data,
say the attributes are :
name, location, age, class, school_name, skill1, skill2, skill3
based ...
4
votes
1answer
369 views
Comparing accuracy of models in ordinal regression / classification
I am looking into creating a model to predict whether an item is "Very Good", "Good", "Bad" or "Very Bad".
After I fit the training data to the models, comparing the accuracy of the models during ...
2
votes
2answers
1k views
R in production
Many of us are very familiar with using R in reproducible, but very much targeted, ad-hoc analysis. Given that R is currently the best collection of cutting-edge scientific methods from world-class ...
2
votes
1answer
380 views
Rank players of any given sport
I've recently become interested in possibly of developing some sort of method for ranking athletes of sports such as American football and determining which players are better than others in terms of ...
