Questions tagged [scoring]
The scoring tag has no usage guidance.
27
questions
0
votes
0answers
12 views
For a multiclass classification problem, how do we find the cohen kappa score?
So I have a multiclass classification problem and I have found the Matthews Correlation Coefficient of that (https://scikit-learn.org/stable/modules/model_evaluation.html#matthews-correlation-...
0
votes
0answers
16 views
Open Set Classification : How to count True Positive, False Positive,
TL;DR : How to compute TP,FP,TN,FN in Open Set Classification setting ?
Even if the problem is simple; the answer may be tricky, so is my question.
Given two sets of clusters : one from experiments, ...
0
votes
0answers
16 views
Is the precision_recall_curve reasonable?
I always get this 'pattern' PR curve when I use precision_recall_curve function to plot PR curve.
The starting point is always from (0,1).
Because when thresholds set up the highest score lend to ...
1
vote
0answers
15 views
Is there a term for measuring error on a second prediction based on the first's?
I have created a dataset which contains six values per row which may be the target value. Two rows for example:
...
2
votes
1answer
61 views
About sklearn.metrics.average_precision_score documentation
There is a example in sklearn.metrics.average_precision_score documentation.
...
1
vote
0answers
16 views
Assessing performance of an agent based on commission rate, market share and revenue
I have a set of data for agents selling properties (apartments) for a company in different states. The company would like to assess the performance of the different agents given the following:
Number ...
2
votes
1answer
44 views
Ranking ATM based on Utilization and Economic Data (Scoring/Rank Model)
I have a sample data of around 10 ATM's Locations along with their Utilization Count (Deposits, Withdrawals and Others) for the past 3 months. I am planning to collect additional data such as nearby ...
1
vote
1answer
846 views
How to get mean test scores from GridSearchCV with multiple scorers - scikit-learn
I'm trying to get mean test scores from scikit-learn's GridSearchCV with multiple scorers.
grid.cv_results_ displays lots of ...
4
votes
1answer
572 views
Credit scoring using scorecardpy with XGBoost
I used XGBoost for scoring creditworthiness. At first I thought I could use predict_proba for scoring but then I saw that there was a module scorecardpy based on WOE to claculate code scoring. I tried ...
3
votes
2answers
66 views
Is there a definitive and more conclusive way of interpreting the R^2 score from a linear regression model in terms of prediction accuracy?
I'm trying to find a definitive way to conclude the R^2 score from a prediction accuracy point of view rather than variance. How should I do it?
Conceptually, most blogs / articles explain R^2 as:
...
8
votes
2answers
382 views
Why is the F-measure preferred for classification tasks?
Why is the F-measure usually used for (supervised) classification tasks, whereas the G-measure (or Fowlkes–Mallows index) is generally used for (unsupervised) clustering tasks?
The F-measure is the ...
4
votes
2answers
43 views
How can I compare classes from clusterings performed on two different data sets?
I have two data sets defined by real valued vectors, and I have performed clustering on both of them. Now I want to compare the classes to see how they map to each other. If I put the data sets ...
1
vote
0answers
61 views
Predict class having only class proportions for every attribute (non labeled data)
I am working with a big data set (millions of observations) where for each observation I am trying to predict a probability (or score it) of being of a class. I haven't any labeled training data and ...
1
vote
1answer
24 views
How to approach model reporting task
I have been tasked to report on an ensemble model that was created in h2o which includes several model subtypes such as Random Forest, GBM, linear models etc. The end goal is to predict churn rates ...
2
votes
1answer
26 views
Is the maximum BDeu Bayesian Network always the empty network?
I'm recently reading a paper about Scoring Mechanisms for Bayesian Networks.
For the BDeu score, it appears that the maximum possible score of BDeu for Bayesian Network structure learning is zero.
...
2
votes
0answers
330 views
Ranking algorithm based on a handful of features
I am trying to determine the apt algorithm for a ranking problem that I am working on. I have social media metrics - engagement, sentiment, audience size etc for several brands and am looking for a ...
1
vote
0answers
52 views
What is an appropriate way to compare classifiers with different sets of classes?
I have three classifiers for language identification:
A: en, de, ru, fr, ij, kl
B: en, de, ru, fr, xy
C: en, de, ru, fr, no, pq, rs
and I have a balanced dataset ...
2
votes
0answers
25 views
Intuitive interpretation of ratios between training set scores and validation set scores
I'm training models with the usual setup where you hold back a portion (in my case, 20%) of the data just to see how your trained model generalizes to unseen data, to see if it's overfitting.
When ...
2
votes
3answers
84 views
How to learn to score new documents based on a existing set of scored documents?
I have a 50 000 documents of 1000 words or more ranked between 0 and 2000. They all deal with a similar topic. I'd like to create an algorithm that can learn to score new documents.
What approach do ...
3
votes
1answer
789 views
Selecting the right algorithm for match probability prediction
Looking for assistance kick-starting a new machine learning scenario. In this case I need to pair one entity (ex. person) with a group of entities (ex. other people) given a history of matching ...
5
votes
2answers
1k views
Converting non-numeric data values into equivalent rank scores
Consider a data-frame similar to the one shown (the actual data-frame is much larger)
...
3
votes
1answer
850 views
Prediction model for marketing to prospective customers (using pandas)
I'm currently working on a part-time project which involves predicting the likelihood of customers going to buy a product using data analytics. The company I'm interning with has given me a customer ...
7
votes
2answers
994 views
Why are precision and recall used in the F1 score, rather than precision and NPV?
In binary classification problems it seems the F1 score is often used as a performance measure. As far as I've understood the idea is to find the best tradeoff between precision and recall. The ...
2
votes
2answers
137 views
Creating validation data for model comparison
I am working on building a scoring algorithm for student data,
say the attributes are :
name, location, age, class, school_name, skill1, skill2, skill3
based ...
5
votes
1answer
408 views
Comparing accuracy of models in ordinal regression / classification
I am looking into creating a model to predict whether an item is "Very Good", "Good", "Bad" or "Very Bad".
After I fit the training data to the models, comparing the accuracy of the models during ...
3
votes
2answers
1k views
R in production
Many of us are very familiar with using R in reproducible, but very much targeted, ad-hoc analysis. Given that R is currently the best collection of cutting-edge scientific methods from world-class ...
4
votes
1answer
478 views
Rank players of any given sport
I've recently become interested in possibly of developing some sort of method for ranking athletes of sports such as American football and determining which players are better than others in terms of ...
