Questions tagged [scoring]
The scoring tag has no usage guidance.
54
questions
0
votes
0
answers
37
views
Scikit-learn with a custom scoring function using a 'feature'
0
I am trying to use a new metric called 'SERA' (Squared Error Relevance Area) as a custom scoring function for imbalanced regression as mentioned in this paper. https://link.springer.com/article/10....
- 1
0
votes
1
answer
46
views
Approaches for matching leads to salesmen
I'm starting to tackle a new problem where we are trying to optimally match new leads (perspective customers) for our product to our sales representatives in the hopes of improving bottom-line metrics ...
0
votes
0
answers
25
views
Develop a Scorecard Model with Orange 3.30
I'm a super fan user of Orange 3.30, actually I've beeing develop some Collection Strategys and some othe of CLI in my actuall work, and everything has been OK with all the decisions that I've being ...
0
votes
0
answers
19
views
How can i adapt accuracy metric for multiclass classification?
I have a problem which is multiclass e.g. That is 4 classes. I would like a custom metric to assess the model where only if class 3 is predicted as class 2 and class 2 is predicted as class 3 (i.e. ...
- 466
1
vote
0
answers
18
views
Right way to compare model scores for Next Best Action
I have around 15 classification models for different products built in different ways (some are RF, some are Gradient Boosting, some were downsampled in one way, others in other way, some are built in ...
- 11
0
votes
0
answers
15
views
calculating score for the sentiment
I am working on the sentiment project. I have used the BERT model. Now I need to generate a score for the sentiment of each sentence.
I don't have any idea what would be the potential approach to do ...
- 321
0
votes
1
answer
135
views
Scikit-learn make_scorer custom metric problem for multiclass clasification
I was doing a churn analysis using:
...
0
votes
0
answers
26
views
Evaluate scoring algorithm
I have several large datasets (100k samples) with label 1 / 0 and scoring algorithms that should score each sample, after the scoring of the dataset I sort it by the score.
I would like to evaluate ...
- 273
0
votes
1
answer
31
views
Data science tools for easing the participation of a business into their scoring system
I'm a working in a small company. The company sells products on a website and they have a python script that runs everyday to attribute a score to each product based on a set of parameters (google ...
1
vote
0
answers
10
views
Scoring samples after clusterings [closed]
I want to assign a score to all points in a group that I cluster several time.
I want the score to indicate how much this point is grouped with the same individuals all time.
I suppose this idea ...
- 291
2
votes
1
answer
164
views
Having trouble scaling scores of logistic regression
I am constructing a credit scorecard using logistic regression, similar to the one shown here. However, when trying to convert the coefficients of logistic regression into score representation (by ...
- 215
0
votes
1
answer
56
views
What is the proper way to bin variables for calculating WoE during credit scoring?
I have read this article about developing a credit scorecard in python, where it is stated that when binning the continuous variables, it needs to be ensured that:
...
- 215
1
vote
1
answer
34
views
How to score the health of a company? [closed]
i'm currently doing dual apprenticeships. My main mission is to represent the health of a company based on accounting records for multiple companies over multiple years. The part of an accounting ...
1
vote
1
answer
90
views
What Machine Learning Technique can I use to judge boxing fights? [closed]
I want to build a machine learning model that judges the fights based on the results of each round. Any suggestions on what techniques can I use?
- 111
1
vote
0
answers
31
views
Standardizing binary decision with other scales (Like 1-5)
In the company I work for there are 2 different evaluation metrics for a song:
Yes / No (Equivalent to like/dislike)
1-5 Scale
Customers can use both to rank songs they like. I would like to create ...
1
vote
0
answers
99
views
xgboost calibration kde plots (isotonic) not smooth
i am training my xgboost model on an imbalanced binary classification problem. It is important to me to have well calibrated probabilities so i have chosen to optimize the brier score. I then plot the ...
- 466
1
vote
2
answers
248
views
What is the best way to calculate score based on input features?
Lets say we have certain products. We also have certain input features regarding these products like inventory, sales, price, cost etc. based on these features we want to score these parts. The score ...
- 111
3
votes
3
answers
148
views
How to do class balancing?
I am working with a really imbalanced dataset ($\approx$ 1% of positive cases) for a classification problem. I know that class balancing is an important step in this scenario.
I have two questions:
...
- 183
0
votes
0
answers
17
views
Weighted Average Score
I have several different scores that I'd like to combine them into a single score that represents all parameters. Score components are:
Nutrition Score (1-10): 1 for the worst quality and 10 for the ...
- 9
1
vote
2
answers
730
views
How to compute AUC in gridsearchSV (multiclass problem)
I'm working on a multiclass classification problem, comparing results from SVM and Random Forest classificators. I would like to use gridsearchCV for hyperparameters tuning and find that AUC is the ...
- 13
1
vote
0
answers
47
views
scaling credit risk scorecard
I need to build a credit risk scorecard using logistic and linear regression. The variables using to predict are all dummies, where each dummy is a bin of some variable.
Let's say the variable age, I ...
- 11
2
votes
1
answer
52
views
Random forest mode scoring
We are using random forest algorithm but having some trouble understanding the scoring method it uses.
take for example the following CM of the test set:
...
- 181
0
votes
1
answer
28
views
1
vote
0
answers
49
views
Scoring products across all stores
I am trying to score each product based off a couple of features. I was thinking I can use the sales rate(7 days prior) as the target column and predict the sales rate using some ensemble technique ...
2
votes
2
answers
60
views
Best practices for scoring hundreds of models on the same massive dataset?
I have 500+ models predicting various things and a massive database of over 400m+ individuals and about 5,000 possible independent variables.
Currently, my scoring process takes about 5 days, and ...
- 205
1
vote
1
answer
815
views
How to create a score for a SWOT analysis (strengths, weaknesses, opportunities, and threats)?
I'm developing a participatory social environmental diagnostic. To do this, I'm using primary (qualitative data from interviews with stakeholders) and secondary data (local socioeconomic data).
From ...
- 113
1
vote
2
answers
2k
views
Which scoring for GridSearchCV is best, when imbalanced multiclass dataset?
I have an unbalanced multiclass dataset (GTSRB) and want to optimize the hyperparameters of an SVM through GridSearchCV.
I know that accuracy is not suitable for scoring in this case.
Which evaluation ...
- 383
3
votes
1
answer
550
views
sklearn.metrics.average_precision_score getting different answers for same data but different formats
I was trying to learn how average precision (AP) is calculated and implemented in scikit-learn. I have read the documentation, but I don't think I fully understand it yet.
Consider the following two ...
- 33
2
votes
1
answer
49
views
what approach to use for find best customer out of data?
I'm working on this project where the objective is to find certain good leads/customers from the existing customer dataset. I tried the RFM method for scoring but there is no data regarding money or ...
- 37
1
vote
0
answers
17
views
Is there a term for measuring error on a second prediction based on the first's?
I have created a dataset which contains six values per row which may be the target value. Two rows for example:
...
- 111
2
votes
1
answer
2k
views
About sklearn.metrics.average_precision_score documentation
There is a example in sklearn.metrics.average_precision_score documentation.
...
- 87
1
vote
0
answers
18
views
Assessing performance of an agent based on commission rate, market share and revenue
I have a set of data for agents selling properties (apartments) for a company in different states. The company would like to assess the performance of the different agents given the following:
Number ...
- 75
2
votes
1
answer
74
views
Ranking ATM based on Utilization and Economic Data (Scoring/Rank Model)
I have a sample data of around 10 ATM's Locations along with their Utilization Count (Deposits, Withdrawals and Others) for the past 3 months. I am planning to collect additional data such as nearby ...
- 65
3
votes
1
answer
8k
views
How to get mean test scores from GridSearchCV with multiple scorers - scikit-learn
I'm trying to get mean test scores from scikit-learn's GridSearchCV with multiple scorers.
grid.cv_results_ displays lots of ...
- 370
2
votes
1
answer
402
views
How to interpret Sum of Squared Error in a classification task
I am working on ANN. I have 2497 training examples and each of them is a vector of 128, so the input size is 128. Number of neurons in hidden layer is 64 and number of output neurons is 6 (since ...
- 21
4
votes
1
answer
3k
views
Credit scoring using scorecardpy with XGBoost
I used XGBoost for scoring creditworthiness. At first I thought I could use predict_proba for scoring but then I saw that there was a module scorecardpy based on WOE to claculate code scoring. I tried ...
- 51
3
votes
2
answers
130
views
Is there a definitive and more conclusive way of interpreting the R^2 score from a linear regression model in terms of prediction accuracy?
I'm trying to find a definitive way to conclude the R^2 score from a prediction accuracy point of view rather than variance. How should I do it?
Conceptually, most blogs / articles explain R^2 as:
...
- 31
11
votes
3
answers
2k
views
Why is the F-measure preferred for classification tasks?
Why is the F-measure usually used for (supervised) classification tasks, whereas the G-measure (or Fowlkes–Mallows index) is generally used for (unsupervised) clustering tasks?
The F-measure is the ...
- 3,258
4
votes
2
answers
164
views
How can I compare classes from clusterings performed on two different data sets?
I have two data sets defined by real valued vectors, and I have performed clustering on both of them. Now I want to compare the classes to see how they map to each other. If I put the data sets ...
- 143
1
vote
0
answers
76
views
Predict class having only class proportions for every attribute (non labeled data)
I am working with a big data set (millions of observations) where for each observation I am trying to predict a probability (or score it) of being of a class. I haven't any labeled training data and ...
2
votes
1
answer
33
views
How to approach model reporting task
I have been tasked to report on an ensemble model that was created in h2o which includes several model subtypes such as Random Forest, GBM, linear models etc. The end goal is to predict churn rates ...
- 244
2
votes
1
answer
53
views
Is the maximum BDeu Bayesian Network always the empty network?
I'm recently reading a paper about Scoring Mechanisms for Bayesian Networks.
For the BDeu score, it appears that the maximum possible score of BDeu for Bayesian Network structure learning is zero.
...
3
votes
1
answer
481
views
Ranking algorithm based on a handful of features
I am trying to determine the apt algorithm for a ranking problem that I am working on. I have social media metrics - engagement, sentiment, audience size etc for several brands and am looking for a ...
- 302
1
vote
0
answers
68
views
What is an appropriate way to compare classifiers with different sets of classes?
I have three classifiers for language identification:
A: en, de, ru, fr, ij, kl
B: en, de, ru, fr, xy
C: en, de, ru, fr, no, pq, rs
and I have a balanced dataset ...
- 17.4k
3
votes
0
answers
31
views
Intuitive interpretation of ratios between training set scores and validation set scores
I'm training models with the usual setup where you hold back a portion (in my case, 20%) of the data just to see how your trained model generalizes to unseen data, to see if it's overfitting.
When ...
- 211
2
votes
3
answers
87
views
How to learn to score new documents based on a existing set of scored documents?
I have a 50 000 documents of 1000 words or more ranked between 0 and 2000. They all deal with a similar topic. I'd like to create an algorithm that can learn to score new documents.
What approach do ...
- 201
3
votes
1
answer
971
views
Selecting the right algorithm for match probability prediction
Looking for assistance kick-starting a new machine learning scenario. In this case I need to pair one entity (ex. person) with a group of entities (ex. other people) given a history of matching ...
- 131
5
votes
2
answers
2k
views
Converting non-numeric data values into equivalent rank scores
Consider a data-frame similar to the one shown (the actual data-frame is much larger)
...
- 1,573
5
votes
1
answer
1k
views
Prediction model for marketing to prospective customers (using pandas)
I'm currently working on a part-time project which involves predicting the likelihood of customers going to buy a product using data analytics. The company I'm interning with has given me a customer ...
- 413
7
votes
2
answers
2k
views
Why are precision and recall used in the F1 score, rather than precision and NPV?
In binary classification problems it seems the F1 score is often used as a performance measure. As far as I've understood the idea is to find the best tradeoff between precision and recall. The ...
- 71