Last call to make your voice heard! Our 2022 Developer Survey closes in less than a week. Take survey.

Questions tagged [scoring]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
37 views

Scikit-learn with a custom scoring function using a 'feature'

0 I am trying to use a new metric called 'SERA' (Squared Error Relevance Area) as a custom scoring function for imbalanced regression as mentioned in this paper. https://link.springer.com/article/10....
user avatar
  • 1
0 votes
1 answer
46 views

Approaches for matching leads to salesmen

I'm starting to tackle a new problem where we are trying to optimally match new leads (perspective customers) for our product to our sales representatives in the hopes of improving bottom-line metrics ...
user avatar
0 votes
0 answers
25 views

Develop a Scorecard Model with Orange 3.30

I'm a super fan user of Orange 3.30, actually I've beeing develop some Collection Strategys and some othe of CLI in my actuall work, and everything has been OK with all the decisions that I've being ...
user avatar
0 votes
0 answers
19 views

How can i adapt accuracy metric for multiclass classification?

I have a problem which is multiclass e.g. That is 4 classes. I would like a custom metric to assess the model where only if class 3 is predicted as class 2 and class 2 is predicted as class 3 (i.e. ...
user avatar
  • 466
1 vote
0 answers
18 views

Right way to compare model scores for Next Best Action

I have around 15 classification models for different products built in different ways (some are RF, some are Gradient Boosting, some were downsampled in one way, others in other way, some are built in ...
user avatar
0 votes
0 answers
15 views

calculating score for the sentiment

I am working on the sentiment project. I have used the BERT model. Now I need to generate a score for the sentiment of each sentence. I don't have any idea what would be the potential approach to do ...
user avatar
  • 321
0 votes
1 answer
135 views

Scikit-learn make_scorer custom metric problem for multiclass clasification

I was doing a churn analysis using: ...
user avatar
0 votes
0 answers
26 views

Evaluate scoring algorithm

I have several large datasets (100k samples) with label 1 / 0 and scoring algorithms that should score each sample, after the scoring of the dataset I sort it by the score. I would like to evaluate ...
user avatar
  • 273
0 votes
1 answer
31 views

Data science tools for easing the participation of a business into their scoring system

I'm a working in a small company. The company sells products on a website and they have a python script that runs everyday to attribute a score to each product based on a set of parameters (google ...
user avatar
1 vote
0 answers
10 views

Scoring samples after clusterings [closed]

I want to assign a score to all points in a group that I cluster several time. I want the score to indicate how much this point is grouped with the same individuals all time. I suppose this idea ...
user avatar
  • 291
2 votes
1 answer
164 views

Having trouble scaling scores of logistic regression

I am constructing a credit scorecard using logistic regression, similar to the one shown here. However, when trying to convert the coefficients of logistic regression into score representation (by ...
user avatar
  • 215
0 votes
1 answer
56 views

What is the proper way to bin variables for calculating WoE during credit scoring?

I have read this article about developing a credit scorecard in python, where it is stated that when binning the continuous variables, it needs to be ensured that: ...
user avatar
  • 215
1 vote
1 answer
34 views

How to score the health of a company? [closed]

i'm currently doing dual apprenticeships. My main mission is to represent the health of a company based on accounting records for multiple companies over multiple years. The part of an accounting ...
user avatar
1 vote
1 answer
90 views

What Machine Learning Technique can I use to judge boxing fights? [closed]

I want to build a machine learning model that judges the fights based on the results of each round. Any suggestions on what techniques can I use?
user avatar
  • 111
1 vote
0 answers
31 views

Standardizing binary decision with other scales (Like 1-5)

In the company I work for there are 2 different evaluation metrics for a song: Yes / No (Equivalent to like/dislike) 1-5 Scale Customers can use both to rank songs they like. I would like to create ...
user avatar
1 vote
0 answers
99 views

xgboost calibration kde plots (isotonic) not smooth

i am training my xgboost model on an imbalanced binary classification problem. It is important to me to have well calibrated probabilities so i have chosen to optimize the brier score. I then plot the ...
user avatar
  • 466
1 vote
2 answers
248 views

What is the best way to calculate score based on input features?

Lets say we have certain products. We also have certain input features regarding these products like inventory, sales, price, cost etc. based on these features we want to score these parts. The score ...
user avatar
  • 111
3 votes
3 answers
148 views

How to do class balancing?

I am working with a really imbalanced dataset ($\approx$ 1% of positive cases) for a classification problem. I know that class balancing is an important step in this scenario. I have two questions: ...
user avatar
  • 183
0 votes
0 answers
17 views

Weighted Average Score

I have several different scores that I'd like to combine them into a single score that represents all parameters. Score components are: Nutrition Score (1-10): 1 for the worst quality and 10 for the ...
user avatar
  • 9
1 vote
2 answers
730 views

How to compute AUC in gridsearchSV (multiclass problem)

I'm working on a multiclass classification problem, comparing results from SVM and Random Forest classificators. I would like to use gridsearchCV for hyperparameters tuning and find that AUC is the ...
user avatar
  • 13
1 vote
0 answers
47 views

scaling credit risk scorecard

I need to build a credit risk scorecard using logistic and linear regression. The variables using to predict are all dummies, where each dummy is a bin of some variable. Let's say the variable age, I ...
user avatar
2 votes
1 answer
52 views

Random forest mode scoring

We are using random forest algorithm but having some trouble understanding the scoring method it uses. take for example the following CM of the test set: ...
user avatar
  • 181
0 votes
1 answer
28 views

Scoring Methodology

Consider a the below provided sample data; ...
user avatar
  • 65
1 vote
0 answers
49 views

Scoring products across all stores

I am trying to score each product based off a couple of features. I was thinking I can use the sales rate(7 days prior) as the target column and predict the sales rate using some ensemble technique ...
user avatar
2 votes
2 answers
60 views

Best practices for scoring hundreds of models on the same massive dataset?

I have 500+ models predicting various things and a massive database of over 400m+ individuals and about 5,000 possible independent variables. Currently, my scoring process takes about 5 days, and ...
user avatar
  • 205
1 vote
1 answer
815 views

How to create a score for a SWOT analysis (strengths, weaknesses, opportunities, and threats)?

I'm developing a participatory social environmental diagnostic. To do this, I'm using primary (qualitative data from interviews with stakeholders) and secondary data (local socioeconomic data). From ...
user avatar
  • 113
1 vote
2 answers
2k views

Which scoring for GridSearchCV is best, when imbalanced multiclass dataset?

I have an unbalanced multiclass dataset (GTSRB) and want to optimize the hyperparameters of an SVM through GridSearchCV. I know that accuracy is not suitable for scoring in this case. Which evaluation ...
user avatar
  • 383
3 votes
1 answer
550 views

sklearn.metrics.average_precision_score getting different answers for same data but different formats

I was trying to learn how average precision (AP) is calculated and implemented in scikit-learn. I have read the documentation, but I don't think I fully understand it yet. Consider the following two ...
user avatar
  • 33
2 votes
1 answer
49 views

what approach to use for find best customer out of data?

I'm working on this project where the objective is to find certain good leads/customers from the existing customer dataset. I tried the RFM method for scoring but there is no data regarding money or ...
user avatar
1 vote
0 answers
17 views

Is there a term for measuring error on a second prediction based on the first's?

I have created a dataset which contains six values per row which may be the target value. Two rows for example: ...
user avatar
  • 111
2 votes
1 answer
2k views

About sklearn.metrics.average_precision_score documentation

There is a example in sklearn.metrics.average_precision_score documentation. ...
user avatar
1 vote
0 answers
18 views

Assessing performance of an agent based on commission rate, market share and revenue

I have a set of data for agents selling properties (apartments) for a company in different states. The company would like to assess the performance of the different agents given the following: Number ...
user avatar
2 votes
1 answer
74 views

Ranking ATM based on Utilization and Economic Data (Scoring/Rank Model)

I have a sample data of around 10 ATM's Locations along with their Utilization Count (Deposits, Withdrawals and Others) for the past 3 months. I am planning to collect additional data such as nearby ...
user avatar
  • 65
3 votes
1 answer
8k views

How to get mean test scores from GridSearchCV with multiple scorers - scikit-learn

I'm trying to get mean test scores from scikit-learn's GridSearchCV with multiple scorers. grid.cv_results_ displays lots of ...
user avatar
  • 370
2 votes
1 answer
402 views

How to interpret Sum of Squared Error in a classification task

I am working on ANN. I have 2497 training examples and each of them is a vector of 128, so the input size is 128. Number of neurons in hidden layer is 64 and number of output neurons is 6 (since ...
user avatar
4 votes
1 answer
3k views

Credit scoring using scorecardpy with XGBoost

I used XGBoost for scoring creditworthiness. At first I thought I could use predict_proba for scoring but then I saw that there was a module scorecardpy based on WOE to claculate code scoring. I tried ...
user avatar
3 votes
2 answers
130 views

Is there a definitive and more conclusive way of interpreting the R^2 score from a linear regression model in terms of prediction accuracy?

I'm trying to find a definitive way to conclude the R^2 score from a prediction accuracy point of view rather than variance. How should I do it? Conceptually, most blogs / articles explain R^2 as: ...
user avatar
  • 31
11 votes
3 answers
2k views

Why is the F-measure preferred for classification tasks?

Why is the F-measure usually used for (supervised) classification tasks, whereas the G-measure (or Fowlkes–Mallows index) is generally used for (unsupervised) clustering tasks? The F-measure is the ...
user avatar
4 votes
2 answers
164 views

How can I compare classes from clusterings performed on two different data sets?

I have two data sets defined by real valued vectors, and I have performed clustering on both of them. Now I want to compare the classes to see how they map to each other. If I put the data sets ...
user avatar
  • 143
1 vote
0 answers
76 views

Predict class having only class proportions for every attribute (non labeled data)

I am working with a big data set (millions of observations) where for each observation I am trying to predict a probability (or score it) of being of a class. I haven't any labeled training data and ...
user avatar
2 votes
1 answer
33 views

How to approach model reporting task

I have been tasked to report on an ensemble model that was created in h2o which includes several model subtypes such as Random Forest, GBM, linear models etc. The end goal is to predict churn rates ...
user avatar
  • 244
2 votes
1 answer
53 views

Is the maximum BDeu Bayesian Network always the empty network?

I'm recently reading a paper about Scoring Mechanisms for Bayesian Networks. For the BDeu score, it appears that the maximum possible score of BDeu for Bayesian Network structure learning is zero. ...
user avatar
3 votes
1 answer
481 views

Ranking algorithm based on a handful of features

I am trying to determine the apt algorithm for a ranking problem that I am working on. I have social media metrics - engagement, sentiment, audience size etc for several brands and am looking for a ...
user avatar
  • 302
1 vote
0 answers
68 views

What is an appropriate way to compare classifiers with different sets of classes?

I have three classifiers for language identification: A: en, de, ru, fr, ij, kl B: en, de, ru, fr, xy C: en, de, ru, fr, no, pq, rs and I have a balanced dataset ...
user avatar
  • 17.4k
3 votes
0 answers
31 views

Intuitive interpretation of ratios between training set scores and validation set scores

I'm training models with the usual setup where you hold back a portion (in my case, 20%) of the data just to see how your trained model generalizes to unseen data, to see if it's overfitting. When ...
user avatar
  • 211
2 votes
3 answers
87 views

How to learn to score new documents based on a existing set of scored documents?

I have a 50 000 documents of 1000 words or more ranked between 0 and 2000. They all deal with a similar topic. I'd like to create an algorithm that can learn to score new documents. What approach do ...
user avatar
  • 201
3 votes
1 answer
971 views

Selecting the right algorithm for match probability prediction

Looking for assistance kick-starting a new machine learning scenario. In this case I need to pair one entity (ex. person) with a group of entities (ex. other people) given a history of matching ...
user avatar
  • 131
5 votes
2 answers
2k views

Converting non-numeric data values into equivalent rank scores

Consider a data-frame similar to the one shown (the actual data-frame is much larger) ...
user avatar
  • 1,573
5 votes
1 answer
1k views

Prediction model for marketing to prospective customers (using pandas)

I'm currently working on a part-time project which involves predicting the likelihood of customers going to buy a product using data analytics. The company I'm interning with has given me a customer ...
user avatar
  • 413
7 votes
2 answers
2k views

Why are precision and recall used in the F1 score, rather than precision and NPV?

In binary classification problems it seems the F1 score is often used as a performance measure. As far as I've understood the idea is to find the best tradeoff between precision and recall. The ...
user avatar