Questions tagged [scoring]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
17 views

How to create a score for a SWOT analysis (strengths, weaknesses, opportunities, and threats)?

I'm developing a participatory social environmental diagnostic. To do this, I'm using primary (qualitative data from interviews with stakeholders) and secondary data (local socioeconomic data). From ...
1
vote
2answers
69 views

Which scoring for GridSearchCV is best, when imbalanced multiclass dataset?

I have an unbalanced multiclass dataset (GTSRB) and want to optimize the hyperparameters of an SVM through GridSearchCV. I know that accuracy is not suitable for scoring in this case. Which evaluation ...
3
votes
1answer
58 views

sklearn.metrics.average_precision_score getting different answers for same data but different formats

I was trying to learn how average precision (AP) is calculated and implemented in scikit-learn. I have read the documentation, but I don't think I fully understand it yet. Consider the following two ...
2
votes
1answer
41 views

what approach to use for find best customer out of data?

I'm working on this project where the objective is to find certain good leads/customers from the existing customer dataset. I tried the RFM method for scoring but there is no data regarding money or ...
0
votes
0answers
51 views

For a multiclass classification problem, how do we find the cohen kappa score?

So I have a multiclass classification problem and I have found the Matthews Correlation Coefficient of that (https://scikit-learn.org/stable/modules/model_evaluation.html#matthews-correlation-...
0
votes
0answers
32 views

Open Set Classification : How to count True Positive, False Positive,

TL;DR : How to compute TP,FP,TN,FN in Open Set Classification setting ? Even if the problem is simple; the answer may be tricky, so is my question. Given two sets of clusters : one from experiments, ...
0
votes
0answers
21 views

Is the precision_recall_curve reasonable?

I always get this 'pattern' PR curve when I use precision_recall_curve function to plot PR curve. The starting point is always from (0,1). Because when thresholds set up the highest score lend to ...
1
vote
0answers
17 views

Is there a term for measuring error on a second prediction based on the first's?

I have created a dataset which contains six values per row which may be the target value. Two rows for example: ...
2
votes
1answer
303 views

About sklearn.metrics.average_precision_score documentation

There is a example in sklearn.metrics.average_precision_score documentation. ...
1
vote
0answers
16 views

Assessing performance of an agent based on commission rate, market share and revenue

I have a set of data for agents selling properties (apartments) for a company in different states. The company would like to assess the performance of the different agents given the following: Number ...
2
votes
1answer
52 views

Ranking ATM based on Utilization and Economic Data (Scoring/Rank Model)

I have a sample data of around 10 ATM's Locations along with their Utilization Count (Deposits, Withdrawals and Others) for the past 3 months. I am planning to collect additional data such as nearby ...
1
vote
1answer
2k views

How to get mean test scores from GridSearchCV with multiple scorers - scikit-learn

I'm trying to get mean test scores from scikit-learn's GridSearchCV with multiple scorers. grid.cv_results_ displays lots of ...
4
votes
1answer
999 views

Credit scoring using scorecardpy with XGBoost

I used XGBoost for scoring creditworthiness. At first I thought I could use predict_proba for scoring but then I saw that there was a module scorecardpy based on WOE to claculate code scoring. I tried ...
3
votes
2answers
84 views

Is there a definitive and more conclusive way of interpreting the R^2 score from a linear regression model in terms of prediction accuracy?

I'm trying to find a definitive way to conclude the R^2 score from a prediction accuracy point of view rather than variance. How should I do it? Conceptually, most blogs / articles explain R^2 as: ...
8
votes
2answers
572 views

Why is the F-measure preferred for classification tasks?

Why is the F-measure usually used for (supervised) classification tasks, whereas the G-measure (or Fowlkes–Mallows index) is generally used for (unsupervised) clustering tasks? The F-measure is the ...
4
votes
2answers
46 views

How can I compare classes from clusterings performed on two different data sets?

I have two data sets defined by real valued vectors, and I have performed clustering on both of them. Now I want to compare the classes to see how they map to each other. If I put the data sets ...
1
vote
0answers
65 views

Predict class having only class proportions for every attribute (non labeled data)

I am working with a big data set (millions of observations) where for each observation I am trying to predict a probability (or score it) of being of a class. I haven't any labeled training data and ...
2
votes
1answer
29 views

How to approach model reporting task

I have been tasked to report on an ensemble model that was created in h2o which includes several model subtypes such as Random Forest, GBM, linear models etc. The end goal is to predict churn rates ...
2
votes
1answer
32 views

Is the maximum BDeu Bayesian Network always the empty network?

I'm recently reading a paper about Scoring Mechanisms for Bayesian Networks. For the BDeu score, it appears that the maximum possible score of BDeu for Bayesian Network structure learning is zero. ...
2
votes
0answers
361 views

Ranking algorithm based on a handful of features

I am trying to determine the apt algorithm for a ranking problem that I am working on. I have social media metrics - engagement, sentiment, audience size etc for several brands and am looking for a ...
1
vote
0answers
53 views

What is an appropriate way to compare classifiers with different sets of classes?

I have three classifiers for language identification: A: en, de, ru, fr, ij, kl B: en, de, ru, fr, xy C: en, de, ru, fr, no, pq, rs and I have a balanced dataset ...
2
votes
0answers
26 views

Intuitive interpretation of ratios between training set scores and validation set scores

I'm training models with the usual setup where you hold back a portion (in my case, 20%) of the data just to see how your trained model generalizes to unseen data, to see if it's overfitting. When ...
2
votes
3answers
84 views

How to learn to score new documents based on a existing set of scored documents?

I have a 50 000 documents of 1000 words or more ranked between 0 and 2000. They all deal with a similar topic. I'd like to create an algorithm that can learn to score new documents. What approach do ...
3
votes
1answer
833 views

Selecting the right algorithm for match probability prediction

Looking for assistance kick-starting a new machine learning scenario. In this case I need to pair one entity (ex. person) with a group of entities (ex. other people) given a history of matching ...
5
votes
2answers
1k views

Converting non-numeric data values into equivalent rank scores

Consider a data-frame similar to the one shown (the actual data-frame is much larger) ...
4
votes
1answer
914 views

Prediction model for marketing to prospective customers (using pandas)

I'm currently working on a part-time project which involves predicting the likelihood of customers going to buy a product using data analytics. The company I'm interning with has given me a customer ...
7
votes
2answers
1k views

Why are precision and recall used in the F1 score, rather than precision and NPV?

In binary classification problems it seems the F1 score is often used as a performance measure. As far as I've understood the idea is to find the best tradeoff between precision and recall. The ...
2
votes
2answers
139 views

Creating validation data for model comparison

I am working on building a scoring algorithm for student data, say the attributes are : name, location, age, class, school_name, skill1, skill2, skill3 based ...
5
votes
1answer
430 views

Comparing accuracy of models in ordinal regression / classification

I am looking into creating a model to predict whether an item is "Very Good", "Good", "Bad" or "Very Bad". After I fit the training data to the models, comparing the accuracy of the models during ...
3
votes
2answers
1k views

R in production

Many of us are very familiar with using R in reproducible, but very much targeted, ad-hoc analysis. Given that R is currently the best collection of cutting-edge scientific methods from world-class ...
4
votes
1answer
542 views

Rank players of any given sport

I've recently become interested in possibly of developing some sort of method for ranking athletes of sports such as American football and determining which players are better than others in terms of ...