Questions tagged [ranking]
Ranking is ordering data from highest to lowest or *vice versa.* For questions about *constructing* scores to use in ranking, please use the "valuation" tag, too.
116 questions
0
votes
0
answers
6
views
Looking for a rank distance with higher weights for initial items
Assume that the results of a race is as follows: ["a", "b", "c", "d", "e"], so "a" is the winner. Before ...
- 111
0
votes
0
answers
7
views
Ranking discriminator and loss
A ranking discriminator D(z1, z2, z3 ) ∈ [0, 1] that should be high if
_z1 ∼ p1 , z2 ∼ p2 , z3 ∼ p3 _ and low otherwise. For finding the loss and optimal discriminator D(z1 , z2 , z3 ) should I modify ...
0
votes
0
answers
8
views
How to judge the model's confidence in Learning-To-Rank (LTR)?
Recently I have been working on Learning-To-Rank model. I think it's very useful for my purpose, but how can we extract information regarding its confidence and accuracy? Specifically, I am using the ...
- 1
0
votes
1
answer
46
views
What is the effect on a LightGBM ranker model of having many groups where all observations have the same target value?
I'm working on a recommender system where users are recomended 5 items at a time.
We build a dataset by saving the 5 items along with feature values and a "relevant" target value (1 if the ...
- 1
0
votes
0
answers
12
views
Commonly used metric in NLP literature to compare ranked weighted results with variable importance for top-k results
I have two different search engines that always return the same results but in different orders. The results consist of websites along with confidence scores, which range from 100 to 10,000. The ...
- 157
0
votes
0
answers
11
views
how to obtain predictions for all observations in a dataset?
I have a dataset comprising 1864 roads with various traffic and safety features, along with the number of accidents at each road. The objective is to rank these roads based on the number of accidents. ...
- 1,305
0
votes
0
answers
23
views
Recommendation System Algorithms for Multi-entity ranking
I'm looking for industry engineering or research papers tacking the problem of universally ranking disparate items. For example, one example is the Doordash recommender, which their team attempted to ...
0
votes
0
answers
8
views
Best model for predicting user ranking of a bag of (travel) options without a fixed size
I need to predict which flights users's will choose, as a %. For example a user searches for flights LHR->JFK. There will be a variable set of available options each with a price, number of legs, ...
- 101
0
votes
1
answer
149
views
Which python libraries do you recommend for label ranking?
I'm currently looking for python libraries that offer models for the label ranking problem. So provided with a context x and a set of Labels Y, the model should output a ranking of those labels.
I'm ...
- 11
0
votes
1
answer
43
views
Using text embeddings directly to compute similarity vs using them as features for a model that predicts similariy
Say you have a problem where you have a query and a set of result documents and you want to rank the result documents according to the query. Say also you have embeddings for the query and for the ...
- 183
0
votes
0
answers
47
views
What are machine learning algorithms which rank films?
Broadly speaking, there are two approaches to producing lists of the best films: human vs machine curation.
Human curation involves someone, a critic, making decisions and/or creating a canon, based ...
- 141
0
votes
1
answer
12
views
How to handle using input feature (clicks) when it is used in target too?
I am trying to create a ranking model, where I am thinking about creating ground truth based on clicks by user. But at same time past clicks made by users seems like a vital input feature too. Any ...
- 1
0
votes
1
answer
20
views
Unsupervised ranking of samples
Say I have a dataset of n samples. I want to maximize every feature’s value. I’m not sure if feature 1 is more important than feature 2, etc. Are there any methods of ranking my samples out there? If ...
- 1
4
votes
0
answers
312
views
How Does the Reward Model in ChatGPT Calculate Losses?
Reading the InstructGPT paper(which seems to be what ChatGPT was built off of), I found this equation for the reward function.
However, I'm struggling to understand how this equation is used to ...
- 141
1
vote
1
answer
432
views
Combining multiple ranked lists
Suppose I'm given two ranked lists, A and B, with each item in the lists being associated with a score:
...
1
vote
0
answers
46
views
How do I train an ML model that can rank pairs?
Consider that I have a dataset consisting of a list of pairs of the titles of blogposts of the form ...
- 111
2
votes
0
answers
80
views
Smallest possible difference between AUC of two ranker [closed]
If there are 10 positive examples, and 90 negative examples in the test set, what is the smallest possible difference in AUC, between two rankers giving different AUC?
- 123
1
vote
0
answers
38
views
Optimizing Rank Aggregation of Two Different Methods in Information Retrieval
As the title suggests, I would like to train a rank-aggregating model. My target problem is to rank text2s from a database as best as possible to a given query, <...
- 11
1
vote
0
answers
42
views
Algorithm Comparison for Influencers Ranking
I am working on ranking social influencers on Instagram according to their influential power with the metrics collected below.
Metrics collected:
username
categories (the niche the influencer is in)
...
- 11
1
vote
2
answers
921
views
How to interpret rank output from BM25?
I'm using the FTS5 functionality in Sqlite3 to compute a search rank, which uses BM25. Maybe I'm just not seeing it, but I can't find any description of how to actually interpret BM25's output ranking....
- 121
2
votes
1
answer
453
views
How to determine the "total number of relevant documents" in calculatiion of Recall in Precision and Recall if it's not known? Can it be estimated?
On Wikipedia there is a practical example of calculating Precision and Recall:
When a search engine returns 30 pages, only 20 of which are relevant,
while failing to return 40 additional relevant ...
- 131
3
votes
0
answers
376
views
Why is NDCG high even for wrongly ranked predictions?
The NDCG (Normalized Discounted Cumulative Gain) metric for ranking is defined as DCG/IDCG, where IDCG is the ideal DCG and is said to take values in [0, 1].
However, since the DCG will always be ...
- 131
1
vote
0
answers
17
views
How to ensamble different ranking models?
I have trained two different models, which give a score to each data point. The score of the models it is not necessarily comparable.
The score is used to give a ranking, and the performance is ...
- 111
2
votes
1
answer
267
views
Learning to Rank with Unlabelled Dataset
I have folder of about 60k PDF documents that I would like to learn to rank based on queries to surface the most relevant results. The goal is to surface and rank relevant documents, very much like a ...
- 51
0
votes
0
answers
25
views
Na values in rankings
Having a table of rankings containing many Na values, how should I deal with Na values while calculating the correlation between those rankings?
- 71
0
votes
1
answer
73
views
Calculating statistical ranks between datasets with unpaired observations
The problem is the following:
I have multiple datasets for which I want to calculate a ranking for each. All observations contained in the datasets can be arbitrarily permuted, so they are unpaired, ...
- 41
0
votes
1
answer
91
views
What are the simplest predictive ranking algorithms?
I want to apply a predictive ranking algorithm to a dataset, so I've been reading about Learning to Rank, RankNet and LambdaMART [1][2].
These methods use neural nets or trees as their core framework....
- 1,616
1
vote
1
answer
122
views
Use dummy variables to create a rank variable. R
I have a series of multiple response (dummy) variables describing causes for a canceled visits. A visit can have multiple reasons for the cancelation. My goal is to create a single mutually exclusive ...
- 37
1
vote
0
answers
114
views
Which algorithm to use for a very simple ranking problem?
Currently, I have a dataset with 10 features that results in a ranking of 4 items, i.e. [1,2,3,4], [4,3,1,2], [3,2,4,1] or any $4!$ permutations that can arise from the ranking.
What algorithms are ...
- 11
0
votes
1
answer
240
views
Combine the ranking
I have a table to decide what is the top rank subgroup. My idea is to rank independently into each feature (score, quality) then sum the rank to have the final score (see picture below)
I am not sure ...
- 1
1
vote
1
answer
32
views
Changing behaviour of an ML model
I am trying to create a ranking system for recommending books to an user. Let's suppose we have some subjects of books like 'A', 'B', 'C', 'D' and from the past behaviour, it is observed that the user ...
- 189
6
votes
1
answer
18k
views
precision@k and recall@k
Normally, I am familiar with precision and recall evaluation metrics but as you know recall@k and precision@k are different things and used in ranking evaluations especially recommendation systems.
I ...
- 161
1
vote
0
answers
119
views
MongoDB Groupby Rank [closed]
Im Working With Mongodb And Wanted to do a query using Aggregate fucntion. Query Is ...
- 11
0
votes
1
answer
28
views
The upper range of a collected dataset is most likely accurate, but the rest may suffer biased omissions: How to call this phenomenon?
Background: In collecting a dataset of a specific unit ordered by a numeric variable, it is possible that the upper 'cloud' of the dataset is correct, while the 'tail' seems inaccurate.
I can thus ...
- 123
1
vote
1
answer
46
views
paired t-test shows no difference between median and Wilcoxon test p value shows that there is a difference between median values ? How to interpret?
I have a dataset. I wanted to do paired t test on it. So I carried out normality test and it showed that it does not follow normal distribution. So I used Wilcoxon test in place of paired t test. The ...
- 151
1
vote
1
answer
396
views
Is there a good alternative to XGBoost for learn to rank?
My problem with XGBoost is that when I load the train dataset into the XGBoost DMatrix, there is a memory spike that is unavoidable, and I can't get my dataset loaded into RAM without crashing first.
...
- 65
1
vote
1
answer
67
views
Ranking of feature sets which will be used for binary classification
I have some feature sets say X1 and X2 ...
Each feature set have some variable amount of features and there is no intersection between different feature sets
Say X1 have 100 features and X2 have 500 ...
2
votes
1
answer
599
views
Cosine Similarity but with weighting for vector indexes
I am very new to NLP and although this seems like a basic question I don't know how to search for an answer online.
This is my problem: I have extracted and ranked keywords from 2 text sources:
A ...
- 21
2
votes
1
answer
1k
views
Ranking problem and imbalanced dataset
I know about the problems that imbalanced dataset will cause when we are working on classification problems. And I know the solution for that including undersampling and oversampling.
I have to work ...
- 331
1
vote
1
answer
30
views
What does "freedom" mean in the RankNet paper?
In the RankNet (Learning to Rank using Gradient Descent) paper, paragraph 3.1, it said
given the consistency requirements, how much freedom is there to choose the pairwise probabilities?
What does ...
- 11
1
vote
1
answer
185
views
What can help decrease outliers' influence on non-tree models?
I have a feature with all the values between 0 and 1 except few outliers larger than 1. I am trying to collect all the methods that can help to decrease outliers' influence on non-tree models:
...
1
vote
1
answer
28
views
What is the best way to pick the optimized configuration from this dataset?
I have about 8000 configurations in an excel sheet. each configuration has four scores as seen in the image below. I would like to choose the best solution that has the highest lighting level score, ...
- 113
1
vote
0
answers
42
views
How to use the position factor in known data as a feature in recommendation surfacing?
The problem is recommending stories on a website, just below each story based on how similar the stories are and some historic data based on what recommended stories were clicked or not clicked. So ...
- 335
1
vote
1
answer
38
views
Dataset recommendation, ordinal target variable
Does anyone have any recommendations for datasets with an ordinal target?
I'm currently using a rounded target from the Boston dataset.
...
- 121
1
vote
1
answer
745
views
Using BM25 to rank words
How effective is it to use BM25 to rank words, to be more specific i have a dictionary of words and i want to rank only words in a document that are also in my dictionary. I want to rank all words in ...
- 41
0
votes
1
answer
35
views
How to adjust/smooth a certain number using constants or rules
Hi, I am handling a dataset with a customer purchase history.
The field ord_cnt represents the purchase without coupon usage, and cpn_ord_cnt represents the purchase with coupon usage.
There are two ...
- 3
1
vote
0
answers
227
views
TF-Ranking transform data to ELWC - ExampleListWithContext
I have read all the guides, videos, and everything, but I have no idea how to convert my feature set to an ELWC datasheet format for TF-Rank ListWise problem. There is no description of this structure....
- 71
1
vote
0
answers
21
views
Recommended papers on Deep Ordinal Regression?
Can people please recommend papers on Deep Ordinal Regression?
I'm looking for both the basic papers and the current important state-of-the-art ones - basically, everything that one "must" know in ...
- 111
1
vote
1
answer
349
views
How to explain a stable NDCG@K in extreme multilabel recommender model
I am working in a multilabel recommender project and I try to evaluate it as a ranking problem.
I calculate recall@k and precision@k which both looks quite well. Recall increases and Precision ...
- 3,940
0
votes
1
answer
71
views
Analytic Hierarchy Process - consistency ratio over 50%
I have such data of pair-wise comparison. I calculated consistency ratio for this data of 56%. That's too high for relevant results.
Whether is possible to modify initial data to improve consistency ...
- 153