Questions tagged [ranking]
Ranking is ordering data from highest to lowest or *vice versa.* For questions about *constructing* scores to use in ranking, please use the "valuation" tag, too.
121
questions
0
votes
1
answer
81
views
Which python libraries do you recommend for label ranking?
I'm currently looking for python libraries that offer models for the label ranking problem. So provided with a context x and a set of Labels Y, the model should output a ranking of those labels.
I'm ...
- 11
0
votes
0
answers
14
views
Ranking risky routes
i'm looking to get the top 10 rank of the most dangerous routes. I have a routes table where each row is a route and it has features such as
avg daily traffic for past three years
No. of times where ...
- 1
0
votes
0
answers
9
views
Search recall optimization - what appropriate loss function to use?
I am studying machine learning and wanted to work on a project of my own so that I have better chances after graduating college. I'm studying the application of ML to improve searches using a toy ...
- 167
0
votes
1
answer
25
views
Using text embeddings directly to compute similarity vs using them as features for a model that predicts similariy
Say you have a problem where you have a query and a set of result documents and you want to rank the result documents according to the query. Say also you have embeddings for the query and for the ...
- 183
0
votes
0
answers
15
views
Is it possible to train an ML model that predicts two different types of variables (numeric, categorial)? Is learning a ranking function possible?
Problem setup:
We are given a set of training instances and an algorithm C, composed of two randomized algorithms, A and B. Algorithm A has no hyperparameters. In contrast, B has only one ...
0
votes
0
answers
37
views
What are machine learning algorithms which rank films?
Broadly speaking, there are two approaches to producing lists of the best films: human vs machine curation.
Human curation involves someone, a critic, making decisions and/or creating a canon, based ...
- 141
0
votes
0
answers
15
views
how to treat tied condition in ndcg calculation?
I am trying to calculate ndcg manually, and not sure how to treat tied condition.
For example, item A, B, C, D, the actual rating is 1, 0, 0,0. Should irank = 1,2,3,4. or 1,2,2,2, or 1,3,3,3?
- 1
0
votes
1
answer
11
views
How to handle using input feature (clicks) when it is used in target too?
I am trying to create a ranking model, where I am thinking about creating ground truth based on clicks by user. But at same time past clicks made by users seems like a vital input feature too. Any ...
- 1
0
votes
0
answers
16
views
Supervised ranking algorithms
I'm working on a problem statement which involves ranking some short-lived items in an order such that the items expected to sell the most in the next n days are ranked on top - basically ranking ...
- 101
0
votes
0
answers
14
views
Policy Gradient training log-derivative un-normalized vs normalized objective
I am implementing a policy gradient training objective for optimizing ranking metrics in a learning-to-rank setting. For a given query $q$, a set of documents $D_q$ (retrieved from a first-stage ...
- 3,775
0
votes
1
answer
15
views
Unsupervised ranking of samples
Say I have a dataset of n samples. I want to maximize every feature’s value. I’m not sure if feature 1 is more important than feature 2, etc. Are there any methods of ranking my samples out there? If ...
- 1
0
votes
0
answers
367
views
Can BM25 be used as an embedding algorithm?
I'v studied about BM25 algorithm. Untill now, I couldn't find an implementation of BM25 to give me an embedding of a text like TfidfTransformer and ...
0
votes
0
answers
8
views
What kind of model would I use to analyze a respondent's probablity of choosing one choice from a finite list of choices?
I have a data set of survey responses (~1500), and I've run chi-square tests for independence and found that certain features are influential on responses to a specific question.
But now I want to ...
0
votes
0
answers
11
views
Learning to Rank Suitability
Say there is an intermediary company that offers different kinds of loans from a number of banks. My job is to determine the order/ranking in which the bids should be displayed to the customer.
For ...
- 101
4
votes
0
answers
224
views
How Does the Reward Model in ChatGPT Calculate Losses?
Reading the InstructGPT paper(which seems to be what ChatGPT was built off of), I found this equation for the reward function.
However, I'm struggling to understand how this equation is used to ...
- 141
0
votes
0
answers
42
views
How to rank terms with bm25 and bo1 pipeline
In pyTerrier I have list of single terms. For example (I choose those tokens to be as relevant as possible and as irrelevant as possible to enhance the effect):
...
- 101
0
votes
0
answers
56
views
How is the relevance score computed for a (pairwise) Lambda Rank model?
I am looking into the Lambda Rank model and got a bit confused, would really appreciate some inputs from experts:
The pairwise Lambda Rank model incorporates the nDCG into its loss function. So this ...
- 2,735
0
votes
0
answers
8
views
Error with LambdaMart capturing the wrong qid
Can you guys help me with my error? Im trying to rank sentences using lambdarank. I want to rank my "clean_msg" and my "y" value is the one providing by pagetext. My data: enter ...
0
votes
0
answers
19
views
Check relationship between ordinal and categorical variable with four categories
how can we check relationship between an ordinal and categorical variable with 4 categories?
I have a variable with satisfaction score from 1-5, and other variable is distance from home like 1) <...
1
vote
1
answer
193
views
Combining multiple ranked lists
Suppose I'm given two ranked lists, A and B, with each item in the lists being associated with a score:
...
1
vote
0
answers
24
views
How do I train an ML model that can rank pairs?
Consider that I have a dataset consisting of a list of pairs of the titles of blogposts of the form ...
- 111
2
votes
0
answers
77
views
Smallest possible difference between AUC of two ranker [closed]
If there are 10 positive examples, and 90 negative examples in the test set, what is the smallest possible difference in AUC, between two rankers giving different AUC?
- 123
1
vote
0
answers
33
views
Optimizing Rank Aggregation of Two Different Methods in Information Retrieval
As the title suggests, I would like to train a rank-aggregating model. My target problem is to rank text2s from a database as best as possible to a given query, <...
- 11
1
vote
0
answers
40
views
Algorithm Comparison for Influencers Ranking
I am working on ranking social influencers on Instagram according to their influential power with the metrics collected below.
Metrics collected:
username
categories (the niche the influencer is in)
...
- 11
1
vote
2
answers
340
views
How to interpret rank output from BM25?
I'm using the FTS5 functionality in Sqlite3 to compute a search rank, which uses BM25. Maybe I'm just not seeing it, but I can't find any description of how to actually interpret BM25's output ranking....
- 121
2
votes
1
answer
238
views
How to determine the "total number of relevant documents" in calculatiion of Recall in Precision and Recall if it's not known? Can it be estimated?
On Wikipedia there is a practical example of calculating Precision and Recall:
When a search engine returns 30 pages, only 20 of which are relevant,
while failing to return 40 additional relevant ...
- 131
3
votes
0
answers
283
views
Why is NDCG high even for wrongly ranked predictions?
The NDCG (Normalized Discounted Cumulative Gain) metric for ranking is defined as DCG/IDCG, where IDCG is the ideal DCG and is said to take values in [0, 1].
However, since the DCG will always be ...
- 131
1
vote
0
answers
16
views
How to ensamble different ranking models?
I have trained two different models, which give a score to each data point. The score of the models it is not necessarily comparable.
The score is used to give a ranking, and the performance is ...
- 111
2
votes
1
answer
165
views
Learning to Rank with Unlabelled Dataset
I have folder of about 60k PDF documents that I would like to learn to rank based on queries to surface the most relevant results. The goal is to surface and rank relevant documents, very much like a ...
- 51
0
votes
0
answers
21
views
Na values in rankings
Having a table of rankings containing many Na values, how should I deal with Na values while calculating the correlation between those rankings?
- 51
0
votes
1
answer
30
views
Calculating statistical ranks between datasets with unpaired observations
The problem is the following:
I have multiple datasets for which I want to calculate a ranking for each. All observations contained in the datasets can be arbitrarily permuted, so they are unpaired, ...
- 41
0
votes
1
answer
58
views
What are the simplest predictive ranking algorithms?
I want to apply a predictive ranking algorithm to a dataset, so I've been reading about Learning to Rank, RankNet and LambdaMART [1][2].
These methods use neural nets or trees as their core framework....
- 1,604
1
vote
1
answer
91
views
Use dummy variables to create a rank variable. R
I have a series of multiple response (dummy) variables describing causes for a canceled visits. A visit can have multiple reasons for the cancelation. My goal is to create a single mutually exclusive ...
- 37
1
vote
0
answers
67
views
Which algorithm to use for a very simple ranking problem?
Currently, I have a dataset with 10 features that results in a ranking of 4 items, i.e. [1,2,3,4], [4,3,1,2], [3,2,4,1] or any $4!$ permutations that can arise from the ranking.
What algorithms are ...
- 11
0
votes
1
answer
179
views
Combine the ranking
I have a table to decide what is the top rank subgroup. My idea is to rank independently into each feature (score, quality) then sum the rank to have the final score (see picture below)
I am not sure ...
- 1
1
vote
1
answer
31
views
Changing behaviour of an ML model
I am trying to create a ranking system for recommending books to an user. Let's suppose we have some subjects of books like 'A', 'B', 'C', 'D' and from the past behaviour, it is observed that the user ...
- 189
6
votes
1
answer
15k
views
precision@k and recall@k
Normally, I am familiar with precision and recall evaluation metrics but as you know recall@k and precision@k are different things and used in ranking evaluations especially recommendation systems.
I ...
- 161
1
vote
0
answers
115
views
MongoDB Groupby Rank [closed]
Im Working With Mongodb And Wanted to do a query using Aggregate fucntion. Query Is ...
- 11
0
votes
1
answer
27
views
The upper range of a collected dataset is most likely accurate, but the rest may suffer biased omissions: How to call this phenomenon?
Background: In collecting a dataset of a specific unit ordered by a numeric variable, it is possible that the upper 'cloud' of the dataset is correct, while the 'tail' seems inaccurate.
I can thus ...
- 123
1
vote
0
answers
37
views
paired t-test shows no difference between median and Wilcoxon test p value shows that there is a difference between median values ? How to interpret?
I have a dataset. I wanted to do paired t test on it. So I carried out normality test and it showed that it does not follow normal distribution. So I used Wilcoxon test in place of paired t test. The ...
- 151
1
vote
1
answer
356
views
Is there a good alternative to XGBoost for learn to rank?
My problem with XGBoost is that when I load the train dataset into the XGBoost DMatrix, there is a memory spike that is unavoidable, and I can't get my dataset loaded into RAM without crashing first.
...
- 55
1
vote
1
answer
52
views
Ranking of feature sets which will be used for binary classification
I have some feature sets say X1 and X2 ...
Each feature set have some variable amount of features and there is no intersection between different feature sets
Say X1 have 100 features and X2 have 500 ...
2
votes
1
answer
393
views
Cosine Similarity but with weighting for vector indexes
I am very new to NLP and although this seems like a basic question I don't know how to search for an answer online.
This is my problem: I have extracted and ranked keywords from 2 text sources:
A ...
- 21
2
votes
1
answer
750
views
Ranking problem and imbalanced dataset
I know about the problems that imbalanced dataset will cause when we are working on classification problems. And I know the solution for that including undersampling and oversampling.
I have to work ...
- 331
1
vote
1
answer
28
views
What does "freedom" mean in the RankNet paper?
In the RankNet (Learning to Rank using Gradient Descent) paper, paragraph 3.1, it said
given the consistency requirements, how much freedom is there to choose the pairwise probabilities?
What does ...
- 11
1
vote
1
answer
166
views
What can help decrease outliers' influence on non-tree models?
I have a feature with all the values between 0 and 1 except few outliers larger than 1. I am trying to collect all the methods that can help to decrease outliers' influence on non-tree models:
...
1
vote
1
answer
27
views
What is the best way to pick the optimized configuration from this dataset?
I have about 8000 configurations in an excel sheet. each configuration has four scores as seen in the image below. I would like to choose the best solution that has the highest lighting level score, ...
- 113
1
vote
0
answers
39
views
How to use the position factor in known data as a feature in recommendation surfacing?
The problem is recommending stories on a website, just below each story based on how similar the stories are and some historic data based on what recommended stories were clicked or not clicked. So ...
- 315
1
vote
1
answer
35
views
Dataset recommendation, ordinal target variable
Does anyone have any recommendations for datasets with an ordinal target?
I'm currently using a rounded target from the Boston dataset.
...
- 121
1
vote
1
answer
542
views
Using BM25 to rank words
How effective is it to use BM25 to rank words, to be more specific i have a dictionary of words and i want to rank only words in a document that are also in my dictionary. I want to rank all words in ...
- 41