Questions tagged [ranking]

Ranking is ordering data from highest to lowest or *vice versa.* For questions about *constructing* scores to use in ranking, please use the "valuation" tag, too.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

Computing probabilities in Plackett-Luce model

I am trying to implement a Plackett-Luce model for learning to rank from click data. Specifically, I am following the paper: Doubly-Robust Estimation for Correcting Position-Bias in Click Feedback for ...
user avatar
1 vote
1 answer
17 views

How to determine the "total number of relevant documents" in calculatiion of Recall in Precision and Recall if it's not known? Can it be estimated?

On Wikipedia there is a practical example of calculating Precision and Recall: When a search engine returns 30 pages, only 20 of which are relevant, while failing to return 40 additional relevant ...
user avatar
  • 11
2 votes
0 answers
39 views

Why is NDCG high even for wrongly ranked predictions?

The NDCG (Normalized Discounted Cumulative Gain) metric for ranking is defined as DCG/IDCG, where IDCG is the ideal DCG and is said to take values in [0, 1]. However, since the DCG will always be ...
user avatar
  • 21
1 vote
0 answers
12 views

How to ensamble different ranking models?

I have trained two different models, which give a score to each data point. The score of the models it is not necessarily comparable. The score is used to give a ranking, and the performance is ...
user avatar
2 votes
1 answer
38 views

Learning to Rank with Unlabelled Dataset

I have folder of about 60k PDF documents that I would like to learn to rank based on queries to surface the most relevant results. The goal is to surface and rank relevant documents, very much like a ...
user avatar
  • 51
0 votes
0 answers
17 views

Na values in rankings

Having a table of rankings containing many Na values, how should I deal with Na values while calculating the correlation between those rankings?
user avatar
  • 51
0 votes
0 answers
10 views

Can we apply multi-criteria decision making algorithms in incomplete data?

I am currently working on a project where a multi criteria decision making algorithm is needed in order to evaluate several alternatives for a given goal. After long research, I decided to use the AHP ...
user avatar
  • 1
0 votes
0 answers
17 views

How to match people in preference ranked survey results?

I'm sending out a survey that I want to use to create pairs. For example, each person indicates whether they want to be a mentor, or a mentee. They then stack rank 10 topics that they're interested in ...
user avatar
0 votes
1 answer
15 views

Calculating statistical ranks between datasets with unpaired observations

The problem is the following: I have multiple datasets for which I want to calculate a ranking for each. All observations contained in the datasets can be arbitrarily permuted, so they are unpaired, ...
user avatar
  • 21
0 votes
0 answers
21 views

AHP score and ranking

I am doing a self-study. In the AHP method Final product is weight of each criteria. In my understanding I just simply times weight to the criteria in order to get score and do ranking. Am I correct? ...
user avatar
  • 111
0 votes
0 answers
28 views

Ranking recommendation system

I have problem with construction recommended system. I have a DataFrame with columns of users , items (books) and the order in users read the book - ...
user avatar
  • 1
0 votes
1 answer
27 views

What are the simplest predictive ranking algorithms?

I want to apply a predictive ranking algorithm to a dataset, so I've been reading about Learning to Rank, RankNet and LambdaMART [1][2]. These methods use neural nets or trees as their core framework....
user avatar
  • 1,494
1 vote
1 answer
35 views

Use dummy variables to create a rank variable. R

I have a series of multiple response (dummy) variables describing causes for a canceled visits. A visit can have multiple reasons for the cancelation. My goal is to create a single mutually exclusive ...
user avatar
1 vote
0 answers
29 views

Which algorithm to use for a very simple ranking problem?

Currently, I have a dataset with 10 features that results in a ranking of 4 items, i.e. [1,2,3,4], [4,3,1,2], [3,2,4,1] or any $4!$ permutations that can arise from the ranking. What algorithms are ...
user avatar
  • 11
0 votes
1 answer
27 views

Combine the ranking

I have a table to decide what is the top rank subgroup. My idea is to rank independently into each feature (score, quality) then sum the rank to have the final score (see picture below) I am not sure ...
user avatar
1 vote
1 answer
27 views

Changing behaviour of an ML model

I am trying to create a ranking system for recommending books to an user. Let's suppose we have some subjects of books like 'A', 'B', 'C', 'D' and from the past behaviour, it is observed that the user ...
user avatar
  • 189
3 votes
1 answer
5k views

precision@k and recall@k

Normally, I am familiar with precision and recall evaluation metrics but as you know recall@k and precision@k are different things and used in ranking evaluations especially recommendation systems. I ...
user avatar
  • 131
1 vote
0 answers
82 views

MongoDB Groupby Rank [closed]

Im Working With Mongodb And Wanted to do a query using Aggregate fucntion. Query Is ...
user avatar
  • 11
0 votes
1 answer
24 views

The upper range of a collected dataset is most likely accurate, but the rest may suffer biased omissions: How to call this phenomenon?

Background: In collecting a dataset of a specific unit ordered by a numeric variable, it is possible that the upper 'cloud' of the dataset is correct, while the 'tail' seems inaccurate. I can thus ...
user avatar
  • 123
1 vote
0 answers
29 views

paired t-test shows no difference between median and Wilcoxon test p value shows that there is a difference between median values ? How to interpret?

I have a dataset. I wanted to do paired t test on it. So I carried out normality test and it showed that it does not follow normal distribution. So I used Wilcoxon test in place of paired t test. The ...
user avatar
  • 151
1 vote
1 answer
205 views

Is there a good alternative to XGBoost for learn to rank?

My problem with XGBoost is that when I load the train dataset into the XGBoost DMatrix, there is a memory spike that is unavoidable, and I can't get my dataset loaded into RAM without crashing first. ...
user avatar
1 vote
1 answer
33 views

Ranking of feature sets which will be used for binary classification

I have some feature sets say X1 and X2 ... Each feature set have some variable amount of features and there is no intersection between different feature sets Say X1 have 100 features and X2 have 500 ...
user avatar
2 votes
1 answer
191 views

Cosine Similarity but with weighting for vector indexes

I am very new to NLP and although this seems like a basic question I don't know how to search for an answer online. This is my problem: I have extracted and ranked keywords from 2 text sources: A ...
user avatar
  • 21
1 vote
1 answer
366 views

Ranking problem and imbalanced dataset

I know about the problems that imbalanced dataset will cause when we are working on classification problems. And I know the solution for that including undersampling and oversampling. I have to work ...
user avatar
  • 321
1 vote
0 answers
15 views

What does "freedom" mean in the RankNet paper?

In the RankNet (Learning to Rank using Gradient Descent) paper, paragraph 3.1, it said given the consistency requirements, how much freedom is there to choose the pairwise probabilities? What does ...
user avatar
1 vote
1 answer
115 views

What can help decrease outliers' influence on non-tree models?

I have a feature with all the values between 0 and 1 except few outliers larger than 1. I am trying to collect all the methods that can help to decrease outliers' influence on non-tree models: ...
user avatar
1 vote
1 answer
25 views

What is the best way to pick the optimized configuration from this dataset?

I have about 8000 configurations in an excel sheet. each configuration has four scores as seen in the image below. I would like to choose the best solution that has the highest lighting level score, ...
user avatar
1 vote
0 answers
29 views

How to use the position factor in known data as a feature in recommendation surfacing?

The problem is recommending stories on a website, just below each story based on how similar the stories are and some historic data based on what recommended stories were clicked or not clicked. So ...
user avatar
  • 283
1 vote
1 answer
22 views

Dataset recommendation, ordinal target variable

Does anyone have any recommendations for datasets with an ordinal target? I'm currently using a rounded target from the Boston dataset. ...
user avatar
1 vote
1 answer
173 views

Using BM25 to rank words

How effective is it to use BM25 to rank words, to be more specific i have a dictionary of words and i want to rank only words in a document that are also in my dictionary. I want to rank all words in ...
user avatar
  • 41
0 votes
1 answer
27 views

How to adjust/smooth a certain number using constants or rules

Hi, I am handling a dataset with a customer purchase history. The field ord_cnt represents the purchase without coupon usage, and cpn_ord_cnt represents the purchase with coupon usage. There are two ...
user avatar
1 vote
0 answers
124 views

TF-Ranking transform data to ELWC - ExampleListWithContext

I have read all the guides, videos, and everything, but I have no idea how to convert my feature set to an ELWC datasheet format for TF-Rank ListWise problem. There is no description of this structure....
user avatar
1 vote
0 answers
19 views

Recommended papers on Deep Ordinal Regression?

Can people please recommend papers on Deep Ordinal Regression? I'm looking for both the basic papers and the current important state-of-the-art ones - basically, everything that one "must" know in ...
user avatar
  • 111
1 vote
0 answers
80 views

How to explain a stable NDCG@K in extreme multilabel recommender model

I am working in a multilabel recommender project and I try to evaluate it as a ranking problem. I calculate recall@k and precision@k which both looks quite well. Recall increases and Precision ...
user avatar
  • 3,730
0 votes
1 answer
36 views

Analytic Hierarchy Process - consistency ratio over 50%

I have such data of pair-wise comparison. I calculated consistency ratio for this data of 56%. That's too high for relevant results. Whether is possible to modify initial data to improve consistency ...
user avatar
  • 153
0 votes
1 answer
41 views

Compare ratings of players over different leagues

I want to compare ratings of players from different leagues and predict rating of player in a league he/she didnt participate in. Rating of a player is estimated within a league where he was playing. ...
user avatar
1 vote
0 answers
13 views

Solving Feature Distribution variance between Training and Prediction for Ranking models

I am building a linear regression model to improve ranking of documents. And trying to identify problems due to which model performance estimates don't match actual impact One major problem is ...
user avatar
  • 111
6 votes
2 answers
2k views

Preparing for a Machine Learning Design Interview

I am not sure if this is a relevant post here but: I made it to the final round for the Machine Learing Engineer position at Facebook. The final round interview is virtual (thanks to Corona) and will ...
user avatar
  • 237
1 vote
1 answer
63 views

Drastic drop in Somers' D ? Why?

I came across to find the correlation between the ratings assigned by two coaches to a same group of 40 players. I have tabulated the results as below: The Somers' D is 50%. However, for the case ...
user avatar
9 votes
1 answer
2k views

What is query id ("qid") in XGBoost

In XGBoost documentation it's said that for ranking applications we can specify query group ID's qid in the training dataset as in the following snippet: ...
user avatar
0 votes
1 answer
45 views

What are some significance tests to rank features(multiple) before training the data

I have 8 features for a classification problem. The target value tells if there was an anomaly or not. I want to run some significance tests to rank each feature, as being a distinctive feature of ...
user avatar
1 vote
1 answer
289 views

Differentiable loss function for ranking problem in regression model

In regression problem, we may need a loss function to measure the relative ranking accuracy between targets $y$ and predicted values $y_{pred}$. Abviously, the simple MSE does not consider such ...
user avatar
1 vote
1 answer
44 views

what metrics to evaluate rank order results?

I have searched on stackexchange and found a couple of topics like this and this but they are not quite relevant to my problem (or at least I don't know how to make them relevant to my problem). ...
user avatar
2 votes
1 answer
38 views

Recommender system that matches similar customers with similar highly rated products?

I have a dataset of 1,000 customers that bought 20 distinct phones and rated them 1-5. I have several demographic attributes for these customers (gender, age). My website offers 100 distinct devices, ...
user avatar
  • 171
5 votes
1 answer
1k views

How to use ndcg metric for binary relevance

I am working on a ranking problem to predict the right single document based on the user query and use the NDCG metric to measure the model. Given the details : Queries ( Q ), Result Document ( D ),...
user avatar
2 votes
2 answers
271 views

How to calculate precision at K and NDCG for ranking algorithms

I am ranking a filtered item list as per user's metadata and historical behaviour. Now how to calculate metrices like precision at K? One approach could be - Divide historical data in training and ...
user avatar
  • 71
3 votes
1 answer
106 views

Is it Possible to Use Machine Learning for Ranking Alternatives?

Right now, I’m working on road and street safety analysis. I have a dataset of dangerous points in four regions of a city. Some of the available variables are road lighting status, ITS, latitude, ...
user avatar
  • 31
1 vote
1 answer
24 views

Calculate a Rank function from Regression features

I am using 3 features (x1, x2, x3) for regression. Some of my features are continuous some are categorical. My dependent variable are lets number of bookings. And I can predict the number of bookings....
user avatar
4 votes
1 answer
62 views

How is "relevance" defined in information retrieval outside the context of systems with user feedback?

I've seen information retrieval systems that return some results from a query, and then the user rates these results as either "relevant" or "not relevant". What can you do if you do not have user ...
user avatar
  • 53
2 votes
1 answer
791 views

How to extract keywords from a list of URLs?

I have a bunch of URLs in a text file like- ...
user avatar
  • 121