Questions tagged [ranking]

Ranking is ordering data from highest to lowest or *vice versa.* For questions about *constructing* scores to use in ranking, please use the "valuation" tag, too.

Filter by
Sorted by
Tagged with
1
vote
1answer
21 views

Preparing for a Machine Learning Design Interview

I am not sure if this is a relevant post here but: I made it to the final round for the Machine Learing Engineer position at Facebook. The final round interview is virtual (thanks to Corona) and will ...
0
votes
0answers
16 views

Drastic drop in Somers' D ? Why?

I came across to find the correlation between the ratings assigned by two coaches to a same group of 40 players. I have tabulated the results as below: The Somers' D is 50%. However, for the case ...
6
votes
1answer
125 views

What is query id (“qid”) in XGBoost

In XGBoost documentation it's said that for ranking applications we can specify query group ID's qid in the training dataset as in the following snippet: ...
0
votes
1answer
21 views

What are some significance tests to rank features(multiple) before training the data

I have 8 features for a classification problem. The target value tells if there was an anomaly or not. I want to run some significance tests to rank each feature, as being a distinctive feature of ...
1
vote
0answers
14 views

Differentiable loss function for ranking problem in regression model

In regression problem, we may need a loss function to measure the relative ranking accuracy between targets $y$ and predicted values $y_{pred}$. Abviously, the simple MSE does not consider such ...
1
vote
1answer
24 views

what metrics to evaluate rank order results?

I have searched on stackexchange and found a couple of topics like this and this but they are not quite relevant to my problem (or at least I don't know how to make them relevant to my problem). ...
2
votes
1answer
17 views

Recommender system that matches similar customers with similar highly rated products?

I have a dataset of 1,000 customers that bought 20 distinct phones and rated them 1-5. I have several demographic attributes for these customers (gender, age). My website offers 100 distinct devices, ...
2
votes
0answers
38 views

How to use ndcg metric for binary relevance

I am working on a ranking problem to predict the right single document based on the user query and use the NDCG metric to measure the model. Given the details : Queries ( Q ), Result Document ( D ),...
2
votes
2answers
75 views

How to calculate precision at K and NDCG for ranking algorithms

I am ranking a filtered item list as per user's metadata and historical behaviour. Now how to calculate metrices like precision at K? One approach could be - Divide historical data in training and ...
1
vote
0answers
51 views

Is it Possible to Use Machine Learning for Ranking Alternatives?

Right now, I’m working on road and street safety analysis. I have a dataset of dangerous points in four regions of a city. Some of the available variables are road lighting status, ITS, latitude, ...
1
vote
0answers
21 views

Calculate a Rank function from Regression features

I am using 3 features (x1, x2, x3) for regression. Some of my features are continuous some are categorical. My dependent variable are lets number of bookings. And I can predict the number of bookings....
4
votes
1answer
37 views

How is “relevance” defined in information retrieval outside the context of systems with user feedback?

I've seen information retrieval systems that return some results from a query, and then the user rates these results as either "relevant" or "not relevant". What can you do if you do not have user ...
2
votes
1answer
40 views

How to extract keywords from a list of URLs?

I have a bunch of URLs in a text file like- ...
0
votes
0answers
31 views

nDCG - choose relevance scores

I am evaluating a recommender system using nDCG. The recommender system predicts similar movies for a given movie. I want to evaluate predicted similarity rankings by comparing them to a ground truth ...
0
votes
0answers
7 views

Alternative ranking evaluation metrics for biased data

What are some alternative evaluation metrics for ranking problems, that could help when evaluation is done on heavily biased data? Example - if we sort items by their price and want to evaluate ...
0
votes
0answers
10 views

Will re-ranking my results slow down my search?

I'm looking into incorporating ranking in my website by using a machine learning model. But to rank my documents I have to calculate all my features (like tf-idf, word count, etc..) on every document ...
1
vote
0answers
24 views

Alternative for ranking of a response

Is there a regression approach for predicting the rankings (ordering) as an endogenous variable? I would like to fit a regression to a dataset where the response is the order of a variable: 1st, 2nd, ...
0
votes
0answers
12 views

How to normalize old accumulated ratings

I am playing around with data-set which contains movies and their ratings by various users. I am trying to rank these movies based on their user ratings. However, as obvious, many of the old movies (...
0
votes
0answers
31 views

Rank by sorting the output probabilities of a binary classifier or by “learning to rank”?

I have a binary classification problem (5M points, only around 0.01% is positive). As output I want to give a list of data points sorted by "positiveness", and then only the top ...
0
votes
1answer
73 views

Compare Rows Within a Group and Rank Best to Worst

I'm interested in an approach for comparing rows "within group" to produce a ranking from best to worst based on the performance relative to other rows within a group. For example, if given this ...
0
votes
0answers
19 views

Any there databases with native support for applying a NN model to produce search rankings?

The Situation: I have a simple neural net with an input vector that consists of the euclidean distance between the attributes of two cars. (For example the attributes would include wheel size, car ...
0
votes
0answers
16 views

How to adjust ranking function

I have a ranking function that ranks particular markets in a way where their features are highly desirable. However, I can look at the results of the rank and see that it's good, but how can I make it ...
0
votes
0answers
23 views

PageRank computation with dumping (scaling) factor

it is not clear to me how to calculate two iterations of PageRank computation on the following network with dumping (scaling) factor s = 1.how can they calculate it correctly?
2
votes
2answers
48 views

Calculate a ranking function from classification features

I am using 3 features (x1, x2, x3) for binary classification. All my feature values are in 0 to 1 range (unit range). I obtained how important each feature was in ...
1
vote
0answers
19 views

Personalised search ranking for hotels

I've built hotel embeddings which gives very satisfactory results in returning similar hotels for each hotel. Now the problem I'm trying to solve is to rank the hotels in order of relevancy to the ...
2
votes
1answer
206 views

Improve results using user input

I've developed a tool that retrieve the closest expressions from a database based on what the user typed. (using word embedding - a comparison is made between each expression from the database and the ...
1
vote
0answers
89 views

What does the ratio of precision/recall to their ideal values mean, and why are they equal?

When evaluating rankings, Normalized Discounted Cumulative Gain (NDCG) normalizes the score to a [0,1] range by dividing by the ideal score. What happens if we take the same idea to Precision (Pre) ...
1
vote
0answers
355 views

Xgboost rank:ndcg learning per group or for all dataset

I'm trying to implement xgboost with an objective of rank:ndcg I want the target to be between 0-3. In my data for most of the groups, there is only 1 event per ...
2
votes
1answer
79 views

What is the difference between nDCG and rank correlation methods?

When do we use one or the other? My use case: I want to evaluate a linear space to see how good retrieval results are. I have a set of data X (m x n) and some weights W (m x 1). I want to measure ...
2
votes
0answers
71 views

Feature selection with information gain (KL divergence) and mutual information yields different results

I'm comparing different techniques for feature selection / feature ranking. Two of the techniques under scrutiny are the mutual information (MI) and the information gain (IG) as used in decision trees,...
1
vote
0answers
98 views

Constructing a graph where any node can have the highest PageRank among all nodes

I'm trying to solve q5.1.4 in mmds chapter5, however, I'm not sure how I can even start. The question is: Construct, for any integer n, a Web such that, depending on β, any of the n nodes can have ...
2
votes
1answer
73 views

Multilabel Classification With Ranking

I have a dataset as below: ...
0
votes
1answer
30 views

Best way to narrow down a list and rank based on attributes?

I have a mortgage/credit data set that contains a list of customers (600k rows) and has a 100 columns inclusive of the customer's general info (address, city, zipcode, etc), income, fico scores, ...
0
votes
1answer
15 views

Predicting sequences newbie question [closed]

I have a ranked list of rows of 100 lines of data 1- 8 4 0 5 9 3 2- 0 3 3 5 3 2 3- 0 0 2 4 0 2 .. 100- 0 2 3 2 2 0 Is it possible to predict a) when given a new ...
1
vote
0answers
183 views

NDCG score is greater than 1

I'm solving a problem of ranking classes for each unique id based on the utilization quantity. I have 6 unique classes in the training and test data. My neural net mode predicts the utilization ...
2
votes
0answers
30 views

Reduce number of vectors in dataset to achieve the “same average dimensions result”?

I have many tests (rows), each with a large set of 3D vectors (features/cols). Each vector complies: Xn + Yn + Zn = 1 Simply averaging all components ...
2
votes
1answer
191 views

Two definitions of DCG measure

I wanted to check the definition of Discounted Cumulative Gain (DCG) measure in the original paper Jarvelin and it seems it differs from the one given in the later literature Wang. Originally, for $n$ ...
1
vote
0answers
103 views

Statistical Significance in Pairwise Ranking Algorithm

Can anyone recommend an algorithm/toolkit to rank items that have been rated in a hot-or-not style that gives statistical significance? For example, out of a set of N images, two images are shown to ...
1
vote
0answers
179 views

Statistical Significance for Pairwise Ranking Algorithm

I am using this library: https://github.com/lucasmaystre/choix/blob/master/notebooks/intro-pairwise.ipynb to rank items that have been rated in a hot-or-not style. For example, two images are shown ...
3
votes
1answer
74 views

Sparse IR with user feedback

I'm considering a problem framing within an information retrieval context. I have a sequence of documents that feature different attributes. In the web context, these would be webpages. One ...
3
votes
2answers
115 views

Learning to Rank Application

If there's a website/app that sells products and my job is to determine the order/ranking in which the products should be displayed. For example : I click on restaurants and a list of restaurants ...
1
vote
0answers
16 views

business ranking based on transaction data

I have some customer (say 10,000 customers) transaction data of many stores, what would be a good approach to rank these stores? The transaction data is of entire year, and the transaction data ...
3
votes
1answer
137 views

Methods for ensembling ranked lists?

I was wondering if there's a good way to use ensembling when I have two or more algoritims producing ranked lists. That is, suppose I have the following datasets consisting of ordered lists (higher ...
4
votes
1answer
700 views

Can feature importance change a lot between models?

I have a random forest classifier and Multinomial Naive Bayes. For feature importance, I used gini index for random forest and for Multinomial Naive Bayes I used the coefficients of each feature. Then ...
2
votes
1answer
104 views

Peformance evaluation of ranking algorithms

I have three questions: How can we assess (or measure) the performance of the ranking algorithms? Are there any specific measures, or performance metrics, for this? More specifically, how can we ...
1
vote
2answers
982 views

XGBoost ranking file format

The xgboost package has two files that must be used for ranking: train.txt with the data ...
1
vote
1answer
4k views

Cosine similarity between query and document confusion

I am going through the Manning book for Information retrieval. Currently I am at the part about cosine similarity. One thing is not clear for me. Let's say that I have the tf idf vectors for the ...
1
vote
0answers
105 views

Information measure of rank changes?

I'm trying to compute the information content of a function that reranks a list. Perhaps more precisely, I'm trying to compare the information content of different arguments to that function. ...
0
votes
1answer
71 views

Model with rank as the dependent variable?

I have a dataset with n records. The input features are f1, f2, f3 ... fm, and the dependent variable (output) is just the rank (not any type of scores). What type of model should I build so that I ...
1
vote
1answer
141 views

Learning to rank: construct absolute ranking using pair-wise ranking approach

I am learning about the "pairwise approach" for learning to rank. As far as I understood, the training output is a partial ranking function $r$ that: given given some query $q$ and two document $d_i$ ...