Questions tagged [search]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
13 views

Matching resumes with jobs in a NoSQL database

I have a series of user resumes and would like to find the best matching jobs per resume, the task seems simple at first except I have a nosql db holding the jobs (1m+), my initial idea was to add a ...
0
votes
0answers
7 views

Will re-ranking my results slow down my search?

I'm looking into incorporating ranking in my website by using a machine learning model. But to rank my documents I have to calculate all my features (like tf-idf, word count, etc..) on every document ...
2
votes
1answer
29 views

How to efficiently iterate a supervised model over the Cartesian product of very large number of records?

The problem: Two large databases, with ~1M records each, "old customer data" and "new customer data". The data came from different sources and was ingested at different times, so there are many ...
0
votes
0answers
7 views

How to vectorize unigrams character to use LSH functions?

I would like to implement fuzzy search based on Bloom Filter and LSH hashing. The problem is that: I have found almost ready package to get ngrams from words, now I don't know how to generate vector ...
4
votes
1answer
73 views

How can one quickly look up people from a large database?

Vocabulary Face detection: Finding all faces in an image. Face representation: The simplest way to represent a face is as an image (pixels / color values). This is not very space efficient and likely ...
1
vote
1answer
176 views

How to dual encode two sentences to show similarity score

I've been trying to grasp the concept of Google's semantic experiences. By using it, I'm planning to implement a semantic query tool. With universal sentence encoder I can first pre-encode all ...
0
votes
0answers
24 views

Efficient Similarity Indexing and Searching in High Dimensions

I found this recent document, containing simple algorithm that properly indexes high dimensional vectors for binary search later. Algorithm basically uses random partition forest that randomly ...
0
votes
1answer
18 views

Is Elastic Search recommended if attribute getting search is not a huge text document?

We are currently developing a system with MEAN stack with Mongodb at backend. We have employees name, and Ids in our system and our client wants to get pretty good (Read: Google Like) search in our ...
2
votes
1answer
225 views

how to improve searching index in dataframe

Given a pandas dataframe with a timestamp index, sorted. I have a label and I need to find the closest index to that label. Also, I need to find a smaller timestamp, so the search should be computed ...
2
votes
0answers
87 views

CNN combined with a competitive search algorithm [closed]

I'm reading some papers about Deep Neural Networks applied for board games, like for Go with AlphaGo, AlphaGo Zero and some other games, like Othello and Chess. Most of the works are using CNN's as a ...
1
vote
0answers
26 views

Search Query Sample Size Determination for validation set

While designing a search system, which searches in N identifiable categories, how many search queries does one need in each category to validate the target metric (DCG) scores accurately (balanced ...
4
votes
1answer
309 views

How to deal with position bias in search?

In search, position of the search result affects the click-through rate a great deal. How do people usually deal with this ? In practice how to remove such bias to create unbiased training data for ...
-3
votes
1answer
33 views

Algorithms/services to know an “iPhone case” is not an “iPhone”, in the context of complex item descriptions? [closed]

We are trying to implement a highly accurate search, based on user-entered search terms, into a large product database. For example, if the user searches for "iPhone", then one of these is ...
-2
votes
1answer
58 views

What is ElasticSearch and how can it be used? [closed]

I am really quite new to this whole world but I keep hearing lots of talk about ElasticSearch and Kibana. I have been to their website but would just like a really simple, plain English explanation ...
1
vote
3answers
166 views

Grid Search and High Variance

I am currently trying to optimise some parameters on my model (15000 samples). What I am finding is a relatively large variance in the loss function 2%-10% which makes it hard to identify which ...
3
votes
1answer
189 views

Best method for similarity searching on 10,000 data points with 8,000 features each in Python?

As mentioned in the title I am attempting to search through 10,000 vectors with 8000 features each, all in Python. Currently I have the vectors saved in their own directories as pickled numpy arrays. ...
5
votes
2answers
466 views

Why do popular search engines not follow the usual AND, OR logic for queries?

I am teaching myself Information Retrieval from Christopher Manning's book (PDF link: http://nlp.stanford.edu/IR-book/pdf/01bool.pdf). I tried Exercise 1.13: "Try using the Boolean search features on ...
0
votes
1answer
37 views

How to search collection of podcasts (.mp3 files)?

I have a collection of podcasts, in particular the "Talkingmachines" Podcasts ("Human conversation about machine learning"), see http://www.thetalkingmachines.com/blog/ There are 22 episodes, each 1 ...
4
votes
1answer
431 views

Why keep vocabulary and posting list separate in a search engine

I am taking a class in information retrieval. We learned that the index of a search engine has (possibly among other things): A vocabulary mapping terms to their statistics (frequency, type, ...) and ...
0
votes
2answers
70 views

Gathering the number of Google results from a large amount of searches.

I am trying to build a simple dataset using Google, mainly because it seems like the best option for what I want. I want to measure fame for a large group of scientists. The quick method is to ...
11
votes
2answers
13k views

How fit pairwise ranking models in xgBoost?

As far as I know, to train learning to rank models, you need to have three things in the dataset: label or relevance group or query id feature vector For example, the Microsoft Learning to Rank ...
7
votes
5answers
809 views

Best way to search for a similar document given the ngram

I have a database of about 200 documents who's ngrams I have extracted. I want to find the document in my database that is most similar to a query document. In otherwords, I want to find the document ...
0
votes
1answer
49 views

Looking for a 'CITY, STATE' within a body of text (from a CITY-STATE database)

I'm looking for an optimal way to search a large body of text for a combination of words that resemble any CITY, STATE combination I have in a separate CITY-STATE database. My only idea would be to ...
2
votes
0answers
627 views

Weighted k nearest neighbor search

I've searched quite a bit and haven't landed on any useful results. The problem statement is: Given a set of vectors, I wish to find its approximate k-nearest neighbors. The caveat here is that each ...
3
votes
1answer
244 views

Ranking Bias in Learning to Rank

Users tend to click on results ranked highly by search engines much more often than those ranked lower. How do you train a search engine using click data / search logs without this bias? I.e. you don'...
4
votes
1answer
4k views

Where is the cost parameter C in the RBF kernel in SVM?

RBF kernel using SVM depends on two parameters C and gamma. If the equation of the kernel RBF as the following: $K(X,X')= \exp(\gamma||X-X'||^2)$ In the equation I can see where can I use gamma, but ...
1
vote
0answers
29 views

large database operation. check for relatedness between entities

I have one small list of entities, such as: Russia Vladimir Moscow Then I have a massive database of JSON indices. For each entry there are multiple alpha-...
0
votes
1answer
35 views

What approaches are there to not display a search result that a user has no permission to see? [closed]

Question What approaches are there to not display a search result that a user is not supposed to see? Example Suppose we have the following situation. High-level view In an enterprise, there are ...
3
votes
2answers
337 views

Algorithm for multiple extended string matching

I need to implement an algorithm for multiple extended string matching in text. Algorithms to match regular expression would be perhaps too slow. Extended means the presence of wildcards (any number ...
4
votes
2answers
192 views

Where did this NY Times op-ed get his Google Search data?

I hope this is a question appropriate for SO. The article in question: http://www.nytimes.com/2015/01/25/opinion/sunday/seth-stephens-davidowitz-searching-for-sex.html As far as I can tell, the only ...
6
votes
3answers
123 views

Can we quantify how position within search results is related to click-through probability?

Suppose, for example, that the first search result on a page of Google search results is swapped with the second result. How much would this change the click-through probabilities of the two results? ...
1
vote
0answers
38 views

General programs/libraries for studying user search behavior?

Are there any general open-source programs or libraries (e.g., a Python library) for analyzing user search behavior? By "search behavior", I mean a user's interaction with a search engine, such as ...
3
votes
1answer
212 views

Query similarity: how much data is used in practice?

I recently read Similarity Measures for Short Segments of Text (Metzler et al.). It describes basic methods for measuring query similarity, and in the paper, the data consists of queries and their ...
4
votes
3answers
412 views

When is there enough data for generalization?

Are there any general rules that one can use to infer what can be learned/generalized from a particular data set? Suppose the dataset was taken from a sample of people. Can these rules be stated as ...
6
votes
1answer
92 views

How can we effectively measure the impact of our data decisions

Apologies if this is very broad question, what I would like to know is how effective is A/B testing (or other methods) of effectively measuring the effects of a design decision. For instance we can ...
8
votes
2answers
177 views

What are some standard ways of computing the distance between individual search queries?

I made a similar question asking about distance between "documents" (Wikipedia articles, news stories, etc.). I made this a separate question because search queries are considerably smaller than ...
5
votes
4answers
494 views

How does Google categorize results from its image search?

While doing a Google image search, the page displays some figured out categories for the images of the topic being searched for. I'm interested in learning how this works, and how it chooses and ...
11
votes
3answers
404 views

How does a query into a huge database return with negligible latency?

For example, when searching something in Google, results return nigh-instantly. I understand that Google sorts and indexes pages with algorithms etc., but I imagine it infeasible for the results of ...