My question here is in regards to best practices and current methods for selecting search models on the fly based on a users query.

Lets say I have four searching models, each optimized for their respective types:

  • Model A: Embedding-based, used for sentence queries about scientific topics
  • Model B: Embedding-based, used for sentence queries about general news topics
  • Model C: TF*IDF-based, used for keyword queries about scientific topics
  • Model D: TF*IDF-based, used for keyword queries about general news topics

When users enter a query such as:

  • Query: "vaccine science"
  • Query: "what caused the stock market to change today"

...what are the best ways to determine the model a search engine should use? Are there any design patterns I can use as a reference, or, is this simply another model that I would need to train?

I tried to google terms like "models that select other models", or "models to determine which models to use", but I have not had much luck there.

  • $\begingroup$ it looks like a good case for training a classification model to determine which of the 4 groups the query belongs to. $\endgroup$
    – Erwan
    Jan 6 at 23:43
  • $\begingroup$ We thought of this, but two issues came up: First, we won't always have a pre-defined number of groups -- these will change often as more users query data, and more data enters the search engine. Second, we are thinking of using combinations of these models to search for results. AKA a user may require both models A and C for example. $\endgroup$
    – Pythoner
    Jan 7 at 0:17
  • $\begingroup$ Using a combination of models is stacking. This is certainly a relevant option but then there's no need to select a model, instead one trains a "meta-model" using the predictions of all the individual learners. Also if the number of possible models is susceptible of changing then there's no point in any kind of selection method, since the selection can only apply to a fixed set of options. $\endgroup$
    – Erwan
    Jan 7 at 11:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.