0
$\begingroup$

I have an NLP model, for example, Sentiment Analysis.

This model serves in production.

I want to detect Data Drift, and specifically Covariate Shift for this model.

I saw that Cosine Similarity may solve this issue, but I'm concerned about:

  • The ability to calculate it - Cosine similarity can be calculated for vectors that are in the same vector space. Are all of the embeddings that a model produces live in the same space?

  • The time complexity - If I have 1M training data points and 1M prediction data points and I want to infer the average Cosine difference, I'll have to find the cosine difference of every prediction compared to every training data point. I can sample, but what is the sampling algorithm?

I'd love to hear your opinion about:

  • Different solutions to calculate Covariate shift for NLP models

  • My concerns about the cosine similarity

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.