3
You could train an embedding model. Each element would be projected onto a location a vector space based on its co-occurrence with other elements. Then finding similar elements could be done with a nearest neighbor search.
2
You have a big dataset and you get new instances//data every 2 months.
First you should select with which data you want to train. Since your data is big and there is the probability than data from 2 years ago is not as relevant as the data from the last month you can consider doing a Roll out// slidding window validation. This way you will only select the ...
2
From what I understand - your problem is "sample selection bias" problem. Any kind of pattern to select a subset out of large data may lead to bias. This raises two question.
How to choose? Random/stratified random (If you have multiple classes) under sampling to obtain a smaller subset.
How big to choose? we can set percentage of undersampling.
Reducing ...
2
Maybe it's a bit overkill and biased toward my own field (neural machine translation), but you could go with a neural network architecture with self-attention in a masked language model-ish (i.e. BERT) configuration
The input to the network would be a fixed-size (40) sequence of discrete symbols meaning whether the element at that position is either present ...
2
The advantages of deploying the model as a package is the code will be part of the monolith application and require no remote calls thus:
No additional operational demands
No external dependencies
Faster
Fewer security issues
Same uptime as the rest of the application
In general, most features stay in a single monolith application until there is are ...
1
Try these two approaches:
First:
Make a model, using any ML algorithm and divide your data into train and test.
Now using the previous features, check the train and test accuracy.
Now add the new features to the previous ones, again divide data into train and test. Check the train and test accuracy.
If the new features help improve the test accuracy ...
Only top voted, non community-wiki answers of a minimum length are eligible
