Questions tagged [machine-learning-model]
A machine learning model is a simplified representation of a dataset, derived from statistics in the data, used to make predictions. It can represent patterns, behaviours or features within this dataset which have been learnt by the algorithm during training.
801
questions
3
votes
2
answers
212
views
What techniques are used to analyze data drift?
I've created a model that has recently started suffering from drift.
I believe the drift is due to changes in the dataset but I don't know how to show that quantitatively.
What techniques are ...
- 511
0
votes
1
answer
42
views
Decision boundary of an neural network
Starting with a).
For the first unit: 0 * x1 + 1 * x2 + 1 > 0 (0, because the threshold is 0) which is the same as x2+1 > 0.
For the second unit: x1 * 1 + x2 * 0 + 1 > 0 (0, because the ...
- 11
0
votes
0
answers
14
views
Reporting and/or evaluation of metrics in deep learning
While I was trying to write a custom training script, I encountered the following doubt.
I see that the loss is evaluated at the end of every forward pass (i.e., a step or with a particular batch of ...
0
votes
0
answers
12
views
What is the best way to whittle down rendundant categorical data?
I'm trying to build a linear regression model in Tensorflow (and preprocessing with pandas) that will help me categorize bank transactions. I'm trying to whittle down the vendor parameter, because the ...
0
votes
0
answers
9
views
Derivative of a KernelRidge regression model based on Coulomb Matrix descriptor
I am trying to take analytical derivatives of a KernelRidge regression model that takes as input a Coulomb Matrix descriptor. A Coulomb Matrix is a way of representing a molecular structure basically ...
- 1
0
votes
1
answer
35
views
How do you retrain a model as new data comes in?
I'm just curious about real ML projects on production.
I was wondering what is the way to go to retrain your models when you get new data? for example, let's suppose you've built a model with 2023 ...
- 103
0
votes
1
answer
11
views
CWRU Bearing fault
I am new to ML. I have been asked to use a pre-trained GRU model for detecting a bearing fault in CWRU. is pre-trained model another name for transfer learned model?
0
votes
1
answer
21
views
Machine learning / statistical model of a deterministic process: how large must my training set be to ensure almost perfect accuracy?
This may be a silly question, but if I got a deterministic process, for instance, a function (in the mathematical sense) that happens to be computationally expensive to evaluate, and I decided to ...
- 103
0
votes
0
answers
20
views
Does order matter in this causal language model?
Say you've implemented a causal language model like so:
...
- 1
0
votes
0
answers
9
views
Which zero-shot models predict the contents of an image?
I'm looking for the class of machine learning models on Hugging Face where you provide an image and a list of options and it returns which of those options appear in the image for example I might ...
- 101
0
votes
1
answer
38
views
Is classification enough for this?
I have a DB with 2 tables connected with a one to many relationship. Let's say one of A is linked to many of B. The tables have both two fields for a date and some text. And both tables get new ...
- 1
0
votes
0
answers
35
views
Do we need to check the training score when we use randomizedsearchcv
given a model and a set of parameters, randomizedsearchCV(or gridsearchCV) gives the mean of the best scores from a list of different folds of the datasets.
Does the model control for overfitting? I ...
- 21
0
votes
0
answers
22
views
Given 20+ input numbers, predict 4-5 output numbers, using past data with the same 20+ inputs / 4-5 output for training
I'm a veteran Python software engineer, but very amateur at this stuff. I've used PyTorch for some NLP (sentiment and classification), and I've spent a bit of time attempting to learn data science, ...
- 101
0
votes
1
answer
36
views
Why is the accuracy on train dataset not always 100% while we use the same dataset to train the model?
Though tree-based ML algorithms give us 100% accuracy on train dataset many times, but why is this not happening every time. I know this results in overfitting but why not 100% accuracy every time on ...
0
votes
1
answer
98
views
How can I convert a numpy array of tensors to tensor of tensors?
It is my first GRU model so pardon the stupidity. I am trying to learn by training a simple GRU network on variable length sequences. The sequences are numpy arrays of tensors. The length of numpy ...
0
votes
0
answers
30
views
Refining AI problem statement - suggestions
I am looking for some guidance. My company is a electronic goods manufacturing company. We work with multiple distributors (around 7 distributors) across specific regions to sell our products. But ...
- 2,449
1
vote
0
answers
61
views
ValueError: Found input variables with inconsistent numbers of samples: [120, 30]
I practice XGBClassifier() to predict the target in iris dataset.
here is the code:
...
0
votes
0
answers
14
views
Which CNN model to use for the classification(20 classes) of gemstones (diamonds, sapphire, ruby etc) based on digital photo images and huge data set?
Im trying to build CNN Model for the classification of precious stones (like diamonds, sapphire, ruby) based on digital images. So I have data set of labeled 150,000 gemstone certifications and the ...
- 1
0
votes
0
answers
12
views
Loss function for model with two-outputs
I have created a model for this Kaggle competition that outputs a classification of the level of the disease (from 0 to 4) from an image of the retina.
I now want to blend the predictions for both ...
- 113
0
votes
0
answers
24
views
How to do forecasting with categorical timeseries?
I have a dataset that is in the form of categorical timeseries: (specifically, we either know or don't know the values of 6 degrees of freedom of an object at any given time). If we know it, it's ...
- 101
3
votes
2
answers
273
views
is there a deep learning model that handle 47800+ classes for classification?
I am trying to build a text classifier with 47893 classes and 1.3 billion (1,302,687,947) data samples.
What would be the best classifier to build with such kind of data? Each data label will contain ...
- 239
0
votes
0
answers
13
views
Machine learning with 2D data table and single outcome to model
I am new to machine learning and am trying to conceptualize how to effectively build a database of sports data for machine learning. I currently have a list of games and outcomes as well as separate ...
- 1
0
votes
1
answer
37
views
Do I need to use always the same "Test" dataset to compare between different models?
I have two datasources A and B, and I want to check how several methods can affect the accuracy of my multi class models:
If I use cross-validation with validate dataset to obtain the best
hyper ...
0
votes
1
answer
52
views
Machine learning | Column names vs number when training/predicting
Been doing machine learning since a few months by now. I've a grounding questions that I couldn't answer by my self. It's possible I'm asking the wrong question:
When training models, like XGBoost, ...
- 3
0
votes
1
answer
30
views
Predicting student exam outcome based upon study patterns
I have a few years of data now for HE students participation in their course(es) including exam results. If I just compare formative exam results with summative results there is good correlation, and ...
- 111
1
vote
1
answer
40
views
Rule based vs predictive maintenance models
I have data for pumps which have one or more sensors to record the air pressure. Apart from the sensor_id and timestamp, with ...
- 185
0
votes
0
answers
33
views
How to visualize a data drift?
I want to show that my data distribution changes between data windows. Is it enough to visualize the mean and variance for every window? Is there any other solution? thank you
- 21
0
votes
0
answers
5
views
How to use this data set for spatial regression?
I want to graph how much a customer spends by region and have hotspots for high spending regions. Here is an example of the csv file.
0
votes
1
answer
17
views
Applying the model on validation data achieves higher performance than on test set. Is this possible?
I trained a binary cross-validated classification model and got high performance (about 90) on the test data but when I apply the model to new unseen data to see how to performs, i get even higher ...
- 11
0
votes
0
answers
6
views
Should I include marginally better results in a writing sample?
I am performing some experiments and my results are only marginally better than the current state-of-the-art, 0.12% increase in accuracy but on the far side of 90's. Should I include the said SOTA ...
- 101
1
vote
1
answer
32
views
Confusion matrix and precision problem
I'm trying to calculate the precision of a trained model. I have generated the right values for the true positive rate and the false positive rate. And I know that the formula should be TP/TP + FP.
...
- 11
1
vote
0
answers
14
views
Looking at feature contribution after classifying groups using components
I have a lot of features and many are correlated, so I performed dimensionality reduction. I then used these components in binary classification and got high accuracy. I also performed feature ...
- 11
0
votes
2
answers
33
views
Predicting probability of reaching a milestone -- How much data should I use from production universe to train/test model?
If I am predicting probability of a business to reach (x) milestone (classification 1), but the only data I have is live production data, how much of the production data should I use to train the ...
0
votes
0
answers
25
views
Machine learning method to detect correlation between bike counters
I am doing a research master's degree in transportation science. I would like to develop a model for one of my classes to detect the dependence between various bicycle counters.
The database I'm using ...
- 1
0
votes
0
answers
9
views
Same precision, specificity and sensitivity values
My Model gives giving same precision, specificity, and sensitivity values when I'm running a loop five times to fill missing values using Random forest regression and for classification using Gradient ...
- 1
1
vote
1
answer
40
views
Interpreting Learning Curves of models
I need some help to understand if the models are overfitting and which of these we can consider "the best". On the internet i only find simple examples with learning curves but in these ...
- 41
2
votes
4
answers
150
views
99% accuracy in train and 96% in test is too much overfitting?
I have a binary classification problem, the classes are quite balanced (57%-43%), with a GridSearch with Random Forest Classifier I obtained the best hyperparameters and I applied the model to train ...
- 41
1
vote
1
answer
39
views
Combining multiple ranked lists
Suppose I'm given two ranked lists, A and B, with each item in the lists being associated with a score:
...
0
votes
0
answers
59
views
Yolov5 model detects everything as cars
I'm facing an issue regarding my yolov5 model to detect cars.
Here is the following procedure I made to train the model:
Downloaded 10000 images from training car dataset (Google Open Images Dataset)....
0
votes
1
answer
46
views
KMeans is not predicting the correct cluster
k-means clustering is done and created 5 optimal number of clusters. (Clustering is done unevenly).
While using them in my model, the model is not choosing the exact cluster which has the exact data. ...
- 1
0
votes
1
answer
49
views
How to train machine learning on sales forecasting problems of almost 10,000 shops?
I have a dataset of almost 10,000 shops, 'dates', 'shop ID' and 'sales amounts' as their features almost 2 years of data. I want to forecast each shop, the sales amount for 30 next days. I want to ...
- 123
0
votes
0
answers
15
views
Image recognition model with CNN for face gestures is really bad
I have a dataset that contains facial expressions and their label, and I am trying to make a classification model for it. Unfortunatly, I can't manage to create a good model with CNN, as the highest ...
0
votes
1
answer
32
views
What can be the reasons for 95% of samples belong to one cluster when there is 5 clusters?
'''I used the k-means algorithm to clustering set of documents which are textual data only.
The document has 2lack records.
Surprisingly the result for the clustering is
90% of records is storing in 1 ...
- 1
0
votes
1
answer
90
views
Is there a way to make keras custom test_step aware of the call being made from model.fit and model.evaluate
I am using keras custom model with custom train_step and test_step methods overwritten. Also, have a need to change certain margin used in the loss function, only for test dataset. In other words I ...
- 1
0
votes
1
answer
66
views
I get 100% on my test set using random forest. What is wrong?
I am getting 100% accuracy on my test set when trained using random forest.
Is there something wrong with my model?
Code:
...
- 3
2
votes
1
answer
45
views
How to implement linear regression
I am having difficulty achieving the same result as in sklearn while implementing linear regression model from scratch.
After adjusting the learning rate, I obtained an AUC of 0.694 for this binary ...
- 151
0
votes
1
answer
26
views
Ideas on how to solve a problem using machine learning
I am fairly new to machine learning. I have been in mechanical simulation field for the past 7-8 years, I realise there are potential areas which I have been doing the same thing day in and day out, ...
0
votes
1
answer
57
views
how to evaluate the combination of tfidf and kmeans
For my nlp problem I'm using a combination of TFIDF and KMeans from the sklearn package. The tfidf gets the vectors and then I use Kmeans to cluster the texts based on the vectors. I have a few ...
- 167
6
votes
2
answers
1k
views
Image classification architecture for dataset with 710 classes, 90,000 subclasses, and anywhere from 10-1000 images per subclass?
Been struggling with finding the best approach to handle this scenario, I'm also a novice when it comes to machine learning. I have a dataset of around 700 classes, 90,000 total subclasses, and ...
- 61
0
votes
0
answers
5
views
How to predict multiple independent routes?
I have an idea in mind but, due to the lack of expertise in the ML domain, I just don't know where to start. I'd really appreciate any hints/advices on which methods to study or how to approach this ...
- 101