Questions tagged [machine-learning-model]

A machine learning model is a simplified representation of a dataset, derived from statistics in the data, used to make predictions. It can represent patterns, behaviours or features within this dataset which have been learnt by the algorithm during training.

Filter by
Sorted by
Tagged with
3 votes
2 answers
212 views

What techniques are used to analyze data drift?

I've created a model that has recently started suffering from drift. I believe the drift is due to changes in the dataset but I don't know how to show that quantitatively. What techniques are ...
  • 511
0 votes
1 answer
42 views

Decision boundary of an neural network

Starting with a). For the first unit: 0 * x1 + 1 * x2 + 1 > 0 (0, because the threshold is 0) which is the same as x2+1 > 0. For the second unit: x1 * 1 + x2 * 0 + 1 > 0 (0, because the ...
  • 11
0 votes
0 answers
14 views

Reporting and/or evaluation of metrics in deep learning

While I was trying to write a custom training script, I encountered the following doubt. I see that the loss is evaluated at the end of every forward pass (i.e., a step or with a particular batch of ...
0 votes
0 answers
12 views

What is the best way to whittle down rendundant categorical data?

I'm trying to build a linear regression model in Tensorflow (and preprocessing with pandas) that will help me categorize bank transactions. I'm trying to whittle down the vendor parameter, because the ...
0 votes
0 answers
9 views

Derivative of a KernelRidge regression model based on Coulomb Matrix descriptor

I am trying to take analytical derivatives of a KernelRidge regression model that takes as input a Coulomb Matrix descriptor. A Coulomb Matrix is a way of representing a molecular structure basically ...
  • 1
0 votes
1 answer
35 views

How do you retrain a model as new data comes in?

I'm just curious about real ML projects on production. I was wondering what is the way to go to retrain your models when you get new data? for example, let's suppose you've built a model with 2023 ...
  • 103
0 votes
1 answer
11 views

CWRU Bearing fault

I am new to ML. I have been asked to use a pre-trained GRU model for detecting a bearing fault in CWRU. is pre-trained model another name for transfer learned model?
0 votes
1 answer
21 views

Machine learning / statistical model of a deterministic process: how large must my training set be to ensure almost perfect accuracy?

This may be a silly question, but if I got a deterministic process, for instance, a function (in the mathematical sense) that happens to be computationally expensive to evaluate, and I decided to ...
  • 103
0 votes
0 answers
20 views

Does order matter in this causal language model?

Say you've implemented a causal language model like so: ...
  • 1
0 votes
0 answers
9 views

Which zero-shot models predict the contents of an image?

I'm looking for the class of machine learning models on Hugging Face where you provide an image and a list of options and it returns which of those options appear in the image for example I might ...
0 votes
1 answer
38 views

Is classification enough for this?

I have a DB with 2 tables connected with a one to many relationship. Let's say one of A is linked to many of B. The tables have both two fields for a date and some text. And both tables get new ...
  • 1
0 votes
0 answers
35 views

Do we need to check the training score when we use randomizedsearchcv

given a model and a set of parameters, randomizedsearchCV(or gridsearchCV) gives the mean of the best scores from a list of different folds of the datasets. Does the model control for overfitting? I ...
0 votes
0 answers
22 views

Given 20+ input numbers, predict 4-5 output numbers, using past data with the same 20+ inputs / 4-5 output for training

I'm a veteran Python software engineer, but very amateur at this stuff. I've used PyTorch for some NLP (sentiment and classification), and I've spent a bit of time attempting to learn data science, ...
  • 101
0 votes
1 answer
36 views

Why is the accuracy on train dataset not always 100% while we use the same dataset to train the model?

Though tree-based ML algorithms give us 100% accuracy on train dataset many times, but why is this not happening every time. I know this results in overfitting but why not 100% accuracy every time on ...
0 votes
1 answer
98 views

How can I convert a numpy array of tensors to tensor of tensors?

It is my first GRU model so pardon the stupidity. I am trying to learn by training a simple GRU network on variable length sequences. The sequences are numpy arrays of tensors. The length of numpy ...
0 votes
0 answers
30 views

Refining AI problem statement - suggestions

I am looking for some guidance. My company is a electronic goods manufacturing company. We work with multiple distributors (around 7 distributors) across specific regions to sell our products. But ...
  • 2,449
1 vote
0 answers
61 views

ValueError: Found input variables with inconsistent numbers of samples: [120, 30]

I practice XGBClassifier() to predict the target in iris dataset. here is the code: ...
0 votes
0 answers
14 views

Which CNN model to use for the classification(20 classes) of gemstones (diamonds, sapphire, ruby etc) based on digital photo images and huge data set?

Im trying to build CNN Model for the classification of precious stones (like diamonds, sapphire, ruby) based on digital images. So I have data set of labeled 150,000 gemstone certifications and the ...
0 votes
0 answers
12 views

Loss function for model with two-outputs

I have created a model for this Kaggle competition that outputs a classification of the level of the disease (from 0 to 4) from an image of the retina. I now want to blend the predictions for both ...
  • 113
0 votes
0 answers
24 views

How to do forecasting with categorical timeseries?

I have a dataset that is in the form of categorical timeseries: (specifically, we either know or don't know the values of 6 degrees of freedom of an object at any given time). If we know it, it's ...
  • 101
3 votes
2 answers
273 views

is there a deep learning model that handle 47800+ classes for classification?

I am trying to build a text classifier with 47893 classes and 1.3 billion (1,302,687,947) data samples. What would be the best classifier to build with such kind of data? Each data label will contain ...
  • 239
0 votes
0 answers
13 views

Machine learning with 2D data table and single outcome to model

I am new to machine learning and am trying to conceptualize how to effectively build a database of sports data for machine learning. I currently have a list of games and outcomes as well as separate ...
  • 1
0 votes
1 answer
37 views

Do I need to use always the same "Test" dataset to compare between different models?

I have two datasources A and B, and I want to check how several methods can affect the accuracy of my multi class models: If I use cross-validation with validate dataset to obtain the best hyper ...
0 votes
1 answer
52 views

Machine learning | Column names vs number when training/predicting

Been doing machine learning since a few months by now. I've a grounding questions that I couldn't answer by my self. It's possible I'm asking the wrong question: When training models, like XGBoost, ...
0 votes
1 answer
30 views

Predicting student exam outcome based upon study patterns

I have a few years of data now for HE students participation in their course(es) including exam results. If I just compare formative exam results with summative results there is good correlation, and ...
1 vote
1 answer
40 views

Rule based vs predictive maintenance models

I have data for pumps which have one or more sensors to record the air pressure. Apart from the sensor_id and timestamp, with ...
  • 185
0 votes
0 answers
33 views

How to visualize a data drift?

I want to show that my data distribution changes between data windows. Is it enough to visualize the mean and variance for every window? Is there any other solution? thank you
  • 21
0 votes
0 answers
5 views

How to use this data set for spatial regression?

I want to graph how much a customer spends by region and have hotspots for high spending regions. Here is an example of the csv file.
0 votes
1 answer
17 views

Applying the model on validation data achieves higher performance than on test set. Is this possible?

I trained a binary cross-validated classification model and got high performance (about 90) on the test data but when I apply the model to new unseen data to see how to performs, i get even higher ...
  • 11
0 votes
0 answers
6 views

Should I include marginally better results in a writing sample?

I am performing some experiments and my results are only marginally better than the current state-of-the-art, 0.12% increase in accuracy but on the far side of 90's. Should I include the said SOTA ...
  • 101
1 vote
1 answer
32 views

Confusion matrix and precision problem

I'm trying to calculate the precision of a trained model. I have generated the right values for the true positive rate and the false positive rate. And I know that the formula should be TP/TP + FP. ...
  • 11
1 vote
0 answers
14 views

Looking at feature contribution after classifying groups using components

I have a lot of features and many are correlated, so I performed dimensionality reduction. I then used these components in binary classification and got high accuracy. I also performed feature ...
  • 11
0 votes
2 answers
33 views

Predicting probability of reaching a milestone -- How much data should I use from production universe to train/test model?

If I am predicting probability of a business to reach (x) milestone (classification 1), but the only data I have is live production data, how much of the production data should I use to train the ...
0 votes
0 answers
25 views

Machine learning method to detect correlation between bike counters

I am doing a research master's degree in transportation science. I would like to develop a model for one of my classes to detect the dependence between various bicycle counters. The database I'm using ...
  • 1
0 votes
0 answers
9 views

Same precision, specificity and sensitivity values

My Model gives giving same precision, specificity, and sensitivity values when I'm running a loop five times to fill missing values using Random forest regression and for classification using Gradient ...
  • 1
1 vote
1 answer
40 views

Interpreting Learning Curves of models

I need some help to understand if the models are overfitting and which of these we can consider "the best". On the internet i only find simple examples with learning curves but in these ...
  • 41
2 votes
4 answers
150 views

99% accuracy in train and 96% in test is too much overfitting?

I have a binary classification problem, the classes are quite balanced (57%-43%), with a GridSearch with Random Forest Classifier I obtained the best hyperparameters and I applied the model to train ...
  • 41
1 vote
1 answer
39 views

Combining multiple ranked lists

Suppose I'm given two ranked lists, A and B, with each item in the lists being associated with a score: ...
0 votes
0 answers
59 views

Yolov5 model detects everything as cars

I'm facing an issue regarding my yolov5 model to detect cars. Here is the following procedure I made to train the model: Downloaded 10000 images from training car dataset (Google Open Images Dataset)....
0 votes
1 answer
46 views

KMeans is not predicting the correct cluster

k-means clustering is done and created 5 optimal number of clusters. (Clustering is done unevenly). While using them in my model, the model is not choosing the exact cluster which has the exact data. ...
0 votes
1 answer
49 views

How to train machine learning on sales forecasting problems of almost 10,000 shops?

I have a dataset of almost 10,000 shops, 'dates', 'shop ID' and 'sales amounts' as their features almost 2 years of data. I want to forecast each shop, the sales amount for 30 next days. I want to ...
  • 123
0 votes
0 answers
15 views

Image recognition model with CNN for face gestures is really bad

I have a dataset that contains facial expressions and their label, and I am trying to make a classification model for it. Unfortunatly, I can't manage to create a good model with CNN, as the highest ...
0 votes
1 answer
32 views

What can be the reasons for 95% of samples belong to one cluster when there is 5 clusters?

'''I used the k-means algorithm to clustering set of documents which are textual data only. The document has 2lack records. Surprisingly the result for the clustering is 90% of records is storing in 1 ...
0 votes
1 answer
90 views

Is there a way to make keras custom test_step aware of the call being made from model.fit and model.evaluate

I am using keras custom model with custom train_step and test_step methods overwritten. Also, have a need to change certain margin used in the loss function, only for test dataset. In other words I ...
  • 1
0 votes
1 answer
66 views

I get 100% on my test set using random forest. What is wrong?

I am getting 100% accuracy on my test set when trained using random forest. Is there something wrong with my model? Code: ...
  • 3
2 votes
1 answer
45 views

How to implement linear regression

I am having difficulty achieving the same result as in sklearn while implementing linear regression model from scratch. After adjusting the learning rate, I obtained an AUC of 0.694 for this binary ...
  • 151
0 votes
1 answer
26 views

Ideas on how to solve a problem using machine learning

I am fairly new to machine learning. I have been in mechanical simulation field for the past 7-8 years, I realise there are potential areas which I have been doing the same thing day in and day out, ...
0 votes
1 answer
57 views

how to evaluate the combination of tfidf and kmeans

For my nlp problem I'm using a combination of TFIDF and KMeans from the sklearn package. The tfidf gets the vectors and then I use Kmeans to cluster the texts based on the vectors. I have a few ...
  • 167
6 votes
2 answers
1k views

Image classification architecture for dataset with 710 classes, 90,000 subclasses, and anywhere from 10-1000 images per subclass?

Been struggling with finding the best approach to handle this scenario, I'm also a novice when it comes to machine learning. I have a dataset of around 700 classes, 90,000 total subclasses, and ...
  • 61
0 votes
0 answers
5 views

How to predict multiple independent routes?

I have an idea in mind but, due to the lack of expertise in the ML domain, I just don't know where to start. I'd really appreciate any hints/advices on which methods to study or how to approach this ...
  • 101

1
2 3 4 5
17