Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [training]

Training is the part of machine learning whereby a model is "trained" on a define portion of a dataset to learn attributes and statistical features of the data. It's counterparts are called Testing and Validation. After training a model is tested and validated on another portion of the dataset.

Filter by
Sorted by
Tagged with
0 votes
0 answers
21 views

Should it be possible for effective batch size and real batch size to produce different results?

I think I have observed a situation where effective batch size is performing better than actual batch size. Here are some details of what I'm doing. I'm finetuning Llama 3 8B using Hugging Face in a ...
Ameen Izhac's user avatar
0 votes
0 answers
8 views

Validation accuracy dip and recovery when restarting training

i was fine-tuning this large language model with Stochastic Gradient Descent and mid epoch i stopped training, and saved the model weights. Then at a later time, reloaded the weights and restarted the ...
clam's user avatar
  • 101
1 vote
0 answers
13 views

Transfer Learning - GoogLeNet - Training Times || Loss not converging || Pytorch

Hi Community and thanks in advance for the help. I am working on transfer learning - specifically GoogLeNet model with the Food101 Dataset. Code is below. I think everything is in order from data ...
James's user avatar
  • 21
1 vote
0 answers
18 views

Recreating results from Research Paper

so I have been trying to recreate the results from this particular paper (Neural Collaborative Filtering). The dataset I use closely resembles this . I understand that I should my data into train and ...
Panos_42's user avatar
1 vote
0 answers
18 views

Train a LLM to learn the entropy of the use case

I want to train a LLM (prefered Llama-2-13b) to learn the entropy of german texts - to be specific sports news. I use perplexity as training metric and want to check the training success after the ...
Christian01's user avatar
1 vote
1 answer
30 views

How do you train a seq2seq model on sequences longer than its sequence length?

I was reading the GPT original paper here and in section 3.5 they mention evaluating on the CoQA dataset. I checked GPT has a sequence length of 512, yet most of the sequences in the CoQA are a few ...
Ameen Izhac's user avatar
1 vote
0 answers
7 views

Is GroupKFold needed if some samples have some of their feature values equal?

I am given a dataset $D$ of 10k enzyme-substrate complexes having a lock-key relationship, with each sample (complex) being characterized by enzyme features $x_e$ and substrate features $x_s$. That is,...
ado sar's user avatar
  • 191
0 votes
0 answers
18 views

Loss increase while accuracy also increase [duplicate]

I'm training a fairly large classification model,and I'm having the below results. ...
WillWu's user avatar
  • 1
0 votes
0 answers
21 views

How to improve Accuracy on dermaMNIST dataset?

Unlike the regular MNIST which gets 97-99% with a fairly basic network, dermaMNIST gets training/validation stuck on 0.69. This tells me the model is underfitting. But, making it bigger seems to have ...
Zwerchhau's user avatar
0 votes
0 answers
14 views

Why can't I increase my GPU utilization?

I have a simple UNet model (~1M params) written in Keras 3.0.1, running with a torch backend. My CUDA version is ...
Savindi's user avatar
0 votes
0 answers
18 views

If I do cross validation do I need to refit the model?

I am making a dual process. I have an initial dataset in which I train (fit) a model, then I do cross validation to get results. Until now everything normal, but additional to that, I create a new ...
Curious student's user avatar
2 votes
0 answers
23 views

Why is it called 'Test' dataset and not 'Testing' Dataset?

Training data is named as such as it is used for 'training'. Then why not the same for 'Testing' dataset? I apologise if this is not very relevant, but its something I believe only the community can ...
Sameervdtc's user avatar
0 votes
0 answers
12 views

Train-test split strategies in sensor time series

i'd like to train a supervised machine learning algorithm on my sensor data (Accelerometer XYZ). I've already segmented the data with a sliding window approach (1s window_size, 50% overlap) and ...
André S's user avatar
0 votes
0 answers
11 views

How are the cross validation and training processes interlinked here?

Please consider the code given below. ...
Masroor's user avatar
  • 101
2 votes
1 answer
38 views

Effect of Sequential Data Quality Variation on Transformer Model Training: Seeking Insights and Experiences

I'm exploring the training efficiency of transformer models against the backdrop of data quality sequencing. Specifically, I ponder whether arranging unlabeled data by presumed quality affects ...
Lukas N.P. Egger's user avatar
0 votes
0 answers
37 views

Why is there a difference in Training Accuracy Output, when the training dataset is the same but the validation dataset is different?

I am looking at the output of a multi-class image segmentation deep learning model. I used U-Net to implement this. I am confused about why the training accuracies are different for a different ...
user10529827's user avatar
-1 votes
1 answer
51 views

Why is my NN model's prediciton for y= sinc(x) function showing symmetric?

...
RimaMonica's user avatar
0 votes
2 answers
39 views

Difference between (None, 2000, 8) and (2000, 8)

Input: a tensorflow dataset with 17000 items: ...
Fabi's user avatar
  • 11
0 votes
0 answers
21 views

Using Transformers on a seq2seq task with sparse labels

I'm new to the ML world and would like to ask for architecture advice for a project I'm building. I want to detect a certain event throughout an audio. For example, if the audio is divided into 10 ...
TigerHix's user avatar
  • 101
0 votes
0 answers
27 views

Training rate of "convergence" vs viability of network architecure

I am still fairly new to modern neural networks (dabbled in the 90's when convolution was called neocognition which I didn't explore regrettably). I read the suggestions after typing in the above ...
user2624395's user avatar
0 votes
0 answers
16 views

Iterating more quickly, shorter feedback cycles, general workflow practices

(caveat: I've not written that much ML code yet) When I'm writing non-ML code I like to create short feedback loops to check the validity of my code. I use unit-tests or sometimes short helper tools (...
Niels Bom's user avatar
  • 101
1 vote
1 answer
21 views

Hyperparameter tuning

Jane trains three different classifiers: Logistic Regression, Decision Tree, and Support Vector Machines on the training set. Each classifier has one hyper-parameter (regularisation parameter, depth-...
Tom's user avatar
  • 11
2 votes
1 answer
30 views

Building machine learning model with variable targets and on the run

I want to add Machine learning predictive capabilities to an existing application; the data that will be used to train the model is stored in a database. My problem is as follows: in a typical machine ...
Aws rayyan's user avatar
1 vote
1 answer
94 views

Optimal Number of Epochs for Training Transformer Network on Time series data? Early Stopping and Model Selection Strategies

I have a transformer network that is trained on time series data. The task is to predict if a variable will increase a certain percentage in the next 7 days. The input is data from the 90 previous ...
QCQCQC's user avatar
  • 113
0 votes
0 answers
14 views

The using of golden dataset in Augmented SBERT Training

I use the training strategy of Augmented SBERT (Domain-Transfer). In the code example they use the golden-dataset (STSb) for the training evaluator. Here two code snippets of the example of sentence-...
Christian01's user avatar
1 vote
0 answers
34 views

Interpretation of Evaluation Values of Augmented SBERT Training with EmbeddingSimilarityEvaluator()

I train a BI-Encoder to get an Augmented SBERT and I get a final training result. How can I interpret the following output of the final training result? ...
Christian01's user avatar
0 votes
0 answers
11 views

How to split the data for speaker verification using AAM Softmax loss?

There are several models for speaker verificaiton task (wavlm-ecapa / xvector / ...). Some of those model where trained with AAM Softmax loss, which gets the number of labels as input. When training ...
user3668129's user avatar
1 vote
1 answer
252 views

How can I leverage machine learning for log analysis?

I am new to data science and trying to find possibilities of using datascience in tasks. I have a set of logs which I want to convert to json. The logs are more or less of same format and I can write ...
SUNITA GUPTA's user avatar
0 votes
0 answers
74 views

why my training and validation loss curve looks like lognormal distribution?

I trained an XGBoost model and my training and validation curve looks like this? Is something weird I am doing? I have always seen it going from high to low or like a U-shape incase of overfitting. ...
learner's user avatar
  • 51
0 votes
1 answer
31 views

Should I need to cure the curve for the model?

I have several classification models that used for image classification. The epochs is set to 100 for both. Model A gave me accuracy 99.7 and stopped at epoch 100 but Model B gave me 99.93 but take ...
user5520049's user avatar
0 votes
0 answers
22 views

Can I modify my training datasets (X_train and Y_train) while fitting the model?

I am new to ML and I am trying to train a forecasting model. The target variable (Y_train) has multiple columns, all of Boolean type. The features table (X_train), according to my approach, in the ...
Mookesh Dash's user avatar
0 votes
0 answers
15 views

How can I tell if my CNN tuning made a difference?

I'm working on a detection CNN, estimates pose for some classes of objects. I am able to compute a bunch of different metrics on performance, things like position error, rotation error, tracking ...
Mr Squid's user avatar
  • 101
0 votes
0 answers
22 views

Training with few samples, dropping training loss but constant validation loss

I am training a resnet50-based model using transfer learning. My dataset has 10 classes and about 10 occurrences per class, so it is very small. The training loss is decreasing steadily to 0.07 for ...
ml_nnoobb's user avatar
0 votes
1 answer
75 views

Holding batch size constant, will a bigger dataset consume more GPU memory?

If you hold (mini) batch size constant (as well as everything else) but increase the number of examples (and therefore the number of training iterations), should you expect a (significant) increase in ...
ubadub's user avatar
  • 101
3 votes
2 answers
681 views

Confusion over training accuracy vs. training loss

I had a small discussion with my friends on overfitting and we became confused over the two terms: "training accuracy" and "training loss (or cost)". This is the first time I've ...
Tran Khanh's user avatar
0 votes
1 answer
63 views

Is my model overfitting based on my accuracy/loss curves?

Do those results indicate that my model is overfitting?
Begnnier's user avatar
0 votes
0 answers
44 views

How was the word2vec model trained?

Let's take the CBOW (continuous bag of words) model as the example. Suppose that, there are $c$ context words, each of which is a one-hot encoding vector. So the total number of elements of input ...
J. Doe's user avatar
  • 56
0 votes
1 answer
46 views

Very basic but how to understand data statistically for machine learning?

I’m trying to solidify my statistics so I really know how to use them in my analysis/models. However my concept of statistical testing gets completely messed up by context. I’m unsure defining exactly ...
donutmonster's user avatar
1 vote
1 answer
54 views

PyTorch ResNet implementation's Training Loss increasing with every Epochs

I'm implementing a ResNet network from scratch using PyTorch. This network is unique to my requirements, since I need to perform Image Classification for Satellite Imagery with 14 different channels ...
Gamma-ray-burst's user avatar
0 votes
0 answers
10 views

How do we modify the early stopping procedure to account for better losses after initial rise in losses?

I have a question regarding the usage of early stopping in the training of my forecasting model. Curious about how the training would go without early stopping, I observed that the test loss seems to ...
Zezimabig's user avatar
0 votes
1 answer
107 views

Workflow when making a machine learning model

I'm new to data science, and kinda confused with the workflow and steps to make a model. Before learning the math and concepts behind the algorithms like SVM, linear regressions, etc, I would just ...
Justin Jonany's user avatar
1 vote
1 answer
783 views

How to calculate the training accuracy of a decision tree?

The hint given was to construct a confusion matrix.
Praveent Thamil Mani's user avatar
2 votes
1 answer
597 views

what is the difference between window size and context length of language model?

is window size and context length of language model one and the same thing? ******** following text is added as question with ONLY above text was not allowed ***** I am trying to understand how GPT ...
Vinay Sharma's user avatar
0 votes
1 answer
40 views

Is it a problem to use the test dataset for the hyperparameter tuning, when I want to compare 2 classification algorithms on the 10 different dataset?

I know that we should use the validation set to perform hyperparameter tuning and that test dataset is not anymore really the test if it is used for hyperparameter tuning. But is this a problem if i ...
John B's user avatar
  • 1
0 votes
0 answers
9 views

Which Frameworks/Libs Best Support Integer-Based Features, Scaling, Training, etc?

Papers such as Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference have interested me in exploring integer-based data science. In particular, I'm thinking of ...
ezekiel68's user avatar
0 votes
0 answers
120 views

Training loss is much higher than validation loss

I am trying to train a neural network with 2 hidden layers to perform a multi class classification of 3 different classes. There is a huge imbalance to the classes, with the distribution being around ...
joseph wong's user avatar
0 votes
0 answers
105 views

Understanding the desired behavior of the loss function of Variational Autoencoders

So I understood that when training VAE, we need to weight the KL part of the loss with a weight less than 1 so that the reconstruction loss can get a chance to learn (avoiding the posterior collapse). ...
user1407562's user avatar
0 votes
2 answers
69 views

Why shouldn't we try to balance the test set?

Most advice I have found online is that we must not balance the test set. The test set should remain to be unseen. However, I failed to see how balancing the test set will cause us to leak knowledge ...
Fraïssé's user avatar
  • 119
0 votes
1 answer
31 views

Is it possible to detect early if a model is bad?

Let's say we have a model and have just started to fit it, the first epoch out of many. The first epoch shows awful results. Does it make sense to continue training hoping the results will be better ...
Putnik's user avatar
  • 105
0 votes
0 answers
82 views

Classification Threshold Optimization after GridSearchCV

In my machine learning problem I am using a CNN to classify images. Since my dataset is imbalanced I want to perform classification probability threshold tuning so I can find the optimal balance ...
Throwaway123's user avatar

1
2 3 4 5
14