The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [hyperparameter-tuning]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
2answers
17 views

Parameters optimization algorithms in Weka

In Weka, I used the Grid and Random search parameters tuning algorithms but unfortunately, their performance (in terms of better prediction accuracy) is observed worst when we use the ML algorithms (...
1
vote
1answer
33 views

Hyperparameter tuning and cross validation

I have some confusion about proper usage of cross-validation to tune hyperparameters and evaluate estimator performance and generalizeability. As I understand it, this would be the process you ...
0
votes
0answers
11 views

Getting worse results after Hyperparameter Tuning(Grid/Random Search/TPOT)

I have a problem with Hyperparameter Tuning. Usually I getting almost the same results(or worse) than before tuning. Usually default parameters of classificator(regressor) give me a best score. ...
0
votes
1answer
31 views

hyperparameter tuning with validation set

For what I know, and correct me if I am wrong, the use of cross-validation for hyperparameter tuning is not advisable when I have a huge dataset. So, in this case it is better to split the data in ...
1
vote
0answers
17 views

Minimizing overfitting when doing hyperparameter Tuning

Generaly when using Sklearn's GridSearchCV (or RandomizedGridSearchCV), we get best model with best test score even if the model overfits a little bit. How can we compute generalization error ...
0
votes
0answers
22 views

Ensure class balanced batches while hyperparameter tuning keras models with grid search

Ensuring class balanced batches while training keras models is possible using fit_generator method. I used imblearn.keras.BalancedBatchGenerator for that and it works fine! But i wanted to do that ...
2
votes
1answer
61 views

how to pass parameters over sklearn pipeline's stages?

I'm working on a deep neural model for text classification using Keras. To fine tune some hyperparameters i'm using Keras Wrappers for the Scikit-Learn API. So I builded a Sklearn Pipeline for that: <...
0
votes
1answer
30 views

CNN for subsets of a dataset - how to tune hyperparameters

I have a dataset and would like to train CNNs on subsets of different size of the dataset. I already have a CNN, which classifies very well if I use the entire dataset. Now the question arises if I ...
1
vote
0answers
19 views

Analyzing the search space of hyperparameter optimization

My goal is to train a CNN via transfer learning on a given dataset and to analyze and document the training process. I selected a few CNN architectures and hyperparameters to perform a random search. ...
1
vote
1answer
27 views

Track underlying observation when using GridSearchCV and make_scorer

I'm doing a GridSearchCV, and I've defined a custom function (called custom_scorer below) to optimize for. So the setup is like this: ...
1
vote
0answers
7 views

Cross-validation for Timeseries Counterfactual Analysis

We are looking to predict counterfactual states from time-series data. In our problem we are looking to determine the energy savings from a grid-installed device that is varied on and off for many ...
0
votes
1answer
23 views

Difference between a log scale and linear scale : np.random.rand()

How and why will a linear np.random.rand() (to generate a linear scale between 0.0001 and 1) not result in better distributed result but ...
2
votes
0answers
46 views

How to measure the stability of hyperparameter selection in a model-building procedure?

For my project I run several model-building-procedures. I use the mean and standard deviation of the test scores in the outer folds as an estimator for the generalizability of the model-building ...
0
votes
1answer
31 views

Hyperparameter Tuning Time Series in Production

I have a time series data that handled using GDBT to predict the next value. I always use previous 30 days data to train daily, but overtime the data to predict and train is increased because the ...
0
votes
1answer
15 views

Do we need to tune same model differently for different window sizes in time series data classification?

I am currently working on the time series data classification problem using deep learning. As we all know that in time series, we process the time-series data sequentially for some time steps at a ...
1
vote
0answers
62 views

Optimizing parameters for CNN autoencoder based on training and validation loss

I have designed an autoencoder with a encoder and decoder consiting of 2D convolutational layers (the input are 40'000 2D images). I train the autoencoder using adam optimizer. The autoencoders has ...
0
votes
0answers
23 views

Hyperopt Model runs with 0 seconds duration

I use Hyperopt for Random Forest Regression hyperparameter tuning. my parameterspace is : ...
1
vote
0answers
38 views

How to put KerasClassifier, Hyperopt and Sklearn cross-validation together

I am performing a hyperparameter tuning optimization (hyperopt) tasks with sklearn on a Keras models. I am trying to optimize KerasClassifiers using the Sklearn cross-validation, Some code follows: <...
2
votes
0answers
92 views

Error while trying to do hyperparameter tuning using hyperas

I am getting a syntax error while using hyperas and am not sure why. My code: ...
0
votes
1answer
26 views

CNNs - Hyperparameter tuning with different training sizes of the same data set

I would like to compare how much the classification performance (test accuracy) of CNNs changes depending on the size of the data set. For this I would like to use a data set like MNIST or Fashion ...
1
vote
0answers
47 views

How to tune the parameters of ANN in R?

I tried below code where I used method as 'mxnet': classifier = train(form = Survived ~ ., data = training_set_scaled, method = 'mxnet') From this code I got ...
5
votes
3answers
219 views

Why my network needs so many epochs to learn?

I'm working on a relation classification task for natural language processing and I have some questions about the learning process. I implemented a convolutional neural network using PyTorch, and I'm ...
0
votes
0answers
24 views

Confused about hyper-parameter search and network architecture search in NAS

I want to understand automatic Neural Architecture Search (NAS). I read already multiple papers, but I cannot figure out what the actual search space of NAS is / how are classical hyper-parameters ...
1
vote
2answers
60 views

Is it a good idea to tune the number of folds for cross validation when tuning hyperparameters of RF

I'm new to data science. I'm trying to get the best model for Random Forest. Unfortunately, I'm not sure if my idea can produce a good generalized model. 1) I have split data to TrainingSet (70%) and ...
0
votes
0answers
40 views

Has anybody used alternative hyperparameter optimization techniques (other than default one) in SK-Learn?

I've been using Sklearn for Gaussian process regression that has L-BFGS-B (“fmin_l_bfgs_b”) as a default optimization algorithm. I want to implement some other ...
1
vote
0answers
300 views

Bayesian optimization for a Light GBM Model

I am able to successfully improve the performance of my XGBoost model through Bayesian optimization, but the best I can achieve through Bayesian optimization when using Light GBM (my preferred choice) ...
1
vote
0answers
24 views

Fast way to tune SVM parameters in R

I have tried to tuned the parameters with the function tune, but it took over 15 hours to tune this (and at that point I had to stop it): ...
3
votes
2answers
233 views

Hyper parameters tuning XGBClassifier

I am working on a highly imbalanced dataset for a competition. The training data shape is : (166573, 14) ...
1
vote
1answer
55 views

When to use BayesianSearchCV and how it works?

Can somebody highlight when to use BayesianSearchCV and how it works? I have seen the implementation of same on kaggle and wanted to explore it further. Below is the link where the implementation ...
1
vote
1answer
98 views

Accuracy improving but, val_acc oscillating in ConvNet. What does it mean?

In my ConvNet model, i'm trying to classify some images. It is malware images and it doesn't contain complex features (i think), as expected model learn to classify images easily. You can see my ...
1
vote
1answer
45 views

Can `k=1` be a good choice for K neighbors classification?

Running sklearn.KNeighborsClassifier() on Kaggle's Leaf Classification sample (set of 99 species, 10 specimen each), with defaults kNN parameters and a grid search ...
0
votes
0answers
578 views

XGBOOST (sklearn interface) REGRESSION error

I am trying to run a GRIDSEARCHCV (sklearn) on XGBRegressor. Documentation on the parameter says that if regression, then objective = reg:squarederror.(see https://...
2
votes
2answers
183 views

Setting best SVM hyper parameters

I have a non linear data set, and I am using SVM (RBF kernel) to build a classification model, but not sure how to set the best hyperparameters of the SVM, C and gamma in Matlab ...
4
votes
1answer
290 views

What happens if GBM parameters (e.g., learning rate) vary as the training progresses?

In neural networks there is an idea of a "learning rate schedule" which changes the learning rate as training progresses. This made me ask the question, what would be the impact of varying ...
4
votes
3answers
84 views

Scaling neural networks

While using Neural Networks (TensorFlow: Deep Neural Regressor), when increasing your training data size from a sample to the whole data (say a 10x larger dataset), what changes should you make to the ...
1
vote
0answers
21 views

Hyperparameter Tuning with Simulated Data

I'm trying to create a SVM classifier which can predict some fault, and to train it I'm using simulated examples of the fault. Of course, the simulations are not perfect, but they appear to be good ...
0
votes
1answer
60 views

review: gradient descent, epochs, validation in neural network training

These days, training data aren't put in gradient descent all at once. Rather, they are put in batch after batch. Gradient descent is run once for each batch of training data. When all batches are ...
8
votes
2answers
421 views

What is the most efficient method for hyperparameter optimization in scikit-learn?

An overview of the hyperparameter optimization process in scikit-learn is here. Exhaustive grid search will find the optimal set of hyperparameters for a model. The downside is that exhaustive grid ...
6
votes
1answer
92 views

How to decide how many n_neighbors to consider while implementing LocalOutlierFactor?

I have a data set with rows: 134000 and columns: 200. I am trying to identify the outliers in data set using LocalOutlierFactor from scikit-learn. Although I ...
1
vote
1answer
39 views

what exactly happens during each epoch in neural network training

Across different epochs, which of the following is/are updated? initial weights (initial ConvNet filter matrices, initial fully connected weights) hyper parameters: number of ConvNet filters, size ...
0
votes
0answers
57 views

How can I recognise if I can improve a random forest model by adding features

I want to tune a random forest model with caret package. I'm tuning it with cross-validation to prevent overfitting and resulted cross-validation accuracy is very ...
0
votes
0answers
42 views

Cross-validation and out-of-bag bootstrap applications

I have a question regarding steps on which a specific resample method should be used in general. As far as I know: out-of-bag bootstrap is the resample method with replacement, which has lower ...
0
votes
0answers
73 views

Hyperparameter optimization when calculating learning curves

I'm selecting a model for a regression problem and want to calculate learning curves. My dataset consists of ~20,000 x-y pairs. I'm using kernel ridge regression with different kernels, different ...
1
vote
0answers
19 views

Any heuristic for minimal DCGAN latent space dimension?

I am highly interested in approaching minimal latent space dimension (as many other may be) for DCGANs or autoencoders. In this example of DCGAN on the MNIST dataset, the person uses a 100-...
0
votes
2answers
63 views

Disadvantages of hyperparameter tuning on a random sample of dataset

I often work with very large datasets where it would be impractical to check all relevant combinations of hyperparameters when constructing a machine learning model. I'm considering randomly sampling ...
3
votes
2answers
49 views

Bayesian optimisation in deeplearning

Has anyone tried using Bayesian optimisation to get best learning rates, and other hyperparameters for deeplearning. How to change the parameters between the training. Any examples on callbacks? Can ...
1
vote
1answer
140 views

Is it necessary to tune the step size, when using Adam?

The Adam optimizer has four main hyperparameters. For example, looking at the Keras interface, we have: ...
1
vote
1answer
165 views

Hyperas LSTM configuration assignment error

I have been working on my trivial keras lstm model trying to implement Hyperas with the following code that gives me an error I cannot resolve. I have just been experimenting around with Hyperas and ...
0
votes
1answer
36 views

hypeparameters tuning neural network according to loss vs according to scoring function

During hyperparameters tuning we select a metric to measure performance of the model. Example of metrics : f1 score, precision, recall, AUC ... In general, for the training of neural networks, back-...