Questions tagged [optimization]

In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

-2
votes
0answers
15 views

How to use Particle swarm optimization as optimizer in autoencoder for Imputation?

There are various optimizers defined in Keras Library like Adam, Adadelta. How to use Particle Swarm optimization AS optimizer.
0
votes
0answers
10 views

Optimizing Market Basket Analysis by limiting threshold

I'm creating a suggestion model through MBA. I observed that in my particular model, that if the min_support was placed as 0, the model would take an insanely long ...
1
vote
2answers
50 views

How to find the right regression model for my project

I tried several things to find the best regression model with the best parameters but i can't go higher than 40% right predictions. So i have 67741 rows in an excel file. the data looks like this ...
1
vote
1answer
46 views

What should I study to find optimal value of best feature combinations in machine learning?

I would like to do production optimization with machine learning and/or optimization problem. My goal is not to find minimizing loss in loss function only to give the best y value. My ultimate goal ...
0
votes
0answers
16 views

Techniques to increase stability of parameters

I have a highly non-linear function $f(x,y,t)$, which I am trying to maximise over different time periods $t$. It is the case that other parameters are of mixed type with $x \in Re$ and $y \in {Z}$, ...
1
vote
0answers
10 views

How can I evaluate the constant parameters of a series of linear functions with varying input parameters?

I have the following setup of linear functions: g0(x0, y0) -> (x1,y1) g1(x1, y1) -> (x2, y2) ... about 30 more cases Where gi is a function parameterized by 6 values (ai-fi). Which can be described ...
0
votes
0answers
18 views

How to optimise a boolean expression

I am working on an optimization problem involving Boolean expressions and wanted some help as I have very little knowledge about the topic. The problem statement is as follows: There are a set of ...
1
vote
1answer
14 views

How Stochastic Gradient Descent used like Mini Batch gradient descent?

As I know Gradient Descent has three variants which are: 1- Batch Gradient Descent: processes all the training examples for each iteration of gradient descent. 2- Stochastic Gradient Descent: ...
0
votes
1answer
17 views

How to find optimized x values (input features) after training in deep learning?

I did deep learning training by Keras. I have done the training part by model.fit If I do model.predict, it only gives me y value. But I want to know x (input features) that gives the best y value, ...
1
vote
2answers
24 views

Is there any advantage in using Particle Swarm Optimization for clustering than K-Means?

I have read some paper about using particle swarm optimization. It doesn't look give much different result than K-Means. I tried to use PSO for clustering but the result is pretty much the same with K-...
0
votes
1answer
24 views

Why are parameter updates downscaled by uncentered variance (instead of centered variance) in Adam optimizer?

In Adam optimizer algorithm, parameter updates are computed as follows: $\theta_t \leftarrow \theta_{t-1} - \alpha \frac{\hat{m}_t}{\sqrt{\hat{v}_t}+\epsilon}$ Where $\hat{m}_t$ is a bias-corrected ...
0
votes
0answers
13 views

Changing the performance metric used to optimize with random-forest

I'm looking to change the performance metric used to optimize the training for my data set because it is highly unbalance. Because it's highly unbalanced, I don't feel like accuracy is the appropriate ...
0
votes
0answers
12 views

Formulation of Optimization Problem in SVM

I need help in verifying/understanding a step in formulating an optimization problem used for support vector machines (though this question doesn't need any background in SVM). Consider a bunch of $m$ ...
2
votes
1answer
27 views

How to optimize the lambdas of a hybrid loss in a deep learning model

I am using a generative adversarial deep learning model (GAN) with a hybrid loss represented by a linear combination of four losses with three $\lambda$'s, something like: $total\_loss = loss_1 + \...
2
votes
2answers
49 views

Neural networks, optimization math intuition

When I look into the following partial derivative, I see it as being the key element of any optimization algorithm out there. Correct me if I'm wrong, but this gets us the slope of the loss function, ...
0
votes
0answers
55 views

Optimization of a custom loss function

I want to implement deep learning model in Keras, but I want to use my own loss function, i.e. custom loss. If I implement some loss function and use Keras Functional API for the model, do I need to ...
0
votes
0answers
15 views

how can I find tuned X feature values and Minimized Y value?

I am starting to work on a smart-factory project. Now we would like to achieve as below: 1: Minimizing cost (y value) 2: getting the best tuned (optimized) value of X features. (e.g., now ...
0
votes
0answers
12 views

comparing genetic algorithm vs particle swarm optimization

I am trying out various optimization techniques. I would like to implement GA and PSO for the same optimization problem and compare them. I have found implementations for each with different examples. ...
0
votes
1answer
31 views

finding optimal solution $w$ and classification accuracy

Suppose you are given $6$ one-dimensional points: $3$ with negative labels $x_1 = −1$, $x_2 = 0$, $x_3 = 1$ and $3$ with positive labels $x_4 = −3$, $x_5 = −2$, $x_6 = 3$. In this question, we first ...
1
vote
0answers
32 views

Optimizing ad placement using historical data

Edit: increased generality. I have an ad placement optimization problem and I am brainstorming to determine which ML techniques are well suited to it. Basically, I have some objective that involves ...
0
votes
1answer
13 views

Finding sequence combinations that impact target variable the most

One can create a time series model to predict a target variable. What I need to do is find the input combinations and sequences that impact the target variable the most. In this case, the input data ...
1
vote
1answer
34 views

Gradient Descent

Can we use Gradient descent to find Global/local maxima? What types of problem needed to maximize the function? Can we do it with GD? Can anybody explain my question with an example relevant to ...
0
votes
1answer
14 views

What's the correct form of use the real coded genetic algorithm?

i'm new with the ga and i can't found specific info about the real coded ga, i want to do an antennas array optimization by using the real values of antennas position , phase and amplitude, but in my ...
0
votes
0answers
9 views

Finding optimal parameter values under higher-order effects

I have a series of computer experiments. In each experiment, I run one of two programs, each with about 6 parameters (4 of which are common to both programs; some parameters are continuous, others ...
0
votes
0answers
213 views

adding constraints in PuLP optimization problems in python? pyschedule required?

I want to create an optimal meal plan with minimum sugar intake for 7 days but the everyday diet plan should include food from 3 different categories. How do I add constraint that I could get food ...
1
vote
1answer
44 views

Can `k=1` be a good choice for K neighbors classification?

Running sklearn.KNeighborsClassifier() on Kaggle's Leaf Classification sample (set of 99 species, 10 specimen each), with defaults kNN parameters and a grid search ...
1
vote
0answers
28 views

Suggestion for choosing (building) loss funciton

I would like to build a supervised learning model M satisfying the following conditions: Training data {X, Y}, where $x \in R^m$ and $y \in R^n$ Assume: ...
0
votes
0answers
10 views

Is there any literature on how the batch-size affects the estimation of parameters of the output distribution in Variational Autoencoder?

I am working on data imputation task. I am using variational autoencoder to estimate the values of missing data. The data has real, categorical and textual features. I assume the output distribution ...
0
votes
0answers
140 views

How to make scipy.optimize.basinhopping find the global optimal point

Question Try to find the global optimal point of the function (reading Python for finance 2nd edition - Chapter 11. Mathematical Tools). ...
1
vote
0answers
35 views

GAN optimizer settings in Keras

I am working on a Generative Adversarial Network, implementing in Keras. I have my generator model, G, and discriminator D, both are being created by two functions, and then the GAN model is created ...
3
votes
1answer
24 views

Is it possible for a neural net to score as high as a different form of supervised learning?

I've been working with the Adult Census Income dataset from UCI http://archive.ics.uci.edu/ml/datasets/adult I've created two different models, one using a gradient boosted classifier with sklearn, ...
1
vote
1answer
36 views

Tuning C hyper parameter in Soft Margin SVM in Matlab

How to tune the C 'BoxConstraint' hyperparameter in soft margin SVM to get the best optimal value?
0
votes
0answers
12 views

Finding optimal region of a continuous search space

I'm working on a problem in which I've got large table of data with two features and three target columns and I need to find the optimal filter parameters which theoretically would give me: 1) most ...
1
vote
0answers
60 views
0
votes
0answers
77 views

SVM hyperparameters using Matlab's fitcsvm and OptimizeHyperparameters

I am building SVM models and will compare their performances, linear vs RBF, and I'm using OptimizeHyperparameters to get best hyperparameters C (BoxConstraints) ...
2
votes
0answers
48 views

Knowing when a GAN is overfitting (sequence classification study)

I have sequences of long, sparse 1_D vectors (3000 digits, made of of 0s and 1s) that I am trying to classify. I have previously implemented a simple CNN to classify them with relative success (with ...
1
vote
1answer
35 views

Gradient descent with infinite gradient value

Given a function $f(x)$ and $\frac{\partial f(x)}{\partial x_i}=\frac{f^2(x1,...,x_i+\pi/2,...,x_n)-f^2(x1,...,x_i-\pi/2,...,x_n)}{f(x)}$. When $f(x)\to0$, $\frac{\partial f(x)}{\partial x_i}$ could ...
1
vote
1answer
254 views

Why is my generator loss function increasing with iterations?

I'm trying to train a DC-GAN on CIFAR-10 Dataset. I'm using Binary Cross Entropy as my loss function for both discriminator and generator (appended with non-trainable discriminator). If I train using ...
3
votes
1answer
37 views

Newton method and Vanishing Gradient

I read the article on Vanishing Gradient problem, which states that the problem can be rectified by using ReLu based activation function. Now I am not able to understand that if using ReLu based ...
2
votes
1answer
51 views

Newton's method optimization for Deep Learning

I'm reading this paper "Deep learning via Hessian-free optimization" by J. Martens, I am having difficulty figure out the following statement: In the standard Newton's method, $q_{\theta}(p)$ is ...
1
vote
1answer
60 views

Understanding general approach to updating optimization function parameters

This question not related to a specific method or technique, rather there is a broader concept that I'm struggling to see clearly. Introduction In machine learning, we have loss functions that we're ...
13
votes
4answers
3k views

Is Gradient Descent central to every optimizer?

I want to know whether Gradient descent is the main algorithm used in optimizers like Adam, Adagrad, RMSProp and several other optimizers.
0
votes
1answer
42 views

How should I tackle this real-life hypermarket problem?

I registered myself in the payback program of the hypermarket I am going to. For every 2$ I get 1 point. I buy the same products every week (Feta 2.19\$, Milk 0.99$, ...). I visit only in weekdays. ...
1
vote
1answer
26 views

Optimization based on validation and not training

Hello neural network programmers, I am currently creating a neural network with keras, as I am not that familiar with tensorflow and it's a bit more difficult. I want my optimization to optimize the ...
0
votes
1answer
24 views

What makes a problem good for an evolutionary strategy vs a genetic algorithm vs particle swarm optimization?

I understand that evolutionary strategies (ES), genetic algorithms (GA), and particle swarm optimization (PSO) are all algorithms used to solve optimization types of problems, but what might make an ...
0
votes
1answer
18 views

Can we use decreasing step size to replace mini-batch in SGD?

As far as I know, mini-batch can be used to reduce the variance of the gradient, but I am also considering if we can achieve the same result if we use the decreasing step size and only single sample ...
0
votes
0answers
21 views

How to predict similarity of unseen data to the training set?

I have a time series of human pose data which are recorded from real humans. I want to train the model with unsupervised learning on the training data. Let's call this the "real" training data. The ...
0
votes
1answer
17 views

General question on the approach to optimise numbers

I have a huge huge model in SQL that nobody knows what it is doing. This model spits out some numbers and those numbers should be optimised to match another batch of 'correct' numbers as much as ...
3
votes
1answer
36 views

Can Adagrad be used to optimize non-differentiable functions?

I am reading a book (TensorFlow For Dummies, Matthew Scarpino), and here it says: Adagrad methods compute subgradients instead of gradients. A subgradient is a generalization of a gradient that ...
0
votes
0answers
22 views

How to optimize function built on top of the classifier?

I have a dataset with classification model build for it for $n$ classes as target. And also using the probabilities for each class, which classificator returns, I built confidence function for each ...