Questions tagged [optimization]

In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

Filter by
Sorted by
Tagged with
0
votes
0answers
3 views

Equation related to Smoothness

If you have a differentiable function $f:\mathbb{R}^d\rightarrow\mathbb{R}$ that is $\beta$-smooth (for all $v$ and all $w$, you have $\|\nabla f(v)-\nabla f(w)\| \leq \beta \|v-w\|$), how can you ...
0
votes
0answers
8 views

Python and CPLEX [on hold]

I am working with a project is about Software Defined networking. We consider for optimization problem. I use python programming language with mathematical optimization solver CPLEX. We use k mean ...
0
votes
0answers
7 views

Backtracking Line search for Multiclass classification gradient descent

For my case i am dealing with multiclass problem and there are total 28 direction component for each class and there are total 5 classes, for given equation above, f(w+nd) and f(w) gives scaler values ...
0
votes
0answers
8 views

How to handle optimization problem when objective function and constraints involve different set of parameters

I am working on this constrained optimization problem. The objective function is the efficiency of the machine which is determined by 6 controllable variables. The constraint is the pressure can't ...
2
votes
1answer
15 views

Quadratic approximation of L1 regularized cost function

I'm reading the Deep Learning book of Goodfellow, but I fail to see why minimization of (7.22) gives (7.23). I tried to compute the gradient w.r.t. the $w_{i}$ and set this to zero, but it doesn't ...
1
vote
1answer
23 views

Is hyperparameter tuning more affected by the input data, or by the task?

I'm working on optimizing the hyperparameters for several ML models (FFN, CNN, LSTM, BiLSTM, CNN-LSTM) at the moment, and running this alongside another experiment examining which word embeddings are ...
0
votes
0answers
8 views

Distributed optimization with a common variable among individual agents

I am new to optimization techniques and have a doubt in my approach. Consider I have an agent which tries to optimize on 3 variables C1,x11,x12 to minimize power. I have 60 such agents which ...
0
votes
1answer
22 views

What is the best way to optimize the parameters in a Sklearn classifier, when I have little data?

What is the best way to optimize the parameters in a Sklearn classifier when I only have a data set with 684 rows and 177 columns, and the column I want to predict has 3 labels? I know I should split ...
1
vote
1answer
19 views

Methodology for parallelising linked data?

If I have some form of data that can have inherent links to all other data in the set but I wish to parallelise out this data in order to increase computation time or to reduce the size of any ...
0
votes
2answers
91 views

How to use Chi-square test in dataset with negative values

I could not fully explain the title. In order to use the Chi-square test in my dataset, I am finding the smallest value and add each cell with that value. (for example, the range of data here is [-8,...
0
votes
2answers
38 views

Python lifelines - ConvergenceWarning: Newton-Raphson failed to converge sufficiently in Cox prop hazard

When calling CoxPHFitter() on my full dataset I'm getting the following error: ...
0
votes
0answers
4 views

What are the main reasons which does not cause the training error of yolov2 to not diminish?

I am using https://github.com/thtrieu/darkflow yolov2 for detecting and classifying the images of passport. There are 8 classes and all the objects are passport details like name, father name, mother ...
5
votes
2answers
51 views

Why is taking the gradient of the average error in SGD not correct, but rather the average of the gradients of single errors?

I am a little confused about taking averages in cost functions and SGD. So far I always thought in SGD you would compute the average error for a batch and then backpropagate it. But then I was told in ...
0
votes
2answers
11 views

Why does discriminiator accuracy falls to 0%, and is there a fix around this?

I am training a Vanilla-GAN(or original GAN 2016) on a pokemon dataset https://www.kaggle.com/kvpratama/pokemon-images-dataset, for few epochs the discriminator has 100% accuracy over the real ...
1
vote
1answer
44 views

scipy.optimize.minimize(method=’trust-constr’) doesn't terminate on xtol condition

I have set up an optimization problem with linear equality constraints as follows ...
0
votes
0answers
9 views

Resources on Non-Linear Programming and Optimization

With respect to the below question. https://math.stackexchange.com/questions/871370/optimizing-independent-variables-to-maximize-dependent-variable/871421#871421 I am currently building a similar ...
0
votes
0answers
11 views

How to optimize MAPE in regression algorithms

I have a regression task where the label is varying from about 0.001 to 1000. One of the feature called group, for example, group A corresponding label from 0-0.1 and group G corresponding label from ...
4
votes
1answer
190 views

Mathematical formulation of Support Vector Machines?

I'm trying to learn maths behind SVM (hard margin) but due to different forms of mathematical formulations I'm bit confused. Assume we have two sets of points $\text{(i.e. positives, negatives)}$ one ...
1
vote
1answer
29 views

RMSProp - To a layman?

Can someone explain RMSprop in layman's terms? I have tried reading various resources but they don't talk about the intuition on couple of things: 1) Why is is different from SGD with momentum? 2) ...
1
vote
0answers
8 views

Why does NAG cause unstable validation loss?

I'm building a neural network for a classification problem. When playing around with some hyperparameters, I was surprised to see that using Nesterov's Accelerated Gradient instead of vanilla SGD ...
0
votes
0answers
3 views

How calculate computation time for each part of the network

I want to report how much times it takes to compute each specific part of the network in a batch (forward and backward time). For example, in this paper they've reported RNN, softmax, and optimization ...
1
vote
2answers
26 views

Parallel hyperparameter optimization techniques?

Most hyperparameter optimization technique want to evaluate points one by one. I have an expensive optimization problem, but i can run hundreds of evaluations in parallel. The dimension of the problem ...
0
votes
0answers
14 views

Find possible fearure values to predict a certain outcome

I have a dataset about patients waiting times in a health district. The data is aggregated by health provider, type of medical procedure, urgency of the procedure (3 classes) and reports the n. of ...
1
vote
3answers
41 views

Different learning rates for each dimension

I have been thinking about why normalization and scaling are done for each feature in the basic context of gradient descent. One thing that got me wondering is that we use a pre-defined set of ...
0
votes
0answers
16 views

Optimizing Market Basket Analysis by limiting threshold

I'm creating a suggestion model through MBA. I observed that in my particular model, that if the min_support was placed as 0, the model would take an insanely long ...
1
vote
2answers
59 views

How to find the right regression model for my project

I tried several things to find the best regression model with the best parameters but i can't go higher than 40% right predictions. So i have 67741 rows in an excel file. the data looks like this ...
1
vote
1answer
50 views

What should I study to find optimal value of best feature combinations in machine learning?

I would like to do production optimization with machine learning and/or optimization problem. My goal is not to find minimizing loss in loss function only to give the best y value. My ultimate goal ...
0
votes
0answers
16 views

Techniques to increase stability of parameters

I have a highly non-linear function $f(x,y,t)$, which I am trying to maximise over different time periods $t$. It is the case that other parameters are of mixed type with $x \in Re$ and $y \in {Z}$, ...
1
vote
0answers
11 views

How can I evaluate the constant parameters of a series of linear functions with varying input parameters?

I have the following setup of linear functions: g0(x0, y0) -> (x1,y1) g1(x1, y1) -> (x2, y2) ... about 30 more cases Where gi is a function parameterized by 6 values (ai-fi). Which can be described ...
0
votes
0answers
20 views

How to optimise a boolean expression

I am working on an optimization problem involving Boolean expressions and wanted some help as I have very little knowledge about the topic. The problem statement is as follows: There are a set of ...
1
vote
1answer
18 views

How Stochastic Gradient Descent used like Mini Batch gradient descent?

As I know Gradient Descent has three variants which are: 1- Batch Gradient Descent: processes all the training examples for each iteration of gradient descent. 2- Stochastic Gradient Descent: ...
0
votes
1answer
19 views

How to find optimized x values (input features) after training in deep learning?

I did deep learning training by Keras. I have done the training part by model.fit If I do model.predict, it only gives me y value. But I want to know x (input features) that gives the best y value, ...
1
vote
2answers
30 views

Is there any advantage in using Particle Swarm Optimization for clustering than K-Means?

I have read some paper about using particle swarm optimization. It doesn't look give much different result than K-Means. I tried to use PSO for clustering but the result is pretty much the same with K-...
0
votes
1answer
46 views

Why are parameter updates downscaled by uncentered variance (instead of centered variance) in Adam optimizer?

In Adam optimizer algorithm, parameter updates are computed as follows: $\theta_t \leftarrow \theta_{t-1} - \alpha \frac{\hat{m}_t}{\sqrt{\hat{v}_t}+\epsilon}$ Where $\hat{m}_t$ is a bias-corrected ...
0
votes
0answers
16 views

Changing the performance metric used to optimize with random-forest

I'm looking to change the performance metric used to optimize the training for my data set because it is highly unbalance. Because it's highly unbalanced, I don't feel like accuracy is the appropriate ...
0
votes
0answers
13 views

Formulation of Optimization Problem in SVM

I need help in verifying/understanding a step in formulating an optimization problem used for support vector machines (though this question doesn't need any background in SVM). Consider a bunch of $m$ ...
2
votes
1answer
31 views

How to optimize the lambdas of a hybrid loss in a deep learning model

I am using a generative adversarial deep learning model (GAN) with a hybrid loss represented by a linear combination of four losses with three $\lambda$'s, something like: $total\_loss = loss_1 + \...
2
votes
2answers
50 views

Neural networks, optimization math intuition

When I look into the following partial derivative, I see it as being the key element of any optimization algorithm out there. Correct me if I'm wrong, but this gets us the slope of the loss function, ...
0
votes
1answer
131 views

Optimization of a custom loss function

I want to implement a deep learning model in Keras, but I want to use my own loss function, i.e. custom loss. If I implement some loss function and use Keras Functional API for the model, do I need to ...
0
votes
0answers
16 views

how can I find tuned X feature values and Minimized Y value?

I am starting to work on a smart-factory project. Now we would like to achieve as below: 1: Minimizing cost (y value) 2: getting the best tuned (optimized) value of X features. (e.g., now ...
0
votes
0answers
13 views

comparing genetic algorithm vs particle swarm optimization

I am trying out various optimization techniques. I would like to implement GA and PSO for the same optimization problem and compare them. I have found implementations for each with different examples. ...
0
votes
1answer
32 views

finding optimal solution $w$ and classification accuracy

Suppose you are given $6$ one-dimensional points: $3$ with negative labels $x_1 = −1$, $x_2 = 0$, $x_3 = 1$ and $3$ with positive labels $x_4 = −3$, $x_5 = −2$, $x_6 = 3$. In this question, we first ...
1
vote
0answers
33 views

Optimizing ad placement using historical data

Edit: increased generality. I have an ad placement optimization problem and I am brainstorming to determine which ML techniques are well suited to it. Basically, I have some objective that involves ...
0
votes
1answer
14 views

Finding sequence combinations that impact target variable the most

One can create a time series model to predict a target variable. What I need to do is find the input combinations and sequences that impact the target variable the most. In this case, the input data ...
2
votes
1answer
44 views

Gradient Descent

Can we use Gradient descent to find Global/local maxima? What types of problem needed to maximize the function? Can we do it with GD? Can anybody explain my question with an example relevant to ...
0
votes
1answer
15 views

What's the correct form of use the real coded genetic algorithm?

i'm new with the ga and i can't found specific info about the real coded ga, i want to do an antennas array optimization by using the real values of antennas position , phase and amplitude, but in my ...
0
votes
0answers
9 views

Finding optimal parameter values under higher-order effects

I have a series of computer experiments. In each experiment, I run one of two programs, each with about 6 parameters (4 of which are common to both programs; some parameters are continuous, others ...
0
votes
0answers
309 views

adding constraints in PuLP optimization problems in python? pyschedule required?

I want to create an optimal meal plan with minimum sugar intake for 7 days but the everyday diet plan should include food from 3 different categories. How do I add constraint that I could get food ...
1
vote
1answer
44 views

Can `k=1` be a good choice for K neighbors classification?

Running sklearn.KNeighborsClassifier() on Kaggle's Leaf Classification sample (set of 99 species, 10 specimen each), with defaults kNN parameters and a grid search ...
1
vote
0answers
29 views

Suggestion for choosing (building) loss funciton

I would like to build a supervised learning model M satisfying the following conditions: Training data {X, Y}, where $x \in R^m$ and $y \in R^n$ Assume: ...