Questions tagged [optimization]

In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

Filter by
Sorted by
Tagged with
-1
votes
0answers
18 views

What approach to solve problem of maximizing # of products sold with the constraint of maintaining an average profitability %?

I am working on a reinforcement learning model which recommends products to customers. The goal is to maximize the # of products sold, while maintaining an rolling average profitability of 2% for the ...
1
vote
0answers
12 views

Finding clusters of 2D points when the metrics changes with spatial spread of points in each cluster (i.e., clustering output)

I am trying to calculate (a given number of) clusters of 2D points. However, I can't apply any conventional algorithm I'm aware of, such as k-means, as I need the clustering metrics to depend on the ...
4
votes
1answer
42 views

How do I minimizie cost for EV charging?

I want to find a charging schedule that minimize cost of charging an EV. The main objective is to have a fully charged car for the next morning, but the sub objective is to minimize cost based these ...
1
vote
0answers
9 views

How can we optimize a model to predict in the no shortest possible time (real time production model)?

I need to put a model in production and I have some questions: How can we measure the time it takes to predict? Let's consider data is ready (real time) and we need first to transform data than to ...
2
votes
2answers
47 views

Warmup steps in deep learning

What do warm steps and warmup proportion mean? how to select the number of warmup steps? Learning rate changes for each batch or each epoch for warmup step=1 ?
0
votes
0answers
30 views

Can XGBoost support vector outputs?

I am interested in fitting data (regression rather than classification) with individual targets which are vectors via an XGBoost type model. However, currently Python's xgboost.XGBRegressor model only ...
1
vote
1answer
40 views

Why does stochastic gradient descent lead us to a minimum at all?

Why do we think that stochastic gradient descent is going to find a minimum at all? I mean on each iteration SGD moves in the direction that reduces only current batch's error (SGD doesn't care about ...
0
votes
0answers
8 views

Formula of momentum gradient descent optimizer

Learning about the optimizers recently, I was confused about the formula for momentum. I mean, I understood the concept but I came across the following 2 formulas while learning. I see that the left ...
1
vote
1answer
37 views

Numpy array vs Pandas DataFrame when training [closed]

https://towardsdatascience.com/speed-testing-pandas-vs-numpy-ffbf80070ee7 (You can open the link in incognito if its locked). Numpy arrays are faster than DataFrame on normal mathematical operations. ...
2
votes
1answer
40 views

What does "regularization" actually refer to?

I am familiar with regularization, where we add a penalty in our cost function to force the model to behave a certain way. But is this a definition of regularization? Typically we regularize to get a &...
1
vote
1answer
22 views

Understanding SVM's Lagrangian dual optimization problem

I was going through SVM section of Stanford CS229 course notes by Andrew Ng. On page 18 and 19, he explains Lagrangian and its dual: He first defines the generalized primal optimization problem: $$ \...
0
votes
0answers
28 views

Crossover Operation for 1-dimensional problems in Differential Evolution

I am using Differential Evolution (DE/best/1/bin) for optimizing a 1-dimensional function i.e. My Population has floating point values (Population size=10, hence 10 floating point numbers) and I have ...
1
vote
1answer
17 views

Understanding Lagrangian equation for SVM

I was trying to understand Lagrangian from SVM section of Andrew Ng's Stanford CS229 course notes. On page 17 and 18, he says: Given the problem $$\begin{align} min_w & \quad f(w) \\ s.t. &...
0
votes
0answers
31 views

Can Adagrad or Adam be used in loss function with l1-norm regularization?

there is one question for me. I want to know that how Adam or Adagrad treat l1-norm regularization in loss-function? (e.g. Lasso) I know that l1-norm is not differentiable function at zero but we can ...
0
votes
1answer
53 views

Understanding Lagrangian for SVM

I was referring SVM section of Andrew Ng's course notes for Stanford CS229 Machine Learning course. On page 22, he says: Lagrangian for optimization problem: $$\mathcal{L}(w,b,\alpha)=\frac{1}{2}\...
1
vote
0answers
43 views

Best way to optimize problem with additively separable fitness function?

I am using a genetic algorithm to maximize a few hundred thousand real-valued variables. Each of the variables, $x_i$, has its own independent boundary condition. The fitness function uses each of ...
1
vote
1answer
34 views

What does it mean (non) convex "constraint"?

I was referring SVM section of Andrew Ng's course notes for Stanford CS229 Machine Learning course. On page 16, he says: SVM optimization problem can be given as follows: $$\begin{align} \max_{\...
1
vote
0answers
10 views

RMSprop in weight update - what if vertical slopes small and horizontal slopes large?

I have a question regarding the intuition behind RMSprop, As shown in the lecture video of Deep Learning Specialization by Andrew Ng, RMSprop helps to reduce the oscillation (the values of the ...
2
votes
2answers
58 views

How to visualize optimization problems' feasible region?

Is there any tool to visualize the feasible region when given a set of Linear equations (equalities and inequalities). If not, can anyone suggest a way to visualize it? If I am going to do it myself ...
1
vote
0answers
7 views

Multiple Time Series Impact

I have a marketing business question where the objective is to learn from my historical data to deduce the best marketing strategy. Input (Leading Indicator)= For each year I have multiple monthly ...
1
vote
1answer
17 views

Significance of Convex Loss Function with Nonlinear Models

When used in a linear model, a convex loss function guarantees a unique global minimum for the parameters, which can be found by local optimization methods. However, when the model is nonlinear (e.g. ...
1
vote
0answers
49 views

Best way to find nearest neighbor distance for large datasets

I am a grad student doing research using generative machine learning with pytorch, and I have generated a set of points. I would like to check how similar these new points are to the points I used in ...
0
votes
0answers
40 views

Desired error not necessarily achieved due to precision loss

I am trying to maximize a function but I am getting a message of precision loss. ...
0
votes
1answer
40 views

Why is my Neural Network having constant loss and always predicting a singular value?

I am trying to make a neural network on a dataset with 257 features and 1 target variable. My code looks like the following: ...
0
votes
0answers
32 views

Gradient descent in linear regression converges but the trend line is incorrect

For the dataset https://physics.info/linear-regression/dash-world.txt, I have been trying to implement linear regression for predicting the men record times as a function of year. I have used gradient ...
0
votes
1answer
23 views

Randomforest code taking longer time every iteration

I have a prediction code that runs RandomForestRegressor and RandomForestClassifier. I call the functions 9 times each ...
0
votes
1answer
21 views

How can I extract an optimized matrix of correlations from a larger data set?

Consider an Excel sheet containing a matrix of correlations between individual stocks and the combined portfolio as a whole: How can I extract an optimized matrix such that most stocks have a low ...
0
votes
0answers
15 views

Optimizing the Loss Function For Another Metric

Suppose I have a machine learning model which is used to improve the profitability of a business. One of the components of the model is a loss function, say for measuring the success of a ...
1
vote
0answers
40 views

How to do online retraining of model on a single new data point/observation?

I am trying to investigate the effect on performance on old data and new data when a classifier is retrained on only the new observation when it is encountered. The aim is to retrain the classifier on ...
0
votes
1answer
19 views

finding winning strategy

For a given asset, I have simulations of the price and implied volatility for T periods in N scenarios. Furthermore, assuming that I know the value of the risk-free asset (and the dividend yield), I ...
0
votes
0answers
9 views

how to shape a formula pattern by optimization

I have built a formula which has a plot as shown bellow in the photo and matlab code. my formula has 8 "a" and 8 "f" coeffients which decide the shape of the plot. How can i do an ...
0
votes
0answers
61 views

Why the error between the measured data and model data is not minimizing in Python?

I want to fit the non-linear experimental data with the model function by estimating some parameters in the function. The model function I have is: ...
0
votes
2answers
45 views

Can I run this job quicker for GridSearchCV?

I am using GridSearchCV for optimising my predictions and its been 5 hours now that the process is running. I am running a fairly large dataset and I am afraid I ...
0
votes
0answers
22 views

Question on hinge loss for GANs

I'm currently experiencing some difficulty with the hinge loss optimizer for GANs. In the equation below, the discriminator is looking to minimize $L_D$ and the ...
1
vote
0answers
22 views

Machine learning with constraint on outputs

I'm experimenting with a 3D pose estimation model that predicts 3D keypoints, given 2D keypoints as input. Since the distance between these predicted joints in 3D is known and constant (we know the ...
0
votes
1answer
23 views

Memorization in deep neural networks, random vs. properly labelled datasets

From about 19:20 in the video here: https://www.youtube.com/watch?v=IHZwWFHWa-w it shows the difference in value of the cost function for randomly labelled data vs. properly labelled data. What do ...
0
votes
1answer
17 views

How does bayesian optimization with gaussian processes work?

Could someone explain in simple words what are gaussian processes how does bayesian optimization work and their combination?
1
vote
0answers
49 views

How does Keras optimization for a network with multiple outputs

I currently have a neural network that takes in 3 numbers as inputs and outputs 3 numbers. I've attached a picture of the network below and my code is accessible through the following link: [Google ...
0
votes
0answers
60 views

using Reinforcement learning for binary classification

I want to build an agent for binary classification. I have a large dataset with two label (0 and 1). I want to build an agent to predict labels. I build a deep model and now I want to build an agent. ...
0
votes
0answers
19 views

Optimise for the sum of regression predictions?

I'm building a machine learning model to forecast the number of students on a course at a University. I'm currently optimising for MAE for each sample (i.e. a ...
1
vote
1answer
29 views

Multiclass data redistribution

I want to redistribute the data in classes according to new proportions and wonder what is the optimal way to do it. For example I have ...
0
votes
0answers
30 views

GAN model with different optimization functions

Building GAN model contains the following steps: Build generator model, and choose ...
0
votes
0answers
13 views

Step size finds by quadratic fitting in steepest descent

I have a function $f=(1-x_1)^2 + (x_2-(x_1^2))^2$ and initial point $[0,5]$. I wonder how I will find step size by quadratic fitting using the (e.g. $0.01$) value in Steepest Descent with Matlab. To ...
0
votes
1answer
22 views

How can I optimise/parallelize my neural network code?

I have a neural network with 784 inputs, 30 hidden neurons and 10 output neurons. The main performance issue is when backpropagating. Currently it takes around 0.1 seconds for one iteration of ...
0
votes
0answers
114 views

Sine curve fitting

I want to fit a a * abs(sin(b*x - c)) + d function for each of the following data. In most of the cases I'm able to get decent accuracy. But for some cases, I'm not ...
1
vote
0answers
64 views

Hyper parameters (window size and vector dimensions) tuning in word2vec using Grey Wolf Optimization

Using Grey wolf Optimization, I want to calculate optimal values of two hyper parameters: context window size and embedding size (vector dimensions) for word2vec skipgram model used for word embedding....
0
votes
0answers
28 views

Determining the right box size for packaging

I was given a very challenging problem at a logistics company and I would appreciate your help. My company ships products from a clothing brand from the warehouse to final e-commerce client. The ...
0
votes
0answers
17 views

ADAM optimizer yields sub-optimal results

I've been writing my own neural network from scratch to get a better understanding of how they work (using MATLAB initially, but plan to port it to C++ afterwards). One major problem for me has been ...
0
votes
0answers
8 views

How does fusing operations lower accuracy for machine learning models?

In this talk the speaker Sachin Joglekar mentions that it's important to consider tradeoffs when choosing delegates for optimizing Tensorflow Lite. One of the tradeoffs he mentions at 10:14 is that ...
0
votes
0answers
62 views

Problem with elastic constraints in PuLP

(This is my first question on Stack Exchange) I am working on a production allocation problem, whereby sales orders have to be allocated over three production plants. I am using PuLP in Python, and ...

1
2 3 4 5
9