# Questions tagged [optimization]

In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

427 questions
Filter by
Sorted by
Tagged with
18 views

### What approach to solve problem of maximizing # of products sold with the constraint of maintaining an average profitability %?

I am working on a reinforcement learning model which recommends products to customers. The goal is to maximize the # of products sold, while maintaining an rolling average profitability of 2% for the ...
12 views

### Finding clusters of 2D points when the metrics changes with spatial spread of points in each cluster (i.e., clustering output)

I am trying to calculate (a given number of) clusters of 2D points. However, I can't apply any conventional algorithm I'm aware of, such as k-means, as I need the clustering metrics to depend on the ...
42 views

### How do I minimizie cost for EV charging?

I want to find a charging schedule that minimize cost of charging an EV. The main objective is to have a fully charged car for the next morning, but the sub objective is to minimize cost based these ...
9 views

### How can we optimize a model to predict in the no shortest possible time (real time production model)?

I need to put a model in production and I have some questions: How can we measure the time it takes to predict? Let's consider data is ready (real time) and we need first to transform data than to ...
47 views

### Warmup steps in deep learning

What do warm steps and warmup proportion mean? how to select the number of warmup steps? Learning rate changes for each batch or each epoch for warmup step=1 ?
30 views

### Can XGBoost support vector outputs?

I am interested in fitting data (regression rather than classification) with individual targets which are vectors via an XGBoost type model. However, currently Python's xgboost.XGBRegressor model only ...
40 views

### Why does stochastic gradient descent lead us to a minimum at all?

Why do we think that stochastic gradient descent is going to find a minimum at all? I mean on each iteration SGD moves in the direction that reduces only current batch's error (SGD doesn't care about ...
8 views

### Formula of momentum gradient descent optimizer

Learning about the optimizers recently, I was confused about the formula for momentum. I mean, I understood the concept but I came across the following 2 formulas while learning. I see that the left ...
37 views

### Numpy array vs Pandas DataFrame when training [closed]

https://towardsdatascience.com/speed-testing-pandas-vs-numpy-ffbf80070ee7 (You can open the link in incognito if its locked). Numpy arrays are faster than DataFrame on normal mathematical operations. ...
40 views

### What does "regularization" actually refer to?

I am familiar with regularization, where we add a penalty in our cost function to force the model to behave a certain way. But is this a definition of regularization? Typically we regularize to get a &...
22 views

I was going through SVM section of Stanford CS229 course notes by Andrew Ng. On page 18 and 19, he explains Lagrangian and its dual: He first defines the generalized primal optimization problem: $$\... 0answers 28 views ### Crossover Operation for 1-dimensional problems in Differential Evolution I am using Differential Evolution (DE/best/1/bin) for optimizing a 1-dimensional function i.e. My Population has floating point values (Population size=10, hence 10 floating point numbers) and I have ... 1answer 17 views ### Understanding Lagrangian equation for SVM I was trying to understand Lagrangian from SVM section of Andrew Ng's Stanford CS229 course notes. On page 17 and 18, he says: Given the problem$$\begin{align} min_w & \quad f(w) \\ s.t. &...
31 views

there is one question for me. I want to know that how Adam or Adagrad treat l1-norm regularization in loss-function? (e.g. Lasso) I know that l1-norm is not differentiable function at zero but we can ...
53 views

10 views

### RMSprop in weight update - what if vertical slopes small and horizontal slopes large?

I have a question regarding the intuition behind RMSprop, As shown in the lecture video of Deep Learning Specialization by Andrew Ng, RMSprop helps to reduce the oscillation (the values of the ...
58 views

### How to visualize optimization problems' feasible region?

Is there any tool to visualize the feasible region when given a set of Linear equations (equalities and inequalities). If not, can anyone suggest a way to visualize it? If I am going to do it myself ...
7 views

### Multiple Time Series Impact

I have a marketing business question where the objective is to learn from my historical data to deduce the best marketing strategy. Input (Leading Indicator)= For each year I have multiple monthly ...
17 views

### Significance of Convex Loss Function with Nonlinear Models

When used in a linear model, a convex loss function guarantees a unique global minimum for the parameters, which can be found by local optimization methods. However, when the model is nonlinear (e.g. ...
49 views

### Best way to find nearest neighbor distance for large datasets

I am a grad student doing research using generative machine learning with pytorch, and I have generated a set of points. I would like to check how similar these new points are to the points I used in ...
40 views

### Desired error not necessarily achieved due to precision loss

I am trying to maximize a function but I am getting a message of precision loss. ...
40 views

### Why is my Neural Network having constant loss and always predicting a singular value?

I am trying to make a neural network on a dataset with 257 features and 1 target variable. My code looks like the following: ...
32 views

### Gradient descent in linear regression converges but the trend line is incorrect

For the dataset https://physics.info/linear-regression/dash-world.txt, I have been trying to implement linear regression for predicting the men record times as a function of year. I have used gradient ...
23 views

### Randomforest code taking longer time every iteration

I have a prediction code that runs RandomForestRegressor and RandomForestClassifier. I call the functions 9 times each ...
21 views

### How can I extract an optimized matrix of correlations from a larger data set?

Consider an Excel sheet containing a matrix of correlations between individual stocks and the combined portfolio as a whole: How can I extract an optimized matrix such that most stocks have a low ...
15 views

### Optimizing the Loss Function For Another Metric

Suppose I have a machine learning model which is used to improve the profitability of a business. One of the components of the model is a loss function, say for measuring the success of a ...
40 views

### How to do online retraining of model on a single new data point/observation?

I am trying to investigate the effect on performance on old data and new data when a classifier is retrained on only the new observation when it is encountered. The aim is to retrain the classifier on ...
19 views

### finding winning strategy

For a given asset, I have simulations of the price and implied volatility for T periods in N scenarios. Furthermore, assuming that I know the value of the risk-free asset (and the dividend yield), I ...
9 views

### how to shape a formula pattern by optimization

I have built a formula which has a plot as shown bellow in the photo and matlab code. my formula has 8 "a" and 8 "f" coeffients which decide the shape of the plot. How can i do an ...
61 views

### Why the error between the measured data and model data is not minimizing in Python?

I want to fit the non-linear experimental data with the model function by estimating some parameters in the function. The model function I have is: ...
45 views

### Can I run this job quicker for GridSearchCV?

I am using GridSearchCV for optimising my predictions and its been 5 hours now that the process is running. I am running a fairly large dataset and I am afraid I ...
22 views

### Question on hinge loss for GANs

I'm currently experiencing some difficulty with the hinge loss optimizer for GANs. In the equation below, the discriminator is looking to minimize $L_D$ and the ...
22 views

### Machine learning with constraint on outputs

I'm experimenting with a 3D pose estimation model that predicts 3D keypoints, given 2D keypoints as input. Since the distance between these predicted joints in 3D is known and constant (we know the ...
23 views

### Memorization in deep neural networks, random vs. properly labelled datasets

From about 19:20 in the video here: https://www.youtube.com/watch?v=IHZwWFHWa-w it shows the difference in value of the cost function for randomly labelled data vs. properly labelled data. What do ...
17 views

### How does bayesian optimization with gaussian processes work?

Could someone explain in simple words what are gaussian processes how does bayesian optimization work and their combination?
49 views

### How does Keras optimization for a network with multiple outputs

I currently have a neural network that takes in 3 numbers as inputs and outputs 3 numbers. I've attached a picture of the network below and my code is accessible through the following link: [Google ...
60 views

### using Reinforcement learning for binary classification

I want to build an agent for binary classification. I have a large dataset with two label (0 and 1). I want to build an agent to predict labels. I build a deep model and now I want to build an agent. ...
19 views

### Optimise for the sum of regression predictions?

I'm building a machine learning model to forecast the number of students on a course at a University. I'm currently optimising for MAE for each sample (i.e. a ...
29 views

### Multiclass data redistribution

I want to redistribute the data in classes according to new proportions and wonder what is the optimal way to do it. For example I have ...
30 views

### GAN model with different optimization functions

Building GAN model contains the following steps: Build generator model, and choose ...
13 views

### Step size finds by quadratic fitting in steepest descent

I have a function $f=(1-x_1)^2 + (x_2-(x_1^2))^2$ and initial point $[0,5]$. I wonder how I will find step size by quadratic fitting using the (e.g. $0.01$) value in Steepest Descent with Matlab. To ...
22 views

### How can I optimise/parallelize my neural network code?

I have a neural network with 784 inputs, 30 hidden neurons and 10 output neurons. The main performance issue is when backpropagating. Currently it takes around 0.1 seconds for one iteration of ...
114 views

### Sine curve fitting

I want to fit a a * abs(sin(b*x - c)) + d function for each of the following data. In most of the cases I'm able to get decent accuracy. But for some cases, I'm not ...
64 views

### Hyper parameters (window size and vector dimensions) tuning in word2vec using Grey Wolf Optimization

Using Grey wolf Optimization, I want to calculate optimal values of two hyper parameters: context window size and embedding size (vector dimensions) for word2vec skipgram model used for word embedding....
28 views

### Determining the right box size for packaging

I was given a very challenging problem at a logistics company and I would appreciate your help. My company ships products from a clothing brand from the warehouse to final e-commerce client. The ...
17 views

### ADAM optimizer yields sub-optimal results

I've been writing my own neural network from scratch to get a better understanding of how they work (using MATLAB initially, but plan to port it to C++ afterwards). One major problem for me has been ...