Skip to main content

Questions tagged [optimization]

In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

Filter by
Sorted by
Tagged with
0 votes
2 answers
188 views

Can we use decreasing step size to replace mini-batch in SGD?

As far as I know, mini-batch can be used to reduce the variance of the gradient, but I am also considering if we can achieve the same result if we use the decreasing step size and only single sample ...
0 votes
1 answer
19 views

Scheduling Production on One Machine with Changeover Costs and due dates

I'm trying to develop a solution to find a local optimum to a combination of manufacturing orders. They have a changeover cost per type, this means that the change between a type 2 order and type 3 ...
1 vote
2 answers
115 views

Reinforcement Learning : Why acting greedily with the optimal value function gives you the optimal policy?

The course of David Silver about Reinforcement Learning explains how you get the optimal policy from the optimal value function. It seems to be very simple, you just have to act greedily, by ...
0 votes
1 answer
176 views

Should deep layers ever have more units than the input layer?

i.e. if a model, with 10 inputs, say,: ...
1 vote
2 answers
466 views

error while running lasso.py

The following is the error code generated while running lasso.py. Can anybody help in fixing the same. Here is the code: ...
3 votes
1 answer
135 views

Reinforcement Learning applied to Optimisation Problem

Problem Statement: We are given an optimisation problem; with production centres, source airport, destination airports, transfer points and finally delivered to the customers. This is better explained ...
0 votes
1 answer
145 views

Machine learning with constraints on features

I am working on a learning to rank problem. I have queries and documents related to every query which I have to rank. I used lightgbm ranker to fit the model. Some of features are very important and ...
0 votes
1 answer
129 views

Marketing Spend Optimization Techniques

I need some help with market spend optimization. I’m working with a client who’s running an offline operation that’s primarily driven by online marketing (fb, google, twitter etc). They had asked me ...
5 votes
2 answers
226 views

Why is each successive tree in GBM fit on the negative gradient of the loss function?

Page 359 of Elements Of Statistical Learning 2nd edition says the below. Can someone explain the intuition & simplify it in layman terms? Questions What is the reason/intuition & math ...
0 votes
1 answer
24 views

Optimal combination of variables to minimise output

To be honest I'm not 100% sure how much this is purely a coding issue or a data science issue, but I'll take my chances. I've developed a matrix which is a mixture of various hyperparameters, the ...
3 votes
1 answer
544 views

Two steps optimization of a credit card limit

I have a problem similar to what is on the title but not the same. The problem on the title allows me to explain the dynamics of my need. I have to determine what the optimal value is for a variable ...
0 votes
1 answer
117 views

"Invalid value" in RMSprop implementation from scratch in Python

Edit 2: The regularization term (reg_term) is sometimes negative due negatative parameters. Hence S[f"dW{l}"] contains some negative values. I realize the reg_term has to be added before ...
6 votes
2 answers
244 views

How sklearn SVM find the initial hyperplane before Optimisation?

The optimization goal of the SVM is to maximize the distance between the positive and negative hyperplanes. But before optimizing, how does sklearn first find the positive and negative support vectors ...
1 vote
1 answer
237 views

GAN optimizer settings in Keras

I am working on a Generative Adversarial Network, implementing in Keras. I have my generator model, G, and discriminator D, both are being created by two functions, and then the GAN model is created ...
3 votes
1 answer
320 views

Support Vector Machines with soft margin: solving the dual form

I am currently struggling with finding an analytical solution for the $\alpha_k$. I have derived the following constrained optimization problem: $$ L = \sum_{i=1}^{N} \alpha_i - \frac{1}{2} \sum_{i=1}^...
2 votes
1 answer
79 views

Software for reweighted L1 minmization?

I am trying to solve a sparsity-promoting optimization problem. It is well known that the L1 norm is a good surrogate to the L0 norm, and it is studied in (Candes et al, 2008: Enhancing sparsity by ...
0 votes
1 answer
124 views

Optimizing decision tree

I have a question regarding the technique/technology which could be applied for the issue: Suppose I have a rule-based tree or decision tree which predicts a variable Y based on variables A,B,C. This ...
2 votes
1 answer
2k views

Setting BATCH SIZE when performing multi-class classification with imbalanced dataset

I have a question regarding BATCH_SIZE on multi-class classification task with imbalanced data. I have 5 classes and a small dataset of around ...
1 vote
1 answer
132 views

Algorithm for rule set optimization

I have hand writed classifiers (there are a lot of them). It's implemented as collection of rule sets IIF - THEN. I want to optimize the % of errors. There some ...
2 votes
1 answer
115 views

Regression problem - too complex for gradient descent

I try to predict temperatures values as function of time and different parameters. The temperature curve look like a "ramp" with some "gauss peaks" on regular intervals. So, I try to build a ...
3 votes
2 answers
184 views

Machine learning model with simultaneous function optimization

Consider the following scenario. I am a sculpturer and customers ask me for what price I am willing to provide them with some statues. Their request for sculptures can vary in difficulty, quantity, ...
1 vote
1 answer
475 views

Computing variance of an SGD iteration

It is known that SGD iteration has huge variance. Given the iteration update: $$ w^{k+1} := w^k - \underbrace{\alpha \ g_i(w^k)}_{p^k}, $$ where $w$ are model weights and $g_i(w^k)$ is gradient of ...
3 votes
3 answers
540 views

How to find optimized x values (input features) after training in deep learning?

I did deep learning training by Keras. I have done the training part by model.fit If I do model.predict, it only gives me y value. But I want to know x (input features) that gives the best y value, ...
1 vote
1 answer
242 views

Does severe multicollinearity affect solving linear regression by gradient descent?

Since OLS may fail when there is severe/near perfect multicollinearity, how would gradient descent perform in such a scenario? Does it converge at the minima? (My guess is, Cost function of linear ...
1 vote
1 answer
62 views

Grouping numeric data into efficient group/pool

I have devices assigned with different data plan. But based on device behaviour amount of data used by device changes during the month. I need to put the device into appropriate data plan based on ...
0 votes
1 answer
78 views

Multiprocessing data loading in colab

I want to convert mozilla common voice dataset from mp3 to wav. But this dataset is large and convertion takes many time. How can I make this convertion in colab with multiprocessing to decrease time ...
0 votes
1 answer
124 views

Gradient descent around optimal loss surface

All the loss surface used in examples have some of bowl shape that decrease drastically far from the optimal and decrease slowly around the optimal flat point. My questions are: Has all the loss ...
0 votes
0 answers
23 views

Using Reinforcement learning for minimisation

I would like to use reinforcement learning for the optimisation of a given function under some contraints. Take for example the following problems: ...
2 votes
1 answer
3k views

Why does Faster R-CNN use SGD optimizer instead of Adam?

I just start learning Faster R-CNN and I have some doubts about the optimizer of this network. In my understanding, Adam optimizer performs much better than SGD in a lot of networks. However, the ...
0 votes
1 answer
59 views

What is "SwarmPackagePy.cso.cso at 0x187cf21e340"

...
1 vote
1 answer
63 views

Calculate the top 5 optimal parcel locker cabinet configurations

Dear Data Science community, I have the following problem to solve and I'd like to learn which algorithm or approach I can use to tackle it. I don't expect a full solution here but I really want to ...
2 votes
1 answer
562 views

Reversing the input and output of an ML algorithm to Optimize

My dataset consists of multiple input variables (X) and multiple output variables (Y). For example: ...
1 vote
1 answer
42 views

How to comment on goodness of loss functions?

I have two loss functions $\mathcal{L}_1$ and $\mathcal{L}_2$ to train my model. The model is predominantly a classification model. Both $\mathcal{L}_1$ and $\mathcal{L}_2$ takes are two variants of ...
1 vote
1 answer
261 views

Visualize n-dimensional bayesian optimization results

I am working on a 6-dimensional bayesian optimization problem using (skopt's gp_minimize). After the optimizer ran for j iterations I would like to somehow visualize the "progress/result" of ...
1 vote
1 answer
99 views

Good chromosome representation in a VRPTW genetic algorithm

I have a genetic algorithm for a vehicle routing problem with time windows and I need to implement certain modifications. I am not sure what would be the best chromosome representations. I have tasks ...
0 votes
1 answer
197 views

Optimum weights for weighted average of 3 prediction models

I have 3 sklearn models which I use to predict a probability score for a binary classification problem. I want to create a weighted average score of all the predictions made by these models. I am ...
1 vote
3 answers
260 views

Why do neural networks with more layers perform better?

Why do neural networks with more layers perform better than a single layer MLP with a number of neurons that leads to the same number of parameters? I read this post: https://www.quora.com/Why-do-...
1 vote
2 answers
137 views

Finding a vector that minimize the MSE of its linear combination

I have been doing a COVID-19 related project. Here is the question: N = vector of daily new infected cases D = vector of daily deaths E[D] = estimation of daily deaths N is a n-dimensional vector, n ...
2 votes
1 answer
156 views

Why do we only care about convex functions when doing Gradient Descent/SGD?

I mean I know why we specifically care about convex functions: it's because their local minimum are also global, and so you just have to "follow a path which goes down" to find the minima of ...
0 votes
1 answer
134 views

What's the correct form of use the real coded genetic algorithm?

I'm new to genetic algorithms, but I haven't found specific info about real-coded GA's. I want to do antenna array optimization by using the real values of antenna position, phase, and amplitude, but ...
3 votes
1 answer
65 views

How to optimize client's portafolio with analytical models?

I have a model in which we want to optimize the probability of an outcome depending on a election of some product (a personalized product for every client amongst three posibilities). The product is ...
1 vote
0 answers
3k views

adding constraints in PuLP optimization problems in python? pyschedule required?

I want to create an optimal meal plan with minimum sugar intake for 7 days but the everyday diet plan should include food from 3 different categories. How do I add constraint that I could get food ...
2 votes
2 answers
1k views

Does settings $\beta_1 = 0$ or $\beta_2 = 0$ means that ADAM behaves as RMSprop or Momentum?

I read on ADAM optimizer, and I saw multiple quotes which say that ADAM is a combination of Momentum and RMSprop optimizers. So if we: Set $\beta_1 = 0$ does it means that ADAM behaves exactly as ...
2 votes
1 answer
697 views

Deep learning test loss curve won't go down

I've been working with Deep Learning projects for this current project that I am working on and it's basically a time series classification problem. Where given an array of time series data I need to ...
0 votes
1 answer
570 views

Randomforest code taking longer time every iteration

I have a prediction code that runs RandomForestRegressor and RandomForestClassifier. I call the functions 9 times each ...
3 votes
1 answer
710 views

Why are parameter updates downscaled by uncentered variance (instead of centered variance) in Adam optimizer?

In Adam optimizer algorithm, parameter updates are computed as follows: $\theta_t \leftarrow \theta_{t-1} - \alpha \frac{\hat{m}_t}{\sqrt{\hat{v}_t}+\epsilon}$ Where $\hat{m}_t$ is a bias-corrected ...
1 vote
1 answer
99 views

AdaGrad: Intuition

The update formula for Adagrad is: \begin{equation} w^i(t)=w^i(t-1) -\frac{\eta}{\sqrt{\epsilon +\sum_{1}^t |\nabla_i\mathcal{L}}|^2} \nabla_i\mathcal{L} \end{equation} It indicates that if the ...
1 vote
1 answer
241 views

Finding sequence combinations that impact target variable the most

One can create a time series model to predict a target variable. What I need to do is find the input combinations and sequences that impact the target variable the most. In this case, the input data ...
2 votes
1 answer
352 views

When is Non-Stochastic Global Optimization Preferable or Necessary?

Background I'm specifically referring to non-convex black-box optimization problems of the form: $ \text{min} f(\vec{x})$ $s.t. \ \ a_i\le x_i \le b_i \ \forall i\in \{1,2,...,n\} \ \ \ \text{and}\ \ ...
7 votes
2 answers
3k views

Why does degradation occur in deep neural networks?

It has been shown that "plain" neural networks tend to have an increased amount training error, and accompanied test error, as more layers are added. I am not quite certain as to why this occurs. In ...

1
2 3 4 5
10