Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [optimization]

In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

0
votes
0answers
9 views

How to adjust deep learning parameters using Particle swarm optimization (PSO)?

As success of deep learning depends upon appropriately setting of its parameters to achieve high-quality results. The number of hidden layers and the number of neurons in each layer of a deep machine ...
1
vote
0answers
23 views

Can anyone explain how fmincg works?

In the Coursera machine learning course by Andrew Ng, fmincg is used for optimization frequently. I'm not getting the mechanism of it. Can anyone give overview of how it finds the minimum of any ...
0
votes
1answer
32 views

Linear optimization problem of $argmin$

Consider a vector $a \in R^n$. I want to know how I can find analytically the solution of the following optimization problem: $x^* = argmin_{x \in R^n} f(x)$, where $f(x) = ||x-a||_{2}^2 + \lambda ...
0
votes
0answers
13 views

Best way to optimize server runtime for multiple jobs?

I am not well versed in the data science field as I am working on devops now so I wanted to ask this community. If you can refer me to a more proper stackoverflow then let me know! Given servers and ...
0
votes
1answer
9 views

Bayesian optimisation in deeplearning

Has anyone tried using Bayesian optimisation to get best learning rates, and other hyperparameters for deeplearning. How to change the parameters between the training. Any examples on callbacks? Can ...
1
vote
1answer
21 views

Strategies for continuously assessing and improving model performance

I am building a supervised machine learning model to generate forecast. So I would have historic data like this: SKU, Month, .... other features, Actual Volume ...
0
votes
0answers
9 views

Storage optimisation of sparse matrix

I have a question about optimisation of the storage of a matrix. Let's say that I have a 3D matrix representing a cartography of a room. In this matrix, I have values representing for example the ...
0
votes
0answers
22 views

iris dataset feature selection using cuckoo search optimisation

I am trying to use a python library SwarmPackagePy to select significant features from a dataset for example iris dataset But i am stuck in dimension and iteration setting. In addition in how to use ...
1
vote
1answer
19 views

Conjugated gradient method. What is an A-matrix in case of neural networks

I am reading about conjugated gradient methods to understand how they exactly work. I understand that a pair of vector $u$ and $v$ are conjugated with respect to $A$ if $u^TAv=0$. I also read that $A$ ...
0
votes
1answer
31 views

What does it mean when someone says “Most of the data science algorithms are optimization problems”

I was trying to understand the Gradient Descent algorithm from this article and the author says Most of the data science algorithms are optimization problems I come from software engineering ...
1
vote
1answer
30 views

Is it necessary to tune the step size, when using Adam?

The Adam optimizer has four main hyperparameters. For example, looking at the Keras interface, we have: ...
1
vote
1answer
43 views

Recurrent Neural Network (LSTM) not converging during optimization

I am trying to train a RNN with text from wikipedia but I having having trouble getting the RNN to converge. I have tried increasing the batch size but it doesn't seem to be helping. All data is one ...
0
votes
1answer
22 views

error while running lasso.py

The following is the error code generated while running lasso.py. Can anybody help in fixing the same. Here is the code: ...
0
votes
0answers
13 views

Incremental Learning in Neural Networks

Are there any known (good) methods to perform incremental learning with neural network models? I know we can use transfer learning to update a trained model for a different task, or perform several ...
0
votes
0answers
28 views

Swap memory size increases when running multiple Keras models

I have the following Python code with Keras ...
1
vote
0answers
11 views

What is the simplest optimization algorithm for a multi-parameter closed system?

I'm working on a minimization problem for a wireless communication link. I want to minimize the total bit error rate (BER) of a closed system. My problem is that the signal has multiple (50) ...
-1
votes
0answers
18 views

What optimization algorithm is more suitable for timetable rescheduling problem?

I'm working on the project where university course is represented as a to-do list, where: course owner (teacher of the course) can add tasks (containing the URL to the resource needs to learn and two ...
1
vote
1answer
25 views

Does policy optimization learn policies to make better actions with higher probability? [closed]

When I talk about policy optimization, it is referred to the following picture, and it is linked to DFO/Evolution plus Policy Gradients. I would like to know is it correct to say: Policy ...
0
votes
0answers
19 views

Per parameter learning rate for AdamOptimizer by scaling gradients

I'm using an AdamOptimizer, and I compute the gradients, but before applying the descent step, I scale (i.e.: multiply) gradients with constants, to mimic having a different learning rate per ...
0
votes
1answer
23 views

Minimization algorithm that can consider gradient close to solution

I want to minimize a function which has sharp gradients close to each local minimum. Due to process tolerances, I want to find solutions which meet some minimum criterion (e.g. lower than x), but have ...
0
votes
1answer
90 views

Should the minimum value of a cost (loss) function be equal to zero?

We know optimization techniques search in the space of all the possible parameters for a parameter set that minimizes the cost function of the model. The most well-known loss functions, like MSE or ...
1
vote
1answer
18 views

Are there any learner-specific optimizers?

In reading about machine learning (ML), and working through some basic examples, it appears to me most learning algorithms use generic optimizers. I am using the word "optimizer" to describe the ...
2
votes
1answer
34 views

How to derive the sum-of squares error function formula?

I'm attending a Machine Learning course and I'm studying linear models for classification right now. Slides present approaches to learn linear discriminants (Least squares, Fisher's linear ...
0
votes
1answer
42 views

How to Define a Cost Fucntion?

I want to define a cost function in python to identify optimum value in days when i should end a marketing campaign to save spend on campaigns not generating traffic good traffic. Problem is I dont ...
1
vote
0answers
12 views

How to optimize the separation of two distributions from binary classfication

Given a sample where for each individual a classification is predetermined (e.g. sick or not) and 5 random variables are measured. The random variables are on the same scale but from differnt bins. E....
0
votes
0answers
20 views

Primal vs Dual SVM Problem for linearly separable data

Given a linearly separable data where i don't need to use kernels. Is there any need to use the dual form? In other words, is primal form enough?
0
votes
0answers
12 views

using regression model to optimize teams working on work items

I have a few work items with these features: WI1, WI2, WI3 which describe these work items. I also know the number of people and how many minutes they spend each ...
1
vote
1answer
24 views

AI that maximizes the storage of rectangular parallelepipeds in a bigger parallelepiped

As you can see in the title, I'm trying to program an AI in Java that would help someone optimize his storage. The user has to enter the size of his storage space (a box, a room, a warehouse etc...) ...
0
votes
0answers
28 views

how to calculate number of frames in epsilon greedy with decay rate?

If starting epsilon is alpha and end epsilon is beta in epsilon greedy algorithm. discount rate is gamma and epsilon decay is lambda, how to calculate the F: number of frames to reach from alpha to ...
0
votes
0answers
55 views

What are the reasons of select a optimizer to be SGD or Adam in DQN?Why?

I saw several comparison between SGD, RMSPROP and ADAM but what I am looking for is their comparsion in DQN algorithm? What is best to select as optimizer SGD or Adam in DQN? Why? Please check the ...
0
votes
1answer
102 views

Which are the latest Optimization techniques in artificial intelligence?

My project work is optimization in power system using artificial intelligence (like fault location and classification,load forecasting and context awareness and IoT etc ) and I have used PSO (particle ...
2
votes
0answers
57 views

clipping the reward for adam optimizer in keras

I would like to clip the reward in keras. I saw it is possible to clip the norm and clip the value is sgd as follows: ...
2
votes
1answer
13 views

Algorithm for campaign optimization (Digital Advertising)

Suppose i am running an Ad thru an Ad exchange A, and i have a set of campaigns running on it. I have The spend of the campaign. The budget allocated to it. The number of hours it took to exhaust it'...
1
vote
1answer
16 views

has number of output layer of DNN any effect in speed of find the optimal answer of DNN?

has number of output layer of DNN any effect in speed of find the optimal answer of DNN? For instance the more episodes is needed to train a DNN when the number of outputs is more? Is it correct?
1
vote
0answers
95 views

Policy gradient: why does this converge with Adam and not SGD?

I am looking into policy gradient methods. I stumbled into this implementation: https://gist.github.com/calclavia/cfcd41ad4e47d7b9b6ab8af15410747a It uses a Nesterov Adam optimizer. If I run it, it ...
8
votes
2answers
103 views

Why aren't Genetic Algorithms used for optimizing neural networks?

From my understanding, Genetic Algorithms are powerful tools for multi-objective optimization. Furthermore, training Neural Networks (especially deep ones) is hard and has many issues (non-convex ...
0
votes
1answer
40 views

What is new population in genetic algorithm?

Here is my (mis?)understanding of genetic algorithm: Create n individuals. This is initial population Calculate fitness of each individual in this population ...
3
votes
1answer
48 views

Why imbalanced data-set will bias the prediction model towards the more common class?

As we know, an imbalanced data-set has a disadvantage of training a model for deep learning. However, I don't know how to explain it with mathematics?
2
votes
3answers
97 views

Small dataset in Time series

I have soccer data with a time series index. 30 seconds interval. So, 194 rows for 90+ minutes per game. I have 1500 games. The dataframe has the following information. Home/Away: • Goal Total. • ...
4
votes
2answers
77 views

What is the class of this optimization problem?

I have the following optimization problem: Find $\mathbf{w}$ such that the following error measure is minimised: $E_u = \dfrac{1}{N_u}\sum_{i=0}^{N_u-1}\lVert \mathbf{w}^Tx(t_{i+1})-\mathbf{F}(\{\...
1
vote
0answers
31 views

RMSProp Optimizer Performing Poorly

I am building an RNN and have decided to try RMSProp as an alternative to sgd. Here is my implementation: ...
1
vote
0answers
31 views

Mean-variance mapping optimization (MVMO) in R

Someone know tell me if there is any package in R about Mean-variance mapping optimization (MVMO) algorithm? I already researched, but don't find anything about this.
0
votes
0answers
34 views

Train the model to increase accuracy rather than to minimise loss

I am currently in a situation of seq2seq training where the cross entropy loss is very low (near zero) but the accuracy is also very low. This made me wondering if there were any loss functions that ...
2
votes
1answer
42 views

Optimising Expensive Functions

I'm trying some different techniques to optimise a Boosted Gradient Regressor by using an evolutionary programming technique to try and find the most efficient set of features. So far I've been having ...
2
votes
2answers
67 views

What is the logic of the epoch?

What is the logic of the epoch? for example, 1 time, 2 times ... I just do not know what else is working to give better results than I know.
1
vote
1answer
25 views

Where can we find the application of bayes's theorem in Bayesian optimiation with gaussian processing

I am trying to learn bayesian optimisation by following this tutorial. However, until now I don't get the relation between bayes's theorem to the gaussian process formalism. Any ideas?
0
votes
0answers
7 views

Implementation of Fisher's extact test in Scikit-learn

How to implement efficiently Fisher's extact test in Scikit-learn to use it with SelectKBest in an optimal way ?
0
votes
0answers
15 views

How to optimize stacking?

I'm wondering if there is a way to find the optimal weights when stacking multiple models? For example if I have five models which perform similarly, how do I find if I should discard some altogether, ...
3
votes
1answer
61 views

Minimizing an upper bound of objective function

In many machine learning problems, you often have an objective function (e.g., cost, loss, error) that you want to minimize. Instead of directly minimizing this objective function, I sometimes see ...
1
vote
0answers
212 views

Optimizing an averaged perceptron algorithm using numpy and scipy instead of dictionaries

So I'm trying to write an averaged perceptron algorithm (page 48 here for the equation) in python. Instead of storing the historical weights, I simply accumulate the weights and then multiply ...