Questions tagged [optimization]

In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

Filter by
Sorted by
Tagged with
0
votes
0answers
4 views

Basket items optimisation minimising constraints

I have a real problem (not home work) when I have to distribute an ordered list by position to respect some constraints eg. 1. 11 2. 15 3. 18 4. 18 5. 1 baskets:...
0
votes
1answer
21 views

Computing variance of an SGD iteration

It is known that SGD iteration has huge variance. Given the iteration update: $$ w^{k+1} := w^k - \underbrace{\alpha \ g_i(w^k)}_{p^k}, $$ where $w$ are model weights and $g_i(w^k)$ is gradient of ...
1
vote
1answer
14 views

Is Neural Network Architecture independent of Data?

If I change my dataset (let's say it is always images), should I change the architecture of my neural network?
0
votes
0answers
6 views

what does “Tree” refer to in Tree-structured Parzen Estimators

I am going through the literature of Hyperparameter optimization techniques and came across TPE. There is very little to no explanation on why the name has "Tree" in it. What is Tree referring to? and ...
0
votes
0answers
6 views

Does severe multicollinearity affect solving linear regression by gradient descent?

Since OLS may fail when there is severe/near perfect multicollinearity, how would gradient descent perform in such a scenario? Does it converge at the minima? (My guess is, Cost function of linear ...
1
vote
1answer
21 views

How does the construction of a decision tree differ for different optimization metrics?

I understand how a decision tree is constructed (in the ID3 algorithm) using criterion like entropy, gini index, variance reduction. But the formulae for these criteria do not care about optimization ...
0
votes
0answers
8 views

Need ideas on how to find the best email send frequency go get maximum desired action

I have got data which contains email id total times email has been send to him (frequerny) and desired action taken (read, clicked etc), non desired action taken(unsubscribed, etc), also count of no ...
0
votes
1answer
29 views

How to Minimize mean square error using Python

I want to minimise mean square error function to find best alpha value (decay rate) for my model. Here is the description of my model: ...
0
votes
0answers
7 views

LSTM Regression TS with >50% of zero outcomes for y

Modelling multivariate Time series with LSTM, the y of the TS are on >50% consist of zeros. the same is true for features I use loss = 'mean_squared_error', optimizer =Adam, and make grid search ...
1
vote
1answer
21 views

Differential Evolution optimal tolerance parameter

I am trying to optimize the parameters of a global optimization system for my set of data, because I will have a bunch of similar data to process so I need to fine tune the global optimizator so that ...
2
votes
1answer
37 views

How many times is backprop used in epoch?

As I understand for the algorithms that use gradient descent we have to pass data to the algorithms multiple times so that the optimum is found. So one epoch means that the forward-backprop (and ...
-3
votes
1answer
36 views

Solve an equation using machine learning [closed]

Imagine we have the following equation: y=xz. We have y but not other ones. Note that y is like a matrix and we could as many sample we want. It is the values obtained from sensors. This means it ...
0
votes
0answers
17 views

Reinforcement Learning : Why acting greedily with the optimal value function gives you the optimal policy?

The course of David Silver about Reinforcement Learning explains how you get the optimal policy from the optimal value function. It seems to be very simple, you just have to act greedily, by ...
0
votes
1answer
17 views

ADAM algorithm for multilayer neural network

I’m trying to touch neural networks without using “in box” algorithms. And so I found out that nowhere is written how to calculate square of gradient for hidden layers in ADAM optimizer. I took the ...
2
votes
0answers
30 views

How to optimize client's portafolio with analytical models?

I have a model in which we want to optimize the probability of an outcome depending on a election of some product (a personalized product for every client amongst three posibilities). The product is ...
0
votes
0answers
12 views

Any good resources for Operational Research and optimization problems

looking for good learning resources for Operational Research and optimization problems(specifically some case studies). If i could get something on digital media optimization then it would be ...
1
vote
0answers
42 views

Can we optimize heterogeneous parameters of RBF Network using Gradient Descent?

There're three parameters in the Radial Basis Function Networks (RBFN). Centers of RBFs Width of RBFs Weights of RBFs It's a fact that Weights can be easily updated using a simple Gradient Descent. ...
1
vote
1answer
43 views

Salesman problem with additional conditions and features

I'd like to specify the kind of a problem I encountered. I need to compose the best route for a car driver who goes to different cities. His aim to check in the most proper car park in a city taking ...
3
votes
3answers
105 views

Constructing function - f(x,y) for the given minimums (Python) [closed]

Problem Statement: I need to construct a function f(x,y) in which there're 3 minimums. 2 local and 1 global which are written below. Locals are: z = f(0.2,0.3) = 0.7 | z = f(0.6,0.8) = 0.8 ...
0
votes
1answer
63 views

What applications does linear programming have in data science?

I'm currently learning about linear programming in my degree. I'm wondering how this is relevant to anything in data science?
5
votes
1answer
42 views

Can I completely cancel the effects of using a smaller batch size by reducing the learning rate?

I'm having the problem that the data from a regular sized batch (e.g., 32, 64) doesn't fit in my GPU. Among other solutions, I'm considering reducing the batch size, as is normally suggested. Of ...
2
votes
1answer
129 views

Is it possible to make F1_Score differentiable and use it directly as a Loss function?

One of the metrics that is widely used in binary classification is the F1 score: $F_1 = 2\cdot \frac{recall \cdot precision}{recall+precision}$ The problem of the F1-score is that it is not ...
1
vote
0answers
15 views

How to build Explanatory Graph for Convolutional Neural Network?

I m reading very interesting paper (https://arxiv.org/pdf/1812.07997.pdf) that aims to interpret convolutional neural network using graph. The general idea is when there are co-related parts in layers ...
2
votes
1answer
51 views

Machine learning model with simultaneous function optimization

Consider the following scenario. I am a sculpturer and customers ask me for what price I am willing to provide them with some statues. Their request for sculptures can vary in difficulty, quantity, ...
2
votes
0answers
26 views

Parameter optimization and selection in dynamic neural networks

I have used a Bayesian optimization to tune machine learning parameters. The optimized parameters are "Hidden layer size" and "...
2
votes
1answer
78 views

Hyperparameter optimization performance comparison

I have used Bayesian optimization for hyperparameter tuning in a machine learning model. What is the best way to compare the performance of network with and without Bayesian optimization? I found some ...
2
votes
1answer
52 views

How to get the best combinations of features for a sale optimization problem?

I have a database of shoes items from the same brand with many variables (features) like the size, the color or the shape. I also have the produced and sold quantity for the last years. This is a ...
3
votes
0answers
79 views

Ising Spin Glass - Optimization

I'm a newbie researcher working on model-based genetic algorithms, mainly linkage learning in both discrete and continuous spaces, using data modeling. I would like to ask you about Ising Spin Glass (...
2
votes
1answer
23 views

Maximize one data point

I am completely new to data science and looking to narrow down the search and reduce the learning curve required to solve problems like the one given below I have a data set with 7 columns , Column ...
2
votes
1answer
22 views

How curvature information in second order optimization methods helps

It is said that second order optimization methods in neural networks work better than first order because they contain information about rate of change of gradient or the curvature. This information ...
1
vote
1answer
24 views

Optimization problem: Given Beta Bounds Maximize sharpe

I would like to maximize a portfolio's Sharpe Ratio while keeping Beta in bounds. Could anyone supply a calculation please? ...
2
votes
1answer
23 views

optimizing a linear optimization function with linear constarints and binary variables

I am new to optimizations and trying to solve a problem, which I feel falls in the umbrella of optimization. I have an ojective function that needs to be maximized ...
3
votes
1answer
29 views

Question on Scipy - Minimize. Adding additional constraints

I am trying to using scipy minimize function for the following optimization: ...
0
votes
0answers
5 views

What is the best approach (and why) to identify a conic section given the the points along its cross-section and its vector magnitudes over time?

I have an N-body simulation that calculates the position, velocity, and acceleration of each body at every frame over the course of some duration. As an example, I tested the algorithm using our solar ...
0
votes
0answers
17 views

How to create a positive definite matrix from Dataset for solving svm dual optimization problem?

I try to implement a SVM from the scratch by myself and facing some issues when solving the dual optimization problem using qpsolvers. So I created linear separable data with sklearn ...
1
vote
2answers
37 views

“Super” Optimizer concept

I was wondering why there isn't a feature built into common-use ML libraries, like Keras, that plugs many different combinations of layers and nodes to multiple models and trains them simultaneously ...
0
votes
0answers
17 views

Assumptions on discounted long-term loss

The infinite horizon discounted long-term loss is defined as: $$ f(\theta) = \mathbb{E}_{\tau \sim \mathbb{P}(.|\theta)}\left[\sum_{t=1}^{\infty}{\gamma^t l_m(s_t,a_t)}\right]$$ where $(s_t,a_t) \in ...
1
vote
0answers
9 views

optimal minimization algorithm for platou

Greeting, I'm trying to solve an optimization problem (minimization, to be specific). My problem is that my function has one major plateau (see example image). I'm using the optimization algorithms ...
0
votes
0answers
15 views

Is using cross-entropy enough to ensure the output is a distribution probability?

I am following along https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html. In this code, the last layers of the pretrained networks are linear. The loss used in this ...
0
votes
0answers
21 views

Broadcast error in optimize.minimize

I've defined a function that references an array and broadcasts variables across that array. When the function is run, it works fine. However, when I attempt to use scipy.minimize to minimize the ...
0
votes
1answer
16 views

Calculating possible number of configuration

I am wondering how did they get the $19200$ possible configurations? Like, $5^6 = 15625$, where $6$ is the number of hyper-parameters:
1
vote
0answers
25 views

Unsupervised Function Optimization using Input and Output for Loss Function?

I have some vectors {$\mathbf{X_1 ... X_n}$} and they are all of dimension 1 x N. Vectors {$\mathbf{X_1' ... X_n'}$} are also 1 x N and are related to {$\mathbf{X_1 ... X_n}$}, but the relation cannot ...
0
votes
1answer
20 views

which scoring function for validation_curve (regression)?

Is there any thumb of rule which scoring function should be used for e.g. the validation_curve? Atm I try to study the difference between several optimizers: ...
0
votes
0answers
28 views

Variability in CNN test results

I'm trying to do some time series analysis on 1-minute forex data using a CNN. I'm new to deep learning and just getting started in building a model. So this is probably a very basic question, but I'...
2
votes
2answers
34 views

Optimization of pandas row iteration and summation

i'm wondering if anyone can provide some input on improving the speed and calculations of a pandas result. What i am trying to obtain is a summation of IDs in one table (player table) based on each ...
1
vote
2answers
764 views

what is the difference between euclidean distance and RMSE?

I'm searching for a loss function that fits my Project. Actually I have two question but they are in the same direction. I take a look at the definition of the root mean squared error and the ...
1
vote
0answers
8 views

How to explain local minima found between two trained Neural networks?

I have trained 2 neural networks with SGD and then I have taken a linear path between their weights. Say W_0 and W_1 are the weight matrices of network 1 and network 2, respectively. Then I compute ...
1
vote
2answers
29 views

How to optimize input parameters given target and scoring parameters

I'm new to machine learning/optimization, so I apologize in advance if this has been answered before. I don't know which search terms to use. I have a large dataset where I have a number of input ...
0
votes
0answers
17 views

Gym Cartpole not solving with Cross Entropy Method?

Cross Entropy Method is considered as one of the simplest optimization algorithm which can be used for training an agent. I tried to train an agent to solve gym's cartpole environment and I have used ...
3
votes
1answer
246 views

Grid search or gradient descent?

Assume we have a neural network and one if its activation functions is a function of parameter a. We want to find the weights and parameter a that leads to the minimum loss on the validation set which ...

1
2 3 4 5
7