Questions tagged [optimization]

In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

Filter by
Sorted by
Tagged with
0
votes
1answer
14 views

Estimating 3D function - f(x,y) according to its minimums

Problem Statement: I was given 3 minimums of Y = f(X1, X2) such that: Local Minimum1: X1 = 0.20; X2 = 0.30; Ymin1 = 0.70 Local Minimum2: X1 = 0.60; X2 = 0.80; Ymin2 = 0.80 Global Minimum:...
0
votes
1answer
24 views

What applications does linear programming have in data science?

I'm currently learning about linear programming in my degree. I'm wondering how this is relevant to anything in data science?
5
votes
1answer
37 views

Can I completely cancel the effects of using a smaller batch size by reducing the learning rate?

I'm having the problem that the data from a regular sized batch (e.g., 32, 64) doesn't fit in my GPU. Among other solutions, I'm considering reducing the batch size, as is normally suggested. Of ...
2
votes
1answer
53 views

Is it possible to make F1_Score differentiable and use it directly as a Loss function?

One of the metrics that is widely used in binary classification is the F1 score: $F_1 = 2\cdot \frac{recall \cdot precision}{recall+precision}$ The problem of the F1-score is that it is not ...
1
vote
0answers
11 views

How to build Explanatory Graph for Convolutional Neural Network?

I m reading very interesting paper (https://arxiv.org/pdf/1812.07997.pdf) that aims to interpret convolutional neural network using graph. The general idea is when there are co-related parts in layers ...
2
votes
1answer
35 views

Machine learning model with simultaneous function optimization

Consider the following scenario. I am a sculpturer and customers ask me for what price I am willing to provide them with some statues. Their request for sculptures can vary in difficulty, quantity, ...
2
votes
0answers
22 views

Parameter optimization and selection in dynamic neural networks

I have used a Bayesian optimization to tune machine learning parameters. The optimized parameters are "Hidden layer size" and "...
1
vote
1answer
39 views

Hyperparameter optimization performance comparison

I have used Bayesian optimization for hyperparameter tuning in a machine learning model. What is the best way to compare the performance of network with and without Bayesian optimization? I found some ...
2
votes
1answer
37 views

How to get the best combinations of features for a sale optimization problem?

I have a database of shoes items from the same brand with many variables (features) like the size, the color or the shape. I also have the produced and sold quantity for the last years. This is a ...
3
votes
0answers
58 views

Ising Spin Glass - Optimization

I'm a newbie researcher working on model-based genetic algorithms, mainly linkage learning in both discrete and continuous spaces, using data modeling. I would like to ask you about Ising Spin Glass (...
2
votes
1answer
23 views

Maximize one data point

I am completely new to data science and looking to narrow down the search and reduce the learning curve required to solve problems like the one given below I have a data set with 7 columns , Column ...
2
votes
1answer
21 views

How curvature information in second order optimization methods helps

It is said that second order optimization methods in neural networks work better than first order because they contain information about rate of change of gradient or the curvature. This information ...
1
vote
0answers
17 views

Optimization problem: Given Beta Bounds Maximize sharpe

I would like to maximize a portfolio's Sharpe Ratio while keeping Beta in bounds. Could anyone supply a calculation please? ...
3
votes
1answer
23 views

optimizing a linear optimization function with linear constarints and binary variables

I am new to optimizations and trying to solve a problem, which I feel falls in the umbrella of optimization. I have an ojective function that needs to be maximized ...
4
votes
1answer
23 views

Question on Scipy - Minimize. Adding additional constraints

I am trying to using scipy minimize function for the following optimization: ...
0
votes
0answers
4 views

What is the best approach (and why) to identify a conic section given the the points along its cross-section and its vector magnitudes over time?

I have an N-body simulation that calculates the position, velocity, and acceleration of each body at every frame over the course of some duration. As an example, I tested the algorithm using our solar ...
0
votes
0answers
16 views

How to create a positive definite matrix from Dataset for solving svm dual optimization problem?

I try to implement a SVM from the scratch by myself and facing some issues when solving the dual optimization problem using qpsolvers. So I created linear separable data with sklearn ...
1
vote
2answers
34 views

“Super” Optimizer concept

I was wondering why there isn't a feature built into common-use ML libraries, like Keras, that plugs many different combinations of layers and nodes to multiple models and trains them simultaneously ...
0
votes
0answers
16 views

Assumptions on discounted long-term loss

The infinite horizon discounted long-term loss is defined as: $$ f(\theta) = \mathbb{E}_{\tau \sim \mathbb{P}(.|\theta)}\left[\sum_{t=1}^{\infty}{\gamma^t l_m(s_t,a_t)}\right]$$ where $(s_t,a_t) \in ...
1
vote
0answers
9 views

optimal minimization algorithm for platou

Greeting, I'm trying to solve an optimization problem (minimization, to be specific). My problem is that my function has one major plateau (see example image). I'm using the optimization algorithms ...
0
votes
0answers
14 views

Is using cross-entropy enough to ensure the output is a distribution probability?

I am following along https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html. In this code, the last layers of the pretrained networks are linear. The loss used in this ...
0
votes
0answers
14 views

Broadcast error in optimize.minimize

I've defined a function that references an array and broadcasts variables across that array. When the function is run, it works fine. However, when I attempt to use scipy.minimize to minimize the ...
0
votes
1answer
15 views

Calculating possible number of configuration

I am wondering how did they get the $19200$ possible configurations? Like, $5^6 = 15625$, where $6$ is the number of hyper-parameters:
1
vote
0answers
25 views

Unsupervised Function Optimization using Input and Output for Loss Function?

I have some vectors {$\mathbf{X_1 ... X_n}$} and they are all of dimension 1 x N. Vectors {$\mathbf{X_1' ... X_n'}$} are also 1 x N and are related to {$\mathbf{X_1 ... X_n}$}, but the relation cannot ...
0
votes
1answer
19 views

which scoring function for validation_curve (regression)?

Is there any thumb of rule which scoring function should be used for e.g. the validation_curve? Atm I try to study the difference between several optimizers: ...
0
votes
0answers
26 views

Variability in CNN test results

I'm trying to do some time series analysis on 1-minute forex data using a CNN. I'm new to deep learning and just getting started in building a model. So this is probably a very basic question, but I'...
2
votes
2answers
29 views

Optimization of pandas row iteration and summation

i'm wondering if anyone can provide some input on improving the speed and calculations of a pandas result. What i am trying to obtain is a summation of IDs in one table (player table) based on each ...
1
vote
2answers
294 views

what is the difference between euclidean distance and RMSE?

I'm searching for a loss function that fits my Project. Actually I have two question but they are in the same direction. I take a look at the definition of the root mean squared error and the ...
1
vote
0answers
8 views

How to explain local minima found between two trained Neural networks?

I have trained 2 neural networks with SGD and then I have taken a linear path between their weights. Say W_0 and W_1 are the weight matrices of network 1 and network 2, respectively. Then I compute ...
1
vote
2answers
27 views

How to optimize input parameters given target and scoring parameters

I'm new to machine learning/optimization, so I apologize in advance if this has been answered before. I don't know which search terms to use. I have a large dataset where I have a number of input ...
0
votes
0answers
12 views

Gym Cartpole not solving with Cross Entropy Method?

Cross Entropy Method is considered as one of the simplest optimization algorithm which can be used for training an agent. I tried to train an agent to solve gym's cartpole environment and I have used ...
3
votes
1answer
109 views

Grid search or gradient descent?

Assume we have a neural network and one if its activation functions is a function of parameter a. We want to find the weights and parameter a that leads to the minimum loss on the validation set which ...
5
votes
3answers
514 views

Genetic algorithms(GAs): to be considered only as optimization algorithms? Are GAs used in machine learning any way?

As a quick question, what are genetic algorithms meant to be used for? I read somewhere else that they should be used as optimization algorithms (similar to the way we use gradient descent to optimize ...
0
votes
0answers
5 views

Is there any relation between binary/ternary quantization using in deep learning and fuzzy?

I am new with binary/ternary quantization but its structure seems to have some relation with fuzzy. Am I in right way? Is there any relation between binary/ternary quantization using in deep learning ...
1
vote
0answers
11 views

If $\ell_0$ regularization can be done via the proximal operator, why are people still using LASSO?

I have just learned that a general framework in constrained optimization is called "proximal gradient optimization". It is interesting that the $\ell_0$ "norm" is also associated with a proximal ...
1
vote
0answers
30 views

XGBoost speed issues

I'm trying to optimize the hyperparameters for XGBoost, thus needing to run it multiple times with different parameters. However the time needed to run single XGBoost with the parameters provided ...
0
votes
1answer
39 views

Non-Convex Constraints for Classification Problems

I am willing to create a hypothetical non-convex constraints for the purpose of practising nonlinear classification using an algorithm. I thought of such constraints in the form: $x^TAx + Bx \leq c$. ...
1
vote
1answer
25 views

How can we conclude that an optimization algorithm is better than another one

When we test a new optimization algorithm, what the process that we need to do?For example, do we need to run the algorithm several times, and pick a best performance,i.e., in terms of accuracy, f1 ...
0
votes
0answers
15 views

How to implement AdamW?

I have implemented AdamW but I am not getting good results, is there some mistake in my implementation? ...
2
votes
1answer
37 views

Optimizer for Function Approximation using Fully connected Neural Network

In short, my query is: Which optimizer(s) should one choose to experiment for a fully connected neural network if she wants perfect fitting (mae < 1e-04) on the training data? Details: In my ...
1
vote
1answer
35 views

Gaussian Mixture Models Clustering

When using the EM algorithm in Gaussian Mixture Models (GMM), in the E-step, we take each x set in the training dataset to calculate and update the "weight" and parameters of each Gaussian ...
0
votes
0answers
31 views

Custom layer in Keras and optimizer

How is optimizer related to our own Keras layer? Do we have to rewrite optimizer for that certain layer? For example, suppose we have ...
0
votes
0answers
29 views

Optimizing parameters for an image-generation algorithm

I have a program that takes an image and a list of parameters and generates a new image. I would like to automate the selection of parameters to produce the 'best' image. 'Best' in this context doesn'...
1
vote
3answers
76 views

Gradient descent in a noisy environment

How to know the right direction in a noisy environment? In the typical example of neural network learning, we can see several local minima. The gradient descent is choosing one local minimum and ...
0
votes
1answer
29 views

Python metaheuristic packages

I need to use a metaheuristic algorithm to solve an optimization problem on a Python codebase. Metaheuristics usually need to be written in C++ or Java as they involve a lot of iterations, while ...
0
votes
0answers
94 views

Why does degradation occur in deep neural networks?

It has been shown that "plain" neural networks tend to have an increased amount training error, and accompanied test error, as more layers are added. I am not quite certain as to why this occurs. In ...
0
votes
0answers
18 views

Algorithm/Model for Power System Optimisation

For my engineering honours project, I'm performing a study of voltage control on the electrical distribution network, using reactive power provision from residential solar inverters. Basically, each ...
0
votes
0answers
13 views

Are there any other recommended optimizers for word2vec/glove than Adagrad and SparseAdam?

adagrad and sparseAdam work great for sparse training because there’s separate sums for each of the parameters. Are there any other recommended optimizers for embedding training?
0
votes
1answer
21 views

Can we Create Neural network(Simple one such as Multi Layer perceptron) that only contains positive weights only?

I was wondering if there is a specific method to create a well performing neural network with only positive weights (I already tried clipping the weight before training or so and initializing the ...
0
votes
1answer
39 views

How to deal with a constant value as an output from neural network?

I am using feedforward neural network for regression and what I get as a result of prediction is a constant value visible on the graph below: Data I use are typical standardised tabular numbers. The ...