# Questions tagged [optimization]

In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

291 questions
Filter by
Sorted by
Tagged with
14 views

### Estimating 3D function - f(x,y) according to its minimums

Problem Statement: I was given 3 minimums of Y = f(X1, X2) such that: Local Minimum1: X1 = 0.20; X2 = 0.30; Ymin1 = 0.70 Local Minimum2: X1 = 0.60; X2 = 0.80; Ymin2 = 0.80 Global Minimum:...
24 views

### What applications does linear programming have in data science?

I'm currently learning about linear programming in my degree. I'm wondering how this is relevant to anything in data science?
37 views

### Can I completely cancel the effects of using a smaller batch size by reducing the learning rate?

I'm having the problem that the data from a regular sized batch (e.g., 32, 64) doesn't fit in my GPU. Among other solutions, I'm considering reducing the batch size, as is normally suggested. Of ...
53 views

### Is it possible to make F1_Score differentiable and use it directly as a Loss function?

One of the metrics that is widely used in binary classification is the F1 score: $F_1 = 2\cdot \frac{recall \cdot precision}{recall+precision}$ The problem of the F1-score is that it is not ...
11 views

### How to build Explanatory Graph for Convolutional Neural Network?

I m reading very interesting paper (https://arxiv.org/pdf/1812.07997.pdf) that aims to interpret convolutional neural network using graph. The general idea is when there are co-related parts in layers ...
35 views

### Machine learning model with simultaneous function optimization

Consider the following scenario. I am a sculpturer and customers ask me for what price I am willing to provide them with some statues. Their request for sculptures can vary in difficulty, quantity, ...
22 views

### Parameter optimization and selection in dynamic neural networks

I have used a Bayesian optimization to tune machine learning parameters. The optimized parameters are "Hidden layer size" and "...
39 views

### Hyperparameter optimization performance comparison

I have used Bayesian optimization for hyperparameter tuning in a machine learning model. What is the best way to compare the performance of network with and without Bayesian optimization? I found some ...
37 views

I have a database of shoes items from the same brand with many variables (features) like the size, the color or the shape. I also have the produced and sold quantity for the last years. This is a ...
58 views

### Ising Spin Glass - Optimization

I'm a newbie researcher working on model-based genetic algorithms, mainly linkage learning in both discrete and continuous spaces, using data modeling. I would like to ask you about Ising Spin Glass (...
23 views

### Maximize one data point

I am completely new to data science and looking to narrow down the search and reduce the learning curve required to solve problems like the one given below I have a data set with 7 columns , Column ...
21 views

### How curvature information in second order optimization methods helps

It is said that second order optimization methods in neural networks work better than first order because they contain information about rate of change of gradient or the curvature. This information ...
17 views

### Optimization problem: Given Beta Bounds Maximize sharpe

I would like to maximize a portfolio's Sharpe Ratio while keeping Beta in bounds. Could anyone supply a calculation please? ...
23 views

### optimizing a linear optimization function with linear constarints and binary variables

I am new to optimizations and trying to solve a problem, which I feel falls in the umbrella of optimization. I have an ojective function that needs to be maximized ...
23 views

I am trying to using scipy minimize function for the following optimization: ...
4 views

### What is the best approach (and why) to identify a conic section given the the points along its cross-section and its vector magnitudes over time?

I have an N-body simulation that calculates the position, velocity, and acceleration of each body at every frame over the course of some duration. As an example, I tested the algorithm using our solar ...
16 views

### How to create a positive definite matrix from Dataset for solving svm dual optimization problem?

I try to implement a SVM from the scratch by myself and facing some issues when solving the dual optimization problem using qpsolvers. So I created linear separable data with sklearn ...
34 views

### “Super” Optimizer concept

I was wondering why there isn't a feature built into common-use ML libraries, like Keras, that plugs many different combinations of layers and nodes to multiple models and trains them simultaneously ...
16 views

25 views

### How can we conclude that an optimization algorithm is better than another one

When we test a new optimization algorithm, what the process that we need to do?For example, do we need to run the algorithm several times, and pick a best performance,i.e., in terms of accuracy, f1 ...
15 views

I have implemented AdamW but I am not getting good results, is there some mistake in my implementation? ...
37 views

### Optimizer for Function Approximation using Fully connected Neural Network

In short, my query is: Which optimizer(s) should one choose to experiment for a fully connected neural network if she wants perfect fitting (mae < 1e-04) on the training data? Details: In my ...
35 views

### Gaussian Mixture Models Clustering

When using the EM algorithm in Gaussian Mixture Models (GMM), in the E-step, we take each x set in the training dataset to calculate and update the "weight" and parameters of each Gaussian ...
31 views

### Custom layer in Keras and optimizer

How is optimizer related to our own Keras layer? Do we have to rewrite optimizer for that certain layer? For example, suppose we have ...
29 views

### Optimizing parameters for an image-generation algorithm

I have a program that takes an image and a list of parameters and generates a new image. I would like to automate the selection of parameters to produce the 'best' image. 'Best' in this context doesn'...
76 views

### Gradient descent in a noisy environment

How to know the right direction in a noisy environment? In the typical example of neural network learning, we can see several local minima. The gradient descent is choosing one local minimum and ...
29 views

### Python metaheuristic packages

I need to use a metaheuristic algorithm to solve an optimization problem on a Python codebase. Metaheuristics usually need to be written in C++ or Java as they involve a lot of iterations, while ...
94 views

### Why does degradation occur in deep neural networks?

It has been shown that "plain" neural networks tend to have an increased amount training error, and accompanied test error, as more layers are added. I am not quite certain as to why this occurs. In ...
18 views

### Algorithm/Model for Power System Optimisation

For my engineering honours project, I'm performing a study of voltage control on the electrical distribution network, using reactive power provision from residential solar inverters. Basically, each ...
13 views

adagrad and sparseAdam work great for sparse training because there’s separate sums for each of the parameters. Are there any other recommended optimizers for embedding training?