Questions tagged [optimization]

In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

Filter by
Sorted by
Tagged with
0 votes
0 answers
17 views

How to interpret arg min in the the following equation?

I am studying the following equation: $\hat{s}_m(n) = \text{arg}\text{min}_{s_m(n)\in A_s}|\frac{\psi_m^H}{||\psi_m^H||^2}y_m(n)-s_m(n)|^2$----(1) here $A_s$ is 1x$N$ vector of QPSK symbols, $s_m(n)$ ...
user avatar
  • 1
0 votes
1 answer
14 views

Optimum weights for weighted average of 3 prediction models

I have 3 sklearn models which I use to predict a probability score for a binary classification problem. I want to create a weighted average score of all the predictions made by these models. I am ...
user avatar
0 votes
0 answers
10 views

When using optuna I should return accuracy or loss as objective value?

I am using optuna for hyperparameter tuning for my segmentation model. At the model, I am returning accuracy as an objective value since I realised that it tries to optimize to get the best result ...
user avatar
0 votes
0 answers
7 views

Project/scale a set of 2D points to keep a set of similarities constraints

I have a problem similar to this one posted here MDS scikit-learn example. I have a set of similarities between 2D points that I want to place in a map/plane while ...
user avatar
  • 143
0 votes
1 answer
34 views

Regularization and loss function

I am currently trying to get a better understanding of regularization as a concept. This leads me to the following question: Will regularization change when we change the loss function? Is it correct ...
user avatar
1 vote
0 answers
16 views

Structured policies in dynamic programming: solving a toy example

I am trying to solve a dynamic programming toy example. Here is the prompt: imagine you arrive in a new city for $N$ days and every night need to pick a restaurant to get dinner at. The qualities of ...
user avatar
-1 votes
0 answers
47 views

Why following functions are NOT reasonable loss functions? Assume we can find the optimal parameters for each loss function

Which of the following functions are NOT reasonable loss functions? Note that is the prediction and y is the true target value. Assume we can find the optimal parameters for each loss function. ...
user avatar
  • 1
0 votes
0 answers
9 views

Plot six variables

I would like to plot a landscape spanned by six variables. The numerical target variable is explained by five numerical variables. Ultimately, it is about to get a visual impression for optima and the ...
user avatar
  • 480
2 votes
0 answers
21 views

Sequential feature selection stopping condition

When using sequential feature selection approach, say forward feature selection, I want to stop adding new features when the improvement in model scores is smaller than a certain level. Is there a ...
user avatar
0 votes
1 answer
17 views

Optimization of a simple M x N dataset

I have a dataset consisting of M questionnaires and N students. Each students replied to some questionnaires. I would like to make the dataset better, by removing some questionnaires and/or some ...
user avatar
0 votes
1 answer
19 views

Implementing a Randomized Neural Network using Tensorflow?

I want to implement a Randomised Neural Network (alt. Neural Network with Random Weights (NNRW)) in keras based on the following paper: https://arxiv.org/pdf/2104.13669.pdf Essentially the idea is the ...
user avatar
0 votes
0 answers
14 views

How Can I Implement MOEA/D Algorithm in Java From Pseudocode?

I want to implement MOEA/D algorithm for a spesific population but I could not figure out how to write the java code from the pseudocode. My population size is 50 and the chromosomes shape is like ...
user avatar
0 votes
1 answer
13 views

Should deep layers ever have more units than the input layer?

i.e. if a model, with 10 inputs, say,: ...
user avatar
  • 21
1 vote
0 answers
14 views

Understanding Learning Rate in depth

I am trying to understand why the learning rate does not work universally. I have two different data sets and have tested out three learning rates 0.001 ,0.01 and 0.1 . For the first data set, I was ...
user avatar
  • 11
0 votes
0 answers
15 views

Combining features from multiple models and optimising features

I have multiple models predicting an outcome (continuous) and I want to take action to optimize the best values of these features to make a decision. Consider a regression model, y1 = m1x1 + m2x2 + ...
user avatar
0 votes
0 answers
12 views

Should hyperparameter optimisation focus on many trials (models) lower epochs first, then a second round with few models, many epochs?

Rather than a hyperparameter optimisation with kt.tuners.RandomSearch, say, that does (option A), say X model trials (e.g. 100), Y epochs each (say 100, so a total of 10,000 epochs across all models) ...
user avatar
  • 21
1 vote
0 answers
16 views

Is there a multi-modal population based metaheuristic that is non-GA?

I have a feature set from which I want to select various combinations and permutations of the features. The length of a solution feature vector can range between , say 5 - 20 features , and the ...
user avatar
  • 11
1 vote
0 answers
14 views

Finding optimal $\theta$ via maximize cost function

I want to find a variable $\theta\in R^n$ subject to maximizing the value of $f(\theta)$ where $||\theta||=1$ and $f$ is a black-box function. Is there any solution or library help me in this case? ...
user avatar
  • 111
0 votes
0 answers
14 views

Function optimization in Neural Network

What kind of function optimization uses a neural network (not deep neural)? I know descent gradient (batch, mini-batch, and stochastic) and momentum, someone can explain what they are and also if ...
user avatar
  • 411
1 vote
0 answers
25 views

Find the right balance between price of a property and agent fee

I would like to know when buying a property when is better for an estate agent to get a higher fee from me compared to the seller if we get a deal with a lower amount. As an example, let's say that: ...
user avatar
  • 111
0 votes
0 answers
37 views

Determine optimal number of layers for a neural network based on the dataset

I have a Neural network architecture where there are N parallel-connected layers (min. 3). Based on the dataset and classes it has, the optimal number of layers differ. Eg. for dataset1 optimal number ...
user avatar
1 vote
0 answers
7 views

Hardware datapaths for weights and operands

A paper, Survey and Benchmarking of Machine Learning Accelerators, mentions Conversely, pooling, dropout, softmax, and recurrent/skip connection layers are not computationally intensive since these ...
user avatar
  • 143
0 votes
0 answers
12 views

How to build a neural network without using keras compile method

I have the following neural network: ...
user avatar
0 votes
0 answers
7 views

What is the impact of an outlier in a dependent variable and independent variable on model performance?

What is the impact of an outlier in a dependent and independent variable on model performance in regression and other machine learning models?. Is outlier in dependent variable more impactful than in ...
user avatar
0 votes
0 answers
21 views

Why does my regression-NN completely fail to predict some points?

I would like to train a NN in order to approximate an unknown function $y = f(x_1,x_2)$. I have a lot of measurements $y = [y_1,\dots,y_K]$ (with K that could be in the range of 10-100 thousands) ...
user avatar
  • 1
0 votes
0 answers
8 views

Analytical gradients from tf.gradients don't match approximate gradients

I have a trained neural network (NN) with independent inputs x1, x2.. xn and a scalar output y. Input ...
user avatar
  • 1
0 votes
0 answers
23 views

How to make a classifier work the opposite way?

if I have a dataset of (x,y) and target f, how do I learn a model based on that dataset that allows me to insert value of f and get the optimal conditions (x,y) that correspond to it ? thanks in ...
user avatar
0 votes
0 answers
8 views

How to Find the Dominant Solution in Multi-Objective Optimizatiın Problems Like Simulated Annealing?

I am trying to run the simulated annealing algorithm for a multi-objective problem but I can not be sure how to check the domination between on two solutions. The objective funchtions are these: funch ...
user avatar
0 votes
0 answers
8 views

Maximizing target y of multiple polynomial regression models by optimizing input variables

I have several polynomial regressors each and everyone with their own x's and y's. I want to maximize the target Y(sum of all y's) by configuring the x's, the x's have to be within a certain interval. ...
user avatar
2 votes
1 answer
118 views

How do you solve strictly constrained optimization problems with pytorch?

I am trying to solve the following problem using pytorch: given a six sided die whose average roll is known to be 4.5, what is the maximum entropy distribution for the faces? (Note: I know a bunch of ...
user avatar
0 votes
1 answer
34 views

How pixel value range is satisfying in CNN

gray-scale images have pixel value range [0,1]. In most imaging task such as Denoising, deblurring, and inpainting we usually calculate mean square error of observed image and denoised image. However, ...
user avatar
  • 21
0 votes
0 answers
41 views

Keras Adam Optimizer minimize function: no gradientes provided

I need to optimize a function with Adam Optimizer (no Neural Network involved). I made a dummy example to understand how it works, using the minimize function but ...
user avatar
1 vote
0 answers
32 views

Is the RNN vanishing gradients problem really a gradients problem?

It is known that RNNs do not have long memory, that is they do not capture long dependencies. Usually this is explained by the vanishing (or explolding) gradients problem - when computing the ...
user avatar
0 votes
0 answers
7 views

Ideas to enforce uniformity of error in linear models

I am looking for ideas to not only solve the least square problem, but to enforce errors to be roughly similar. One idea I had is to add the variance of errors in the classical Ordinary Least Square ...
user avatar
  • 1
0 votes
0 answers
18 views

Is learning_rate linear with the time to converge using AdamOpt?

Say that both learning rates 1e-3,1e-4 leading to the same solution (not too high or too small). In terms of convergence by the amount of epochs, does ...
user avatar
0 votes
1 answer
26 views

Recommended number of features for regression problem

In the following link the answer recommends a feauture amount of N/3 for regression (or it is quoted). Where N corresponds to the sample size: How many features to sample using Random Forests Is there ...
user avatar
0 votes
0 answers
16 views

Minimizing W in SVM

When using SVM, we need to solve an optimization problem that maximizes the margin. Considering both positive and negative hiperplanes, we get something like: ...
user avatar
  • 125
0 votes
0 answers
10 views

Genetic Algorithm Optimization of RF Model

I am working on a problem set in which I have "Cost Per Click" and "Day of the week" as independent variables and "Profit" as dependent variable. I want to use this ...
user avatar
0 votes
0 answers
17 views

Non-linear solver with RNN for MPC

Is it possible to use a non-linear solver to optimize the output of a recurrent neural network (RNN) by using a solver to find the optimal RNN inputs? For example, I want to optimize a RNN to a cost ...
user avatar
0 votes
1 answer
34 views

Finding global optimum of unknown and expensive function

I would like to find optimal combination of parameters for the algorithm affecting the disk space used by some storage. Therefore, several algorithm parameters (...
user avatar
0 votes
0 answers
75 views

Dual Optimization SVM in Python using Numpy

I need to implement the dual function of SVMs optimization problem with numpy in Python and I am pretty stuck since I am not a Python or a numpy pro at all. Dual function evaluation What I want to ...
user avatar
0 votes
2 answers
90 views

Determining the optimal number of clusters by elbow method

I have a dataset that consists of 700 categorical columns and around 6000 rows. I created 2-50 clusters with the k-mode algorithm and plotted the cost function to determine the optimal number of ...
user avatar
0 votes
2 answers
91 views

Is reinforcement learning analogous to stochastic gradient descent?

Not in a strict mathematical formulation sense but, would there be there any key overlapping principals for the two optimisation approaches? For example, how does $$\{x_i, y_i, \mathrm{grad}_i \}$$ (...
user avatar
  • 1,668
2 votes
1 answer
36 views

SVM - Why we use the dual theorem?

Why in SVM we use the dual theorem? I can't understand why we cannot minimize the norm of the weights w directly.
user avatar
0 votes
0 answers
14 views

regression quality with meta score using R2 and MAE for optimisation

Considering quality of regression models I currently try to compare two types of information: The $R^2$ score that give me the information about the tendency of the predictor The $MAE$ (or $RMSE$) ...
user avatar
0 votes
0 answers
24 views

Which approach is beneficent for identifying the fake news detection?

The problem is to identify the fake news detection, As this is text classification problem . Constraints are basically that we cannot use traditional machine learning and deep learning approaches. If ...
user avatar
  • 219
0 votes
0 answers
9 views

Using Genetic Algorithms to create SSDP packets

for a part of my project we have to create request packets for dns, ssdp and ntp protocols. My professor has said to use genetic algorithms to come up with the field values for the request packet the ...
user avatar
  • 1
3 votes
1 answer
34 views

Reinforcement Learning applied to Optimisation Problem

Problem Statement: We are given an optimisation problem; with production centres, source airport, destination airports, transfer points and finally delivered to the customers. This is better explained ...
user avatar
  • 31
0 votes
0 answers
24 views

Finding function minima by neural network

I'm looking for articles that use neural network as genetic algorithms and solving optimisation problems by it. Any reference are welcome, books, lectures, YouTube, etc. Basically I'm looking for ...
user avatar
1 vote
1 answer
34 views

Tuning a multivariate process automatically

I have a process to optimize which involves multiple algorithms. These algorithms are mostly interchangeable, but can have different performance benefits depending upon the input, and depending upon ...
user avatar
  • 13

1
2 3 4 5
10