Questions tagged [optimization]

In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

Filter by
Sorted by
Tagged with
0
votes
0answers
7 views

ADAM optimizer yields sub-optimal results

I've been writing my own neural network from scratch to get a better understanding of how they work (using MATLAB initially, but plan to port it to C++ afterwards). One major problem for me has been ...
0
votes
0answers
5 views

How does fusing operations lower accuracy for machine learning models?

In this talk the speaker Sachin Joglekar mentions that it's important to consider tradeoffs when choosing delegates for optimizing Tensorflow Lite. One of the tradeoffs he mentions at 10:14 is that ...
0
votes
0answers
5 views

Problem with elastic constraints in PuLP

(This is my first question on Stack Exchange) I am working on a production allocation problem, whereby sales orders have to be allocated over three production plants. I am using PuLP in Python, and ...
1
vote
1answer
23 views

Optimal points of $f(x,y)=x^2 + y^2 + \beta xy + x + 2y$

I am self-learning basic optimization theory and algorithms from "An Introduction to Optimization" by Chong and Zak. I would like someone to verify my solution to this problem, on finding ...
0
votes
0answers
6 views

Is it okay to train Triplet Loss where the anchor and the positive are the same?

Triplet loss roughly defined as total of$$ max (- distance(Anchor,Negative) + distance(Anchor,Positive) +margin,0)$$ I'm working with triplets data, and it turns out that some triplets(just small ...
0
votes
0answers
12 views

How to optimize AUPRC for imbalanced data given a precision or recall bias?

My general understanding is that when optimizing a model in an imbalanced class case with a small preferred target class one should optimize first for a model with the best AUPRC (assuming one doesn't ...
0
votes
0answers
13 views

Model selection for multiple groups to maximize overall f1 score

Model 1 Model 2 .. Model m Group 1 tp11/act1/pred11 .. .. Group 2 tp21/act2/pred21 .. .. .. .. .. .. .. Group n tpn1/actn/predn1 .. tpnm/actn/prednm I have a multiple groups model selection for ...
0
votes
0answers
13 views

Identifying subset of feature set to represent remaining features using pca or any other techniques

How can you use PCA to identify a subset of feature set to represent remaining features in a data set? Suppose there are 10 features given by F={ f1,f2......f10}. How to identify a subset of F such ...
0
votes
0answers
7 views

Qunatify total time saved by prioritizing tasks based on the failure rate probability of each task

I am trying to solve a problem where I am trying to prioritize the tasks in a job based on the failure rates of each task. For ex: ...
0
votes
0answers
11 views

Cluster fusion techniques for global optimal solution

Suppose you have a huge data set. You divide that data set into N number of blocks. Each block is then clustered into M blocks (using any clustering algorithms like K means). Now you have N local ...
1
vote
0answers
39 views

BERT MLM overfitting [closed]

We are training the BERT model on masked language modeling task for the Russian Language. Our dataset consists of 60 mln texts with (128 tokens for each text) from online social networks, ...
0
votes
0answers
21 views

Evolutionary search mutation range

Im currently writing an evolutionary program that has the task to optimize a weight set for a neural network for a specific and well defined environment. It is performing excellently but i am getting ...
1
vote
0answers
14 views

Ant colony optimization for clustering [closed]

What do you mean by applying ant colony optimization (ACO) to clustering? What is the output one would get after it? Could you explain it using a two dimesional data set which is clustered into 3 ...
2
votes
1answer
30 views

avoiding premature convergence with neural networks (EA's)

I am currently writing a program that would be able to play snake on an 25*25 grid. It works by optimizing a set of weights of 300 different solutions (each solution would be a different neural ...
0
votes
1answer
13 views

Custom thresholds on categorical classification

When assessing a binary classification task, it is possible to search for particular threshold in order to have better score on some metrics (f1,recall,etc) through numerous methods. Unfortunately, it ...
0
votes
0answers
15 views

Why is my accuracy 0 on the first epoch in continuous training on MNIST

I'm trying an experiment where I first train my model on MNIST labels [0-4], and then I freeze the first conv layers and continue training my model on labels [5-9]. When I dont re-initialize my Fully ...
0
votes
1answer
35 views

Would a neural network trained on extracted features have the same accuracy as a full network with frozen layers?

Let's say that I train two neural networks on the exact same dataset. The first network is a VGG19 model with frozen convolutional layers so only the top dense ...
1
vote
1answer
11 views

Do I load all files at once or one at a time?

I currently have $1700+$ CSV files. Each of them is in the same format and structure, give or take a row or possibly a column at the end. Each CSV is $\approx 3.8$ MB. I need to perform a ...
0
votes
0answers
18 views

Loss function for points inside polygon

I am trying to optimize some parameters that used to transform 2d points from a place to another (you may think of that as rotation & translation parameter for simplicity) The parameters are ...
1
vote
0answers
35 views

How to maximize a log linear regression equation satisfying a constraint?

I have a log linear equation of the form $y = w_1(\log{X1}) + w_2(\log{X2}) + ... + w_n(\log{Xn})$. How can I find the value of X's that maximize the value of y subject to a constraint $(X_1+X_2+...+...
0
votes
0answers
15 views

Curve fitting (or other techniques) for revenue and price optimization

My question is as follows: with historical data we can model the relationship between discounts and sales (number of sold units). So we can address the question "how are sales boosted by ...
1
vote
1answer
34 views

What method/algorithm for constrained multi-target regression

I am working with three dimensional measurement data and want to model them using a multivariate linear regression. I have already implemented a simple gradient descent algorithm to solve the classic ...
1
vote
0answers
17 views

Finding the dual to an optimization problem on an unsupervised dataset [closed]

We consider the unsupervised dataset $x_1,..x_N \in R^d$ and the optimization problem: $$min_w \,\frac{1}{2}{\left\lVert w \right\rVert}^2,$$ subject to constraints:$$\forall_{i=1}^N: \phi(x_i)^Tw\...
0
votes
0answers
83 views

Need help to understand backpropagation for gated recurrent units (GRU)

I'm stuck regarding the implementation of backpropagation in a binary classification task using a GRU. I wanted to know if someone could tell me how to proceed. I was able to understand how BP works ...
1
vote
1answer
99 views

Terminology to distinguish between ML methods and optimization methods (PSO, ACO..)

I am currently writing a scientific thesis which consists of two parts. In the first part I am building ML models with neural networks, support vectors etc. and the second part is about finding global ...
1
vote
0answers
18 views

Optimize Yahoo Finance Code for Analysis [closed]

I am trying to analyze a number of companies using financial data I gathered from Yahoo Finance. I am also using the yfinance API to get some more details about the company using functions. Since I am ...
0
votes
1answer
22 views

Gradient descent around optimal loss surface

All the loss surface used in examples have some of bowl shape that decrease drastically far from the optimal and decrease slowly around the optimal flat point. My questions are: Has all the loss ...
0
votes
0answers
13 views

Notation for anti-optimal

What's the notation for the worst element/solution? The best "something" can be denoted with *, i.e.: $R^* = R_{best}$ $R^? = R_{worst}$
0
votes
0answers
28 views

Solve for the set of coordinates that reduces the average distances between request and server in half

I generate a DataFrame with coordinates and distances to 3 servers. ...
3
votes
1answer
177 views

Two steps optimization of a credit card limit

I have a problem similar to what is on the title but not the same, the problem on the title allows me to explain the dynamics of my need. I have to determine how much is the optimal value for a ...
2
votes
1answer
38 views

Adam Optimiser First Step

Plotting the paths on the cost surface from different gradient descent optimisers on a toy example, I found that the Adam algorithm does not initially travel in the direction of steepest gradient (...
0
votes
0answers
9 views

Word2Vector multicontext CBOW model with Adam optimization

In cbow multiword context word2vec model there are two weights matrixes $$ W,W^{'}$$ Where $W$ is $I$ -> $H$ weight matrix, and $W^{'}$ is $H$ -> $U$ weight matrix and output is just softmax ...
0
votes
1answer
28 views

Keras weird loss and metrics during train

I am doing some testing with tensorflow, and I bumbed into a very weird behaviour. Here is my code ...
1
vote
1answer
26 views

Find parameters to maximise output score [closed]

Not sure this is the right place to ask. Lets say there is a function f() where its implementation is unknown but it returns a score. I would like to get the ...
0
votes
0answers
11 views

How can CSO(cuckoo search optimization), PSO(particle swarm opt.) algorithms be utilized for this dataset?

How can such data be optimized using CSO, PSO algorithms so that it gives a result like which products to buy for a budget of 600$ ...
0
votes
1answer
29 views
1
vote
0answers
12 views

Optimizing a model for three different metrics that have different ranges

I have a multiple object tracker that I apply on a specific object in an image series. The tracker has several parameters that can be adjusted which affects the performance of the tracking. I am ...
0
votes
0answers
10 views

Weigthing discrimnator and generator loss in GAN networks?

Training of good generator model in vanilla GAN (Generative adversarial networks) https://papers.nips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf is achieved via minimax game, where <...
2
votes
1answer
46 views

Deep learning test loss curve won't go down

I've been working with Deep Learning projects for this current project that I am working on and it's basically a time series classification problem. Where given an array of time series data I need to ...
1
vote
0answers
11 views

Predicting quality results from operating data

Background: I have process data (table 1) that is "batch" in the chemical engineering sense of the word. Each batch ID represents the start and end of a run. Throughout the batch, different ...
1
vote
1answer
38 views

Is there a rule of thumb for a sufficient number of trials for hyperparameter search

I am implementing a quite complicated Bayesian hyperparameter search in hyperopt library on a CNN. Is there a rule of thumb for a "sufficient" number of ...
1
vote
1answer
39 views

Visualize n-dimensional bayesian optimization results

I am working on a 6-dimensional bayesian optimization problem using (skopt's gp_minimize). After the optimizer ran for j iterations I would like to somehow visualize the "progress/result" of ...
3
votes
1answer
75 views

ML/NN as Function Evaluator for further Optimization (maximization) - Practical Example

I am working on a production optimization problem; a very similar idea to what is described by Vegard Flovik How to use machine learning for production optimization. The following image, taken from ...
0
votes
1answer
15 views

Tuning the model parameters vs the parameter of optimizer for Deep Neural Networks?

I understand that there are rarely general recipes in field of machine learning and the many results can be achieved only by trial and error, and are task specific as well. My question is, if the ...
1
vote
1answer
28 views

Optimizing regression weights for NN outputs with PyTorch

So I'm basically trying to fit a regression on the relation of the input and output of a neural network model. Then the idea is, that these estimated regression weights should be optimized to some ...
0
votes
0answers
12 views

Rating Sports Teams: How to perform least squares minimization with a constraint?

I am trying to rate NFL teams by minimizing the sum of squared errors subject to a constraint. My data looks like: ...
0
votes
0answers
73 views

Levenberg–Marquardt or Adam optimization

In RBF (radial basis functions)-Neural Networks, which method (Levenberg–Marquardt or Adam optimization) is more efficient for optimizing the parameters (centers, widths, and output weights) in ...
2
votes
0answers
22 views

How to get the maximum likelihood estimate of the categorical distribution parameters using Lagrange optimization?

Let's say our data is discrete-valued and belongs to one of $K$ classes. The underlying probability distribution is assumed to be a categorical/multinoulli distribution given as $p(\textbf{x}) = \...
0
votes
0answers
87 views

Meaning of absolute tolerance and relative tolerance

Hello Data science community, I am using a library nnet in R. The documentation, page no. 4-5, shows that it has default values for ...
0
votes
0answers
6 views

How to get accurate estimates on Neural Networks Hessian?

I need to get not only accurate estimates on the neural network output itself but also on its second order derivatives in order to use the NN for optimization problems. With Adam optimizer I can't get ...

1
2 3 4 5
8