Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [optimization]

In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

Filter by
Sorted by
Tagged with
0 votes
1 answer
20 views

Optimal combination of variables to minimise output

To be honest I'm not 100% sure how much this is purely a coding issue or a data science issue, but I'll take my chances. I've developed a matrix which is a mixture of various hyperparameters, the ...
Dante Saint-Germain's user avatar
0 votes
0 answers
15 views

Scheduling Production on One Machine with Changeover Costs and due dates

I'm trying to develop a solution to find a local optimum to a combination of manufacturing orders. They have a changeover cost per type, this means that the change between a type 2 order and type 3 ...
Victor Ruiz's user avatar
0 votes
0 answers
23 views

Using Reinforcement learning for minimisation

I would like to use reinforcement learning for the optimisation of a given function under some contraints. Take for example the following problems: ...
Ach Raf's user avatar
1 vote
1 answer
42 views

How to comment on goodness of loss functions?

I have two loss functions $\mathcal{L}_1$ and $\mathcal{L}_2$ to train my model. The model is predominantly a classification model. Both $\mathcal{L}_1$ and $\mathcal{L}_2$ takes are two variants of ...
Aleph's user avatar
  • 123
0 votes
1 answer
29 views

Each person gets their top, or second choice of activity over a period of 6 slots

We are running a camp for 130 children, and on 3 days they can pick different activities to do. One activities for slot 1 (45min), the other for slot 2 (another 45min), enabling them to do 6 ...
Pascal's user avatar
  • 9
0 votes
0 answers
17 views

Bulding Deep Learning model for multiclassification case

I am soo confused i read a lot of information in forumas and still cna't get what is wrong. my data is around 500.000 rows and 32 columns. my target variables consists of 3 classes (0, 1, 2). Hyperopt ...
Shamkhal Mammadov's user avatar
0 votes
1 answer
27 views

Learning the gradient descent stepsize with RL [closed]

Problem statement: I've been working on a project to accelerate the convergence of gradient descent using reinforcement learning (RL). I want to learn a policy that can map the current state of ...
CodeGuy's user avatar
0 votes
1 answer
38 views

How to prevent update a pretrained model if a model is optimized with backpropagation in Pytorch?

I use Pytorch exclusively to develop my model, and these are components in my model and how it works: A generator An encoder: a pretrained, and should not updated. A loss function. Input is passed to ...
Jesse's user avatar
  • 101
0 votes
0 answers
39 views

Learning the "surface" of a function

Given a continuous non-convex function and assuming knowledge of all extremum points, is it possible to learn all initialization points from which performing classic GD lead to global minimum? Pure ...
codeplay's user avatar
1 vote
0 answers
20 views

How to force my model heads to learn different things?

I have an Seq2Seq model that has 2 generative LM heads. I want the two heads to focus on different features/styles while decoding. The approach that I was thinking of is adding a distance cost to the ...
Tathagato Roy's user avatar
0 votes
0 answers
9 views

Dual of the SCM square hinge loss

Let $x_1,\dots,x_n\in \mathbb{R}^n$, $y_1,\dots,y_n\in \{-1,1\}$, $\lambda \ge 0$ and $K$ be the invertible Gram matrix $K=(x_i\cdot x_j)_{ij}$. Consider $$ (P) \qquad \qquad \min_{a\in \mathbb{R}^n} \...
Smilia's user avatar
  • 101
0 votes
0 answers
42 views

Setting derivative to 0 when minimizing expected loss over possible data samples

Suppose, given $x\in \mathbb{R}^d$, you want $\theta$, which is the solution of $$ \text{argmin}_{\theta} \mathbb{E}_a[L(\theta;x,a)] $$ where $a \sim \mathcal{N}(x,\Sigma)$ and $L$ measures a (convex)...
user9781778's user avatar
1 vote
1 answer
113 views

Which machine learning models are rational to use on NP-hard and NP-complete "theoretical" problems?

Time and time again I run into "surprising" NP-hard problems that seem naturally simpler than they are. I recently worked on a weighted graph theoretical problem where the point is to ...
me9hanics's user avatar
  • 113
5 votes
1 answer
575 views

What do we mean by optimizer.zero_grad()

This should be a simple question. But it is vague to me. What do we mean by optimizer.zero_grad(). Consider SGD as an example: $W^{t+1}= W^{t}- \lambda g_t$. Which one becomes zero for each batch. It ...
Ali.A's user avatar
  • 71
0 votes
0 answers
24 views

Is there anyway to evaluate the estimation results of least square

Consider the scenario where a practical problem is tackled utilizing the method of least squares. Upon each iteration, an estimation of the parameter $\theta$ is derived via $\hat{\theta} = (X^\top X)^...
yangtzech's user avatar
0 votes
0 answers
20 views

Feeding more data to a neural network

I watched a video on Tesla's FSD where the drive was really smooth but required one intervention when the traffic light changed to green but the car wouldn't go because it looked like the light was ...
Noale's user avatar
  • 1
0 votes
1 answer
102 views

Optimizing decision tree

I have a question regarding the technique/technology which could be applied for the issue: Suppose I have a rule-based tree or decision tree which predicts a variable Y based on variables A,B,C. This ...
DannyV's user avatar
  • 1
0 votes
1 answer
33 views

Optimization problem of find a pair of values (x, y) such that they produce an output z

Problem I am trying to create a process that determines the rent and purchase price of a home that gives me a capitalization rate of 0.08 for instance. Background Normally I calculate the ...
Olek's user avatar
  • 11
0 votes
0 answers
43 views

How to work with multiple feature types on autoencoder?

This is my first post here. I am working on an adversarial autoencoder that receives different features, encodes them, and decodes them. For instance, suppose you have a dataset from a large survey ...
Umberto Mignozzetti's user avatar
1 vote
2 answers
98 views

weighting voting classifier (MAE and MSE)

I am trying to optimize the weights of a Voting Regressor problem. To achieve the best score, I am considering both MAE and MSE as parameters, using the following formula: score = w * MAE + (w-1) * ...
Guilherme Raibolt's user avatar
0 votes
0 answers
12 views

Team building algorithm based on rank order preferences

I have a dataset with employees at four levels of training: junior employee, senior employee, junior manager, senior manager. I am looking to match them into teams of 4, each team with 1 person from ...
Zach Eisner's user avatar
0 votes
0 answers
10 views

Are Julia ML libraries inherently parallelized and optimized?

I am trying to decide if I should learn Julia or stick with Python and its libraries, e.g. TensorFlow and FastAI. One important consideration is whether Julia has a library with similar capabilities ...
Joachim Rives's user avatar
1 vote
2 answers
183 views

Gradient Descent: Is the magnitude in Gradient Vectors arbitrary?

I am only just getting familiar with gradient descent through learning logistic regression. I understand the directional component in the gradient vectors is correct information derived from the slope ...
MrHunda's user avatar
  • 11
0 votes
0 answers
35 views

How does non-negative constrained optimization work?

I am dealing with a machine learning problem in which a logistic regression model is trained with under-bound constrained optimization for model interpretability purposes. In a simple 2-dimensional ...
hypothesisusable's user avatar
0 votes
0 answers
72 views

PySpark Logistic regression model weights are inconsistent between runs

I am training a pyspark logistic regression model using pyspark mllib. I am noticing that the weights are not being consistent in between runs. I have set the random seed in the training script and ...
hypothesisusable's user avatar
0 votes
0 answers
7 views

How to interpet the bolded lines in bayesopt

I am using bayesopt to maximize a function that is everywhere less or equal to zero $(f(x) \leq 0)$. The score is essentially a negated Mean Absolute Error because the default behavior of bayesopt is ...
Enk9456's user avatar
  • 105
1 vote
0 answers
15 views

Question about Keras implementation of ADAM

I have a question regarding this line in the Keras implementation of Adam: alpha = lr * tf.sqrt(1 - beta_2_power) / (1 - beta_1_power) From the algorithm here, is this step doing the bias correction?...
Yandle's user avatar
  • 231
2 votes
0 answers
20 views

Consistency error in visualization of policy improvement in Sutton & Barto's book?

Sutton & Barto introduce in their foundational book on "Reinforcement Learning: An Introduction" in the context of Dynamic Programming algorithms for policy evaluation and improvement. ...
Steve's user avatar
  • 41
1 vote
2 answers
38 views

Simplex method for equality optimization

I have a linear model that goes as 0.1*x1 + 0.8*x2 + 3.4*x3 + 5.0*x4 + c and this linear model was generated by using a Linear Regression. MAE is ~ 0.4 MSE is ~ 0....
anthino12's user avatar
  • 111
1 vote
1 answer
41 views

Is it common that a model only learns in a few training runs and doesn't learn in the majority of training runs

I have a model that does binary classification. I train the same model many times, say, 20 times. In these 20 training runs: roughly 10% of the training attempts learn fast and the model performs ...
D.J. Elkind's user avatar
0 votes
0 answers
64 views

Equivalence between K-Means objective functions proof

I am working on a project involving the K-Means clustering algorithm, and I am trying to prove the equivalence between different formulations of the objective function. Specifically, I want to show ...
Oren Ben Eliyahu's user avatar
0 votes
0 answers
40 views

Hyperparameters values changing on every run

...
Akshita's user avatar
  • 13
0 votes
1 answer
73 views

Multiprocessing data loading in colab

I want to convert mozilla common voice dataset from mp3 to wav. But this dataset is large and convertion takes many time. How can I make this convertion in colab with multiprocessing to decrease time ...
randomuser228's user avatar
2 votes
1 answer
31 views

What can we do in order to find customers that follow a specific pre-defined pattern?

Let's say I have a few customers buying product A. How can I find other customers that have the same characteristics, or something close to, these A buyers? I was thinking about using clustering ...
Aldla E Aoepql's user avatar
1 vote
1 answer
16 views

What are the constants in this formula for polytrophic head?

Polytrophic head can be expressed as H = b1N^2 + b2NQ + b3Q^2 where b1, b2 and b3 are constants, N is the speed of the compressor (rmp), and Q is the volumetric flow rate of natural gas at the ...
Starbucks's user avatar
  • 175
0 votes
1 answer
29 views

AdaGrad denominator exponents comparison

Adagrad adapts the learning rate $ \alpha $ all along the gradient descent process, by dividing each weight on a quantity based on the sum of the previous squared gradient up to time $ t $. Therefore, ...
The Limit Does Not Exist's user avatar
0 votes
1 answer
62 views

Statistical test for comparing number of clusters in data

I am performing $K$-means clustering on a dataset consisting of $n$ observations and $d$ variables, and I'm trying to determine the optimal number of clusters. Is there a test that can determine the ...
RyRy the Fly Guy's user avatar
1 vote
1 answer
85 views

Which of 2 options is better practice for model optimization: 1) Nested CV wrongly averaging inner CV scores. 2) Two successive CVs on X_all. Altrntv?

Goal: Compare preprocessing methods, models, and hyperparameters without leaking into the final generalization estimate, applying cross-validation (cv), i.e. NOT applying any fixed train/test splits. ...
le8rning's user avatar
1 vote
1 answer
298 views

Solve optimization problem with machine learning algorithm

I have an optimization problem that I solved with grid search using hyperopt in python. In this problem, I have some parameters and a score. I want to find the best ...
Mohammadreza Riahi's user avatar
2 votes
0 answers
99 views

How can I make my neural network learn faster?

I would like to train an LSTM-based variational autoencoder on a large dataset (37 million sentences). However, I have calculated that my training speed as of now is too slow (on Google Colab). I am ...
postnubilaphoebus's user avatar
0 votes
1 answer
340 views

Understanding the stochastic average gradient (SAG) algorithm used in sklearn

For pedagogical purposes I've been trying to create my own implementation of the stochastic average gradient (SAG) algorithm in a logistic regression framework. Page 10 of the associated paper ...
hillard28's user avatar
0 votes
1 answer
37 views

What is the best way to use three different losses on two classifiers?

Two classifiers need to be trained simultaneously, and I have three losses, as shown in the figure. Classifiers 1 and 2 will be updated by losses 1 and 2. Furthermore, loss 3 should update the two ...
phantrang's user avatar
0 votes
0 answers
20 views

How to decide a State for Deep Q Learning for Production Line scheduling

There is a production floor with W workstations and N jobs with M operations( different processing times per operation ). A job is completed only if its M Operations are completed. Objective is to ...
ArchanaR's user avatar
1 vote
0 answers
262 views

How Adam optimizer influence the learning rate? [closed]

I read some papers about how ADAM optimizer works, and there are some issues which seems that are confusing: ADAM equations are: ...
user3668129's user avatar
2 votes
2 answers
974 views

Does settings $\beta_1 = 0$ or $\beta_2 = 0$ means that ADAM behaves as RMSprop or Momentum?

I read on ADAM optimizer, and I saw multiple quotes which say that ADAM is a combination of Momentum and RMSprop optimizers. So if we: Set $\beta_1 = 0$ does it means that ADAM behaves exactly as ...
user3668129's user avatar
0 votes
1 answer
27 views

Are there any error functions with imbalanced negative/positive impact

I have a regression task, where positive error should be much worse than negative one. It means the importance of positive error bigger. For example, If real value is less than predicted one weights ...
Timofey's user avatar
  • 23
1 vote
3 answers
231 views

Why do neural networks with more layers perform better?

Why do neural networks with more layers perform better than a single layer MLP with a number of neurons that leads to the same number of parameters? I read this post: https://www.quora.com/Why-do-...
user3668129's user avatar
0 votes
1 answer
521 views

Why does Adam outperform SGD in logistic regression?

I am training a logistic regression model. In case it matters, the features are 1376-dimensional embeddings output from a neural network. I tried both SGD and Adam with a learning rate of $10^{-3}$ ...
nalzok's user avatar
  • 113
2 votes
0 answers
86 views

Can I find the input that maximises the output of a Neural Network?

So I trained a 2 layer Neural Network for a regression problem that takes $D$ features $(x_1,...,x_D)$ and outputs a real value $y$. With the model already trained (weights optimised, fixed), can I ...
puradrogasincortar's user avatar
0 votes
1 answer
46 views

Quadratic approximation, second-order optimization method, Newton method

I am learning Newton's method for second-order optimization in ML. I encountered this formula, but I do not understand how we get it. I guess it is from the Taylor series, but I still cannot fully ...
eons_stills_0r's user avatar

1
2 3 4 5
10