Skip to main content

Questions tagged [optimization]

In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

Filter by
Sorted by
Tagged with
0 votes
1 answer
38 views

Recommended number of features for regression problem

In the following link the answer recommends a feauture amount of N/3 for regression (or it is quoted). Where N corresponds to the sample size: How many features to sample using Random Forests Is there ...
qwertzi's user avatar
6 votes
2 answers
248 views

How sklearn SVM find the initial hyperplane before Optimisation?

The optimization goal of the SVM is to maximize the distance between the positive and negative hyperplanes. But before optimizing, how does sklearn first find the positive and negative support vectors ...
user3363813's user avatar
0 votes
1 answer
70 views

Finding global optimum of unknown and expensive function

I would like to find optimal combination of parameters for the algorithm affecting the disk space used by some storage. Therefore, several algorithm parameters (...
Vitaly Isaev's user avatar
3 votes
2 answers
2k views

Gradient descent implementation of logistic regression

Objective Seeking for help, advise why the gradient descent implementation does not work below. Background Working on the task below to implement the logistic regression. Gradient descent Derived the ...
mon's user avatar
  • 731
1 vote
2 answers
545 views

Determining the optimal number of clusters by elbow method

I have a dataset that consists of 700 categorical columns and around 6000 rows. I created 2-50 clusters with the k-mode algorithm and plotted the cost function to determine the optimal number of ...
Alexander Jaenisch's user avatar
0 votes
2 answers
2k views

Is reinforcement learning analogous to stochastic gradient descent?

Not in a strict mathematical formulation sense but, would there be there any key overlapping principals for the two optimisation approaches? For example, how does $$\{x_i, y_i, \mathrm{grad}_i \}$$ (...
hH1sG0n3's user avatar
  • 2,058
2 votes
1 answer
83 views

SVM - Why we use the dual theorem?

Why in SVM we use the dual theorem? I can't understand why we cannot minimize the norm of the weights w directly.
Giorgio Martinez's user avatar
0 votes
0 answers
24 views

regression quality with meta score using R2 and MAE for optimisation

Considering quality of regression models I currently try to compare two types of information: The $R^2$ score that give me the information about the tendency of the predictor The $MAE$ (or $RMSE$) ...
lelorrain7's user avatar
0 votes
0 answers
29 views

Which approach is beneficent for identifying the fake news detection?

The problem is to identify the fake news detection, As this is text classification problem . Constraints are basically that we cannot use traditional machine learning and deep learning approaches. If ...
Hamza's user avatar
  • 240
3 votes
1 answer
135 views

Reinforcement Learning applied to Optimisation Problem

Problem Statement: We are given an optimisation problem; with production centres, source airport, destination airports, transfer points and finally delivered to the customers. This is better explained ...
Alpha's user avatar
  • 31
1 vote
1 answer
46 views

Tuning a multivariate process automatically

I have a process to optimize which involves multiple algorithms. These algorithms are mostly interchangeable, but can have different performance benefits depending upon the input, and depending upon ...
ofcsub's user avatar
  • 13
1 vote
2 answers
118 views

How to find criterion that best separates two populations in a dataset?

I have a dataset of two identified populations that contains various parameters for each data point. I would like to find the best criterion, i.e. the relation between e.g. three of those parameters, ...
mapf's user avatar
  • 111
4 votes
1 answer
163 views

How do I minimizie cost for EV charging?

I want to find a charging schedule that minimize cost of charging an EV. The main objective is to have a fully charged car for the next morning, but the sub objective is to minimize cost based these ...
NorwegianClassic's user avatar
1 vote
0 answers
17 views

How can we optimize a model to predict in the no shortest possible time (real time production model)?

I need to put a model in production and I have some questions: How can we measure the time it takes to predict? Let's consider data is ready (real time) and we need first to transform data than to ...
rebar's user avatar
  • 31
1 vote
2 answers
6k views

Warmup steps in deep learning [closed]

What do warm steps and warmup proportion mean? how to select the number of warmup steps? Learning rate changes for each batch or each epoch for warmup step=1 ?
SS Varshini's user avatar
1 vote
1 answer
183 views

Why does stochastic gradient descent lead us to a minimum at all?

Why do we think that stochastic gradient descent is going to find a minimum at all? I mean on each iteration SGD moves in the direction that reduces only current batch's error (SGD doesn't care about ...
mathgeek's user avatar
  • 121
1 vote
1 answer
715 views

Numpy array vs Pandas DataFrame when training [closed]

https://towardsdatascience.com/speed-testing-pandas-vs-numpy-ffbf80070ee7 (You can open the link in incognito if its locked). Numpy arrays are faster than DataFrame on normal mathematical operations. ...
Ayazzia01's user avatar
  • 103
2 votes
1 answer
140 views

What does "regularization" actually refer to?

I am familiar with regularization, where we add a penalty in our cost function to force the model to behave a certain way. But is this a definition of regularization? Typically we regularize to get a &...
jtb's user avatar
  • 168
1 vote
1 answer
297 views

Understanding SVM's Lagrangian dual optimization problem

I was going through SVM section of Stanford CS229 course notes by Andrew Ng. On page 18 and 19, he explains Lagrangian and its dual: He first defines the generalized primal optimization problem: $$ \...
Mahesha999's user avatar
1 vote
1 answer
238 views

Understanding Lagrangian equation for SVM

I was trying to understand Lagrangian from SVM section of Andrew Ng's Stanford CS229 course notes. On page 17 and 18, he says: Given the problem $$\begin{align} min_w & \quad f(w) \\ s.t. &...
Mahesha999's user avatar
0 votes
1 answer
85 views

Understanding Lagrangian for SVM

I was referring SVM section of Andrew Ng's course notes for Stanford CS229 Machine Learning course. On page 22, he says: Lagrangian for optimization problem: $$\mathcal{L}(w,b,\alpha)=\frac{1}{2}\...
Rnj's user avatar
  • 225
1 vote
0 answers
106 views

Best way to optimize problem with additively separable fitness function?

I am using a genetic algorithm to maximize a few hundred thousand real-valued variables. Each of the variables, $x_i$, has its own independent boundary condition. The fitness function uses each of ...
João Bravo's user avatar
1 vote
1 answer
76 views

What does it mean (non) convex "constraint"?

I was referring SVM section of Andrew Ng's course notes for Stanford CS229 Machine Learning course. On page 16, he says: SVM optimization problem can be given as follows: $$\begin{align} \max_{\...
Rnj's user avatar
  • 225
3 votes
2 answers
4k views

How to visualize optimization problems' feasible region?

Is there any tool to visualize the feasible region when given a set of Linear equations (equalities and inequalities). If not, can anyone suggest a way to visualize it? If I am going to do it myself ...
Mina Ashraf's user avatar
1 vote
0 answers
11 views

Multiple Time Series Impact

I have a marketing business question where the objective is to learn from my historical data to deduce the best marketing strategy. Input (Leading Indicator)= For each year I have multiple monthly ...
datascientist's user avatar
1 vote
1 answer
80 views

Significance of Convex Loss Function with Nonlinear Models

When used in a linear model, a convex loss function guarantees a unique global minimum for the parameters, which can be found by local optimization methods. However, when the model is nonlinear (e.g. ...
user1337's user avatar
  • 103
1 vote
0 answers
746 views

Best way to find nearest neighbor distance for large datasets

I am a grad student doing research using generative machine learning with pytorch, and I have generated a set of points. I would like to check how similar these new points are to the points I used in ...
Amateur Coding Bird's user avatar
0 votes
1 answer
194 views

Why is my Neural Network having constant loss and always predicting a singular value?

I am trying to make a neural network on a dataset with 257 features and 1 target variable. My code looks like the following: ...
bballboy8's user avatar
  • 101
0 votes
0 answers
63 views

Gradient descent in linear regression converges but the trend line is incorrect

For the dataset https://physics.info/linear-regression/dash-world.txt, I have been trying to implement linear regression for predicting the men record times as a function of year. I have used gradient ...
Bhavya Jain's user avatar
0 votes
1 answer
591 views

Randomforest code taking longer time every iteration

I have a prediction code that runs RandomForestRegressor and RandomForestClassifier. I call the functions 9 times each ...
PyNoob's user avatar
  • 83
0 votes
1 answer
52 views

How can I extract an optimized matrix of correlations from a larger data set?

Consider an Excel sheet containing a matrix of correlations between individual stocks and the combined portfolio as a whole: How can I extract an optimized matrix such that most stocks have a low ...
Zesty's user avatar
  • 103
1 vote
0 answers
75 views

How to do online retraining of model on a single new data point/observation?

I am trying to investigate the effect on performance on old data and new data when a classifier is retrained on only the new observation when it is encountered. The aim is to retrain the classifier on ...
Abanoub Ghobrial's user avatar
1 vote
1 answer
25 views

finding winning strategy

For a given asset, I have simulations of the price and implied volatility for T periods in N scenarios. Furthermore, assuming that I know the value of the risk-free asset (and the dividend yield), I ...
Alessio Martone's user avatar
0 votes
0 answers
118 views

Why the error between the measured data and model data is not minimizing in Python?

I want to fit the non-linear experimental data with the model function by estimating some parameters in the function. The model function I have is: ...
N_T's user avatar
  • 1
0 votes
2 answers
236 views

Can I run this job quicker for GridSearchCV?

I am using GridSearchCV for optimising my predictions and its been 5 hours now that the process is running. I am running a fairly large dataset and I am afraid I ...
PyNoob's user avatar
  • 83
0 votes
1 answer
303 views

Memorization in deep neural networks, random vs. properly labelled datasets

From about 19:20 in the video here: https://www.youtube.com/watch?v=IHZwWFHWa-w it shows the difference in value of the cost function for randomly labelled data vs. properly labelled data. What do ...
mLstudent33's user avatar
1 vote
1 answer
349 views

How does bayesian optimization with gaussian processes work?

Could someone explain in simple words what are gaussian processes how does bayesian optimization work and their combination?
Ben's user avatar
  • 560
0 votes
0 answers
464 views

using Reinforcement learning for binary classification

I want to build an agent for binary classification. I have a large dataset with two label (0 and 1). I want to build an agent to predict labels. I build a deep model and now I want to build an agent. ...
sdbvuf sbjdsfdib's user avatar
1 vote
1 answer
43 views

Multiclass data redistribution

I want to redistribute the data in classes according to new proportions and wonder what is the optimal way to do it. For example I have ...
James Flash's user avatar
0 votes
0 answers
36 views

GAN model with different optimization functions

Building GAN model contains the following steps: Build generator model, and choose ...
user3668129's user avatar
0 votes
1 answer
33 views

How can I optimise/parallelize my neural network code?

I have a neural network with 784 inputs, 30 hidden neurons and 10 output neurons. The main performance issue is when backpropagating. Currently it takes around 0.1 seconds for one iteration of ...
Tiltlord's user avatar
0 votes
0 answers
2k views

Sine curve fitting

I want to fit a a * abs(sin(b*x - c)) + d function for each of the following data. In most of the cases I'm able to get decent accuracy. But for some cases, I'm not ...
Vedanshu's user avatar
  • 101
1 vote
0 answers
270 views

Hyper parameters (window size and vector dimensions) tuning in word2vec using Grey Wolf Optimization

Using Grey wolf Optimization, I want to calculate optimal values of two hyper parameters: context window size and embedding size (vector dimensions) for word2vec skipgram model used for word embedding....
Anil Sharma's user avatar
1 vote
1 answer
359 views

Optimal points of $f(x,y)=x^2 + y^2 + \beta xy + x + 2y$

I am self-learning basic optimization theory and algorithms from "An Introduction to Optimization" by Chong and Zak. I would like someone to verify my solution to this problem, on finding ...
Quasar's user avatar
  • 113
1 vote
0 answers
472 views

BERT MLM overfitting [closed]

We are training the BERT model on masked language modeling task for the Russian Language. Our dataset consists of 60 mln texts with (128 tokens for each text) from online social networks, ...
ilia's user avatar
  • 111
1 vote
0 answers
15 views

Ant colony optimization for clustering [closed]

What do you mean by applying ant colony optimization (ACO) to clustering? What is the output one would get after it? Could you explain it using a two dimesional data set which is clustered into 3 ...
AI_Revolt's user avatar
2 votes
1 answer
164 views

avoiding premature convergence with neural networks (EA's)

I am currently writing a program that would be able to play snake on an 25*25 grid. It works by optimizing a set of weights of 300 different solutions (each solution would be a different neural ...
Nick Stevens's user avatar
0 votes
1 answer
47 views

Custom thresholds on categorical classification

When assessing a binary classification task, it is possible to search for particular threshold in order to have better score on some metrics (f1,recall,etc) through numerous methods. Unfortunately, it ...
nprime496's user avatar
0 votes
1 answer
50 views

Would a neural network trained on extracted features have the same accuracy as a full network with frozen layers?

Let's say that I train two neural networks on the exact same dataset. The first network is a VGG19 model with frozen convolutional layers so only the top dense ...
FloppyC0de's user avatar
1 vote
1 answer
110 views

Do I load all files at once or one at a time?

I currently have $1700+$ CSV files. Each of them is in the same format and structure, give or take a row or possibly a column at the end. Each CSV is $\approx 3.8$ MB. I need to perform a ...
Jonathan Miller's user avatar

1 2
3
4 5
10