# Questions tagged [optimization]

In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

499 questions
Filter by
Sorted by
Tagged with
38 views

### Recommended number of features for regression problem

In the following link the answer recommends a feauture amount of N/3 for regression (or it is quoted). Where N corresponds to the sample size: How many features to sample using Random Forests Is there ...
248 views

### How sklearn SVM find the initial hyperplane before Optimisation?

The optimization goal of the SVM is to maximize the distance between the positive and negative hyperplanes. But before optimizing, how does sklearn first find the positive and negative support vectors ...
• 261
70 views

### Finding global optimum of unknown and expensive function

I would like to find optimal combination of parameters for the algorithm affecting the disk space used by some storage. Therefore, several algorithm parameters (...
• 103
2k views

### Gradient descent implementation of logistic regression

Objective Seeking for help, advise why the gradient descent implementation does not work below. Background Working on the task below to implement the logistic regression. Gradient descent Derived the ...
• 731
1 vote
545 views

### Determining the optimal number of clusters by elbow method

I have a dataset that consists of 700 categorical columns and around 6000 rows. I created 2-50 clusters with the k-mode algorithm and plotted the cost function to determine the optimal number of ...
2k views

### Is reinforcement learning analogous to stochastic gradient descent?

Not in a strict mathematical formulation sense but, would there be there any key overlapping principals for the two optimisation approaches? For example, how does $$\{x_i, y_i, \mathrm{grad}_i \}$$ (...
• 2,058
83 views

### SVM - Why we use the dual theorem?

Why in SVM we use the dual theorem? I can't understand why we cannot minimize the norm of the weights w directly.
24 views

### regression quality with meta score using R2 and MAE for optimisation

Considering quality of regression models I currently try to compare two types of information: The $R^2$ score that give me the information about the tendency of the predictor The $MAE$ (or $RMSE$) ...
• 195
29 views

### Which approach is beneficent for identifying the fake news detection?

The problem is to identify the fake news detection, As this is text classification problem . Constraints are basically that we cannot use traditional machine learning and deep learning approaches. If ...
• 240
135 views

### Reinforcement Learning applied to Optimisation Problem

Problem Statement: We are given an optimisation problem; with production centres, source airport, destination airports, transfer points and finally delivered to the customers. This is better explained ...
• 31
1 vote
46 views

### Tuning a multivariate process automatically

I have a process to optimize which involves multiple algorithms. These algorithms are mostly interchangeable, but can have different performance benefits depending upon the input, and depending upon ...
• 13
1 vote
118 views

### How to find criterion that best separates two populations in a dataset?

I have a dataset of two identified populations that contains various parameters for each data point. I would like to find the best criterion, i.e. the relation between e.g. three of those parameters, ...
• 111
163 views

### How do I minimizie cost for EV charging?

I want to find a charging schedule that minimize cost of charging an EV. The main objective is to have a fully charged car for the next morning, but the sub objective is to minimize cost based these ...
1 vote
17 views

### How can we optimize a model to predict in the no shortest possible time (real time production model)?

I need to put a model in production and I have some questions: How can we measure the time it takes to predict? Let's consider data is ready (real time) and we need first to transform data than to ...
• 31
1 vote
6k views

### Warmup steps in deep learning [closed]

What do warm steps and warmup proportion mean? how to select the number of warmup steps? Learning rate changes for each batch or each epoch for warmup step=1 ?
• 249
1 vote
183 views

### Why does stochastic gradient descent lead us to a minimum at all?

Why do we think that stochastic gradient descent is going to find a minimum at all? I mean on each iteration SGD moves in the direction that reduces only current batch's error (SGD doesn't care about ...
• 121
1 vote
715 views

### Numpy array vs Pandas DataFrame when training [closed]

https://towardsdatascience.com/speed-testing-pandas-vs-numpy-ffbf80070ee7 (You can open the link in incognito if its locked). Numpy arrays are faster than DataFrame on normal mathematical operations. ...
• 103
140 views

### What does "regularization" actually refer to?

I am familiar with regularization, where we add a penalty in our cost function to force the model to behave a certain way. But is this a definition of regularization? Typically we regularize to get a &...
• 168
1 vote
297 views

• 189
85 views

• 225
4k views

### How to visualize optimization problems' feasible region?

Is there any tool to visualize the feasible region when given a set of Linear equations (equalities and inequalities). If not, can anyone suggest a way to visualize it? If I am going to do it myself ...
• 131
1 vote
11 views

### Multiple Time Series Impact

I have a marketing business question where the objective is to learn from my historical data to deduce the best marketing strategy. Input (Leading Indicator)= For each year I have multiple monthly ...
1 vote
80 views

### Significance of Convex Loss Function with Nonlinear Models

When used in a linear model, a convex loss function guarantees a unique global minimum for the parameters, which can be found by local optimization methods. However, when the model is nonlinear (e.g. ...
• 103
1 vote
746 views

### Best way to find nearest neighbor distance for large datasets

I am a grad student doing research using generative machine learning with pytorch, and I have generated a set of points. I would like to check how similar these new points are to the points I used in ...
194 views

### Why is my Neural Network having constant loss and always predicting a singular value?

I am trying to make a neural network on a dataset with 257 features and 1 target variable. My code looks like the following: ...
• 101
63 views

### Gradient descent in linear regression converges but the trend line is incorrect

For the dataset https://physics.info/linear-regression/dash-world.txt, I have been trying to implement linear regression for predicting the men record times as a function of year. I have used gradient ...
591 views

### Randomforest code taking longer time every iteration

I have a prediction code that runs RandomForestRegressor and RandomForestClassifier. I call the functions 9 times each ...
• 83
52 views

### How can I extract an optimized matrix of correlations from a larger data set?

Consider an Excel sheet containing a matrix of correlations between individual stocks and the combined portfolio as a whole: How can I extract an optimized matrix such that most stocks have a low ...
• 103
1 vote
75 views

### How to do online retraining of model on a single new data point/observation?

I am trying to investigate the effect on performance on old data and new data when a classifier is retrained on only the new observation when it is encountered. The aim is to retrain the classifier on ...
1 vote
25 views

### finding winning strategy

For a given asset, I have simulations of the price and implied volatility for T periods in N scenarios. Furthermore, assuming that I know the value of the risk-free asset (and the dividend yield), I ...
118 views

### Why the error between the measured data and model data is not minimizing in Python?

I want to fit the non-linear experimental data with the model function by estimating some parameters in the function. The model function I have is: ...
• 1
236 views

### Can I run this job quicker for GridSearchCV?

I am using GridSearchCV for optimising my predictions and its been 5 hours now that the process is running. I am running a fairly large dataset and I am afraid I ...
• 83
303 views

### Memorization in deep neural networks, random vs. properly labelled datasets

From about 19:20 in the video here: https://www.youtube.com/watch?v=IHZwWFHWa-w it shows the difference in value of the cost function for randomly labelled data vs. properly labelled data. What do ...
• 594
1 vote
349 views

### How does bayesian optimization with gaussian processes work?

Could someone explain in simple words what are gaussian processes how does bayesian optimization work and their combination?
• 560
464 views

### using Reinforcement learning for binary classification

I want to build an agent for binary classification. I have a large dataset with two label (0 and 1). I want to build an agent to predict labels. I build a deep model and now I want to build an agent. ...
1 vote
43 views

### Multiclass data redistribution

I want to redistribute the data in classes according to new proportions and wonder what is the optimal way to do it. For example I have ...
• 301
36 views

### GAN model with different optimization functions

Building GAN model contains the following steps: Build generator model, and choose ...
• 393
33 views

### How can I optimise/parallelize my neural network code?

I have a neural network with 784 inputs, 30 hidden neurons and 10 output neurons. The main performance issue is when backpropagating. Currently it takes around 0.1 seconds for one iteration of ...
2k views

### Sine curve fitting

I want to fit a a * abs(sin(b*x - c)) + d function for each of the following data. In most of the cases I'm able to get decent accuracy. But for some cases, I'm not ...
• 101
1 vote
270 views

### Hyper parameters (window size and vector dimensions) tuning in word2vec using Grey Wolf Optimization

Using Grey wolf Optimization, I want to calculate optimal values of two hyper parameters: context window size and embedding size (vector dimensions) for word2vec skipgram model used for word embedding....
1 vote
359 views

### Optimal points of $f(x,y)=x^2 + y^2 + \beta xy + x + 2y$

I am self-learning basic optimization theory and algorithms from "An Introduction to Optimization" by Chong and Zak. I would like someone to verify my solution to this problem, on finding ...
• 113
1 vote
472 views

### BERT MLM overfitting [closed]

We are training the BERT model on masked language modeling task for the Russian Language. Our dataset consists of 60 mln texts with (128 tokens for each text) from online social networks, ...
• 111
1 vote
15 views

### Ant colony optimization for clustering [closed]

What do you mean by applying ant colony optimization (ACO) to clustering? What is the output one would get after it? Could you explain it using a two dimesional data set which is clustered into 3 ...
164 views

### avoiding premature convergence with neural networks (EA's)

I am currently writing a program that would be able to play snake on an 25*25 grid. It works by optimizing a set of weights of 300 different solutions (each solution would be a different neural ...
47 views

### Custom thresholds on categorical classification

When assessing a binary classification task, it is possible to search for particular threshold in order to have better score on some metrics (f1,recall,etc) through numerous methods. Unfortunately, it ...
I currently have $1700+$ CSV files. Each of them is in the same format and structure, give or take a row or possibly a column at the end. Each CSV is $\approx 3.8$ MB. I need to perform a ...