Questions tagged [optimization]

In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

Filter by
Sorted by
Tagged with
0 votes
1 answer
22 views

How do I know that my weights optimizer have found the best weights?

I am new to deep learning and my understanding of how optimizers work might be slightly off. Also, sorry for a third-grader quality of images. For example if we have simple task our loss to weight ...
  • 3
2 votes
0 answers
17 views

Discussion about modern deep learning training strategies

Previously I have put a lot of effort into training networks appropriately. However, talking to colleagues, a lot of the things I did may be redundant due to novel optimizers and the theory of deep ...
1 vote
1 answer
33 views

Strategy to choose maximum value from an unknown array of n numbers

Suppose you have an array of n normally distributed numbers whose values are initially unknown(and the probability parameters are unknown too). You must choose one number and you want it to have ...
0 votes
0 answers
7 views

Optimization of a dispatch strategy

I am working at a project where I have to optimize the makespan of a system. The briefly description of the system: -I have an Input Output Machine which sends tubes to analyzers which has to execute ...
0 votes
0 answers
15 views

Jenks goodness of variance fit - Interpretation

I am working on clustering/grouping 1D data. I am trying to find bins of multiple variables seperately. So, I tried the jenks natural breaks algorithm. Based on the ...
  • 2,337
0 votes
0 answers
11 views

Recent research on solving "inverse" ODE problems with neural networks?

I come from a physics background, and I am not familiar with the state-of-the-art research in solving ODE optimization problems with NNs. Let me briefly introduce this so-called "inverse" ...
1 vote
0 answers
16 views

Hyperparameter Tuning Guidelines Across Lots of Models

I am performing a set of experiments in which I need to tune a fairly small set of hyperparameters but over a very large space of models trained on relatively related datasets and I'm trying to ...
  • 111
0 votes
0 answers
9 views

Isotonic regression with constraints on X threshold spacing

I am experimenting with using https://scikit-learn.org/stable/modules/generated/sklearn.isotonic.IsotonicRegression.html to approximate some noisy monotonic data. The generated curves have "jumps&...
  • 101
1 vote
0 answers
38 views

modelling- Regression tree with constraints on depth and monotonicity

My goal is to build a regression tree model that takes as input (the z-score: the odds of the logistic regression) and a binary variable (default/non default) as target but with constraint on the ...
0 votes
1 answer
19 views

How to solve Nonlinear least squares problem?

Initial idea is to use euclidean distances. But I do not understand how should I solve this task.
0 votes
1 answer
57 views

Optimize a non-linear function in Python

I am trying to optimize a function using scipy.optimize, but it does not converge. I have a trading strategy with a default stop-loss based on the lowest price over 20 days. I want to optimize this ...
0 votes
0 answers
12 views

Gradient descent/Adam converging to suboptimal solutions

I am using neural nets to find the minimum of a complex function to which I compute the mean (crit in my code). Here is my net : ...
  • 1
0 votes
0 answers
23 views

Dual function for kernel learning

I am asked to derive the dual loss function for the kernel logisitic regression because i should learn how to apply the kernel trick/representer theorem. Now the loss function is given by $L(\theta) = ...
  • 1
1 vote
2 answers
36 views

What DS/AI technique should I try for this optimization problem?

The dataset stores the prices of different stores of each item: Item Store1_price Store2_price Store3_price Apple 2.00 3.23 2.48 Table Salt 1.52 5.20 2.53 There will be around 10 stores and an ...
1 vote
0 answers
40 views

How to guide exploration in reinforcement leanring/model predictive control/dual control problem

Consider the following optimization/control problem: We aim to maximize the cumulative reward R during the horizon H by every day allocating a portion of total budget B to our two different investment ...
0 votes
0 answers
55 views

Multicollinearity in optimal control/reinforcement learning learning/resource allocation problem

Consider the following optimization/control problem: We aim to maximize the cumulative reward R during the horizon H by every day allocating a portion of total budget B to our two different investment ...
0 votes
0 answers
19 views

Suggestions for a python optimization library

I have a dataset with thousands of rows and at least 100 features. I need to be able to subset this dataset with a set of filters (>, <, <>) applied to my features so that my target column ...
0 votes
0 answers
31 views

Which optimization algorithm or method do I use?

I have a specific problem to solve, but not sure where to start my research as I'm not sure what the problem is called. I have an app with many users, that can be segmented based on various criteria (...
  • 101
0 votes
0 answers
12 views

How to create an A* Algorithm on geospatial data using custom heuristics

So my ultimate objective is to create an algorithm that can calculate the SAFEST route between two points on a map. I have created a dataset of unique lat,lng values and their "crime score" ...
1 vote
1 answer
80 views

Using scipy.minimize to find the maximum likelihood estimates for multivariate gaussian

Let's say I have a 100x2 normally distributed array of data. ...
  • 31
0 votes
0 answers
3 views

Adding a human insights into the Vehicle Routing Problem (VRP)

I have the following setting: Every week a similar VRP is solved, this solution is sent to the handler. The handler makes some changes to the suggested solution based on business knowledge (school ...
  • 121
0 votes
0 answers
16 views

What does Optuna actually do in each trial?

I am currently using Optuna library in python to optimize hyperparamaters in my LightGBM model as follows. ...
  • 1
0 votes
0 answers
25 views

Maximize Target Sum By Filtering Input Features

Feature 1 Feature 2 ... Target 0.7 0.3 ... 1.4 0.4 0.45 ... -2 0.7 0.15 ... -2.5 0.8 0.9 ... -3 1 0.4 ... 1.5 -1.5 0.1 ... 0.25 Imagine I have a dataset with almost 100 features containing 80....
0 votes
0 answers
8 views

Can we use a prediction algorithm like random forest or svm as an objective function for a optimization algorithm?

I've predicted target(y) for a dataset using random forest algorithm. Now I need to set a particular value for y and optimise x variables to get that y using a optimisation algorithm. In this case, ...
1 vote
0 answers
8 views

Optimizing utility and square root of divergence via SGD

I am trying to optimize a objective for learning-to-rank, which tries to max(min) the utility(risk) of a ranking function with logged user feedback data. The idea is to learn a ranking policy which ...
0 votes
0 answers
35 views

How to find the best optimized combination on generalized assignment problem and multi-dimensional knapsack problem?

Let say that company [X] has these following features product: A, B, C total target production a year: 100 target for each product: A: 30 B: 30 C: 40 total employees: 10 To achieve the target, ...
0 votes
1 answer
47 views

Understanding Conjugate Gradient Optimization methods

As a beginner in ML, I find it hard to understand how Conjugate Gradient Optimization methods work. The sources I've looked up online have a very complicated explanation. Can someone explain in a ...
  • 263
0 votes
0 answers
39 views

How does exactly eval_set and RandomizedSearchCV work for LightGBM?

How does RandomizedSearchCV form the validation sets, while I also defined an evaluation set for LGBM? Is it formed from the train set I gave or how does the evaluation set comes into the validation? ...
2 votes
1 answer
24 views

Choosing Right Optimiser and Data Scaling

The choice of optimiser and how data is scaled are both very important things in machine learning, yet they are not hyperparameters (as far as I am aware). It is also not necessarily obvious which ...
  • 71
1 vote
1 answer
30 views

Need advice on possible DS techniques for a problem

I am a former engineer, and a newbie data scientist, and am looking for advice on techniques to solve a particular problem for a personal project. Suppose that a store is selling 2 items, A and B, ...
  • 11
1 vote
1 answer
38 views

Optimize daily ice cream profit beased on simulation of all combinations input variables

I have an ice cream sales simulator for which I can simulate ice cream sales on any given day in the past. I want to optimize daily profit. The dependent variables for my ice cream shop which I have ...
0 votes
0 answers
13 views

Capacity planning and modelling

I have a business case in which I am going to model how many devices are required given the predicted workload in a series of monthly cohorts in the next ten years. The work could come from multiple ...
  • 763
0 votes
0 answers
3 views

Gradient Ascent and directional derivative

Suppose that you want to estimate a local maximum of the real function $f(x,y,z)$ with gradient ascent. Given a starting point $(x_0, y_0, z_0)$, the approach is to compute the gradient at this ...
  • 119
0 votes
1 answer
16 views

change parameterization to eliminate weight constraints in neural networks

I am wondering if it makes sense to use a parameterization to eliminate simple weight inequalities, for example if the weights should be $w\geq 0$, one cound train $\exp w$ over the unconstrained set ...
0 votes
0 answers
28 views

Necessity of Hessian Matrix

Hessian Matrix helps determine the saddle points, and the local extremum of a function. Source: https://machinelearningmastery.com/a-gentle-introduction-to-hessian-matrices/ Hessian Matrix is used in ...
  • 263
0 votes
0 answers
26 views

Optimizing classifier wrt. the given gain matrix problem

I have a classifier that classifies if a music is happy, angry, sad and relaxed. The dataset is comprised of low and high-level features only. Now I have trained a classifier, which performs better ...
0 votes
1 answer
24 views

Marketing Spend Optimization Techniques

I need some help with market spend optimization. I’m working with a client who’s running an offline operation that’s primarily driven by online marketing (fb, google, twitter etc). They had asked me ...
0 votes
0 answers
9 views

Input Signal Shape Optimization

I have a system, described by a black-box (a fully connected neural network), that takes as input a signal in time (let's say something similar to a sine wave) and returns a single scalar as output. ...
0 votes
1 answer
117 views

How to make XGBOOST capture trend in time series forecasting?

I am trying to forecast some sales data with monthly values, I have been trying some classical models as well ML models like XGBOOST. My data with a feature set looks like this with a length of 110 ...
0 votes
0 answers
20 views

uncertainties in non-convex optimization problems (neural networks)

How do you treat statistical uncertainties coming from non-convex optimization problems? More specifically, suppose you have a neural network. It is well known that the loss is not convex; the ...
  • 3
0 votes
0 answers
14 views

Is it beneficial to use a batch size > 1 even when all computing power can be used?

In regards to training a neural network, it is often said that increasing the batch size decreases the network's ability to generalize, as alluded to here. This is due to the fact that training on ...
  • 101
1 vote
2 answers
129 views

'Solvers' in Machine Learning

What role do 'Solvers' play in optimization problems? Surprisingly, I could not find any definition for 'Solvers' online. All the sources I've referred to just explain the types of solvers & the ...
  • 263
0 votes
0 answers
22 views

How to interpret arg min in the the following equation?

I am studying the following equation: $\hat{s}_m(n) = \text{arg}\text{min}_{s_m(n)\in A_s}|\frac{\psi_m^H}{||\psi_m^H||^2}y_m(n)-s_m(n)|^2$----(1) here $A_s$ is 1x$N$ vector of QPSK symbols, $s_m(n)$ ...
  • 1
0 votes
1 answer
43 views

Optimum weights for weighted average of 3 prediction models

I have 3 sklearn models which I use to predict a probability score for a binary classification problem. I want to create a weighted average score of all the predictions made by these models. I am ...
0 votes
0 answers
103 views

When using optuna I should return accuracy or loss as objective value?

I am using optuna for hyperparameter tuning for my segmentation model. At the model, I am returning accuracy as an objective value since I realised that it tries to optimize to get the best result ...
0 votes
0 answers
10 views

Project/scale a set of 2D points to keep a set of similarities constraints

I have a problem similar to this one posted here MDS scikit-learn example. I have a set of similarities between 2D points that I want to place in a map/plane while ...
  • 143
0 votes
1 answer
75 views

Regularization and loss function

I am currently trying to get a better understanding of regularization as a concept. This leads me to the following question: Will regularization change when we change the loss function? Is it correct ...
1 vote
0 answers
16 views

Structured policies in dynamic programming: solving a toy example

I am trying to solve a dynamic programming toy example. Here is the prompt: imagine you arrive in a new city for $N$ days and every night need to pick a restaurant to get dinner at. The qualities of ...
0 votes
0 answers
12 views

Plot six variables

I would like to plot a landscape spanned by six variables. The numerical target variable is explained by five numerical variables. Ultimately, it is about to get a visual impression for optima and the ...
  • 490
2 votes
0 answers
24 views

Sequential feature selection stopping condition

When using sequential feature selection approach, say forward feature selection, I want to stop adding new features when the improvement in model scores is smaller than a certain level. Is there a ...

1
2 3 4 5
10