# Questions tagged [cost-function]

The tag has no usage guidance.

67 questions
Filter by
Sorted by
Tagged with
38 views

### What's the correct cost function for Linear Regression

As we all know the cost function for linear regression is: Where as when we use Ridge Regression we simply add lambda*slope**2 but there I always seee the below as cost function of linear Regression ...
26 views

### Training seems to be plateauing at every learning rate

So firstly I have a network that I'm using to approximate the value of a function. Recently, at about 50000 trains, it began to show no further advancement in training, at any learning rate. The ...
15 views

### Is the Cross entropy cost function the same as the Cross entropy loss?

Is the Cross entropy cost function defined as $J(\Theta) = -\frac{1}{m}\sum_{i=1}^{m}\sum_{k=1}^{K}y_{k}^{(i)}log(\hat{p}_{k}^{(i)})$ the same as the one implemented in ...
59 views

### Implementation cost function in logistic regression in python using numpy

I am implementing the cost function for logistic regression and have a question. The formulation for cost function is $J = -\frac{1}{m}\sum_{i=1}^{m}(y^{(i)}\log(a^{(i)})+(1-y^{(i)})\log(1-a^{(i)}))$ ...
353 views

### Why does the MAE still remain, at all?

This may seem to be a silly question. But I just wonder why the MAE doesn't reduce to values close to 0. It's the result of an MLP with 2 hidden layers and 6 neurons per hidden layer, trying to ...
10 views

### Cost-complexity pruning: what the method does exactly?

The tree::prune.tree R function has a method parameter, described in the guide as: character string denoting the measure of ...
9 views

### cost function for probability estimation on reduced range

the data: tabular data (low number of columns). the target is binary, available on the train data only. the task: i need to predict probability of a the target to be "safe" the way the ...
24 views

### Cost Function Binary Classification

I have imbalance dataset for binary classification problem. I want to create a custom cost function that takes into account not only the actual class and probability, but another variable "...
18 views

### Optimising character pair substitution costs in Levenshtein distance

In a typical edit distance algorithm - say Levenshtein - there are hardcoded costs for specific operations, such as insertion, substitution, and deletion. This is obviously a bad assumption (the ...
41 views

50 views

### Cost function - ideas

I build xgboost model for regression problem. By the default xgboost optimize $(y - y_{pred})^2$, so the RMSE will be the best eval metric to measure performance. But my task is to build the best ...
533 views

### How does cost function change by choice of activation function (ReLU, Sigmoid, Softmax)?

I am new to ML and as I take courses for the area DL, I am wondering, by our choice of activation function for the last layer, whether we take sigmoid, relu or softmax, would the formula for ...
11k views

### Why do we have to divide by 2 in the ML squared error cost function? [duplicate]

I'm not sure why you need to multiply by $\frac1{2m}$ in the beginning. I understand that you would have to divide the whole sum by $\frac1{m}$, but why do we have to multiply $m$ by two? Is it ...
24 views

### Decision Tree Optimize Deviation From Objective

I have the following problem: I have three classes/modes, let's call them car, bike, and walking. For any given test data instance with some environmental variables such as distance, road quality etc, ...
43 views

### How is the linear regression cost function evolved?

A couple of weeks ago I joined the Standford University machine learning course on Coursera. In that course, they directly gave the cost function formula without telling how this formula was evolved. ...
96 views

### How to get a rebalance strategy with a cost matrix?

In the case of a classification problem where a cost matrix is used to maximize the model performance, it is common to do a rebalance technique. Let's say for example that I have the following costs ...
2k views

### Cost sensitive classification with individual cost

I'm currently sitting on a problem, where i'm uncertain if there is not a much simpler solution. I'm trying to train a DNN with a dataset for a classification task that should be cost sensitive. ...
365 views

### Logitic Regression cost function - what if ln(0)?

I am building logistic regression from scrap. The simplified cost function I am using is (from machine learning course on coursera): in specific case during learning, one observation in training ...
72 views

### Understanding minimizing cost correctly

I cannot wrap my head around this simple concept. Suppose we have a linear regression, and there is a single parameter theta to be optimized (for simplicity purposes): $h(x) = \theta \cdot x$ The ...
18k views

### What do "compile", "fit", and "predict" do in Keras sequential models?

I am a little confused between these two parts of Keras sequential models functions. May someone explains what is exactly the job of each one? I mean ...
300 views

### Weights not converging while cost function has converged in neural networks

My cost/loss function drops drastically and approaches 0, which looks a sign of convergence. But the weights are still changing in a visible way, a lot faster than the cost function. Should I ensure ...
14k views

### How does Gradient Descent and Backpropagation work together?

Please forgive me as I am new to this. I have attached a diagram trying to model my understanding of neural network and Back-propagation? From videos on Coursera and resources online I formed the ...
216 views

### What is an intuitive explanation for the log loss cost function?

I would really appreciate if someone could explain the log loss cost function And the use of it in measuring a classification model performance. I have read a few articles but most of them ...
590 views

### Logistic regression cost function

In Aurelien Geron's book I found this line ...
4k views

### Should the minimum value of a cost (loss) function be equal to zero?

We know optimization techniques search in the space of all the possible parameters for a parameter set that minimizes the cost function of the model. The most well-known loss functions, like MSE or ...
120 views

### How to Define a Cost Fucntion?

I want to define a cost function in python to identify optimum value in days when i should end a marketing campaign to save spend on campaigns not generating traffic good traffic. Problem is I dont ...
573 views

### boosting an xgboost classifier with another xgboost classifier using different sets of features

What I would like to do, is train a first model $f_{1}(\underline{x})$, where $\underline{x}$ is a set of features, fix what model 1 has learned, and then train a second model $f_{2}(\underline{y})$ ...
166 views

### How to resolve the instability of average reward per episode in training of DQN (Deep Q-Network)?

what is shown when average reward per episode in training is unstable? If there is big difference between average reward per episode and final reward by test section, what we can say? For instance in ...
259 views

### Policy Gradient Methods - ScoreFunction & Log(policy)

In Policy Gradient Methods, Lecture 7 (34:15), David describes a Score Function as being the Gradient of the Log of the policy Question: If we have a Neural ...
577 views

### Should the cost function be zero using TensorFlow's sigmoid_cross_entropy_with_logits?

I'm building a CNN to make a binary classification (1 or zero). For this, I'm using the cost function sigmoid_cross_entropy_with_logits. But for some reason, the ...
11k views

### What is the Time Complexity of Linear Regression?

I am working with linear regression and I would like to know the Time complexity in big-O notation. The cost function of linear regression without an optimisation algorithm (such as Gradient descent) ...
151 views

### Why do we double the number in a quadratic cost function or MSE?

$$C(w,b) = \frac{1}{2n}\sum_{x}||y(x)-a||^2$$ Where y is a 10-dimensional vector, a is the output, w is the weight and b is the bias and n is the number of inputs. If this is the MSE, shouldn't it ...
6k views

### What are the cases where it is fine to initialize all weights to zero

I've taken a few online courses in machine learning, and in general, the advice has been to choose random weights for a neural network to ensure that your neurons don't all learn the same thing, ...
393 views

### Cost/loss functions for multi-tasking regression neural networks

The mean square loss function is the standard for regression neural networks. However, if I have a neural network learning two tasks (two outputs) at once, is it more advisable to train on the sum of ...
I am building a deep neural network based binary classifier, with single output. The loss function I actually want to minimize is  \mathcal L(\hat y,y) = \begin{cases} 0, & \text{if $\hat y$ = ...