Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [cost-function]

The tag has no usage guidance.

0
votes
0answers
18 views

ML / Multivariable cost minimization problems / approach summary?

General Problem Class Minimize the scalar cost function: $f(\vec r) = f(r_1, r_2, ..., r_n)$ The type of problem I'm trying to solve has many local minimum and the global minimum has not been proven ...
0
votes
0answers
19 views

What is the difference in the two implementations of cost functions below, in numpy?

cost = -1/m * np.sum( np.dot(Y.T, np.log(A)) + np.dot((1-Y.T), np.log(1-A))) versus ...
6
votes
2answers
69 views

What is an intuitive explanation for the log loss cost function?

I would really appreciate if someone could explain the log loss cost function And the use of it in measuring a classification model performance. I have read a few articles but most of them ...
0
votes
0answers
33 views

MSE over Euclidean Distance for cost function

When calculating the cost (the distance between two vectors), why is MSE used over Euclidean distance? Is it the case that you could use ED but MSE is better, if so - why?
3
votes
2answers
67 views

Logistic regression cost function

In Aurelien Geron's book I found this line ...
0
votes
1answer
88 views

Should the minimum value of a cost (loss) function be equal to zero?

We know optimization techniques search in the space of all the possible parameters for a parameter set that minimizes the cost function of the model. The most well-known loss functions, like MSE or ...
0
votes
1answer
40 views

How to Define a Cost Fucntion?

I want to define a cost function in python to identify optimum value in days when i should end a marketing campaign to save spend on campaigns not generating traffic good traffic. Problem is I dont ...
1
vote
1answer
74 views

boosting an xgboost classifier with another xgboost classifier using different sets of features

What I would like to do, is train a first model $f_{1}(\underline{x})$, where $\underline{x}$ is a set of features, fix what model 1 has learned, and then train a second model $f_{2}(\underline{y})$ ...
2
votes
0answers
59 views

How to resolve the instability of average reward per episode in training of DQN (Deep Q-Network)?

what is shown when average reward per episode in training is unstable? If there is big difference between average reward per episode and final reward by test section, what we can say? For ...
0
votes
1answer
63 views

Policy Gradient Methods - ScoreFunction & Log(policy)

In Policy Gradient Methods, Lecture 7 (34:15), David describes a Score Function as being the Gradient of the Log of the policy Question: If we have a Neural ...
0
votes
1answer
183 views

Should the cost function be zero using TensorFlow's sigmoid_cross_entropy_with_logits?

I'm building a CNN to make a binary classification (1 or zero). For this I'm using the cost function sigmoid_cross_entropy_with_logits. But for some reason the ...
2
votes
1answer
619 views

What is the Time Complexity of Linear Regression?

I am working with linear regression and I would like to know the Time complexity in big-O notation. The cost function of linear regression without an optimisation algorithm (such as Gradient descent) ...
1
vote
2answers
37 views

Why do we double the number in a quadratic cost function or MSE?

$$ C(w,b) = \frac{1}{2n}\sum_{x}||y(x)-a||^2 $$ Where y is a 10-dimensional vector, a is the output, w is the weight and b is the bias and n is the number of inputs. If this is the MSE, shouldn't it ...
0
votes
0answers
13 views

How Metacost operator act on Knn

Intuitively, the metacost operator ( in a case where we have only 2 classification classes) allow to increase the cost associated in making a misclassification error. In rapidminer software, i can ...
4
votes
2answers
421 views

What are the cases where it is fine to initialize all weights to zero

I've taken a few online courses in machine learning, and in general, the advice has been to choose random weights for a neural network to ensure that your neurons don't all learn the same thing, ...
0
votes
0answers
420 views

Softmax regression cost function and Logistic regression cost function not giving same value?

I'm writing a python program for Softmax regression using equations found here. According to this, the cost function of a Softmax classifier \begin{align} J(\theta) = - \frac{1}{m} \left[ \sum_{i=1}^{...
2
votes
0answers
264 views

Cost/loss functions for multi-tasking regression neural networks

The mean square loss function is the standard for regression neural networks. However, if I have a neural network learning two tasks (two outputs) at once, is it more advisable to train on the sum of ...
0
votes
0answers
53 views

Gradient descent - can I draw function that I will minimize? Linear regression

I'm new in machine learning. I started from linear regression with gradient descent. I have python code for this and I understad this way. My question is: Gradient descent algorithm minimize function, ...
0
votes
0answers
32 views

What is the approximate function for this surface plot shown in picture?

I am looking to repurpose the surface plot attached. I can't seem to figure out the function that represents this. I've tried many time in pyplot code.
0
votes
0answers
566 views

cost function for logistic regression implementation in Python

I used the following function to implement my cost function for logistic regression in python ...
0
votes
0answers
155 views

is it a normal behavior of WGAN-GP to have this cost values?

I'm training WGAN-GP on some large images where they are not normalized. based on their research papers i never saw this cost values as what Im encountering right now. using this implementation WGAN-...
1
vote
1answer
231 views

asymmetric cost function for deep neural network binary classifier

I am building a deep neural network based binary classifier, with single output. The loss function I actually want to minimize is $$ \mathcal L(\hat y,y) = \begin{cases} 0, & \text{if $\hat y$ = ...
2
votes
2answers
390 views

logistic regression algorithm fails to work

I'm trying to code my own logistic regression algorithm using Andrew NG's machine learning using Octave. lectures. So what I did was make a csv file, the first row being some parameter and the second ...
0
votes
1answer
146 views

Derivation of the cross-entropy equation in Michael Nielsen's book

I am reading the book http://neuralnetworksanddeeplearning.com/chap3.html by Michael Nielsen. So this is a question mostly for the people familiar with the book and understanding the material. In the ...
0
votes
0answers
45 views

tuning a binary classifier for desired tradeoff bewteen type I / type II errors

Imagine a [neural network] binary classifier. Presented person's photo, it should output "positive" if person is wearing dark eyeglasses and "negative" otherwise, including 'beeing unsure'. Tricky ...
0
votes
0answers
35 views

Loss function that is robust to a shifting binary target

I'm trying to squeeze as much performance out of a very noisy data set as possible. Initially, I tried to predict a continuous variable, but my feature set performed no better than randomly chosen ...
1
vote
1answer
277 views

Gradient Descent in logistic regression

Logistic and Linear Regression have different cost functions. But I don't get how the gradient descent in logistic regression is the same as Linear Regression. We get the Gradient Descent formula by ...
2
votes
2answers
633 views

Softmax classifier never allows for 100% probability in LSTM?

When working with LSTM I am using a softmax classifier and a one-hot encoded vector approach. The softmax looks like this: $$S(h_i) = \frac{e^{h_i}}{\sum e^{h_{total}}}$$ notice, LSTM's result is a $...
0
votes
1answer
97 views

Custom c++ LSTM slows down at 0.36 cost is usual?

Am I missing a part of the puzzle? I have implemented LSTM in c++ which steadily decreases in error, but slows down at the certain error value. It also seems to predict most of the characters, but ...
1
vote
0answers
75 views

How to decrease the learning rate only when cost function is stagnant?

I'm training a very complex function in tensorflow. Is there a way to decrease the learning rate only when the cost isin't decreasing very much?
1
vote
1answer
494 views

Logistic Regression : Solving the cross-entropy cost function analytically

Logistic regression cost function is cross-entropy. It is defined as below: This is a convex function. To reach the minimum, scikit-learn provides multiple types of solvers such as : ‘liblinear’ ...
7
votes
1answer
1k views

Cost function for Ordinal Regression using neural networks

What is the best cost function to train a neural network to perform ordinal regression, i.e. to predict a result whose value exists on an arbitrary scale where only the relative ordering between ...
1
vote
1answer
790 views

Understanding Logistic Regression Cost function

Linear Regression cost function: $$J(\theta) = \frac{1}{2 m} \sum_{i=1}^m (h_{\theta}(x^{(i)}) - y^{(i)})^2$$ where: $$h_{\theta}(x) = \theta_0 + \theta_1 x_1$$ Logistic Regression cost function $...
12
votes
2answers
13k views

Python implementation of cost function in logistic regression: why dot multiplication in one expression but element-wise multiplication in another

I have a very basic question which relates to Python, numpy and multiplication of matrices in the setting of logistic regression. First, let me apologise for not using math notation. I am confused ...
0
votes
0answers
249 views

My cost function doesn't decrease when running minimization algorithm

I've been noticing that when running my optimization algorithm (SGD, momentum SGD or Adadelta), my cost function doesn't necessarily decrease, in fact, sometimes, it only increases over my epoches. I ...
1
vote
1answer
623 views

Is cross-entropy a good cost function if I'm interested in the probabilities of a sample belonging to a certain class?

I'm training a neural network that, for each of six classes, tries to predict the probability that a sample belongs to it. After that, I want to use these probabilities as fractions of the sample ...
0
votes
1answer
83 views

Any Neural Network implementations that allow for a cost function of more than just the network output?

I have an application of a straightforward MLP, for which the cost function is a function of both the network output, in addition to another value calculated from the network weights (actually the ...
0
votes
2answers
4k views

XGBoost change loss function

I'm using XGBoost (through the sklearn API) and I'm trying to do a binary classification. False Positives are much worse for me than False Negatives, how can I take this into account? The API ...
2
votes
2answers
2k views

Does MLP always find local minimum

In linear regression we use the following cost function which is a convex function: We Use the following cost function in ...
0
votes
1answer
622 views

Cost functions penalizing certain types of misclassification more heavily in tensorflow

I have 2 classes of data A and C. I want to create a NN with 3 output nodes, classifying data as either A, B, or C. Classifying an A or C correctly should have zero cost (very good). Classifying ...
0
votes
1answer
108 views

Computing weights in batch gradient descent

I have a reasonably large set of images that I want to classify using a neural network. I can't fil them all into memory at once, so I decided to process them in batches of 200. I'm using an cross-...
1
vote
0answers
170 views

Altered priors for classification trees

I am using the rpart function (in the "rpart" package) in R to fit a cost-sensitive classification tree. I have read the corresponding vignette, which says that misclassication costs are incorporated ...
1
vote
1answer
812 views

Cost of greater than 1, is there an error?

I'm computing cost in the following way: cross_entropy = tf.nn.softmax_cross_entropy_with_logits(y, y_) cost = tf.reduce_mean(cross_entropy); For the first ...