# Questions tagged [sgd]

Stochastic Gradient Descent (SGD) is an iterative algorithms used for objective function optimization used in machine learning models.

19 questions
Filter by
Sorted by
Tagged with
81 views

### Why does SGDRegressor with partial_fit not converge to the same R2 as RidgeCV

I have a dataframe of about 200 features and 1M rows that I can train a RidgeCV model and get an R2 of about 0.01 I'd like to scale up my training to 5M or 10M rows but that won't fit in memory for me ...
1 vote
23 views

### How to calculate gradient of MSE in backpropagation? [duplicate]

I want to implement a neural network from scratch to solve linear regression by using backpropagation. I don't understand how to compute the gradient of the MSE cost function with respect to each ...
7 views

### Solving constraint optimization with alternate gradient descent optimization

Problem Setup: Offline contextual bandit with logs of form - <input_context, action, reward>. Model: A logging/behavior policy $\pi_0$ is used to collect the log data, with context $x_i$, action ...
48 views

### SGD for Graph Neural Networks

I was going through some research papers about Graph Neural Networks; what struck me is that very often SGD is used as optimiser (as in PointGNN, DGCNN and Graphsage). I figured that for "regular&...
276 views

### Difference between sklearn's LogisticRegression and SGDClassifier?

What is the difference between sklearn's LogisticRegression classifier and its SGDClassifier? I understand that the SGD is an ...
201 views

I am trying to understand the working of gradient descent, stochastic gradient descent and mini-batch gradient descent. In case of gradient descent, gradient is computed on the entire dataset at each ...
1 vote
789 views

### Understanding SGD for Binary Cross-Entropy loss

I'm trying to describe mathematically how stochastic gradient descent could be used to minimize the binary cross entropy loss. The typical description of SGD is that I can find online is: \$\theta = \...
1 vote
117 views

### How variable alpha changes SGDRegressor behavior for outlier?

I am using SGDRegressor with a constant learning rate and default loss function. I am curious to know how changing the alpha parameter in the function from 0.0001 to 100 will change regressor behavior....
1 vote
28 views

### Understanding Learning Rate in depth

I am trying to understand why the learning rate does not work universally. I have two different data sets and have tested out three learning rates 0.001 ,0.01 and 0.1 . For the first data set, I was ...
1 vote
32 views

### Multiple models have extreme differences during evaluation

My dataset has about 100k entries, 6 features, and the label is simple binary classification (about 65% zeros, 35% ones). When I train my dataset on different models: random forest, decision tree, ...
1 vote
56 views

### How exactly do you implement SGD with momentum?

I am looking up sources to implement SGD with momentum, but they are giving me different equations. (beta is the momentum hyper-parameter, ...
791 views

### Can't use The SGD optimizer

I am using the following code: ...
1 vote
259 views

### Why does using Gradient descent over Stochatic gradient descent improve performance?

Currently, I'm running two types of logistic regression. logistic regression with SGD logistic regression with GD implemented as follows ...
1k views

### Learning rate of 0 still changes weights in Keras

I just trained a model (SGD) with keras and was wondering why the change of accuracy and loss from epoch to epoch doesn't really decrease that much when I lower the learning rate. So I tested what ...
3k views

### Changing the batch size during training

The choice of batch size is in some sense the measure of stochasticity : On one hand, smaller batch sizes make the gradient descent more stochastic, the SGD can deviate significantly from the exact ...
1 vote
2k views

### input shape of keras Sequential model

i am new to neural networks using keras, i have the following train samples input shape (150528, 1235) and output shape is (154457, 1235) where 1235 is the training examples, how to put the input ...
153 views

### Problem of multi class classification (Sklearn TfidfVectorizer and SGDClassifier)

I do the (text) topic classification using TfidfVectorizer and SGDClassifier, literally I want to classify the website into categories (like Sport, Business etc). Now, the problem is, that each ...
1 vote