Increase number of iterations in a logistic regression

Question

I have achieved 68% accuracy using glm with family = 'binomial' while doing logistic regression in R. I don't have any idea on how to specify the number of iterations through my code. Any suggestions on it? Will application of stochastic gradient descent work for epoch. If so, how do I apply it in my code? I want to increase the accuracy of my model. Following is my code:

rm(list=ls())
library(dplyr)
data1 <- read.csv("~/hj.csv", head`enter preformatted text here`er=T)
train<- data1[1:116,]
VALUE<-as.numeric(rownames(train))
testset<- data1[1:116,]
mylogit <- glm(VALUE ~ POINT1 + POINT2 + POINT3 + POINT4 , data = data1, family ="binomial")
testset$predicted.value = predict(mylogit, newdata = testset, type="response")
for (i in 1: nrow(testset)){
  if(testset$predicted.value[i] <= 0.50)
    testset$outcome[i] <- 0 
  else testset$outcome[i] <- 1
}
print(testset)
tab = table(testset$VALUE, testset$outcome) %>% as.matrix.data.frame()
accuracy = sum(diag(tab))/sum(tab)
print(accuracy)
print(tab)
table(testset$VALUE, testset$outcome)

Answer 1

Logistic regression in R uses the iterative re-weighted least squares algorithm. You can specify the maximum iterations and accuracy with:

m <- glm(..., family = "binomial", control = list(maxit = 2, epsilon=1))

Please read the documentation for glm.control here.

You can check the actual number of iterations used in fitting:

m$iter

BUT... The default settings should be enough.

> glm.control()

$epsilon
[1] 1e-08

$maxit
[1] 25

$trace
[1] FALSE

The default maximum number of iterations is 25, and I **doubt** you will get anything by changing it to anything larger. The accuracy is 1e-08, which is already very small.

You shouldn't blindly adjust the iteration number, most likely it won't help. Try this:

> glm(admit ~ gre + gpa + rank, data = mydata, family = "binomial", control = list(trace=TRUE))

Deviance = 458.875865177 Iterations - 1
Deviance = 458.517518881 Iterations - 2
Deviance = 458.517492476 Iterations - 3
Deviance = 458.517492476 Iterations - 4

I got the dataset from http://www.ats.ucla.edu/stat/r/dae/logit.htm (because I don't have your data file).

Do you see I have trace=TRUE? I print off the deviance for each iteration. R stops because the deviance difference between iteration 3 and 4 is too small. My model only requires 4 iterations.

My recommendation:

• Print your deviance for each iteration and convince yourself you don't need more than 25 iterations
• Check your model properly. Do you need more independent variables? Do you need to transform your variables to improve prediction accuracy?
• Ask yourself is 60% accuracy enough? In many fields 60% is a huge success.
• Does your model require interaction? Note a saturated model will give you prefect accuracy, but it has all the interaction terms. Your model assumes the effects of your independent variables are additive in predicting the log-odd. Is this a fair assumption?
• Do you want to run automatic stepwise model selection algorithm in R? You might get a better model than manually done it yourself.

Some useful links you might want to consider:

• https://stats.stackexchange.com/questions/5354/logistic-regression-model-does-not-converge
• https://stats.stackexchange.com/questions/60958/logistic-regression-fishers-scoring-iterations-do-not-match-the-selected-itera