In the Coursera course machine learning in the section on Multivariate Linear Regression, Andrew Ng provides the following tips on gradient descent:
- Use Feature Scaling to converge quicker
- Get feature into an approx -1 < x < 1 range
- Mean normalization
Andrew Ng also provides some other tips:
- Plot cost vs iterations
- to ensure cost decreases on every iteration (try smaller alpha)
- to identify if convergence is too slow (try larger alpha)
- to identify approximately the number of iterations to converge
Are these tips applicable to all problems involving gradient descent using different machine/deep learning algorithms or just to Multivariate Linear Regression?