# line search in gradient descent applied to a convex function

I have been working on implementing a line search method for gradient descent where I made an assumption that at any given point on my surface of the loss function I can reach the minima by the single correct value of the learning rate $$\eta$$ which I should choose. I have been trying to find this learning rate using binary search but after the entire implementation, I came to realize that my assumption I made is wrong which means I cant directly reach my minima from any given point on the surface of the loss function for any given learning rate in a single step. Can I get a more intuitive explanation of why my initial assumption is wrong?

Edit: my loss function is convex and has a large number of parameters I am trying to learn ( multidimensional)

• my assumption was that as my function is convex I compute just a single gradient and one right $\eta$ will take me directly from this point to the minimum without visiting any intermediate points and computing gradients at them. so I was finding this " one right" $\eta$ using binary search. but this assumption turned out to be wrong as even with the right $\eta$ I am landing in a point with a lesser loss than the starting point but not the minimum. I was expecting an explanation to why this is happening – venkat Mar 8 '20 at 8:55
• From a given point I am finding three points $w_{1},w_{2},w_{3}$ that bracket my minima and am finding those corresponding $\eta_{1}$,$\eta_{2}$,$\eta_{3}$, now I find the best $\eta$ that takes me to the minima which lies in between these three etas by a method similar to binary search. This is where my assumption is failing because this best $\eta$ is taking me to a lower loss but not to the minima directly. – venkat Mar 9 '20 at 5:52