Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
why is an optimizer used in something that is not a neural network?
in the neural style transfer architecture the image is updated instead of the neural network weights.
in stylegan encoder (find image in latent space), the latent space is updated instead of the weights of a neural network.
Why does it work in these cases, since it is not the weights of a neural network that is updated?
In general, there is more than one kind of optimiser. The gradient-based optimisers such as gradient descent, stochastic gradient descent, Adam, Adagrad, RMSProp, form one broad category of optimiser that can find minima or maxima of scalar functions provided it is possible to calculate a gradient.
In your question however, you are effectively focusing just on gradient-based optimisers - why can they be used to alter input images or latent spaces, as well as neural network weights?
The answer is that these optimisers are not specific to neural network weights, or even to neural networks. They are appropriate whenever:
You can measure success at a task with a scalar function - typically this is a loss or cost function to minimise, but it can also be a numerical score that you wish to maximise.
The task involves parameters that you can control. This is a set of variables that can be changed in order to solve a problem. The weights of a neural network are one example, but so are the pixels in an image if your goal is to find an image that matches some criteria (such as looking like a photo of a face).
You have a way to estimate gradients of that function with respect to the parameters that you can control. Usually some form of back propagation is used to get from the loss function to the parameters you care about. That is the case in style transfer or stylegan, but it is not a requirement.
The basic requirement is being able to estimate the gradient of your scoring function with respect to parameters that can be changed. Then you can minimise or maximise that function - or at least find local minima and maxima. Neural network weights are just one common example of the approach.
Likewise, using minibatches of input/output pair examples is one common way of estimating gradients. It is commonly used when learning how to generalise from a data set, but is not the only possible way to get gradients - which approaches are available and make sense depends on the optimisation problem you need to solve.
