The main distinction to be made here is between a parameter and a hyperparameter; once we have this clarified, the rest is easy: grid search is not used for tuning the parameters (only the hyperparameters), which are tuned with gradient descend.
Now, roughly speaking, a parameter is something that changes during training; in a neural network, the only parameters are the weights and the biases, and they are tuned with gradient descend.
A hyperparameter can be thought of as something "structural", e.g. the number of layers, the number of nodes for each layer (notice that these two determine indirectly also the number of parameters, i.e. how many weights and biases there are in our model), i.e. things that do not change during training. Hyperparameters are not confined to the model itself, they are also applicable to the learning algorithm used (e.g. optimization algorithm, learning rate, etc). A specific set of hyperparameters defines a family of models, which differ among themselves in the exact values of their parameters; in contrast, a specific parameter set (e.g. weights & biases in a NN) defines a unique model.
Having clarified the above, it should be easy to see that a in your example above is a hyperparameter and not a parameter; as such, it would normally be optimized using grid search.
To me, the second option is better since it can lead to the optimal point
Not so fast; gradient descend is not applicable everywhere: there are mathematical prerequisites for a function in order to be eligible for optimization using gradient descend, i.e. to be continuous & differentiable. This is indeed the case for the loss as a function of the weights & biases, but not clear if it is also the case for the loss as a function of your parameter a. And if it isn't, we simply cannot use gradient descend for tuning a.
On the contrary, techniques like grid search do not have such prerequisites, thus they can be used for a much wider range of cases.
in grid search, how do we pick the range?
As we pick almost everything else in ML & DL: empirically, and with trial & error (there is actually not much theory behind it).
UPDATE (after comment):
Now assume that the prerequisites are met for a so that we can use gradient descent. Which one is better now? Grid search or gradient descent?
Sorry, I didn't realize that this was a "Superman vs. Batman" question... Well, gradient descend is a disciplined mathematical operation, which is guaranteed to find the global minimum for convex functions (although NN loss functions are not convex), while grid search is actually just an ad hoc, quick-and-dirty procedure which does not guarantee anything (as you have already correctly suspected), so...
Beware though, especially if this is part of any homework or exam: the question, despite its phrasing, is not actually asking you which one is "better" in an abstract & hypothetical context, but which one is better in a very specific context, i.e a neural network model; and if something is not even applicable, it cannot be actually better, right?
In other words, if someone is actually testing your ML & NN knowledge here, they are certainly more interested in the exposition above, rather than which one is "better" in general; and choice (2) would be a huge mistake (after all, assume that the NN loss is a continuous & differentiable function of a is not part of the question).
