As a quick question, what are genetic algorithms meant to be used for? I read somewhere else that they should be used as optimization algorithms (similar to the way we use gradient descent to optimize the best parameters search with a linear regression, neural network...). If so, why are these GAs not so much present in machine learning (or at least I did not see it too much in the literature)?
Yes, as you found they (evolutionary algorithms such as genetic algorithms) are using for optimization tasks. One reason that they are not using mostly in machine learning could be their performance to converge to the optimum point. Also, implementation of the GAs for some domains could be problematic and it could not be generalized like gradient descent, as it's involved in at least 5 phases (mutation, crossover, ...).
I think you downplayed too much about GA. GA is still used actually in machine learning although not as widespread. You can see some implementation by code bullet and a more academic one for example. One of the issue with GA is it is most of the time it is computationally expensive and to some degree requires luck (because you are mutating instances randomly), hence it is usually not popular when we already know a more deterministic solution to a problem. But there is a set of problems where GA could excel, one of that is when you don't have differentiable objective e.g. parameter search, or reinforcement learning.
Tackling the second part of the question, yes GAs can be used in machine learning but are not the commonly used method.
Mainly, to optimize parameters on (parametric) models, it is common to use either a closed solution (like in least squares) or Gradient Descent (and it's variants) but the choice of Hyper-parameters are usually open to taste, GA's could be used in here, but people usually try things like Random Search, Grid Search and so on.