Wikipedia lists some well-known approaches to hyper-parameter searches.
The brute-force scan/search, or a grid search across multiple parameters, is still a very common and workable approach. As is random search, just trying some variations of parameters automatically and picking the best result from cross validation.
A guided search by intuitive feel of what might best work is also still quite common approach, despite being characterised as "optimisation by graduate descent" (a pun based on the fact that a senior researcher will hand off the tuning work to their smart junior researchers who make educated guesses on hyperparameter values) - for regularisation parameters you can look at training curves and difference between training loss vs cv loss to help decide where to look. You generally want to increase regularisation if there is a large difference and decrease it if the values become close but not very good. What counts and as "large difference" and "not very good" is subjective though, and typically you only get a feel for what works after a few random attempts and immersing yourself in the problem so that you recognise good vs bad behaviour of a model.
There are some claimed "smart" schemes that apply Bayesian optimisation or similar, and even some automated services or paid tools that attempt to make these available for tuning models automatically. However, I don't see these used much in practice.
Unless you are trying to win a Kaggle competition, you quickly hit diminishing returns attempting to fine-tune your parameter. I've rarely found tuning the second significant digit of a meta-param like L2 loss to be worth much attention. Maybe with the exception of testing between e.g. 1.0 and 1.5 - often useful to think in terms of geometric progression, not linear, even when fine-tuning.
You also have to be careful that you are not simply tuning to your cross-validation set, with no meaningful or justifiable improvement for test cases or in production. You can mitigate this in part by using k-fold cross validation (which can reduce error on metrics estimates effectively by using more data to estimate them), although of course that increases time/CPU costs.