I have been working on online learning for a few weeks now, especially with Vowpal Wabbit and logistic regression. My understanding of the online learning algorithms and the problem is alright but I can't get my head straight about the regularisation issue.
In a standard machine learning problem, one uses a validation dataset in order to tune regularisations parameters for L1 and L2. But how do you choose those regularisation parameters in an online setting? Do you just fix them from start, or should I update them while training occurs?