The first solution might give you a lot of headache as some convergence problem may arise from changing the dataset every n iterations. The second one seems the way to go, but before considering a more complex approach with generators, you might want to have some considerations at your memory usage.
Basically memory usage is : number of rows * number of columns * average memory usage. As it happens there are simple solutions to reduce each of those terms independently that often makes wonder as their reduction is multiplicative.
Average memory usage: this one has to be done on each datasets. Basically, working on your datatypes you can reduce the memory usage. This need to be done careffuly so as not to loose too much information. But basically reducing your float types from float64 to float32 already divide your memory usage by a factor of 2. This factor can go up to 8 as you could use small int8 or categories for some variables. (Plus it speeds up calculations, which might end up also giving you statistical performance boost). In the case of multiple csv files it is best to have a dictionnay of types that ensure general coherence.
Number of columns: with Machine Learning there is always the temptation to dump the whole database in the SotA algo and let it sort things out. This is not really a good approach. Usually it is best to start with a simple model (linear / logistic regression), including a smaller set of features. Selecting a subset of pertinent variables means that you will have to do some explanatory analysis. It will help you to handle your problem : having a better understanding of the relationship between features and target, observing problems like outliers and missing values, and being able to explain it. You can even use things like glmnet that do a bit of feature selection for you. With a dozen variables we are talking a factor 10 in memory usage reduction.
Number of rows : when building a model it is often practical to downsample it significantly, like at a 10% rate. This allows you to check you model building step by step without having to actually do all the calculations. Then you can reduce the downsampling. depending on your problem learning on 25% or 50% of your dataset might be more than enough, while giving a memory reduction of a factor 2-4.
Depending on your problem - if there is a variable that mainly drive the underlying behavior - you might want to build multiple models, one for each category of said variable (or buckets of a continuous variable). It's not always intuitive as most Data Scientist will prefer having one model, but sometimes it is more practical to use / handle / explain. And in your case it would allow for handling smaller databases.
All in all, those simple considerations would be enough to reduce your memory usage by a factor of 40-80.