3
$\begingroup$

I am currently dealing with a time-series data set with cyclical gaps every 30 minutes (30 minutes of data, 30 minutes of no data). Is there a relatively simple way of using scikit-learn (or some other Python library) to predict the missing data using the available data to train it? I assume this would involve a 'supervised training' approach? I have provided a graph for visual reference, the orange line is a 20-minute centralized moving average of the gap data set, and I want to fill in the data to look more like the green 60-minute centralized moving average from a separate complete data set. Thanks!

enter image description here

$\endgroup$
  • $\begingroup$ Can you plot the raw data (rather than a moving average of it)? The orange lines don't seem to be the raw data; they are only about 10 minutes out of every hour. Also, what makes you think the green line is the right way to fit that data? It seems to match very poorly all of the orange lines in the middle and only match the endpoints; it's not clear what error model would lead to this behavior. $\endgroup$ – D.W. Nov 7 at 8:03
4
$\begingroup$

You can approximate a time series with a polynomial of degree n_degree by using ridge regression. You can try different degree numbers (e.g. [2,3,4,5,6]) and choose the best one. Keep in mind that higher degree models, always get lower error values. So you should somehow penalize higher degrees.

from sklearn.linear_model import Ridge
from sklearn.preprocessing import PolynomialFeatures
from sklearn.pipeline import make_pipeline

for count, degree in enumerate([2, 3, 4, 5, 6]):
    model = make_pipeline(PolynomialFeatures(degree), Ridge())
    model.fit(Time, y)

More details here.

$\endgroup$
1
$\begingroup$

What you are trying to do is called imputing (i.e. filling gaps) in the time-series.

A guide to imputing time-series using python is available here:

https://medium.com/@drnesr/filling-gaps-of-a-time-series-using-python-d4bfddd8c460

The guide uses a few algorithms to impute data, but a more extensive list of algorithms can be found here:

https://otexts.com/fpp2/regression.html

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.