0
$\begingroup$

I have a question related on how to use the GridSearch to find the best models for my problem with time series data.

Every 3 rows is 1 one row in the original dataset. To make my time series problem a supervised one, I parsed like the one below. This was resolved from one of my previous question.

id Age gender m1 m2 m3 Label
1 20 M 12.4 34 12 0
2 20 M 13 324 34 0
3 20 M 34 232 12 0
4 45 F 1.3 32 19 1
5 45 F 14 132 19 1
6 45 f 94 232 19 1

My question is: How can I use GridSearch for example to find my best machine learning model configuration using time series data? As far as I understand, using cross validation wouldn't work in this case because of the time series nature of the dataset.

I'm not sure how to proceed with this.

$\endgroup$
4
  • $\begingroup$ Sorry but I dont understand it. What is the dependent variable/column? $\endgroup$ – martin Feb 22 at 20:34
  • $\begingroup$ The original record would be like this. id,age,gender,m1_1, m2_1, m3_1, m1_2, m2_2, m3_2, m1_3, m2_3, m3_3, label. the m variables are measurement taking in a fixed time, for instance 2 hours. To make this time series problem a supervised one, I parsed the dataset as show above on the table. All this measurements indicates if a patient had an issue 1 or not 0. Does it make sense? $\endgroup$ – bws Feb 22 at 21:34
  • $\begingroup$ So it is a labeling problem now. Is that binary with 0 and 1, or is it a multilabel problem and there are more than 0 and 1 labels? $\endgroup$ – martin Feb 22 at 22:31
  • $\begingroup$ Just a binary problem with 0s and 1s. $\endgroup$ – bws Feb 22 at 22:40
0
$\begingroup$

So it´s a classification problem with a grid-search, without cross-validation. Yes, don´t use cv in time series data. There is an option, in which you can use cv, when you slowly start with less data and put more and more data during the process. But it´s complex.

For the grid-search are 2 opportunities. Either use GridSearchCV and define cv as none, or you use ParameterGrid(). For my interest I used this method:

https://stackoverflow.com/questions/44636370/scikit-learn-gridsearchcv-without-cross-validation-unsupervised-learning/44682305#44682305

in which is GridSearchCV defined as none.

import pandas as pd
test = pd.DataFrame({"id":[1,2,3,4,5,6,7,8,9], "age":[20,30,32,40,55,32,20,41,38], "gender":[0,1,0,1,0,0,1,1,0], 
            "m1":[12.4, 30,9.4,14,19,20,34,31,16], 'm2':[34,36,22,16,22,27,42,65,13], 'label':[0,0,1,1,0,1,1,1,0]})
test.head()

from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import GridSearchCV
from sklearn.metrics import accuracy_score

X = test.drop('label', axis=1)
y = test.label

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=42)

rf_params = {'n_estimators': [100, 200],
             'max_features': ['auto', 'sqrt'],
             'max_depth': [10, 50],
             'min_samples_split': [2, 20]}

cv=[(slice(None), slice(None))]

rf_clf = GridSearchCV(RandomForestClassifier(random_state=42),rf_params, n_jobs=-1, verbose=2, cv=cv)
rf_clf.fit(X_train, y_train)

#best parameters of model
print(rf_clf.best_params_)

#make predictions
rf_pred = rf_clf.predict(X_test)
print('Accuracy', accuracy_score(rf_pred, y_test))

The GridSearchCV part shows me:

GridSearchCV(cv=[(slice(None, None, None), slice(None, None, None))], estimator=RandomForestClassifier(random_state=42), n_jobs=-1, param_grid={'max_depth': [10, 50], 'max_features': ['auto', 'sqrt'], 'min_samples_split': [2, 20], 'n_estimators': [100, 200]}, verbose=2)

So this method works.

Here I used random forest, because in my own experience, random forest is in most cases very good. In big datasets, the SVC takes too much time.

PS: Before I forget, I changed the gender into numbers. You can use one-hot encoding for that or catboost, which can do this automatically. But with catboost you get different results in comparison with rf or other algorithms. So I prefer to transfer gender into numbers.

$\endgroup$
11
  • $\begingroup$ Thanks! I'm going to try to on my real dataset in a few. One question. The way you set up the dummy data is different than what I have. For instance, the first 3 records of the on my table above is for the same patient. so each column has a measurement that was taken within x amount of hours. Will it make a difference in this case? $\endgroup$ – bws Feb 23 at 3:33
  • $\begingroup$ It seems to work fine and it does return the best hyper-parameter configuration. Regarding the dataset, could you please clarify if I should break the original record into different rows or just keep all the time series data in the original format for a classification problem? $\endgroup$ – bws Feb 23 at 6:14
  • $\begingroup$ to answer this, I need your original given question and data. Now I dont understand whats really going on. Also my code is just an example. Would be gratefull for a voting :) $\endgroup$ – martin Feb 23 at 13:11
  • $\begingroup$ Here is the original question: datascience.stackexchange.com/questions/89668/… The dataset there is just a fake representation of my true dataset. If you need more information, I'd be happy to share it with you. Just let me know You got the vote! :) $\endgroup$ – bws Feb 23 at 18:20
  • $\begingroup$ I dont understand. You wrote, that these variables are 8 entries for measurement data during a day. And the label data (1 and 0) are the status of the patient. Either I need more information, or I think, this is just a classification problem, in which you dont use the CV because of time data. So just a prediction of the status with given dataset. Other question, why you dont check best models like arima or moving average?/ why must it be as a classification problem and not a time series problem? $\endgroup$ – martin Feb 23 at 20:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.