Can GridSearchCV be used for unsupervised learning?

Question

im trying to build an outlier detector to find outliers in test data. That data varies a bit (more test channels, longer/shorter testing).

First im applying the train test split because i want to use grid search for hypertuning. This is timeseries data from multiple sensors and i removed the time column beforehand.

X shape : (25433, 17)
y shape : (25433, 1)

X_train, X_test, y_train, y_test = train_test_split(X,
                                                    y,
                                                    test_size=0.33,
                                                    random_state=(0))

Standardize afterwards and then i changed them into an int Array because GridSearch doesnt seem to like continuous data. This surely can be done better, but i want this to work before i optimize the coding.

'X'
mean = StandardScaler().fit(X_train)
X_train = mean.transform(X_train)
X_test = mean.transform(X_test)

X_train = np.round(X_train,2)*100
X_train = X_train.astype(int)
X_test = np.round(X_test,2)*100
X_test = X_test.astype(int)

'y'
yeah = StandardScaler().fit(y_train)
y_train = yeah.transform(y_train)
y_test = yeah.transform(y_test)
y_train = np.round(y_train,2)*100
y_train = y_train.astype(int)
y_test = np.round(y_test,2)*100
y_test = y_test.astype(int)

I chose the IForrest because its fast, has pretty good results and can handle huge data sets (i currently only use a chunk of the data for testing). Setting Up the GridSearchCV:

clf = IForest(random_state=47, behaviour='new',
          n_jobs=-1)

param_grid = {'n_estimators': [20,40,70,100], 
              'max_samples': [10,20,40,60], 
              'contamination': [0.1, 0.01, 0.001], 
              'max_features': [5,15,30], 
              'bootstrap': [True, False]}

fbeta = make_scorer(fbeta_score,
                    average = 'micro',
                    needs_proba=True,
                    beta=1)

grid_estimator = model_selection.GridSearchCV(clf, 
                                              param_grid,
                                              scoring=fbeta,
                                              cv=5,
                                              n_jobs=-1,
                                              return_train_score=True,
                                              error_score='raise',
                                              verbose=3)

grid_estimator.fit(X_train, y_train)

The Problem:

I cant fit the grid_estimator. GridSearchCV needs an y_argument, without y its passing me the "missing y_true" error. What should be used as a target here ? Atm i just passed an important data column to y for testing, but im getting this error that i dont understand:

ValueError: Classification metrics can't handle a mix of multiclass and continuous-multioutput 
targets

I also got the advice that the i need a scoring function and the iForest doesnt have one. I couldnt find useful information for this, are there any helpful guides or info that can help me ?

Answer 1

The goal of GridSearchCV is to iterate over (hence search) all possible combinations (hence grid) of hyper parameters and evaluate a model on a cross-validation (hence CV). You do need some score to compare models with different sets of hyper parameters. If you can come out with some reasonable way to score a model after the fit, you can write a custom scoring function. If this scoring function does not require target (y) to be computed, you can simply pass an array of zeros to GridSearchCV. The example of such scorer is given here.

Otherwise, if you use some supervised model on a filtered (by IsolationTrees) data, you can do that using Pipelines, and run GridSearchCV on that, see examples in sklearn docs:

from sklearn.pipeline import Pipeline
from sklearn.ensemble import IsolationForest

estimators = [('filter_data_it', IsolationForest()),
              ('clf', LogisticRegression())]

pipe = Pipeline(estimators)
param_grid = dict(filter_data_it__max_features=[5,15,30], clf__C=[0.1, 10])
grid_search = GridSearchCV(pipe, param_grid=param_grid)

recall, that when you use Pipelines you need to prepend param_grid with the name of the pipeline step.

UPD1. As stated in the comments IF don't have method transform, thus simple chaining will not work. The way IF works is by predicting outliers and not by filtering the data (you are supposed to filter outliers afterwards). However, there is a way around this problem. We need to create a new class with transform method, which will run IF, and filter the data based on its predictions. I will update the code snippet.

It turns out there is no clear way to adapt sklearn API for that purpose, as stated in these questions, 1, 2, also this answer suggest a solution, however it is relatively complex. Thus, I suggest you proceed with scorer example.