0
$\begingroup$

I want to use GridSearchCV for search best parameters for my CNN models to detect ECG anomaly. I have two dataframe which defined as train and test datasets, since want to follow previous research, which divide train and test not by number of data proportion but record proportion( a record can contain one or more data on it, since it is an ECG record).

However i get problem with GridSearchCV since it only use single unified dataset.

This is the code that i have try: I declare a function to make the set of my validation data as same as i expected:

def folding_maker(train,valid):
    t = [train,valid]
    t = pd.concat(t)
    tY = t.pop('CLASS').to_numpy()
    tY = to_categorical(tY)
    t = t.drop(columns=['RECORD_NAME','Minute']).to_numpy()
    t = np.expand_dims(t,axis=-1)
    folded = [-1 if x in train.index else 0 for x in valid.index]
    return t,tY,folded

Then build the model using Keras library:

def GetModel(dropout_rate=0,
           feature_len=0,
           wconv=0,
           epc=0,
           poolsize = 0,
           stride = 0):
    model = Sequential([
        Conv1D(filters=feature_len,kernel_size=wconv,activation='relu'),
        ...
        Flatten(),
        Dense(2,activation='softmax')
    ])
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy','mse'])
    return model

and I set my parameters in GridSearchCV like this:

dropout_rate = [0.3, 0.5, 0.7]
feature_len = [64,128,256]
wconv = [3,5,7,9]
epc = [100,200,500,1000]
stride = [1,2,3]
poolsize = [2,3,4]
para = dict(dropout_rate=dropout_rate,
           feature_len=feature_len,
           wconv=wconv,
           poolsize = poolsize,
            stride = stride,
           epc=epc)

and call it on my training flow like this:

 model = KerasClassifier(build_fn=GetModel,verbose=0)
        pds = PredefinedSplit(test_fold = folding)
        grid = GridSearchCV(estimator=model, param_grid=para, n_jobs=1, cv=pds)
        gs = grid.fit(tX, tY)

When i run the GridSearchCV, it always return this warning:

...
WARNING:tensorflow:11 out of the last 11 calls to <function Model.make_test_function.<locals>.test_function at 0x7ff1bf205598> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING:tensorflow:11 out of the last 11 calls to <function Model.make_test_function.<locals>.test_function at 0x7ff1bf205598> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING:tensorflow:8 out of the last 11 calls to <function Model.make_test_function.<locals>.test_function at 0x7ff1bed31268> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING:tensorflow:8 out of the last 11 calls to <function Model.make_test_function.<locals>.test_function at 0x7ff1bed31268> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING:tensorflow:5 out of the last 11 calls to <function Model.make_test_function.<locals>.test_function at 0x7ff1beed6620> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING:tensorflow:5 out of the last 11 calls to <function Model.make_test_function.<locals>.test_function at 0x7ff1beed6620> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
...

Which is pretty confusing, and at the end, it gives me the result that the first value of each list in para dict are the best ones. Is there something wrong or i miss something?

$\endgroup$
5
  • $\begingroup$ What is the problem? $\endgroup$ – Ben Reiniger Dec 28 '20 at 2:14
  • $\begingroup$ I want to used GridSearchCV but with my own validation set, since i have prepared it before. But i don't know how to apply it without merge it with my train test. Both train and validation are separated pandas dataframe $\endgroup$ – Yosafat Vincent Saragih Dec 28 '20 at 8:16
  • $\begingroup$ What's wrong with the code you've provided? $\endgroup$ – Ben Reiniger Dec 28 '20 at 14:38
  • $\begingroup$ ...or is the point just that you'd prefer not to join the training and validation frames? Whatever the case, please edit the question with that information. $\endgroup$ – Ben Reiniger Dec 28 '20 at 18:34
  • $\begingroup$ I have update the problem. It seems i have done it right and the cv runs but there's warning and the Gridsearch always return the first value of each params with score 1.000 $\endgroup$ – Yosafat Vincent Saragih Jan 14 at 14:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.