Intro
I'm quite new to all the subjects in this question. This is my very first shot at Keras, Tensorflow, NNs or Time Series. If you're not a newbie, you'll notice that immediately.
Problem
I have 15 samples from an industrial process that reads temperatures, weights and heatflows every 2 seconds for roughly an hour (1876 rows). Samples look like this:
temp weight heat
t
0 31.9661 9.09606 -0.834733
2 31.9905 9.09503 -0.839906
4 32.0967 9.09374 -0.854042
6 32.3021 9.09315 -0.877126
8 32.5371 9.09390 -0.917037
Each of the 15 samples has been analysed by an expert that has produced QA, rating 5 of them as lousy (0), another 5 as medium (5) and the third lot as perfect (10). My goal is to analyse further samples and produce an equivalent assessment.
Statistical analysis on the data revealed that the only feature showing differences is heatflow, so I'm focusing on that column. The column name is calor (heat in Spanish). I also detected very synchronous activity around the tails but big discrepancies over the middle section, so I'm also focusing on that section (rows 1250 to 3250).
Solution
I'm reading the CSVs into a dictionary and then passing it to a Keras network with the following code:
sequences = [data[k][['calor']].loc[1250:3250].values for k in data.keys()]
final_seq = np.array(sequences)
targets = [5, 5, 5, 5, 5, 0, 0, 0, 0, 0, 10, 10, 10, 10, 10]
groups = [1, 1, 1, 2, 3, 1, 1, 1, 2, 3, 1, 1, 1, 2, 3]
train = np.array([final_seq[i] for i in range(len(groups)) if groups[i] == 1])
test = np.array([final_seq[i] for i in range(len(groups)) if groups[i] == 2])
validation = np.array([final_seq[i] for i in range(len(groups)) if groups[i] == 3])
train_target = np.array([targets[i] for i in range(len(groups)) if groups[i] == 1])
test_target = np.array([targets[i] for i in range(len(groups)) if groups[i] == 2])
validation_target = np.array([targets[i] for i in range(len(groups)) if groups[i] == 3])
# Building the model
model = Sequential()
model.add(LSTM(11, input_shape=train[0].shape))
model.add(Dense(11, input_shape=(11,), activation='softmax'))
adam = Adam(lr=0.001)
chk = ModelCheckpoint('best_model.pkl', monitor='val_accuracy', save_best_only=True, mode='max', verbose=1)
model.compile(loss='categorical_crossentropy', optimizer=adam, metrics=['accuracy'])
train_binary = to_categorical(train_target)
validation_binary = to_categorical(validation_target)
model.fit(train, train_binary, batch_size=len(train), epochs=100, callbacks=[chk], validation_data=(validation, validation_binary))
model = load_model('best_model.pkl')
test_preds = model.predict_classes(test)
print(f'test_target: {test_target}\ntest_preds: {test_preds}')
acc = accuracy_score(test_target, test_preds)
print(f'Accuracy score: {acc}')
I get different results, but usually around the following:
Model testing...
test_target: [ 5 0 10]
test_preds: [10 0 10]
Accuracy score: 0.6666666666666666
Accuracy of 66% is the best I ever got so far.
Questions
Am I doing this right?
I'm not sure about some hyperparameters. I kind of arbitrated 100 epochs, the batch size as the total sample, the categorical_crossentropy loss function and the groupings as per the above groups list because of the targets (which are fixed).
I'm also not sure about the LSTM thing. My categories are not only numeric but ordinal. I could actually have a regression result here.
Should I even be using NN for this? Basically, any input is welcome.
I'm going to need much more data, correct?
I have 5 samples each. How many more should I have?
Thank you very much, experts!
