I am trying to predict "sales" from this dataset:
https://www.kaggle.com/c/rossmann-store-sales
There are>1,000,000 rows, I use 10 features from the dataset to predict sales
I merged two datasets into one in advance. I created a code in Keras to predict "sales". Firstly I created some new variables, threw away some unneeded data. Then I applied one hot encoding on categorical variables, split the dataset into train and test parts, scaled variables of X_train and X_test with StandardScaler. After that, I created a Keras model that looks like this:
model = Sequential()
model.add(Dense(units = 64, kernel_initializer = 'uniform', activation = 'relu', input_dim = 31))
model.add(Dropout(p = 0.1))
model.add(Dense(units = 64, kernel_initializer = 'uniform', activation = 'relu'))
model.add(Dropout(p = 0.1))
model.add(Dense(units = 64, kernel_initializer = 'uniform', activation = 'relu'))
model.add(Dropout(p = 0.1))
model.add(Dense(units = 64, kernel_initializer = 'uniform', activation = 'relu'))
model.add(Dropout(p = 0.1))
model.add(Dense(units = 1, kernel_initializer = 'uniform', activation='linear'))
model.compile(loss='mse', optimizer='adam', metrics=['mse', 'mae', 'mape'])
history = model.fit(X_train, y_train, batch_size = 10000, epochs = 15)
It is a pretty basic model: 4 layers, each has 64 neurons, small dropout to prevent overfitting, relu as an activator, mean squared error as loss function, adam as an optimizer, 15 epochs.
The results of this model:
- R-squared: 0.86
- MSE: 20841
- MAE: 103
I suppose it is doing a good job, this is a comparison of real and predicted values
y_test final_preds
0 1495.0 1737.393188
1 970.0 763.265747
2 660.0 696.281006
3 695.0 884.019226
4 802.0 620.464294
5 437.0 413.590912
6 599.0 564.844177
7 426.0 507.872650
8 1163.0 934.790405
9 563.0 591.833313
10 798.0 729.736572
11 507.0 422.795746
12 447.0 546.338440
13 437.0 437.536194
14 599.0 643.752441
15 607.0 667.271423
16 836.0 793.968872
17 568.0 599.968262
18 522.0 508.874084
19 350.0 395.198883
20 1160.0 1277.464111
I tried to "mimic" the same structure of neural network with the same configurations in Tensorflow by using DNNRegressor. The results were not even close to what Keras achieved. My code for TF is:
Creating feature columns
DayOfWeek_vocab = [4, 3, 1, 5, 6, 2, 7] DayOfWeek_column = tf.feature_column.categorical_column_with_vocabulary_list( key="DayOfWeek", vocabulary_list=DayOfWeek_vocab Open_vocab = [1] Open_column = tf.feature_column.categorical_column_with_vocabulary_list( key="Open", vocabulary_list=Open_vocab) Promo_vocab = [1,0] Promo_column = tf.feature_column.categorical_column_with_vocabulary_list( key="Promo", vocabulary_list=Promo_vocab) StateHoliday_vocab = ['0', 'b', 'a', 'c'] StateHoliday_column = tf.feature_column.categorical_column_with_vocabulary_list( key="StateHoliday", vocabulary_list=StateHoliday_vocab) SchoolHoliday_vocab = [1, 0] SchoolHoliday_column = tf.feature_column.categorical_column_with_vocabulary_list( key="SchoolHoliday", vocabulary_list=SchoolHoliday_vocab) StoreType_vocab = ['a', 'd', 'c', 'b'] StoreType_column = tf.feature_column.categorical_column_with_vocabulary_list( key="StoreType", vocabulary_list=StoreType_vocab) Assortment_vocab = ['a', 'c', 'b'] Assortment_column = tf.feature_column.categorical_column_with_vocabulary_list( key="Assortment", vocabulary_list=Assortment_vocab) month_vocab = [10, 3, 4, 2, 9, 6, 5, 7, 1, 8, 12, 11] month_column = tf.feature_column.categorical_column_with_vocabulary_list( key="month", vocabulary_list=month_vocab) Season_vocab = ['Autumn', 'Spring', 'Winter', 'Summer'] Season_column = tf.feature_column.categorical_column_with_vocabulary_list( key="Season", vocabulary_list=Season_vocab) feature_columns = [ tf.feature_column.indicator_column(DayOfWeek_column), tf.feature_column.indicator_column(Open_column), tf.feature_column.indicator_column(Promo_column), tf.feature_column.indicator_column(StateHoliday_column), tf.feature_column.indicator_column(SchoolHoliday_column), tf.feature_column.indicator_column(StoreType_column), tf.feature_column.indicator_column(Assortment_column), tf.feature_column.numeric_column('CompetitionDistance'), tf.feature_column.indicator_column(month_column), tf.feature_column.indicator_column(Season_column), ]
The model itself
input_func = tf.estimator.inputs.pandas_input_fn(x=X_train,y=y_train ,batch_size=10000,num_epochs=15, shuffle=True) model = tf.estimator.DNNRegressor(hidden_units=[64,64,64,64],feature_columns=feature_columns, optimizer=tf.train.AdamOptimizer(learning_rate=0.0001), activation_fn = tf.nn.relu) model.train(input_fn=input_func,steps=1000000)
The structure is the same as in Keras, 4 layers, 64 neurons. relu, adam and mse as cost (it is a default for DNNRegressor), but tf does not work as good as Keras
Results are a mess, MSE is 44303762026251.3, MAE is 3809120.3086946052, R-squared is even negative, -4598900.028032559
What did I do wrong here? Did I forget something in Tensorflow? Keras is using TF, so I suppose that results should be similar if the model is tuned in the same way.
I randomly put numbers in layers, neurons, learning rate, epochs, but it does not work as well Thank you in advance!
edit1
Thanks for your comments! I tried to apply what you recommended. I totally abanded DNNRegressor and tried to "manually" create everything with tf.layers.dense. I, again, copied the structure of keras (changed to glorot in keras as well). Thats how it looks now:
import tensorflow as tf
import numpy as np
import uuid
x = tf.placeholder(shape=[None, 30], dtype=tf.float32)
y = tf.placeholder(shape=[None, 1], dtype=tf.float32)
dense = tf.layers.dense(x, 30, activation = tf.nn.relu,
bias_initializer = tf.zeros_initializer(),
kernel_initializer = tf.glorot_uniform_initializer())
dropout = tf.layers.dropout(inputs = dense, rate = 0.1)
dense = tf.layers.dense(dropout, 64, activation = tf.nn.relu,
bias_initializer = tf.zeros_initializer(),
kernel_initializer = tf.glorot_uniform_initializer())
dropout = tf.layers.dropout(inputs = dense, rate = 0.1)
dense = tf.layers.dense(dropout, 64, activation = tf.nn.relu,
bias_initializer = tf.zeros_initializer(),
kernel_initializer = tf.glorot_uniform_initializer())
dropout = tf.layers.dropout(inputs = dense, rate = 0.1)
dense = tf.layers.dense(dropout, 64, activation = tf.nn.relu,
bias_initializer = tf.zeros_initializer(),
kernel_initializer = tf.glorot_uniform_initializer())
dropout = tf.layers.dropout(inputs = dense, rate = 0.1)
dense = tf.layers.dense(dropout, 64, activation = tf.nn.relu,
bias_initializer = tf.zeros_initializer(),
kernel_initializer = tf.glorot_uniform_initializer())
dropout = tf.layers.dropout(inputs = dense, rate = 0.1)
output = tf.layers.dense(dropout, 1, activation = tf.nn.sigmoid)
cost = tf.losses.absolute_difference(y, output) #mae
optimizer = tf.train.AdamOptimizer(learning_rate=0.0001).minimize(cost)
init = tf.global_variables_initializer()
tf.summary.scalar("cost", cost)
merged_summary_op = tf.summary.merge_all()
with tf.Session() as sess:
sess.run(init)
uniq_id = "/tmp/tensorboard-layers-api/" + uuid.uuid1().__str__()[:6]
summary_writer = tf.summary.FileWriter(uniq_id, graph=tf.get_default_graph())
x_vals = X_train
y_vals = y_train
#for step in range(673764):
for step in range(673764):
_, val, summary = sess.run([optimizer, cost, merged_summary_op],
feed_dict={x: x_vals, y: y_vals})
if step % 20 == 0:
print("step: {}, value: {}".format(step, val))
summary_writer.add_summary(summary, step)
TF model is slower, so I cannot check precisely the output, but first steps of TF are close to results of a first epoch of keras:
Epoch 1/15
673764/673764 [==============================] - 13s 19us/step - loss: 57019592.1866 - mean_squared_error: 57019592.1866 - mean_absolute_error: 6883.4074 - mean_absolute_percentage_error: 2668499.3291
TF:
step: 0, value: 6957.24365234375
step: 20, value: 6957.2373046875
step: 40, value: 6957.23583984375
step: 60, value: 6957.22998046875
So MAE of both models are close, around 6900. I suppose that the issue is solved now.
I just have one question left, how to apply batches in this type of tensorflow? It is the first time I ever built tf like this and I haven't found an obvious solution online. Thanks!