I was looking at a couple of keras tutorials and something struck me as odd. So in these two tutorials, both on housing data, the number of nodes in the first layer of the network were larger than the number of columns in the dataset itself. NOTE: I am talking about tabular data and not image data or autoencoders. I understand that autoencoders will widen during the decoding phase, and I know that segmentation models will widen during the deconvolution phase. Both of these examples seem to be different than what I am seeing in these tutorials.
The first tutorial used data on Kings County, CA housing prices to create a simple neural network. In this case, the Kings County data only has 19 variables but the network itself has an initial dense layer of size 100 nodes. Here is the code:
def basic_model_1(x_size, y_size): t_model = Sequential() t_model.add(Dense(100, activation="tanh", input_shape=(x_size,))) t_model.add(Dense(50, activation="relu")) t_model.add(Dense(y_size)) print(t_model.summary()) t_model.compile(loss='mean_squared_error', optimizer=Adam(), metrics=[metrics.mae]) return(t_model)
In the second example I was looking at Aurelion Geron's recent book on Keras and Tensorflow. In chapter 10 he uses some different California housing data to estimate a keras neural network. Again in that case he has 8 variables in the dataset, but he starts with a Dense layer of 30.
As a caveat, I did not see any columns get converted over to one-hot encodings or some sparse formulation.
This situation where the network starts wider than the data seemed odd to me, but perhaps that is just me coming from the statistics world. Can anyone explain this? Thanks.