I'm trying to build a deep learning regression model for price prediction of AirBnB listings.
As a baseline, I started with a simple 3-layer NN as follows:
import tensorflow as tf import tensorflow.keras as keras from keras.models import Sequential from keras.layers import Dense from tensorflow.keras.optimizers import Adam, SGD epochs=100 batch_size=64 model1 = Sequential() model1.add(Dense(units=32, activation='relu', input_shape=(X_train.shape,))) model1.add(Dense(units=32, activation='relu')) model1.add(Dense(units=32, activation='relu')) model1.add(Dense(units=1, activation='linear')) # Compile the model model1.compile(optimizer=Adam(), loss='mse', metrics=['mse']) # Training the model history = model1.fit(X_train, Y_train, epochs=epochs, batch_size=batch_size, verbose=1, validation_data=(X_test, Y_test))
Training RMSE: 57.5531 Validation RMSE: 60.5903 Training r2: -0.0345 Validation r2: -0.0767
As you can see, the model predicts all the listing in the 50 to 100 range with an RMSE of about $60.
What I have tried is:
- Tried several batch sizes
- Added extra layers
- Added dropout after each hidden layer
- Used kernel regularizers
- Learning rate optimization
- Callbacks (ReduceLROnPlateau, EarlyStopping)
- Many more...
All the results stayed roughly the same.
I was wondering if there is any other fundamental flaw in my code/thinking?