0
$\begingroup$

I'm trying to build a deep learning regression model for price prediction of AirBnB listings.
As a baseline, I started with a simple 3-layer NN as follows:

import tensorflow as tf
import tensorflow.keras as keras
from keras.models import Sequential
from keras.layers import Dense
from tensorflow.keras.optimizers import Adam, SGD

epochs=100
batch_size=64

model1 = Sequential()
model1.add(Dense(units=32, activation='relu', input_shape=(X_train.shape[1],)))
model1.add(Dense(units=32, activation='relu'))
model1.add(Dense(units=32, activation='relu'))
model1.add(Dense(units=1, activation='linear'))

# Compile the model
model1.compile(optimizer=Adam(), loss='mse', metrics=['mse'])

# Training the model
history = model1.fit(X_train, Y_train, epochs=epochs, batch_size=batch_size, verbose=1, validation_data=(X_test, Y_test))

This produced the following results: prediction vs ground truth
loss function

Training RMSE: 57.5531
Validation RMSE: 60.5903

Training r2: -0.0345
Validation r2: -0.0767

As you can see, the model predicts all the listing in the 50 to 100 range with an RMSE of about $60.

What I have tried is:

  • Tried several batch sizes
  • Added extra layers
  • Added dropout after each hidden layer
  • Used kernel regularizers
  • Learning rate optimization
  • Callbacks (ReduceLROnPlateau, EarlyStopping)
  • Many more...

All the results stayed roughly the same.

The only issue I see myself is that the dataset is way too small for this task. Dataset size

I was wondering if there is any other fundamental flaw in my code/thinking?

$\endgroup$
2
  • $\begingroup$ Did you check against some „naive“ model such as linear regression or random forest. Small samples can be a problem with NN. Did you normalise training data? Otherwise your model looks okay. $\endgroup$
    – Peter
    Dec 29, 2021 at 20:28
  • $\begingroup$ The lack of normalization on the input data was indeed the problem here. Thank you for reminding me! $\endgroup$ Dec 30, 2021 at 10:46

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy