0
$\begingroup$

As the title states, my validation accuracy isn't changing when I try to train my model. I've built an NVIDIA model using tensorflow.keras in python. I have absolutely no idea what's causing the issue.

Here is a link to the google colab I'm writing this in. But I've also put all my code below, below the model summary and Epoch history.

Please help. I'm slowly going insane.

Model Summary:

Model: "sequential_3"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_15 (Conv2D)           (None, 31, 98, 24)        1824      
_________________________________________________________________
conv2d_16 (Conv2D)           (None, 14, 47, 36)        21636     
_________________________________________________________________
conv2d_17 (Conv2D)           (None, 5, 22, 48)         43248     
_________________________________________________________________
conv2d_18 (Conv2D)           (None, 3, 20, 64)         27712     
_________________________________________________________________
conv2d_19 (Conv2D)           (None, 1, 18, 64)         36928     
_________________________________________________________________
dropout_8 (Dropout)          (None, 1, 18, 64)         0         
_________________________________________________________________
flatten_3 (Flatten)          (None, 1152)              0         
_________________________________________________________________
dense_13 (Dense)             (None, 100)               115300    
_________________________________________________________________
dropout_9 (Dropout)          (None, 100)               0         
_________________________________________________________________
dense_14 (Dense)             (None, 50)                5050      
_________________________________________________________________
dropout_10 (Dropout)         (None, 50)                0         
_________________________________________________________________
dense_15 (Dense)             (None, 10)                510       
_________________________________________________________________
dropout_11 (Dropout)         (None, 10)                0         
_________________________________________________________________
dense_16 (Dense)             (None, 1)                 11        
=================================================================
Total params: 252,219
Trainable params: 252,219
Non-trainable params: 0
_________________________________________________________________
None

Epoch History:

Epoch 1/30
18/18 [==============================] - 14s 716ms/step - loss: 0.4594 - accuracy: 0.1236 - val_loss: 0.1029 - val_accuracy: 0.1287
Epoch 2/30
18/18 [==============================] - 13s 702ms/step - loss: 0.1315 - accuracy: 0.1369 - val_loss: 0.0946 - val_accuracy: 0.1287
Epoch 3/30
18/18 [==============================] - 13s 701ms/step - loss: 0.1042 - accuracy: 0.1397 - val_loss: 0.0842 - val_accuracy: 0.1287
Epoch 4/30
18/18 [==============================] - 13s 697ms/step - loss: 0.0972 - accuracy: 0.1392 - val_loss: 0.0753 - val_accuracy: 0.1287
Epoch 5/30
18/18 [==============================] - 13s 699ms/step - loss: 0.0887 - accuracy: 0.1397 - val_loss: 0.0692 - val_accuracy: 0.1287
Epoch 6/30
18/18 [==============================] - 13s 698ms/step - loss: 0.0837 - accuracy: 0.1392 - val_loss: 0.0653 - val_accuracy: 0.1287
Epoch 7/30
18/18 [==============================] - 13s 699ms/step - loss: 0.0775 - accuracy: 0.1397 - val_loss: 0.0604 - val_accuracy: 0.1287
Epoch 8/30
18/18 [==============================] - 13s 700ms/step - loss: 0.0721 - accuracy: 0.1392 - val_loss: 0.0593 - val_accuracy: 0.1287
Epoch 9/30
18/18 [==============================] - 13s 698ms/step - loss: 0.0741 - accuracy: 0.1397 - val_loss: 0.0603 - val_accuracy: 0.1287
Epoch 10/30
18/18 [==============================] - 13s 700ms/step - loss: 0.0730 - accuracy: 0.1397 - val_loss: 0.0573 - val_accuracy: 0.1287
Epoch 11/30
18/18 [==============================] - 13s 705ms/step - loss: 0.0705 - accuracy: 0.1403 - val_loss: 0.0558 - val_accuracy: 0.1287
Epoch 12/30
18/18 [==============================] - 13s 701ms/step - loss: 0.0691 - accuracy: 0.1403 - val_loss: 0.0544 - val_accuracy: 0.1287
Epoch 13/30
18/18 [==============================] - 13s 705ms/step - loss: 0.0619 - accuracy: 0.1397 - val_loss: 0.0538 - val_accuracy: 0.1287
Epoch 14/30
18/18 [==============================] - 12s 694ms/step - loss: 0.0653 - accuracy: 0.1403 - val_loss: 0.0541 - val_accuracy: 0.1287
Epoch 15/30
18/18 [==============================] - 13s 695ms/step - loss: 0.0612 - accuracy: 0.1409 - val_loss: 0.0516 - val_accuracy: 0.1287
Epoch 16/30
18/18 [==============================] - 13s 696ms/step - loss: 0.0594 - accuracy: 0.1403 - val_loss: 0.0500 - val_accuracy: 0.1287
Epoch 17/30
18/18 [==============================] - 13s 701ms/step - loss: 0.0577 - accuracy: 0.1403 - val_loss: 0.0483 - val_accuracy: 0.1287
Epoch 18/30
18/18 [==============================] - 13s 701ms/step - loss: 0.0552 - accuracy: 0.1420 - val_loss: 0.0539 - val_accuracy: 0.1287
Epoch 19/30
18/18 [==============================] - 13s 695ms/step - loss: 0.0563 - accuracy: 0.1415 - val_loss: 0.0488 - val_accuracy: 0.1287
Epoch 20/30
18/18 [==============================] - 13s 699ms/step - loss: 0.0556 - accuracy: 0.1409 - val_loss: 0.0502 - val_accuracy: 0.1287
Epoch 21/30
18/18 [==============================] - 13s 709ms/step - loss: 0.0534 - accuracy: 0.1403 - val_loss: 0.0475 - val_accuracy: 0.1287
Epoch 22/30
18/18 [==============================] - 13s 704ms/step - loss: 0.0579 - accuracy: 0.1397 - val_loss: 0.0488 - val_accuracy: 0.1287
Epoch 23/30
18/18 [==============================] - 13s 697ms/step - loss: 0.0514 - accuracy: 0.1420 - val_loss: 0.0467 - val_accuracy: 0.1287
Epoch 24/30
18/18 [==============================] - 13s 696ms/step - loss: 0.0490 - accuracy: 0.1415 - val_loss: 0.0479 - val_accuracy: 0.1287
Epoch 25/30
18/18 [==============================] - 13s 695ms/step - loss: 0.0472 - accuracy: 0.1403 - val_loss: 0.0477 - val_accuracy: 0.1287
Epoch 26/30
18/18 [==============================] - 13s 698ms/step - loss: 0.0495 - accuracy: 0.1415 - val_loss: 0.0455 - val_accuracy: 0.1287
Epoch 27/30
18/18 [==============================] - 13s 711ms/step - loss: 0.0473 - accuracy: 0.1415 - val_loss: 0.0463 - val_accuracy: 0.1287
Epoch 28/30
18/18 [==============================] - 13s 703ms/step - loss: 0.0466 - accuracy: 0.1415 - val_loss: 0.0451 - val_accuracy: 0.1287
Epoch 29/30
18/18 [==============================] - 13s 700ms/step - loss: 0.0439 - accuracy: 0.1426 - val_loss: 0.0448 - val_accuracy: 0.1287
Epoch 30/30
18/18 [==============================] - 13s 698ms/step - loss: 0.0446 - accuracy: 0.1415 - val_loss: 0.0482 - val_accuracy: 0.1287

Code:

# Utility
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import pandas as pd
from sklearn.utils import shuffle
from sklearn.model_selection import train_test_split
import cv2
 
# For saving the model and reading in data
import os
import ntpath
from tensorflow.keras.models import load_model
from tensorflow.keras.utils import plot_model
 
# Copy the repository of images
from git import Repo
if not os.path.isdir("track"):
    Repo.clone_from("https://github.com/Grant-Allan/Track", "track")
 
# For the model
from tensorflow.keras.models import Sequential
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.layers import Conv2D
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import Flatten
from tensorflow.keras.layers import Dropout
 
 
# Split off everything that isn't the name of the image from the path
def path_leaf(path):
    head, tail = ntpath.split(path)
    return tail
 
 
# Read in the driving data
def read_data(datadir):
    # Name the columns of data
    columns = ['center', 'left', 'right', 'steering', 'throttle', 'reverse', 'speed']
 
    # Read in the data from the cvs while naming the columns
    data = pd.read_csv(os.path.join(datadir, "driving_log.csv"), names=columns)
 
    # Fix the paths to only have the images
    data['center'] = data['center'].apply(path_leaf)
    data['left'] = data['left'].apply(path_leaf)
    data['right'] = data['right'].apply(path_leaf)
 
    # Set the width of cells in the table to show the entire name
    pd.set_option('display.max_colwidth', 12)
 
    # Display the top five rows of the data
    # print(data.head())
 
    return data
 
 
# Get the intervals of the steering angles
def intervals(data, num_bins):
    # Balance the data in order to prevent the data from being overly
    # biased towards one angle or another.
    samples_per_bin = 300
 
    # The values and bins of the steering angles
    hist, bins = np.histogram(data['steering'], num_bins)
 
    # Balance the data, then redo the histogram
    data = balance_data(data, bins, num_bins, samples_per_bin)
    hist, bins = np.histogram(data['steering'], num_bins)
 
    # Centers the center bin by adding the two bins next to it into one bin
    # This doubles all the other values, requiring us to divide by 2
    center = (bins[:-1] + bins[1:])/2
 
    # Graph it
    plt.title('Balanced Data Histogram')
    plt.bar(center, hist, width=0.05)
 
    return data
 
 
def balance_data(data, bins, num_bins, samples_per_bin):
    remove_list = []
 
    # For each bin
    for j in range(num_bins):
        list_ = []
        # For each steering angle
        for i in range(len(data['steering'])):
            # If the steering angle is in the current bin
            if data['steering'][i] >= bins[j] and data['steering'][i] <= bins[j+1]:
                list_.append(i)
 
        # Shuffle the data so that when we cut off part of it, we don't cut off
        # an entire section of data (since the input is ordered, not random).
        list_ = shuffle(list_)
 
        # Cut off the tail end of the list; all elements beyond the samples_per_bin element
        # We use a new list since we use list_ for a number of more loops
        list_ = list_[samples_per_bin:]
        remove_list.extend(list_)
    
    data.drop(data.index[remove_list], inplace=True)
    
    return data
 
 
def load_img_steering(datadir):
    # Add the IMG folder to the directory path
    datadir = os.path.join(datadir, "IMG")
 
    # Create lists for later
    image_path = []
    steering = []
 
    for i in range(len(data)):
        # iloc allows us to perform a selection on a row of data in the data
        # frame based on a specified index. In this case, we go row by row.
        indexed_data = data.iloc[i]
 
        # The indexes correspond to center, left, right (0, 1, 2)
        # We only want the center values for the moment.
        # strip gets rid of any spaces in the name
        image_path.append(os.path.join(datadir, indexed_data[0].strip()))
 
        # Get the steering angle corresponding to the image
        steering.append(float(indexed_data[3]))
    
    # Convert image_path and steering into arrays for easy manipulation
    image_paths = np.asarray(image_path)
    steerings = np.asarray(steering)
    return image_paths, steerings
 
 
def set_histograms(y_train, y_val, num_bins):
    fig, axes = plt.subplots(1, 2, figsize=(12, 4))
    axes[0].hist(y_train, bins=num_bins, width=0.05, color='blue')
    axes[0].set_title('Training Set')
    axes[1].hist(y_val, bins=num_bins, width=0.05, color='red')
    axes[1].set_title('Validation Set')
    plt.show()
 
 
# Preprocess the images
def preprocessing(img):
    # Use the image path to get an actual image
    img = mpimg.imread(img)
 
    # Get rid of noise chunks
    # Crop the top of the image to remove the sky. Crop the
    # bottom of the image to get rid of the hood of our car.
    img = img[60:135, :, :]
 
    # Convert the images to YUV (recommended for the NVIDIA neural network model)
    # Y is the luminance/brightness/grayscale
    # UV are chrominance (color for the image) (they're color difference)
    img = cv2.cvtColor(img, cv2.COLOR_RGB2YUV)
 
    # Gaussian blur the image to reduce noise
    # kernel size and standard deviation.
    img = cv2.GaussianBlur(img, (3, 3), 0)
 
    # Resize the image to decrease size to allow for faster computations
    # (200, 66) matches the input size of the NVIDIA architecture, which
    # is a nice bonus
    img = cv2.resize(img, (200, 66))
 
    # Normalize the data
    img = img/255
 
    return img
 
 
# Create the NVIDIA model
# The model is discussed in detail in "End to End Learning For Self-Driving Cars"
def create_model():
    # Define the model using Sequential
    model = Sequential()
 
    # We use elu instead of relu to avoid the dead relu problem, which we had an issue with.
    model.add(Conv2D(24, kernel_size=(5,5), strides=(2,2), activation='elu', input_shape=(66,200,3)))
    model.add(Conv2D(36, kernel_size=(5,5), strides=(2,2), activation='elu'))
    model.add(Conv2D(48, kernel_size=(5,5), strides=(2,2), activation='elu'))
    model.add(Conv2D(64, kernel_size=(3,3), activation='elu'))
    model.add(Conv2D(64, kernel_size=(3,3), activation='elu'))
    model.add(Dropout(0.5))
 
    model.add(Flatten())
    # model.add(Dense(1164, activation='elu'))
    model.add(Dense(100, activation='elu'))
    model.add(Dropout(0.5))
 
    model.add(Dense(50, activation='elu'))
    model.add(Dropout(0.5))

    model.add(Dense(10, activation ='elu'))
    model.add(Dropout(0.5))
    
    model.add(Dense(1))
 
    # Learning rate of 0.0001
    model.compile(optimizer=Adam(learning_rate=1e-3), loss='mse', metrics=['accuracy'])
    return model
 
 
# Either load or create the model and model history
def get_model(X_train, y_train,  X_val, y_val):
    if os.path.isfile('model.h5') and os.path.isfile('history.npy'):
        # Load the saved model
        model = load_model('model')
 
        # Print the architecture summary
        print("\n=================================================================")
        print(model.summary())
        print("=================================================================\n")
 
        # Load the saved model history
        history = np.load('history.npy', allow_pickle='True').item()
    else:
        # Create the model
        model = create_model()
 
        # Print the architecture summary
        print("")
        print(model.summary())
        print("")
 
        # Train the model and get the History() object
        history_obj = model.fit(x=X_train, y=y_train, validation_data=(X_val, y_val), batch_size=100, epochs=30, verbose=1, shuffle=1)
        print("\n")
 
        # Save the model
        model.save('model.h5')
 
        # Save the model summary visualization
        plot_model(model, to_file='model.png', show_shapes=True, show_dtype=False, show_layer_names=True, rankdir='TB', expand_nested=False, dpi=96)
 
        # Set and save the history of the loss and accuracy values of the model
        history = history_obj.history
        np.save('history.npy', history)
    
    return model, history
 
 
# Plot the model loss and accuracy
def loss_and_accuracy(history):
    fig, axes = plt.subplots(1, 2, figsize=(12, 4))
 
    axes[0].set_title("Loss")
    axes[0].plot(history["loss"])
    axes[0].plot(history["val_loss"])
    axes[0].legend(["Loss", "Validation Loss"])
    axes[0].set_xlabel ("Epoch")
 
    axes[1].set_title("Accuracy")
    axes[1].plot(history["accuracy"])
    axes[1].plot(history["val_accuracy"])
    axes[1].legend(["Accuracy", "Validation Accuracy"])
    axes[1].set_xlabel ("Epoch")
    plt.show()
 
 
# Main driver code
if __name__ == "__main__":
    # Set parent directory
    datadir = "track"
 
    # Get the driving data
    data = read_data(datadir)
 
    # Plot the intervals
    num_bins = 25
    data = intervals(data, num_bins)
 
    # Load the image paths and corresponding steering angles
    image_paths, steerings = load_img_steering(datadir)
 
    # Split into training and validation sets
    # test_size is the proportion of test to valid; this splits 80% into test and 20% into valid
    # random_state is the seed for the random number generator
    X_train, X_val, y_train, y_val = train_test_split(image_paths, steerings, test_size=0.2, random_state=6)
    print("Training Samples: {}\nValidation Samples: {}".format(len(X_train), len(X_val)))
 
    # Show the steering angle histograms of each set
    set_histograms(y_train, y_val, num_bins)
 
    # Display a sample image to make sure the preprocessing is working
    image = image_paths[100]
    original_image = mpimg.imread(image)
    prep_img = preprocessing(image)
 
    fig, axes = plt.subplots(1, 2, figsize=(15, 10))
    fig.tight_layout()
    axes[0].imshow(original_image)
    axes[0].set_title("Original Image")
    axes[1].imshow(prep_img)
    axes[1].set_title("Preprocessed Image")
    plt.show()
 
    # Preprocess the images
    X_train = np.array(list(map(preprocessing, X_train)))
    X_val = np.array(list(map(preprocessing, X_val)))
 
    # Get the model and history
    model, history = get_model(X_train, y_train, X_val, y_val)
 
    # Plot model loss and accuracy
    loss_and_accuracy(history)
$\endgroup$
0
$\begingroup$

There are several reasons for that. Try increasing your training dataset or begin with smaller initial learning rate.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.