Recommending college degrees based on high school subjects

Question

I have a list of our college students' high school courses.

I want to recommend college degrees to current high school students based on their courses - that is, predict a class based on a vector. For instance someone who studied algebra, calculus and statistics could be recommended software engineering, accountancy or mathematics.

There seem to be a few approaches I could take: market basket analysis, collaborative filtering, clustering or even neural networks.

I can structure my data in a sparse matrix of courses, with each row having a class representing the student's eventual degree, e.g.:

DEGREE       English    Calc    Algebra    Geography    History ... etc
Soft.Eng.    0          1       1          0            0
Comms        1          0       0          1            1
Mech.Eng.    1          1       1          0            0

How should I approach this?

Answer 1

This problem is perfectly suited for a neural network. Your model will have 40 input nodes (this is fine), then you will have some arbitrary hidden layers, you need to tune this, and 20 outputs. After the training process you can even get a probability for each of them. This can be used to rank suggestions for potential students!

---

How to do this

Load the data to memory

Depending on the means by which your data is stored this step will be different. However, the goal is to go from the raw file source to either Python Numpy array or a Pandas DataFrame. I will assume your data is structured as follows

and is saved as a .csv file.

Let's load our data into an X and Y matrix. We will be encoding the labels for 'degree' as values. Make sure these are all well spelled otherwise a new label will be created for the mispelled ones.

import pandas as pd
import numpy as np

df = pd.read_csv('test1.csv')
df['Degree'] = df['Degree'].astype('category')
df["Degree_encoding"] = df["Degree"].cat.codes

X = np.asarray(df.loc[:, df.columns != 'Degree'])
Y = df['Degree_encoding']

print(X.shape)
print(Y.shape)

(39, 7)
(39,)

Applying a neural network to this dataset

First we will split the data randomly into a training and testing set. This is used to evaluate our model while maintaining that we are not overfitting. Then we identify the number of output classes. Then we reshape the input matrices such that they have a channel in their last dimension, this is how data flows through the model. Then we will categorize our outputs as one-hot encoded vectors.

from sklearn.model_selection import train_test_split
import keras

# Split the data
x_train, x_test, y_train, y_test = train_test_split(X, Y, test_size=0.33, shuffle= True)

# The known number of output classes.
num_classes = len(set(df["Degree_encoding"]))
# Input dimensions
shape = X.shape[1::]

# We need to add a channels dimension to our data
# Channels go last for TensorFlow backend in Keras
x_train_reshaped = x_train.reshape((x_train.shape[0],) + shape)
x_test_reshaped = x_test.reshape((x_test.shape[0],) + shape)
input_shape = shape
print(input_shape)

# Convert class vectors to binary class matrices. This uses 1 hot encoding.
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)

We then design our model

model = Sequential()
model.add(Dense(32,
                 activation='relu',
                 input_shape=input_shape))
model.add(Dense(128, activation='relu'))
model.add(Dense(num_classes, activation='softmax'))

model.compile(loss=keras.losses.categorical_crossentropy,
              optimizer=keras.optimizers.Adadelta(),
              metrics=['accuracy'])

You can use model.summary() to get a description of these layers. Then we are ready to train our model!

epochs = 100
batch_size = 128
model.fit(x_train_reshaped, y_train,
          batch_size=batch_size,
          epochs=epochs,
          verbose=1,
          validation_data=(x_test_reshaped, y_test))

Answer 2

On such data, naive bayes (and maybe non-naive Bayes variants) should perform extremely well. Because all your inputs are binary.

It's also incredibly cheap to train, evaluate, and explain.

You can combine it with frequent itemset to make it less naive, but I wouldn't be surprised if that does not improve the accuracy much. You would use FIM to find dependencies worth modeling, and build an optimal Bayes for non-overlapping frequent subsets (e.g. engineering subjects vs. languages). Then combine these partitions assuming independence.