1
$\begingroup$

I have a list of our college students' high school courses.

I want to recommend college degrees to current high school students based on their courses - that is, predict a class based on a vector. For instance someone who studied algebra, calculus and statistics could be recommended software engineering, accountancy or mathematics.

There seem to be a few approaches I could take: market basket analysis, collaborative filtering, clustering or even neural networks.

I can structure my data in a sparse matrix of courses, with each row having a class representing the student's eventual degree, e.g.:

DEGREE       English    Calc    Algebra    Geography    History ... etc
Soft.Eng.    0          1       1          0            0
Comms        1          0       0          1            1
Mech.Eng.    1          1       1          0            0

How should I approach this?

$\endgroup$
6
  • $\begingroup$ The method you should select depends heavily on these following questions, please answer them. How many instances do you have (number of students)? How many features (number of courses)? Number of output labels (number of degree programs)? $\endgroup$
    – JahKnows
    Commented Apr 20, 2018 at 8:16
  • $\begingroup$ Very generally, given the questions above are answered in an expected way, I would assume a Random Forests approach would do quite well here. $\endgroup$
    – JahKnows
    Commented Apr 20, 2018 at 8:17
  • $\begingroup$ Can you link to your data please so we can see the entirety of it? $\endgroup$
    – JahKnows
    Commented Apr 20, 2018 at 8:17
  • $\begingroup$ @JahKnows There's no way I can release the actual data. But it is on the order of 50,000 instances with 40 features, belonging to one of 20 output labels. Each instance will only have 5 or 6 features, because you can only study 5 or 6 topics in high school, out of 40. $\endgroup$
    – Tim
    Commented Apr 20, 2018 at 10:02
  • $\begingroup$ @TimBennett - please I am interested in knowing how you eventually solved this. Thanks $\endgroup$
    – Z Z
    Commented Mar 20, 2019 at 8:18

2 Answers 2

-1
$\begingroup$

This problem is perfectly suited for a neural network. Your model will have 40 input nodes (this is fine), then you will have some arbitrary hidden layers, you need to tune this, and 20 outputs. After the training process you can even get a probability for each of them. This can be used to rank suggestions for potential students!


How to do this

Load the data to memory

Depending on the means by which your data is stored this step will be different. However, the goal is to go from the raw file source to either Python Numpy array or a Pandas DataFrame. I will assume your data is structured as follows

enter image description here

and is saved as a .csv file.

Let's load our data into an X and Y matrix. We will be encoding the labels for 'degree' as values. Make sure these are all well spelled otherwise a new label will be created for the mispelled ones.

import pandas as pd
import numpy as np

df = pd.read_csv('test1.csv')
df['Degree'] = df['Degree'].astype('category')
df["Degree_encoding"] = df["Degree"].cat.codes

X = np.asarray(df.loc[:, df.columns != 'Degree'])
Y = df['Degree_encoding']

print(X.shape)
print(Y.shape)

(39, 7)
(39,)

Applying a neural network to this dataset

First we will split the data randomly into a training and testing set. This is used to evaluate our model while maintaining that we are not overfitting. Then we identify the number of output classes. Then we reshape the input matrices such that they have a channel in their last dimension, this is how data flows through the model. Then we will categorize our outputs as one-hot encoded vectors.

from sklearn.model_selection import train_test_split
import keras

# Split the data
x_train, x_test, y_train, y_test = train_test_split(X, Y, test_size=0.33, shuffle= True)

# The known number of output classes.
num_classes = len(set(df["Degree_encoding"]))
# Input dimensions
shape = X.shape[1::]

# We need to add a channels dimension to our data
# Channels go last for TensorFlow backend in Keras
x_train_reshaped = x_train.reshape((x_train.shape[0],) + shape)
x_test_reshaped = x_test.reshape((x_test.shape[0],) + shape)
input_shape = shape
print(input_shape)

# Convert class vectors to binary class matrices. This uses 1 hot encoding.
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)

We then design our model

model = Sequential()
model.add(Dense(32,
                 activation='relu',
                 input_shape=input_shape))
model.add(Dense(128, activation='relu'))
model.add(Dense(num_classes, activation='softmax'))

model.compile(loss=keras.losses.categorical_crossentropy,
              optimizer=keras.optimizers.Adadelta(),
              metrics=['accuracy'])

You can use model.summary() to get a description of these layers. Then we are ready to train our model!

epochs = 100
batch_size = 128
model.fit(x_train_reshaped, y_train,
          batch_size=batch_size,
          epochs=epochs,
          verbose=1,
          validation_data=(x_test_reshaped, y_test))

enter image description here

$\endgroup$
4
  • $\begingroup$ Judging by your curves, you are overfitting like crazy, probably because of duplicate records in your toy data. $\endgroup$ Commented Apr 25, 2018 at 21:58
  • $\begingroup$ @Anony-Mousse, these curves are in fact not indicative of overfitting. If I were overfitting you would see a degradation in performance in the validation set. There would be a large mismatch between the performance of the training set and the validation set. I encourage you to run some test samples through the network and find one which will be wrongly classified. I have tried they are all correctly classified, thus this network would be perfectly suitable for this task. I always encourage people to remember the true purpose of applying machine learning, it's to get answers. $\endgroup$
    – JahKnows
    Commented Apr 26, 2018 at 2:05
  • $\begingroup$ Moreover, this is a sample example with only like 40 instances, because I do not have access to the original data. With the OPs 50,000 instances he will be able to use the same framework as I have outlined above for this purpose. The performance will then be dependent on the ability of the features to discriminate between the output classes. $\endgroup$
    – JahKnows
    Commented Apr 26, 2018 at 2:10
  • 1
    $\begingroup$ @JahKnows I wanted to come back and say thank you for the time you put into this answer. I'm still working on implementing various suggestions, I haven't forgotten your effort. $\endgroup$
    – Tim
    Commented Jun 8, 2018 at 5:23
-2
$\begingroup$

On such data, naive bayes (and maybe non-naive Bayes variants) should perform extremely well. Because all your inputs are binary.

It's also incredibly cheap to train, evaluate, and explain.

You can combine it with frequent itemset to make it less naive, but I wouldn't be surprised if that does not improve the accuracy much. You would use FIM to find dependencies worth modeling, and build an optimal Bayes for non-overlapping frequent subsets (e.g. engineering subjects vs. languages). Then combine these partitions assuming independence.

$\endgroup$
1
  • $\begingroup$ @JahKnows why did you downvote this, without even commenting what you do not like about the answer? Do you have any reason to believe that Naive Bayes won't work here? $\endgroup$ Commented Apr 26, 2018 at 15:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.