1
$\begingroup$

I am starting a machine learning project (for fun!), but I am not sure where to start from... I am fairly new to ML so any hints are appreciated.

I have a relatively large data-set where each input is a list of roughly 300 integers (mainly zeros). The output is a list of 20ish integers. The goal is to predict the output given a random input (obviously). And I am not sure what is the best method for that. I have started reading a bit into neural networks which seem like it could be a good way to solve such problem, but there seems to be a whole range of different activation etc. (not sure how it's actually called) so I am not really sure what to do.

Any hints on what direction to look into?

Thanks a lot!

$\endgroup$
2
  • $\begingroup$ So for each sample you have 20 different values (targets) which you want to predict? $\endgroup$
    – Peter
    Commented Dec 4, 2021 at 20:40
  • $\begingroup$ Yes, for each sample I'll have an output of 20 values (integers, most of them 0), and I want to be able to predict this output for any input. $\endgroup$
    – Jylu
    Commented Dec 4, 2021 at 20:53

1 Answer 1

1
$\begingroup$

In case you have several output columns (targets), you have a limited choice of models since most models predict one output column. See the sklearn docs for a good overview. In essence you deal with a multioutput problem.

The sklearn models for regression simply aim at predicting each output separately. The RegressionChain trys to successively predict one target value after another, taking the previous prediction into account. From the docs:

Multioutput regression support can be added to any regressor with MultiOutputRegressor. This strategy consists of fitting one regressor per target. Since each target is represented by exactly one regressor it is possible to gain knowledge about the target by inspecting its corresponding regressor. As MultiOutputRegressor fits one regressor per target it can not take advantage of correlations between targets.

Regressor chains (see RegressorChain) is analogous to ClassifierChain as a way of combining a number of regressions into a single multi-target model that is capable of exploiting correlations among targets.

If you want to try neural nets, you can use Keras' "functional API" which allows you to define several outputs (columns). Find a minimal example here. In a nutshell you need to define some input (the $X$ matrix, aka the explanatory variables) and you need to define all of the outputs separately. Inputs and outputs can be feeded into the model definition and the model fit statement as shown below.

# Input and model architecture
Input_1=Input(shape=(13, ))
x = Dense(1024, activation='relu', kernel_regularizer=regularizers.l2(0.05))(Input_1)
x = Dense(512, activation='relu', kernel_regularizer=regularizers.l2(0.05))(x)
x = Dense(256, activation='relu', kernel_regularizer=regularizers.l2(0.05))(x)
x = Dense(128, activation='relu', kernel_regularizer=regularizers.l2(0.05))(x)
x = Dense(8, activation='relu', kernel_regularizer=regularizers.l2(0.05))(x)

# Outputs
out1 = Dense(1)(x)
out2 = Dense(1)(x)

# Compile/fit the model
model = Model(inputs=Input_1, outputs=[out1,out2])
model.compile(optimizer = "rmsprop", loss = 'mse')

# Add actual data here in the fit statement
model.fit(train_data, [train_targets,train_targets2], epochs=500, batch_size=4, verbose=0, validation_split=0.2)
$\endgroup$
1
  • $\begingroup$ Thanks! I'll read more into this. This definitely helps! $\endgroup$
    – Jylu
    Commented Dec 6, 2021 at 17:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.