2
$\begingroup$

As an ML newbie, I have a question. I have a set of data with 2 inputs and 1 output. I'm trying to predict the output.

input1 is an integer number, input2 is like a category between 1-5. Output is also a number.

input1=25 input2=2 output=25
input1=34 input2=2 output=35
input1=12 input2=5 output=29
input1=3 input2=4 output=48
input1=45 input2=1 output=36 

With this data, I want to predict the output for input1=27 and input2=2

I have a small set of data (10-20 items). I wonder which ML algorithm should I learn for this kind of multiple inputs and single output small sets of data?

Edit

With a high probability, while calculating the output, there is a mathematical relation between input1 and input2 like:

output = (input1)*x + (input2)*y (x and y is unknown of course and the equation can be linear or logarithmic or something else. No idea.)
| improve this question | | | | |
$\endgroup$
  • 1
    $\begingroup$ This task is called multiple regression. $\endgroup$ – Emre Jun 7 '18 at 16:37
2
$\begingroup$

Since you believe the output can be predicted by a linear combination of the inputs, a reasonable approach to try is Linear Regression, specifically Multiple Regression since you have more than one input variable.

Linear regression will attempt to fit the best parameters $\beta_0$ and $\beta_1$ to model your output as a weighted sum of your inputs, ie $\beta_0*input_1 + \beta_1*input_2$. This is exactly the same as the expression you gave, but it's more standard to call the weights $\beta_i$s instead of $x$ and $y$.

The most standard form of linear regression using Ordinary Least Squares will find $\beta_0$ and $\beta_1$ that minimize the sum of the squared errors over your dataset, which are the differences between the actual values of output and the predicted values generated by computing $\beta_0*input_1 + \beta_1*input_2$ for each row.

EDIT: To answer your question in the comments:

It is always reasonable to try linear model first since it is simple and efficient, and it will give you a good baseline.

However, if you suspect there is a non-linear relationship between your inputs and outputs you can also try more flexible models such as gradient boosting regression trees or a neural network.

You do not need to know what the exact relationship is to use these models - they will learn it for you. In theory a neural network can fit any function.

As you use more complex models, however, you should be increasingly wary of overfitting.

| improve this answer | | | | |
$\endgroup$
  • $\begingroup$ Hello Imran, thank you for your answer. What if the equation is not linear but it's logarithmic? or something else? Is regression is the correct algorithm for my case? I'm sure there is a kind of equation, the output is based on numerical input values but I have no idea how they are related. linear or logaritmic or power. $\endgroup$ – Eray Jun 7 '18 at 16:09
  • 1
    $\begingroup$ Thanks for the follow-up question. I have edited my answer to respond. $\endgroup$ – Imran Jun 7 '18 at 16:18
  • $\begingroup$ Hello @Imran, as a last question for my case, I tried Neural Networks as you suggested.There are 30 data in my training set. And two of the records are input1=0.6 input2=0.65 output=0.48 and input1=0.6 input2= 0.90 output=0.56. In this case when I try to predict input1=0.6 input2=0.75 I'm expecting a value between 0.48-0.56 but it's giving me 0.46. It's lower than my training set data. Is it means I'm using wrong actiation function? For full of my code (in JavaScript) paste.ubuntu.com/p/jPmVxVbYdR (using brain.js github.com/BrainJS/brain.js#node) $\endgroup$ – Eray Jun 9 '18 at 12:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.