What are some examples of linear and non-linear machine learning models (algorithms) for purposes of comparison between the two categories? Which are the parameters (or scalars in a linear algebraic sense) and which are the predictors/factors (or vectors in a linear algebraic sense again)?

  • 4
    $\begingroup$ Welcome to DS.SE! Your question seems quite broad, could you sharpen it to contain only one well defined one? $\endgroup$
    – mico
    Apr 15 '18 at 13:16

Actually, there are many linear and non-linear machine learning algorithms. Selecting a right algorithm highly depends on your data-set and the nature of your data. for instance, in machine-learning era, it was customary to estimate functions by assigning a typical model to the problem and reducing the error by predicting the appropriate coefficients using a cost function for regression tasks. In such cases, you should have bias-variance trade-off. That means you should not fit the data nor miss it. you should find a good estimate, a good model that has generalization capability. For doing all of these you have to choose features which help you describe the problem better for making a model. In deep-learning, we usually do not have this trade-off. If you increase the size of your training set, you can almost be sure that you can have better results. In machine-learning, you can always be sure that by making complex non-linear models, you overfit your data while using complex deep-learning models does not necessarily mean that if you employ generalization techniques which avoid overfitting. For choosing a right model, it is customary to use a simple linear model and make it complicated step by step. Problems which their inputs have numerous features, you can not see and visualize the data-set to check whether they are linearly separable in that space or not. Consequently, beginning with a simple model and making it complex step by step is a logical solution. For measuring performance there are different solutions that all depend on your goal. Take a look at here.


Not the answer you're looking for? Browse other questions tagged or ask your own question.