Is there such thing as linear and non-linear data?

Question

While doing machine learning projects we've heard that logistic regression works well with "Linear data" and decision tree works well with "non-linear data"

However concept of linear and non-linear data does not make sense. To me only linearly separable data and non-linearly separable data makes sense to me, it only makes sense to say logistic regression works well with "Linearly separable data" since it is a linear function. In mathematics linear functions are polynomials with degree one and all other functions that are not linear are considered non-linear function.

What exactly is linear data and non-linear data?

Answer 1

Naming linear data or non-linear data is a bit misleading and wrong I would say. Instead, there is linear relation and non-linear relation between variables would be better and correct naming. It can easily be explained by example from real life.

• Non-linear relation: Any kind of relationship that is not linear can be put forward as an example (e.g. quadratic relation: y = intercept + x^2, square root relation: y = intercept + 6 * sqrt(x) etc.) Let's imagine a world where we are not aware of the free fall equation and we use ML to learn and predict it :). In that case (specific and simplified case), the algorithm would need to learn an equation in the following structures = v₀t + (1/2)gt², where t is our independent variable, s is our target or dependent variable and v₀, g would be our coefficients to be learned. In other words, you would want to predict free fall (distance or height) using time. In that fictional world, Newton’s Law of Universal Gravitation (F = G * (M1 * M2) / r^2), Area of a circle (A = π * r^2) would also be a quadratic problem for ML. Linear functions do not meet the requirements of such cases unless you use some kind of transformation, let's say take the square root of r^2 from Newton's law or the square root of t^2 from free fall then use them in a linear function. The most known example of non-linear real-life relation is a prediction of height using age. Height changes continuously, however, in different rates: until 13 gradually increases, between 13-18 significantly increases, between 18-25 slightly changes, and after 25 it does not change. Thus, it is impossible to fit it into a linear equation, because it is not dependent on a single coefficient or i.o.w. it does not fit into the formula height = intercept + b * age because coefficient (b) is not constant over time (age). Moreover, Graph, Tree, and other similar structures are also fitted into a non-linear relationship. In cases where the result is highly dependent on ifs, linear algorithms are useless because you cannot fit the relationship into y = a + b * x. Let's say, you want to predict who will win a chess game basing on the moves they make, you would use a tree-like (e.g.: alpha-beta pruning) algorithm to predict it.
• Linear relation: It is simply the relations where those relations can be explained by the y = a + b * x formula or even y = a + b1* x1 + b2*x2 + .... The simplest example is predicting the cost that is spent in the bar (or any kind of entertainment venue). cost = intercept (let's imagine you pay money for entering to the bar) + (the price of a drink) * (the number of drinks bought) + (the price of appetizer) * (the number of appetizers bought) would be our formula.