11
Decision Tree, KNN, & Random Forest (Methods that are suitable for overlapping data)
This statement is false. All those methods are good when the decision surface (separating surface) has a highly nonlinear form. They act as a non-parametric local approximation - all parameters are not in fact parameters of the decision function but are meta parameters ...
6
Your data is multidimensional, it is possible that any two dimensional projection overlaps while still existing an hyperplane on the original dimensionality that separates the two classes well
Say for instance you have 3 data points from 2 labels in 2d that are linearly separable
X:(0,-1) O:(1,2) X:(4,3)
X
O
X
In the x axis they look ...
1
Yes the L2 regularization will shrink the irrelevant feature coefficients to zero and hence it doesn't require feature selection. In fact it IS a commonly used feature selection technique. So basically you are performing feature selection!!
1
We have, $p = (1 - p)e^{b + b_1x_1 + \ldots}$
Let $y= {b + b_1x_1 + \ldots}$
So, $p = (1 - p)e^y$
or, $p = e^y - pe^y$
or, $p+pe^y = e^y$
or, $p(1+e^y) = e^y$
or, $p = e^y/(1+e^y)$
or, $p = 1/(e^{-y}+1)$ (Dividing both denominator and numerator by $e^y$ on the RHS)
or, $p = 1/(e^{-{b + b_1x_1 + \ldots}}+1)$
or, $p = 1/(1+e^{-{b + b_1x_1 + \ldots}})$
Let me ...
1
$p = (1 - p)e^{b + b_1x_1 + \ldots}$, then $p(1 + e^{b + b_1x_1 + \ldots}) = e^{b + b_1x_1 + \ldots}$, thus
\begin{align}
p &= \frac{e^{b + b_1x_1 + \ldots}}{1 + e^{b + b_1x_1 + \ldots}} \\
&= \frac{1}{1 + e^{-(b + b_1x_1 + \ldots)}}
\end{align}
1
In the linked answer, it is convenient to have the $1/2$ in the loss function so it cancels when we bring down the $2$ in the derivative, and this is okay since we just want to optimize the parameters. I do not see something that should cancel out in your equation, but there could be another reason to divide through.
In your case, unless you pick a silly ...
Only top voted, non community-wiki answers of a minimum length are eligible