Suppose I have n features $(x_1, x_2, ...., x_n)$ (all float) and want to classify $y$ (0 or 1)

For now I have a legacy expert system to do the classification.

Expert system rules:

Categorize all $x_i$

If $x_1 \lt 1$, set $x_1' = a_{11} $,

If $x_1 \in [1,10]$, set $x_1' = a_{12} $,

If $x_1 \gt 10$, set $x_1' = a_{13} $,


If $x_n \gt 100$, set $x_n' = a_{nk} $,

If $w_1 x_1'+...+w_n x_n' \geq z$, then predict $y = 1$, otherwise $0$

$a_{11}, a_{12}, .... a_{nk}, w_1, ..., w_n, z$ are determined by intuition instead of calculation, and I wanted to improve the expert system. How to do it? logistic regression cannot be simply applied on this case...

  • 1
    $\begingroup$ I understand that you don't want (or need) to use $a$, $w$ and that you only want to model $y(x_i)$. So why should logit not work? I don't get what the problem is here. $\endgroup$ – Peter Sep 25 '19 at 11:21

As Peter said, this is exactly the sort of problem logistic regression can work on directly. But, there are a few more things to say.

The binning that happens on your raw variables suggests there's some strong nonlinearity. You could do some univariate analysis to find suitable transformations, but in particular you could try binning, possibly with weight-of-evidence scoring and/or tree-based recursive partitioning.

If you want to insist on preserving the binning structure and the $x'$ values, then it's just straightforward logistic regression with these transformed predictors.

If you insist on preserving the binning structure but not their $x'$ values, you could again go with WOE or, one-hot encode for these bins and learn the $w_i x_i'$ products as the coefficients of the encoded variables in a logistic regression.

Finally, of course, you can replace almost any time I mentioned logistic regression with your favorite classifier.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.