I'm trying to build a classification model where instances are classified based not on some linear combination of their features, but instead based on whether the features are all within some (unknown) ranges. So, for example, if I had two features, and my instances (features, followed by known classification) are:

  • (0, 5): 0
  • (2, 7): 1
  • (3, 2): 0
  • (4, 4): 1
  • (3, 9): 0
  • (6, 6): 0

then perhaps the model would learn that the first feature should be between 1 and 5, and the second feature should be between 3 and 8. (I only want "and"s in the model; I'm not looking for any cases where feature 1 is between a and b OR feature 2 is between c and d.) In essence, the model should find some hyperrectangle in feature space with all edges parallel to feature axes, where 0s are expected to be outside the rectangle and 1s are expected to be inside it. (In the example data, the data points are separable; this is not the case in my real problem. In the spirit of an SVM or logistic regression, the model should do the best it can.)

My first stab at this involved trying to do least-squares regression with a hypothesis function made up of two sigmoids for each feature (one "forward", one "backward"), parameterized to slide back and forth, all of which were multiplied together. Maybe my scipy-fu is weak, but above four features, I started having convergence issues with that approach. Is there some tried-and-true way to solve this kind of classification problem? (Even just a name for this kind of problem would be helpful!)

  • 1
    $\begingroup$ Are you aware of decision tree learning, en.m.wikipedia.org/wiki/Decision_tree_learning ? $\endgroup$
    – aivanov
    Feb 9, 2018 at 17:16
  • $\begingroup$ I'm aware of, but not experienced with it. It seems like that's solving a different (more general?) problem than what I have; a decision tree model could carve out arbitrary rectilinear regions of space, and I'm not interested in solutions that aren't simple hyperrectangles. Do you think I'm missing something? $\endgroup$
    – user45996
    Feb 9, 2018 at 17:53

1 Answer 1


The problem you are are describing is precisely what a decision tree does.

Decision trees find "hyperrectangle in feature space with all edges parallel to feature axes". A decision tree will automatically learn the learn conditional hyperrectangle by finding the range based on previous and current splits. The goal of classification is purity in the hyperrectangle, having as much as possible a single category inside the region.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.