1
$\begingroup$

I have a varaible distance which is continous until a "hard stop" at which we stop measuring the distance itself and just label the distance as "out of range". Example:

distances: 10.1, 11.3, 20.2, 36.5, 39.6, out_of_range, out_of_range

Is there a best practise approach for encoding this data that is continous until a point? I have thought about just setting: out_of_range = max(distances) so that all out_of_range data is set at the same value but I'm not sure if this could have implications with an ML model assuming that an example with a longer distance that is in-range is close to an example that is out-of-range.

This out_of_range data is useful so I don't want to just remove it from the model but I would like to be able to differentate within the model between examples that are in_range vs out_of_range.

For context I'm planning on using this data as in input into a tree-based ML model such as Random Forrest

$\endgroup$

1 Answer 1

1
$\begingroup$

When you are building features for training an ML algorithm, you almost always have an upper and lower limit on your numerical features - sometimes more visible than others.

However, what you are dealing with is a problem that resembles that of outliers. You can deal with outliers in mainly two ways:

  1. Define upper/lower cutoff limits and replace the outlier values with one of these limits:
    You would adopt this method when you know that you can not/ do not care about the values which are outside this limit. For example a person can only see upto a distance and anything farther than that distance is just invisible and you do not have any other reason to get into further detail.
  2. Drop the data points that have outlier values:
    Sometimes, depending on your domain, you can make a reasonable conclusion that those outliers are result of a faulty data capture process. In such cases it may make more sense to discard such data points altogether.

As you can guess, the choice among the two approaches depends upon the nature of data and the problem you are dealing with.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.