0
$\begingroup$

Say I've created a random forest model for binary-classification prediction target of either "Pass" or "Fail" for a group of students based upon numerical features "Hours Studied" and "Amount of Sleep obtained." Is there a way to tell what SPECIFIC number of hours studied or amount of sleep obtained is the classification threshold for the model's prediction? For example, <6 hours studied = Fail while >6 hours of studied = Pass. Is there a way to determine this with a random forest model? Is this even possible or is my understanding off?

$\endgroup$
1
  • $\begingroup$ You can from an heuristic point of view, i.e. trial and error; probably not what you are looking for. $\endgroup$
    – Memristor
    Jul 22, 2023 at 2:40

2 Answers 2

1
$\begingroup$

I think this is not possible since a RF is an ensemble of potentially many decision trees (DTs).

Basically, each DT has its own "decisional path" which is kind of "averaged" in the RF. So you loose the interpretability of DTs, which may haved allowed you to understand at which value of a given feature the decision, at a given point, is made. Anyway, what you still get by a RF is an estimate of the importance of each feature which can help you understand which of them is more important (impactful) in the decision process.

$\endgroup$
0
$\begingroup$

You can try using partial dependence plots. I give you an example for binary classification problem:

from sklearn.inspection import PartialDependenceDisplay
from sklearn.ensemble import RandomForestClassifier
import matplotlib.pyplot as plt


# Create some samples and behavior
samples = 1000

X = [[i, i*2] for i in range(0, samples)]
y = []

for x0, x1 in X:
    if x0 > 700:
        y.append(1)
        continue
    elif x0 > 300 and x1 > 700:
        y. append(1)
        continue
    else:
        y.append(0)

rf = RandomForestClassifier()
rf.fit(X, y)

# Make partial dependence plots for features 0 and 1
features = [0, 1]
PartialDependenceDisplay.from_estimator(rf, X, features)

plt.show()

The plot is:

Partial dependence plot for binary classification problem

where you can see the value at which the change is produced. Hope it helps.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.