0
$\begingroup$

I have built a random forest classification model with Python, which works really well. However, for the implementation, I wanna build it from scratch on SQL. Does a Random Forest Classification model have coefficients? Are the feature impotance values the same as coefficients?

$\endgroup$

4 Answers 4

0
$\begingroup$

Random forest (RF) is based on the "majority vote" and it is a non-parametric method. Thus, there are no parameters that need to be estimated. The feature importance is based on the permutation and again no parameters that need to be estimated. There for your questions:

  1. The 1st question: NO
  2. The 2nd question: NO
$\endgroup$
0
$\begingroup$

What does it mean to build on SQL? Does this mean you will re-implement all of the trees in the model into SQL? That will be a lot of case statements and joins. A quick thought says each tree could be a temp table or a common table expression with a bunch of nested case statements then joined over a unique value of the record and another set of case statements to vote. This will be a lot of work and not very debuggable or maintainable. When the model is refit, all of the work to develop the SQL needs to be redone. Consider there may be 500, 1,000 or more trees with 10 or more levels in each tree (some Random Forrests can be big). Some RDBMS cannot join 1,000 tables so you might need to split the final joins into multiple. Some RDBMS might have a limit on nested case statements.

This sounds like an opportunity to re-evaluate the production environment. If scoring must be in SQL, then a model with coefficients seems the better option.

If the RDBMS in use supports remote procedures, then examine linking to the tool that built the random forrest and can score directly.

$\endgroup$
0
$\begingroup$

SQL is not a programming language, rather a (human-understandable) query language; hence, it is very uncommon to have, for example, any implementation of Random Forests (RFs). Nevertheless, (declarative and procedural) SQL is Turing-complete and, as such, RFs could be implemented if wanted.

There are no coefficients if the intended meaning is similar to those of neural networks. If we consider the regression case with RFs (the classification task can be viewed as a particular case), the output is the average of each (base, decision tree) regressor output, that is: $$ \hat{y} = RF(\vec{x}) = \frac{1}{K} \cdot \sum_{k = 1}^K DT_k(\vec{x}) = \frac{1}{K} \cdot DT_1(\vec{x}) + \frac{1}{K} \cdot DT_2(\vec{x}) + \ldots + \frac{1}{K} \cdot DT_K(\vec{x}), $$ where $\hat{y}$ is the predicted output, $K$ is the number of trees in the RF model and $DT_i(\vec{x})$, for $1 \le i \le K$, is the predicted output of the $i$-th tree on the input vector $\vec{x}$. Then, the $K$-dimensional coefficients' vector can be learned by, for example, a linear regression model.

Finally, as DaCard said, feature importance is not related to coefficients.

$\endgroup$
0
$\begingroup$

I'm not sure what you mean by "coefficients". A random forest classifier is made up of decision trees. Each decision tree has split points where it selects a feature and value to split at.

In a random forest, feature importance can be estimated by the impurity which is computed as the (normalized) total reduction of the criterion brought by that feature. That is an additional step beyond the splitting step.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.