• Solve a query about the possibility of use prediction/classification variable importance tools in a time series type dataframe.
  • Collect the largest number of variable importance algorithms. If you know another, please post it! (R & Python) (in: classification/prediction/forecasting problems).

Actually, in prediction & classification problems, there exist many algorithms to know the variable importance of each observable characteristic on results. An example can be:

In R: #y ~. <=> y = x1 + x2 + x3 +...

model = train(formula = y ~., data = train_set, method = ...)

model = randomForest(formula = y ~., data = train_set, ..., importance = TRUE) 

model = xgboost(formula = y ~., data = train_set, ...)
xgb.importance(colnames(train_set), model = model)

model = catboost.train(catboost_pool, params = as.list(...))
catboost.get_feature_importance(model, pool = NULL, type = 'FeatureImportance', thread_count = -1)

In Python:

from xgboost import XGBClassifier
from xgboost import plot_importance
from matplotlib import pyplot as plt
model = XGBClassifier(...)
model.fit(X_train, y_train)

from catboost import CatBoostClassifier, Pool
model = CatBoostClassifier(...)
feature_importances = model.get_feature_importance(train_pool)
feature_names = X_train.columns
for score, name in sorted(zip(feature_importances, feature_names), reverse = True):
   print('{}: {}'.format(name, score))

from sklearn.ensemble import RandomForestRegressor
from matplotlib import pyplot as plt
model = RandomForestRegressor(...)
model.fit(X_train, y_train)
plt.barh(X.feature_names, model.feature_importances_)

#Using Permutation
from sklearn.inspection import permutation_importance
perm_importance = permutation_importance(model, X_test, y_test)
plt.barh(model.feature_names, perm_importance.importances_mean)

#Using SHAP Values
import shap
explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_test)
shap.summary_plot(shap_values, X_test, plot_type=...)

#Here you can use it in Deep Learning with Keras/Tensorflow models too
import tensorflow
import tensorflow.keras.backend 
model = Sequential()
explainer = shap.DeepExplainer(model,  X_train)
shap_values = explainer.shap_values(X_test)
shap.summary_plot(shap_values, X_test, plot_type=...)

My post is focused on the 'context where I can use this tools'.

PROBLEM: The problem that I must to resolve is determine the behavior of asset prices of a company. I will have the historical data with many exogenous variables. But the structure of this data is like a 'time series set' where I only have the prices' evolution across time (!= cross-section database).
INTENTION: Use the tools above.
DISAGREEMENT: Forecasting != (prediction & clasification) because your target depend also the time.

  • Can I use those variable importance tools in time series problems? (to solve as prediction problem). If not,
  • Are there computational-theorical tools in R or Python that help in this context?
  • What would be the theoretical basis for why not? I'm thinking that maybe is for the dataframe is not a cross-section database. Right?
  • In addition to using variable importance algorithms, what would you do to solve something like this in time series?

EXAMPLE: For example, a posted solution can be [1], but in my opinion, this is wrong. Is it?

I know it can be possible [2]. Like feature selection [3] in R [4]. But I don't find any code-package-library where I can use it like codes above. Furthemore, R include packages that use XGBoost in forecasting problems on newly packages [5]. So is really confuse. Do you have a preferred library?

Assume that you have the best forecasting algorithm choose in a set of algorithms. The function (as 'caret' in R) bring you a ranked table with variables.

[1] https://www.r-bloggers.com/2021/03/time-series-forecasting-with-xgboost-and-feature-importance/
[2] https://arxiv.org/pdf/2005.00259.pdf
[3] https://link.springer.com/chapter/10.1007%2F11430919_60
[4] https://cran.r-project.org/web/packages/fsMTS/fsMTS.pdf
[5] https://cran.r-project.org/web/packages/modeltime/modeltime.pdf

Thank you very much for your time. Have a nice day.
Best regards, Mirko.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.