How to control a decision tree?

Question

This is my R script for a decision tree:

    library(caret)
    library(rpart.plot)
    library(plyr)
    library(dplyr)
    library(rpart)
    
    data("iris")
    
    names(iris) = tolower(names(iris))
    
    table(iris$species)
    suppressMessages(library(caret))
    
    index = createDataPartition(y=iris$species, p=0.7, list=FALSE)
    
    train = iris[index,]
    test = iris[-index,]
    
    trainctrl <- trainControl(method = "cv", number = 5, verboseIter = FALSE)

dt.model <- train(species~., data=train, method = "rpart", 
                  tuneLength = 10,
                  preProcess = c("center", "scale"),
                  trControl = trainctrl,
                  metric="Kappa")

dt.predict <-predict(dt.model, test)
confusionMatrix(dt.predict, test$species)

rpart.plot(dt.model$finalModel)
varImp(dt.model)

my feature importance are:

> varImp(dt.model)
rpart variable importance

             Overall
petal.width   100.00
petal.length   96.95
sepal.length   45.08
sepal.width     0.00

Is there a way to consider less petal.width? For example, I want that my tree use more petal.length and sepal.lenght than petal.width. Is it possible?

---

My problem is that in my dataset that I am using the decision tree on, one variable x is used more than the others. This x variable, however, affects the value of another y variable which is the one that actually characterizes my classes. Using an example with Iris: it's like saying that petal.length is related proportionally to petal.width, ie:

petal.width petal.length class
1            4           Virginica
1            6           Setosa
2            6            Virginica
2            7            Setosa

my model discriminates more the classes in base to the "petal.width" of my dataset rather than to the "petal.lenght". For this I would want to give a various weight to the characteristics

Answer 1

In the comments you wrote:

I can't exclude it because it is related on an other feature

If you mean related as in a function of two other features, you could add columns to explicitly show those relations, then delete the original column. E.g. add a column called petal.width.plus.length and then remove petal.width.

Taking a step back, I think your problem could be rephrased as decision trees end up in a local minima, and so fail to find the best solution? If so, consider another algorithm? E.g. random forest, GBMs (e.g. XGBoost) or even deep learning.

If other algorithms also don't do what you expect, you should also consider that the data is not saying what you think it is. Maybe you need to collect more data, or oversample some rows, or look for bugs (e.g. some data has turned into NAs).

Finally (and I suppose this is the answer to the question you are asking) you can manually partition the data on the field you believe is most important, and then build a decision tree for each partition. As you need to do that each time you want to do predict you definitely want to write your own wrapper class to store the partitions.

Answer 2

As far as I know there would be only two ways to use a particular feature "less" than its normal use by the decision tree:

• Remove the feature completely.
• Modify the data so that the feature becomes less important compared to the others (don't do it, this is complex and pointless)

It's important to understand that the decision tree learning algorithm decides which feature to use at every node by picking the feature which is the most discriminative in the data, i.e. the one which is the most informative to predict the target label. So using a particular feature "less" would mean that the choice of which feature to use is not based on the best way to predict the target according to the data... So it would not really follow the logic of training a decision tree.