Skip to main content
deleted 2 characters in body
Source Link
MartinM
  • 308
  • 2
  • 10

Well, sometimes the features simply do not provide enough information to get 100% accuracy (like in your case), even with a model a flexible as the Decision Tree.
The Decision Tree works by trying to split the data using condition statements (e.g. A < 1), but how does it choose which condition statement is best? Well, it does this by measuring the "purity" of the split (conditional statements split the data in two, so we call it a "split"). The purity is simply how well the statement split up the target variable (so, a good split would be [2, 0], which basically translates to "If I did this split, I would get two $D = 0$ instances, and zero $D = 1$ instances.", and a bad split would be [1, 1], saying "With this split I would get one $D = 0$ instance and one $D = 1$ instance"). If the model finds that no further splits can reduce the purity, it stops.
If you want to look into it further, there are a couple of algorithmsmeasures for measuring purity (or rather, impurity), the main ones being gini and entropy.

Well, sometimes the features simply do not provide enough information to get 100% accuracy (like in your case), even with a model a flexible as the Decision Tree.
The Decision Tree works by trying to split the data using condition statements (e.g. A < 1), but how does it choose which condition statement is best? Well, it does this by measuring the "purity" of the split (conditional statements split the data in two, so we call it a "split"). The purity is simply how well the statement split up the target variable (so, a good split would be [2, 0], which basically translates to "If I did this split, I would get two $D = 0$ instances, and zero $D = 1$ instances.", and a bad split would be [1, 1], saying "With this split I would get one $D = 0$ instance and one $D = 1$ instance"). If the model finds that no further splits can reduce the purity, it stops.
If you want to look into it further, there are a couple of algorithms for measuring purity (or rather, impurity), the main ones being gini and entropy.

Well, sometimes the features simply do not provide enough information to get 100% accuracy (like in your case), even with a model a flexible as the Decision Tree.
The Decision Tree works by trying to split the data using condition statements (e.g. A < 1), but how does it choose which condition statement is best? Well, it does this by measuring the "purity" of the split (conditional statements split the data in two, so we call it a "split"). The purity is simply how well the statement split up the target variable (so, a good split would be [2, 0], which basically translates to "If I did this split, I would get two $D = 0$ instances, and zero $D = 1$ instances.", and a bad split would be [1, 1], saying "With this split I would get one $D = 0$ instance and one $D = 1$ instance"). If the model finds that no further splits can reduce the purity, it stops.
If you want to look into it further, there are a couple of measures for measuring purity (or rather, impurity), the main ones being gini and entropy.

Source Link
MartinM
  • 308
  • 2
  • 10

Well, sometimes the features simply do not provide enough information to get 100% accuracy (like in your case), even with a model a flexible as the Decision Tree.
The Decision Tree works by trying to split the data using condition statements (e.g. A < 1), but how does it choose which condition statement is best? Well, it does this by measuring the "purity" of the split (conditional statements split the data in two, so we call it a "split"). The purity is simply how well the statement split up the target variable (so, a good split would be [2, 0], which basically translates to "If I did this split, I would get two $D = 0$ instances, and zero $D = 1$ instances.", and a bad split would be [1, 1], saying "With this split I would get one $D = 0$ instance and one $D = 1$ instance"). If the model finds that no further splits can reduce the purity, it stops.
If you want to look into it further, there are a couple of algorithms for measuring purity (or rather, impurity), the main ones being gini and entropy.