# Why Decision Tree boundary forms a square shape and SVM a circular/oval one?

I was going through a Udacity tutorial wherein a few data points were given and the exercise was to test which of the following models best fit the data: linear regression, decision tree, or SVM. Using sklearn, I was able to determine that that SVM is the best fit followed by decision tree. I got a very distinct decision boundary when these two algorithms were applied:

Is there any specific reason for the said shapes or does it just depend on the data sets?

The code was quite straightforward; just reading the CSV, separating the features and then applying the algorithms as shown below:

from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC

import pandas
import numpy

# Split the data into X and y
X = numpy.array(data[['x1', 'x2']])
y = numpy.array(data['y'])

# import statements for the classification algorithms
from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC

# Logistic Regression Classifier
classifier = LogisticRegression()
classifier.fit(X,y)

# Decision Tree Classifier
classifier.fit(X,y)

# Support Vector Machine Classifier
classifier = SVC()
classifier.fit(X,y)


Shape of the SVM decision boundary depends on the kernel (similarity function) used. The "standard" version of SVM has linear decision boundary. The one displayed could be using Gaussian kernel.

Decision boundary of a decision tree is determined by overlapping orthogonal half-planes (representing the result of each subsequent decision) and can end up as displayed on the pictures.

See more here:

https://shapeofdata.wordpress.com/2013/07/02/decision-trees/

https://www.quora.com/What-are-Kernels-in-Machine-Learning-and-SVM

I found this slide very useful in understanding the rectangular decision boundaries generated by decision trees .