I'm pursuing a computer science minor at my university, and one class I'm in is Machine Learning. We have a project to utilize a few algorithms we have learned so far. I've been using SciKit-Learn to perform these algorithms, but when it comes to decision trees I keep getting a feeling I am overfitting.

I'm using a dataset about weather, giving characteristics such as city, state, month, year, wind direction, wind speed, etc... where the target variable is the average temperature for the day. Now I know this is hard to classify, as it is pretty much a continuous variable space, but I've simplified it to the predicted being within a range of 5 to the actual.

Here is a link to the csv I'm using

The following is my code:

address2 = 'C:/.../weather.csv'
weather = pd.read_csv(address2)

cityCode= le.fit_transform(weather.iloc[:,2])
windDirection = le.fit_transform(weather.iloc[:,3])
month = le.fit_transform(weather.iloc[:,8])
precip = le.fit_transform(weather.iloc[:,9])
windSpeed = le.fit_transform(weather.iloc[:,10])
state = le.fit_transform(weather.iloc[:,11])
week = le.fit_transform(weather.iloc[:,12])
year = le.fit_transform(weather.iloc[:,13])

Xweather = list(zip(cityCode,windDirection,month,precip,windSpeed,state,week,year))
yweather= weather.iloc[:,0]

yweather_test = train_test_split(Xweather, y, test_size = 0.2, random_state=413)

cWeather = tree.DecisionTreeClassifier()
cWeather.fit(Xweather_train,yweather_train)
accu_train_weather=np.sum(abs(cWeather.predict(Xweather_train)-yweather_train)<=5)/float(yweather_train.size)*100
accu_test_weather=np.sum(abs(cWeather.predict(Xweather_test)-yweather_test)<=5)/float(yweather_test.size)*100
print("Classificaton accuracy on training set", accu_train_weather, "%")
print("Classificaton accuracy on test set", accu_test_weather, "%")

My training set constantly gets 100% training accuracy, but the test set is constantly 57% accurate, which leads me to believe the tree is overfitting to the training set.

I know I'm not doing any pruning, but even when I do, I can get the same test accuracy as unpruned at best. By pruning I mean setting the tree classifier to have a maximum number of leaves, minimum samples per leaf, and maximum depth.

I'm not an advanced coder (as you can probably tell by my code), but any help would be great.

We have a project to utilize a few algorithms we have learned so far. I've been using SciKit-Learn to perform these algorithms, but when it comes to decision trees I keep getting a feeling I am overfitting.

I'm using a dataset about the weather, giving characteristics such as city, state, month, year, wind direction, wind speed, etc... where the target variable is the average temperature for the day. Now I know this is hard to classify, as it is pretty much a continuous variable space, but I've simplified it to the predicted being within a range of 5 to the actual.

Here is a link to the csv I'm using

The following is my code:

address2 = 'C:/.../weather.csv'
weather = pd.read_csv(address2)

cityCode= le.fit_transform(weather.iloc[:,2])
windDirection = le.fit_transform(weather.iloc[:,3])
month = le.fit_transform(weather.iloc[:,8])
precip = le.fit_transform(weather.iloc[:,9])
windSpeed = le.fit_transform(weather.iloc[:,10])
state = le.fit_transform(weather.iloc[:,11])
week = le.fit_transform(weather.iloc[:,12])
year = le.fit_transform(weather.iloc[:,13])

Xweather = list(zip(cityCode,windDirection,month,precip,windSpeed,state,week,year))
yweather= weather.iloc[:,0]

yweather_test = train_test_split(Xweather, y, test_size = 0.2, random_state=413)

cWeather = tree.DecisionTreeClassifier()
cWeather.fit(Xweather_train,yweather_train)
accu_train_weather=np.sum(abs(cWeather.predict(Xweather_train)-yweather_train)<=5)/float(yweather_train.size)*100
accu_test_weather=np.sum(abs(cWeather.predict(Xweather_test)-yweather_test)<=5)/float(yweather_test.size)*100
print("Classificaton accuracy on training set", accu_train_weather, "%")
print("Classificaton accuracy on test set", accu_test_weather, "%")

My training set constantly gets 100% training accuracy, but the test set is constantly 57% accurate, which leads me to believe the tree is overfitting to the training set.

I know I'm not doing any pruning, but even when I do, I can get the same test accuracy as unpruned at best. By pruning I mean setting the tree classifier to have a maximum number of leaves, minimum samples per leaf, and maximum depth.