# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     language: python
#     name: python3
# ---
# # Exercise 42: Classification Using a CART Decision Tree
#
# In this activity we will be using the well known Iris Species dataset [https://en.wikipedia.org/wiki/Iris_flower_data_set](https://en.wikipedia.org/wiki/Iris_flower_data_set) created in 1936 by botanist <NAME>.  The dataset contains Sepal and Petal Length and Width measurements of three different Iris flower species *Iris setosa*, *Iris versicolor*, *Iris virginica*.  Throughout this activity we will use the measurements provided in the dataset to classify the different flower species using a CART decision tree.
#
# ## Import the Required Packages
#
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.tree import DecisionTreeClassifier
# ## Load the Data
# Load the Iris dataset using Pandas and examine the first 5 rows
df = pd.read_csv('iris-data.csv')
df.head()
# ## Train the model
# Train the decision tree model using all available data.  **Remember that in practice decision trees tend to overfit the training set and thus cross validation should be used (see Lesson Six)**
#
# As we have yet to cover cross validation we will extract a random sample of 10 rows to with-hold from training and will provide the data as "unseen" samples to get an idea of the performance of the Decision Tree.
np.random.seed(10)
samples = np.random.randint(0, len(df), 10)
df_test = df.iloc[samples]
df = df.drop(samples)
# Fit the model
model = DecisionTreeClassifier()
model = model.fit(df[['Sepal Length', 'Sepal Width', 'Petal Length', 'Petal Width']], df.Species)
model.score(df[['Sepal Length', 'Sepal Width', 'Petal Length', 'Petal Width']], df.Species)
# The model achieved 100% accuracy when compared against the training set.  Why is this?  Will this always occur for decision trees?
#
# Compare against the test set:
model.score(df_test[['Sepal Length', 'Sepal Width', 'Petal Length', 'Petal Width']], df_test.Species)
# ## Visualising the Model
# Depending on your system setup you may need to first install the graphviz package before you can visualise the decision tree.  With Anaconda you can enable and run the next cell to install graphviz
# !conda install python-graphviz
import graphviz
from sklearn.tree import export_graphviz
df.columns[3]
dot_data = export_graphviz(model, out_file=None) 
graph = graphviz.Source(dot_data) 
graph