The AUSTRALIAN data set

Table 6: Performance comparison between the MT model and other classification methods on the AUSTRALIAN dataset [Michie, Spiegelhalter, Taylor 1994]. The results for mixtures of factorial distribution are those reported by [Kontkanen, Myllymaki, Tirri 1996] .

Method	% correct		Method	% correct
Mixture of trees $m=20, \beta=4$	87.8		Backprop	84.6
Mixture of factorial distributions	87.2		C4.5	84.6
Cal5 (decision tree)	86.9		SMART	84.2
ITrule	86.3		Bayes Trees	82.9
Logistic discrimination	85.9		K-nearest neighbor $k=1$	81.9
Linear discrimination	85.9		AC2	81.9
DIPOL92	85.9		NewID	81.9
Radial basis functions	85.5		LVQ	80.3
CART	85.5		ALLOC80	79.9
CASTLE	85.2		CN2	79.6
Naive Bayes	84.9		Quadratic discrimination	79.3
IndCART	84.8		Flexible Bayes	78.3

This dataset has 690 examples each consisting of 14 attributes and a binary class variable [Blake, Merz 1998]. In the following we replicated the experimental procedure of [Kontkanen, Myllymaki, Tirri 1996] and [Michie, Spiegelhalter, Taylor 1994] as closely as possible. The test and training set sizes were 70 and 620 respectively. For each value of $m$ we ran our algorithm for a fixed number of epochs on the training set and then recorded the performance on the test set. This was repeated 20 times for each $m$, each time with a random start and with a random split between the test and the training set. Because of the small data set size we used edge pruning with $\beta \propto 1/m$. The best performance of the mixtures of trees is compared to the published results of [Kontkanen, Myllymaki, Tirri 1996] and [Michie, Spiegelhalter, Taylor 1994] for the same dataset in Table 6.