Intertwining Binary Decision Trees and Probabilistic Neural Networks for Maximising Accuracy and Efficiency in Classification Tasks

A Pilot /Proof–of–Concept Demonstration on the Iris Benchmark Classification Dataset

Abstract

Intertwining binary decision trees (BDTs) and probabilistic neural networks (PNNs) is put forward as a powerful custom–made methodology for simultaneously maximising accuracy and efficiency in classification tasks. The proposed methodology brings together the complementary strengths of (I) the parametric, global, recursive, efficient as well as maximal dichotomisation of the BDT, and (II) the non–parametric, accurate–to–local–detail, multi–class identification of the PNN. The BDT/PNN combination evolves level–by–level, with each level comprising two steps: (step 1) the optimal BDT (hyper–)linear bi–partitioning of the entire dataset for the first level, and of every non–terminal partition thereof for the next levels, is determined, with each resulting partition being assigned to a new node of the next BDT level; (step 2) as many PNNs are created as there are  multi–class partitions resulting from (step 1), implementing the non–parametric creation of the hyperplane that maximises class separability over each corresponding partition of the problem–space. BDT/PNN expansion is applied recursively – and independently – to each non–terminal partition of the dataset, concluding once the cumulative classification accuracy (CCA) of the terminal – BDT and PNN – nodes of the BDT/PNN is maximised. A proof–of–concept demonstration on the iris benchmark classification dataset (IBCD) illustrates the details of the BDT/PNN combination, attesting to its simplicity, effectiveness, and efficiency of operation. Follow–up research shall focus upon the classification of additional benchmark datasets, as well as of “hard” and “Big” real–world problems, for further evaluating and validating the proposed methodology.