Performing hierarchical clustering on distance matrices in OptiML

Abstract

Since it became evident that domain-specific languages are able to follow the trends of language design evolving demands, they became increasingly popular. This is resulting in the need for a convenient host languages, that can satisfy and ease the aspects of DSL development. Scala is highly flexible, virtualised language, serving as a great ground for different DSLs. In this paper, we used some of the languages, developed on top of Scala, namely Lightweight Modular Staging, Delite and OptiML, to start the development of our own data-mining DSL. Our main aim is to develop a highly efficient DSL for some important data-mining algorithms. As a starting point, we implemented hierarchical clustering in OptiML, with Scala code generation. We also performed testing, based on experimental data, gained from a psychological experiment, related to human behavior analysis and artificial agent development. We compared the results of our hierarchical clustering, to results gained from R on the same data set. This algorithm serves as a starting point towards parallel code generation for a set of data-mining algorithms.