A strategy for finding relevant clusters; with an application to microarray data

Summary

Cluster analysis is a helpful tool for explorative data analysis of large and complex data. Most clustering methods will, however, find clusters also in random data. An important aspect of cluster analysis is therefore to distinguish real and artificial clusters, as this will make interpretation of the clusters easier. In some cases, certain types of clusters are more interesting than others. When working with gene expression data, examples of such clusters are gene clusters with high between sample variability, and clusters with a certain expression profile. Here we present a strategy with the ability to search for such clusters. The clustering is done sequentially. For each sequence, the data is separated into "interesting" and "rest" using the fuzzy c-means algorithm with noise clustering. The interesting cluster is defined by adding a penalty function to the usual clustering criterion. The penalty function is constructed in such a way that clusters without the interesting properties are given a high penalty. The strategy is presented in a general frame, and can be adjusted by defining different criteria for each type of cluster that is of interest. The methodology is presented in the context of microarray data but can be used for any type of data where cluster analysis may be a helpful tool. The methodology is illustrated with simulated data and microarray data.