3.6 General remarks As mentioned before,the measures presented in this section can all be cal- culated by means of the confusion matrix M(and the cluster sizes);this is either obvious from the formula (e.g.for the Fowlkes-Mallow Index)or can be seen after some transformation (e.g.for the Rand Index,which can be transformed into a variation of the Mirkin Metric,see 3.4). For different reasons,these measures do not seem to be very appealing. Some of them are sensitive to certain parameters(cluster sizes,number of clusters);think of a pair of clusterings with similarity a E 0,1]and replace each element in the underlying set by two elements.Why should the result- ing pair of clusterings have a similarity other than a?This behavior,as well as sensitivity to the number of clusters,are undesirable. Other measures,like the Fowlkes-Mallows Index,suffer from another draw- back:they make use of a very strong null hypothesis,that is,independance of the clusterings,fixed number of clusters,and fixed cluster sizes.When comparing results provided by clustering algorithms these assumptions are apart from the number of clusters that is fixed for some algorithms-vio- lated.None of the algorithms works with fixed cluster sizes.Furthermore, in practice it would be against the intuition to compare two clusterings when assuming that there is no relationship between them.In fact,we compare clusterings because we suppose a certain relationship and we want to know how strong it is [12]. 4 Measures based on set overlaps Another kind of measure tries to match clusters that have a maximum ab- solute or relative overlap.This is also a quite intuitional approach,however the assymetry of some of the measures makes them difficult to use. 4.1 F-Measure The F-Measure has its origin in the field of document clustering where it is used to evaluate the accuracy of a clustering solution.Each cluster of the first clustering is a(predefined)class of documents and each cluster of the second clustering is treated as the result of a query [14,[13].The F- Measure for a cluster C with respect to a certain class Ci indicates how "good"the cluster C describes the class Ci by calculating the harmonic 83.6 General remarks As mentioned before, the measures presented in this section can all be cal￾culated by means of the confusion matrix M (and the cluster sizes); this is either obvious from the formula (e.g. for the Fowlkes-Mallow Index) or can be seen after some transformation (e.g. for the Rand Index, which can be transformed into a variation of the Mirkin Metric, see 3.4). For different reasons, these measures do not seem to be very appealing. Some of them are sensitive to certain parameters (cluster sizes, number of clusters); think of a pair of clusterings with similarity α ∈ [0, 1] and replace each element in the underlying set by two elements. Why should the result￾ing pair of clusterings have a similarity other than α? This behavior, as well as sensitivity to the number of clusters, are undesirable. Other measures, like the Fowlkes-Mallows Index, suffer from another draw￾back: they make use of a very strong null hypothesis, that is, independance of the clusterings, fixed number of clusters, and fixed cluster sizes. When comparing results provided by clustering algorithms these assumptions are - apart from the number of clusters that is fixed for some algorithms - vio￾lated. None of the algorithms works with fixed cluster sizes. Furthermore, in practice it would be against the intuition to compare two clusterings when assuming that there is no relationship between them. In fact, we compare clusterings because we suppose a certain relationship and we want to know how strong it is [12]. 4 Measures based on set overlaps Another kind of measure tries to match clusters that have a maximum ab￾solute or relative overlap. This is also a quite intuitional approach, however the assymetry of some of the measures makes them difficult to use. 4.1 F-Measure The F-Measure has its origin in the field of document clustering where it is used to evaluate the accuracy of a clustering solution. Each cluster of the first clustering is a (predefined) class of documents and each cluster of the second clustering is treated as the result of a query [14], [13]. The F￾Measure for a cluster C 0 j with respect to a certain class Ci indicates how ”good” the cluster C 0 j describes the class Ci by calculating the harmonic 8