Proposal: Indices for Clustering.

[ntnshrav]

I have implemented cluster validation indices both internal and external Indices as part of my package.
I have 40 such indices which are tested and packaged as such into a Package called CRAVED. I would like to merge these indices as a part of SKlearns metrics for Clustering. Please let me how i can get started with this.

[jnothman]

I don't think it is in our interest to implement and maintain numerous, rarely used metrics, when we're not able to also advise what the benefits or uses of them are. See related comments on inclusion criteria at our FAQ. I can also immediately see that you are missing several clustering metrics (many of which I know from the related coreference resolution evaluation literature)

I'd be interested in potentially supporting:

• a generic metric for transforming a clustering into binary classification through a pairwise (co-clustered samples) transformation
• a generic metric for transforming a clustering into binary classification through max-sum assignment (see also [MRG]Clusters-Class auto-match #10604)
• avoiding repeated work by allowing contingency matrices to be precomputed (Allow clustering metrics to reuse a once-calculated contingency matrix #8103)
• perhaps a couple more core "internal" measures (some have been recently proposed in other issues: Score Function as a metric for unsupervised clustering algorithms in Scikit-Learn #11351, Dunn Index as a metric for unsupervised clustering algorithms  #11304, and I would appreciate your feedback on whether these are indeed some of the most important metrics to implement)

[jnothman]

Every feature we include has a maintenance cost. Our maintainers are mostly
volunteers. For a new feature to be included, we need evidence that it is
often useful and, ideally, well-established in the literature or in
practice. That doesn't stop you implementing it for yourself and publishing
it in a separate repository.

[ntnshrav]

Score Function : works good for hyper-spheroidal data..s. It is shown to work well on multidimensional
data sets and is able to accommodate unique and sub-cluster
cases.

Davies–Bouldin index - validation of how well the clustering has been done is made using quantities and features inherent to the dataset.

Dunn index -- ratio between the minimal intracluster distance to maximal intercluster distance
Drawbacks : computationally expensive and sensitive to noisy data.

Hartigan Index : generally used to find no. cluster in a dataset (used only for K-Means Algorithm)
advantage: can be used in combination with silhouette or GAP index to find no.of.classes in dataset more accurately than any other methodREF: [Hartigan Indexx]

and about external indices ,

Entropy : degree to which each cluster contains objects of a single class.

Kulczynski_index : arithmetic mean of the precision and recall coefficients.
They reveal some aspects of the clustering done.
Advantage of these External Indices are that They can be Used over In classification metrics also.
So ,I have already Implemented these Indices .Would like to Know what should i do next.

[BradKML]

Also dropping notes from

• Add more unsupervised clustering metrics #6654 (from @pambot)
• unsupervised scoring methods should be a value addition in score.py #20773 (from @BharadwajEdera)
• Dunn Index as a metric for unsupervised clustering algorithms  #11304 (from @53X)
• Gap Statistic and Davies-Bouldin Index DistrictDataLabs/yellowbrick#1205 (others relying on this software)

Firstly, some of the people outside already implemented them in other repos

• https://github.com/online-ml/river-extra/blob/main/river_extra/metrics/cluster
• https://github.com/clslabMSU/clustGUI/blob/master/internal_validation.py
• https://github.com/Simon-Bertrand/Clusters-Features

And there are more papers noting on its differences

• https://www.semanticscholar.org/paper/Comparison-of-Internal-Clustering-Validation-for-H%C3%A4m%C3%A4l%C3%A4inen-Jauhiainen/441b8745311c9a849bd7e41a9bf36d2cfdc133d4/figure/0
• https://www.semanticscholar.org/paper/Understanding-of-Internal-Clustering-Validation-Liu-Li/f504e1c8fd0fb7c19c73a5759a5fa369131c573e/figure/6
• http://www2.uaem.mx/r-mirror/web/packages/clusterCrit/vignettes/clusterCrit.pdf
• http://dx.doi.org/10.3390/w12051372
• http://csjournals.com/IJCSC/PDF6-2/53.%20Kp2.pdf

To note where to insert the code, check https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/metrics/cluster/_unsupervised.py and for prosperity add a test in https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/metrics/cluster/tests/test_unsupervised.py

[BradKML]

@cmarmo Thanks for the RFC. the best internal indices reference implementation is in https://github.com/Simon-Bertrand/Clusters-Features/blob/main/ClustersFeatures/src/_score_index.py

Also for external indices, I have noted some other table for external evaluation (which is suitable for clustering and community detection) GiulioRossetti/cdlib#147 (comment) calling back to #1362