Adding support for other types of distances

[Aceticia]

Hi, it's nice discovering the package and I really enjoy using it so far. There are some great new works that propose new metrics for measuring dissimilarity, for example the generalized shape metrics (code here) and the generalized Riemannian metric mentioned here. It would be really nice to integrate those into the package. I'll be happy to submit a PR if people like this idea.

Alternatively, maybe there could be a unified interface for the user to provide a callable like many sklearn apis.

A downside to this would be that we will have to deal with a large number of built-in distances without clear knowledge about when to use which. A good idea might be to use some sample data to create a preliminary benchmark.

As a side note, I'm not sure how, but I really wish I knew about this project sooner. By this point I've written a lot of rsa code in python myself, but luckily not enough to make me think I should start a package.

[JasperVanDenBosch]

Welcome @Aceticia ! A custom distance metric is something I've considered as well, but we are a little swamped right now with some goals for the release, including proper demo's for all modalities. We'll be sure to be in touch about plans in this direction.

[HeikoSchuett]

Hi @Aceticia,
by the way: The Riemannian metric is actually implemented here:

rsatoolbox/src/rsatoolbox/rdm/compare.py

Line 246 in 69f27f9

def compare_neg_riemannian_distance(rdm1, rdm2, sigma_k=None):

and you can use it by using the method string "neg_riem_dist"

The shape metric methods are also interesting, especially when you want to compare many models to each other or want to use the similarities between models as a distance. We have not converged on whether and how to incorporate them though. The problem is that the shape metric approach only sometimes maps on the schema of first computing an RDM and them comparing them between models. In other cases, you never compute an equivalent of the RDM, which would make implementing these cases awkward in this toolbox.

[Aceticia]

@HeikoSchuett Ha! Thank you. I didn't notice this choice being available. Yeah, I agree that this would be a little problematic in terms of interface. My two cents:

We can directly interact with datasets that generate them and have more general methods that can be used to "compare" models, where shape metrics and compare_rdms could both be options. But this means a very big change to the code and a drastically different user experience.

An alternative that doesn't break the interface: When we calculate the RDMs, we keep a reference to the raw dataset. Let's say we have a brain dataset A and a model B. We can have a method shape_metric for comparing them and under the hood we go back to the dataset to do the comparison. This keeps with the interface.