Ship citation info with our packages

[jl-wynen]

Since we publish scientific packages, it makes sense to make those packages easily citable. We currently have DOIs for all packages and refer to them in our docs. This is good but requires users to manually search for it in the docs. We can expose this information in more places to make it easier to find. And ideally also programmatically findable and introspectable.

Previous work

There is no standard or convention in Python.

Some (many) packages provide a __citation__ attribute: https://github.com/search?q=__citation__&type=Code. The contents of that attribute are diverse and range from a simple DOI to a full BibTeX entry. This seems to generally be borne out of astropy which provides a BibTex entry.

Different attempts have been made but none got anywhere. It looks like they weren't rejected, in fact responses are generally positive. They just seem to have run out of steam.

• [Python-ideas] Add a cite method for scientific packages
• Convention for encouraging citation of python packages

There are also some packages that aim to help with citation:

• cff-from-612 builds a CFF file from a pyproject.toml file.
• duecredit injects citation data into a package and can extract it from an installed package.

Citation data can be provided by a separate file. GitHub can handle many such files and display a link on a project's home page: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-citation-files.
There is a dedicated format for storing citation information that is both human readable and easy to parse by a program: https://citation-file-format.github.io/

Assessment

It is unfortunate that there is no consensus within the Python community. The closest we have is the __citation__ attribute. But its diverse content makes it difficult to handle. In particular, BibTeX entries are difficult to parse.

3rd party packages can help but adoption is low and relying on them presents a risk.

Providing a citation file in the repo is good. But it does not address the introspection issue. Even if we ship it with our packages, it is difficult to handle. Citing a package should be as easy as possible to not scare away users.

Ideas

We should definitely provide citation files in our projects. Even if for no other reason than to provide one more way for people to find that information.

Further, we can make our own in-house convention¹ for storing citation info in packages. We can start with storing a DOI in __citation__ as there is precedent for that. This would already cover some use cases.
If we want more, we can package a CITATION.cff file. These files appear to be YAML files even though this isn't stated in the docs. So they are easy to parse if we want to access the info programmatically.

Example CFF file for ScippNeutron

Here is an incomplete(!) CFF file for ScippNeutron for reference:

# This CITATION.cff file was generated with cffinit.
# Visit https://bit.ly/cffinit to generate yours today!

cff-version: 1.2.0
title: ScippNeutron
message: Please cite this software using these metadata.
type: software
authors:
  - given-names: Simon
    family-names: Heybrock
    affiliation: 'European Spallation Source ERIC'
  - given-names: Jan-Lukas
    family-names: Wynen
    email: [email protected]
    affiliation: European Spallation Source ERIC
    orcid: 'https://orcid.org/0000-0002-3761-3201'
identifiers:
  - type: doi
    value: 10.5281/zenodo.14410489
repository-code: 'https://github.com/scipp/scippneutron'
url: 'https://scipp.github.io/scippneutron/'
repository-artifact: 'https://pypi.org/project/scippneutron/'
abstract: >-
  Neutron scattering tools for Data Reduction powered by
  Scipp.
keywords:
  - neutron
  - scattering
license: BSD-3-Clause
commit: 25.01.0
version: v25.01.0
date-released: '2025-01-20'

Footnotes

1. By 'in-house' I mean ideally all of DMSC, not just Scipp. ↩

[MridulS]

Thanks for the overview!

Further, we can make our own in-house convention1 for storing citation info in packages. We can start with storing a DOI in citation as there is precedent for that.

I assume this means users can access this information with something like ?

>>> import scippneutron as scn
>>> scn.__citation__
 'https://doi.org/10.5281/zenodo.7760280'

I do like the idea of keeping everything in a citation.cff as we can have a single source of truth and the .cff file can be programtically read to inject this information into __citation__ too.

[jl-wynen]

Trying that out now.

[jl-wynen]

Got it to work. But there are some tricky bits.

Location of CITATION.cff

GH wants CITATION.cff in the project root. But to include it in the wheel, it has to be in src/<package>/. It looks like we can make a symlink from ./src/<package>/CITATION.cff to ./CITATION.cff. Then we only have one actual file and both GH and setuptools are happy.

Reading CITATION.cff

This is easy when the file is part of the wheel:

def _read_citation():
    from importlib import resources
    import yaml
    record = yaml.safe_load(resources.read_text(__package__, "CITATION.cff"))
    for identifier in record['identifiers']:
        if identifier['type'] == 'doi':
            return identifier['value']
    raise RuntimeError("No DOI")

__citation__ = _read_citation()

Keeping the citation record up to date

This is probably the biggest problem. We face the same issue as with version numbers in that we have some version-specific data in the file that needs to be kept up to date every time we make a release. See the example from the OP. The CFF format examples also show two DOIs, one for a concrete version and one for all versions collectively. We also have both on Zenodo.

We could conceivably write a script to inject the current version number and release date as part of the release process. We might even be able to hook directly into setuptools with a custom plugin. But this may not be able to get the DOI for the current release because that only gets minted after the package has been released. So I don't know how we could make this work. (Maybe it gets minted as soon as we click the release button on GH and so the release workflow can read it from Zenodo. Something to find out...)

[MridulS]

But to include it in the wheel, it has to be in src//

Can't we use MANIFEST.in to control the files without symlinks?

But this may not be able to get the DOI for the current release because that only gets minted after the package has been released.

Zenodo does have the concept of pre-reserving DOIs but unfortunately it doesn't work with github integration https://support.zenodo.org/help/en-gb/24-github-integration/73-can-i-pre-reserved-a-doi-before-a-github-release

Also interestingly zenodo supports cff https://support.zenodo.org/help/en-gb/24-github-integration/96-how-does-a-citation-cff-file-affect-metadata-of-my-github-release

[jl-wynen]

Can't we use MANIFEST.in to control the files without symlinks?

MANIFEST.in only affects source distributions, not wheels: https://setuptools.pypa.io/en/latest/userguide/miscellaneous.html

Zenodo does have the concept of pre-reserving DOIs but unfortunately it doesn't work with github integration https://support.zenodo.org/help/en-gb/24-github-integration/73-can-i-pre-reserved-a-doi-before-a-github-release

Thanks for finding this! Then maybe we need to upload differently. This can't be too hard, can it? :-)
Maybe we can use https://github.com/marketplace/actions/zenodo-upload

[jl-wynen]

I tried using the action https://github.com/marketplace/actions/zenodo-upload to upload a test project to the Zenodo sandbox: https://github.com/jl-wynen/test_software_citation But it fails and I don't know why. In general, the action does not see a lot of activity or stars. So we may not be able to rely on it.

While looking into this, I came across the CodeMeta Project which defines an alternative codemeta.json file that is largely equivalent to CITATION.cff. It seems to have some traction and good support.

[jl-wynen]

If Zenodo doesn't work out for us, we could also look into the Software Heritage project for uploading our software. They have our code anyway: swh:1:dir:2143375f73b6417410f348a3251c9c6a062ae060

But they don't seem to mint DOIs. Instead they have their own identifier: SWHID (see the link above for an example). So this makes it more difficult to cite the software until (and if) publishers offer the option to cite SWHIDs.

I also don't know how to keep a CITATION.cff or codemeta.json file up to date with Software Heritage. But at least, Software Heritage supports codemeta.json.