[MRG] Travis: Deploy releases to pypi

[manics]

Release tagged builds to PyPi using Travis.

A matrix build will normally deploy multiple times. The usual way of deploying once is to use either skip_existing: true or a conditional that matches one of the jobs. The downside is that the deploy only depends on whether that particular job passes, other jobs in the matrix may fail.

I think using build stages is better because the deploy only happens if all matrix jobs pass. It's also clearer in travis since the deploy runs as it's own job.

I've tested this on testpypi:

• manics@aa53d6d
• https://test.pypi.org/project/oauthenticator-testpypi/0.0.1.dev3/
• https://travis-ci.com/manics/oauthenticator/builds/135826346
  

Next steps:

• Either add a secret environment variable PYPI_TOKEN to Travis or add an encrypted secret to travis.yml

Related:

• Make a release #299
• Add RELEASE.md #294 (I'll help update this if we decide to go ahead with the automatic deploy for this release)
• Systematization of CD in our repos team-compass#213 (comment)

[betatim]

Looks good to me!

[consideRatio]

I really like having build stages, +100 for using them!

I'd like to see that the nightly build is allowed to fail, and that it is put last to make everything else complete faster as a final job that is allowed to fail doesn't need to finish for the travis build as a whole can report success. See for example https://docs.travis-ci.com/user/customizing-the-build/#allow_failures-examples for a not obvious example, and combine that with the fast_finish thing I documented in the z2jh .travis.yml: https://github.com/jupyterhub/zero-to-jupyterhub-k8s/blob/04f8a1e1546dc79a4fecc13b62bb4fc15c2992ba/.travis.yml#L68-L72

[manics]

@consideRatio How does this look?

[consideRatio]

@manics this LGTM! Is the token stuff setup?

[manics]

@consideRatio No! Someone with access tothe PyPi project needs to add the token either in the Travis web ui or by adding a secret to .travis.yml. We can either wait, or merge and sort out the token separately since it only takes effect on a tag. I tested it on testpypi but probably best to test again with an RC just in case.

[consideRatio]

@manics i was a maintainer but not an owner so I cannot create such token. What path do you recommend to take regarding adding it to the web ui of travis or adding it as a encrypted secret in .travis.yml ?

@choldgraf or @minrk can perhaps help out with permissions

[choldgraf]

@consideRatio you are now an owner!

[manics]

@consideRatio
Creating the token:
Go to https://pypi.org/manage/project/{pypi-repo}/settings/
API Token → Create a token for {pypi-repo}, Name can be anything, Scope: just the repository for now

Either add in Travis web UI:
Go to https://travis-ci.com/{organisation}/{repo}/settings and scroll down to Environment Variables. Add PYPI_TOKEN:

Or use an encrypted secret inside `.travis.yml
https://docs.travis-ci.com/user/environment-variables#defining-encrypted-variables-in-travisyml

There's an ongoing discussion on jupyterhub/team-compass#213 (comment) about pypi accounts and token scopes, but if you're ready to release might as well use your own token and update later?

[consideRatio]

Important insights:

1. Tag builds, which are triggered by a refs/tags/ object in git is pushed, does not belong to a branch. That makes sense, as a tag is a tagged commit, but this can be part of multiple branches or none at all.
2. If we limit an env var to to a certain branch, that means we exclude tagged builds as they are not part of any branch. It is not possible in the TravisCI web settings to declare if they should be part of tagged builds but not branch builds.
3. With travis encrypt we could get a nice entry in TravisCI relating to an encrypted environment variable that would allow us to specifically make the PYPI_PASSWORD environment available when it is needed, on tagged builds. But, we can't use it due to a limiting behavior we experience.

Anyhow, we now have a PYPI_PASSWORD setup, which is making the variable accessible on all pushes to the git repo. It is tied to a deployment token specifically for jupyterhub/oauthenticator. Note that I opted for PYPI_PASSWORD over PYPI_TOKEN as this documentation suggest that: https://docs.travis-ci.com/user/deployment-v2/providers/pypi. You can therefore remove the password: $PYPI_TOKEN fields and instead document that the PYPI_PASSWORD will be set indirectly by the PYPI_PASSWORD environment variable.

It sais that the default distribution is sdist, perhaps it should be bdist_wheel or similar also? I'm super noob about these matters really but I think it is very relevant that we inspect the varioable options for PyPI deployment here: https://docs.travis-ci.com/user/deployment-v2/providers/pypi#known-options

[manics]

@consideRatio Done!

[consideRatio]

@manics ❤️