Problem: archivematica-storage-service does not use Python 3

[sallain]

Please describe the problem you'd like to be solved.
archivematica-storage-service needs to use Python 3 as per the Python 3 ADR.

Describe the solution you'd like to see implemented.
Refactor archivematica-storage-service to use Python 3.

Additional context

Python 2.7 final release in July 2020: here
Ubuntu 20.04 (LTS) and Python 2.7: here
Python3 may be default in CentOS8/RHEL8: here

---

For Artefactual use:
Please make sure these steps are taken before moving this issue from Review to Done:

• All PRs related to this issue are properly linked 👍
• All PRs related to this issue have been merged 👍
• Test plan for this issue has been implemented and passed 👍
• Documentation regarding this issue has been written and it has been added to the release notes, if needed 👍

[ross-spencer]

 

[ross-spencer]

SWORD2 issue logged with the repo maintainers: swordapp/python-client-sword2#16

NB. I left #806 (comment) blank in-case this ticket becomes its own epic so codetree can modify that field as it needs.

[ross-spencer]

From internal discussions we have the following gaps in the Py3 migration of the Storage Service:

• Sword2 client library,
• LOCKSS-o-matic model,
• DSpace model.

As such, for 1.11 we will attempt to cherry-pick and merge the pieces of the Storage Service migration that are compatible with both Python 2 and Python 3, and we will continue to run the Storage Service as Python 2 in 1.11. and try and plug the gaps for 1.12.

Notes on Python 2.7 in 2020:

Python 2.7 final release in July 2020: here
Ubuntu 20.04 (LTS) and Python 2.7: here
Python3 may be default in CentOS8/RHEL8: here

[sallain]

@ross-spencer I don't think this needs any testing, right? SS in Python 3 is not ready to be run in production, we plan to get it finished and test for the next release.

[ross-spencer]

Good question @sallain. Definitely nothing we can do around Python 3 here. Where the changes we've made are across the code-base, I don't think there is anything specific. I think some of the more important aspects will be tested through our regression testing. Check-marks below represent things I tested and know Douglas was thinking about as we worked on this, + things we get for free from Travis):

• Storage service can be deployed and migrations are run: ✔️
• Storage service unit tests run: ✔️
• The range of AMAUAT tests run as anticipated (these test both the UI interaction + API): ✔️
• Transfers and integrations work as anticipated: ✔️ (looking promising)

We'll see pretty quickly (imo) if anything strange has been introduced.

With regards to Github/Kanban mechanics/process, do you want to create a new ticket for that final tranche of storage service work and we can add notes etc. about remaining work? Or keep this open post 1.11?

[sallain]

@ross-spencer Okay, perfect - then from a manual tester's perspective, there's nothing specific to do here.

I think it would be best to open a new ticket, so that we can close this one as part of the release; otherwise we'd need to de-milestone which seems odd since work has been done and merged on this issue.

[ross-spencer]

Closing with note from Slack re: PY3 readiness:

I think given the extensive use through testing, and the fact anything works on any of the servers, they can be signed-off. As working baby-steps to Py3 🐍.

And to answer your question -- Dashboard: unfortunately not because of Gearman. But once the Gearman problem is solved, then theoretically yes. But it's a moving goalpost for now until we commit to making a Py3 virtual environment available (or docker image) in a future release + Gearman 3.

Storage Service - we'd need to fill in those storage service specific gaps first. But it's looking good so far.

[ross-spencer]

NB. Complete Py3 readiness will need to be handled in a separate Issue per #806 (comment)