Extend Iris loading to accept S3 URLs

[JoshuaWiggs]

✨ Feature Request

Add capability to the Iris loading and io modules to utilise a S3 URL to load a data cube into memory.

Proposed used would be something like
cube = iris.load("s3://some-bucket/some-object")

Motivation

This feature would allow us to make more optimal use of our AWS cloud based platforms by removing the need to copy data files from our object store to a mount file system before working with them.

Additional context

Click to expand this section...

This feature is required in order to allow us to reduce remove the duplication of the s3 input and output data. This will allow us to just have the one instance input data in the s3 input bucket being used directly by our science workflows. This will allow the reduction of our fsx storage capacity which is currently our greatest spend on AWS infrastructure.

This could be accomplished by adding a S3 loading method into Iris utilising the boto3 (https://boto3.amazonaws.com/v1/documentation/api/latest/index.html) Python library. Therefore, this would add that library as an optional dependency to use this loading method.

The scope of this feature is limited to loading data from an S3 bucket. Additionally, this feature will assume that you have already correctly configured your environment to access the S3 bucket that is being targeted.