allow cloud run job overrides

[sleithart]

I'm trying to run a cloud run function that processes a GCS folder based on a parameter that's passed in. I'm not able to run the function because Goblet doesn't allow arbitrary arguments passed into the CLI.

Ideally, this would allow someone to capture an args parameter or similar that's passed into the job, but it's easy enough to use something like sys.argv to grab any arguments that are passed in.

Here's the RunJobRequest documentation that specifies allowing overrides: https://cloud.google.com/python/docs/reference/run/latest/google.cloud.run_v2.types.RunJobRequest

[anovis]

@sleithart how are you calling your cloudrun job? Are you using the goblet cli or calling the deployed cloudrun job directly via the run endpoint.

I think you actually should be able to achieve you use case with goblet as is, by overriding environment variables when calling the run endpoint and having your script use those env variables as the parameter to process.

so you can deploy your goblet cloudrun job as is and then use a container override to set your desired environment variable (https://cloud.google.com/run/docs/reference/rest/v2/projects.locations.jobs/run#ContainerOverride) .

let me know if this makes sense and works for you.

separately i can look into how we could override the args parameter and pass that in through goblet.

[sleithart]

I'm using ContainerOverride via the Python client, but using args instead of env vars. I'll try out env vars and see if that works. It seems like a good short term solution, but also feels like a bit of an anti-pattern long-term. If I have time, I may submit a PR for the args change.

Thanks for the help! Love using Goblet so far.

[sleithart]

just for anyone else finding this... using env vars worked well for me and should be a pretty decent holdover.