PostgreSQL cannot report an error when it cannot connect.

[shuangkun]

Pre-requisites

• I have double-checked my configuration
• I have tested with the :latest image tag (i.e. quay.io/argoproj/workflow-controller:latest) and can confirm the issue still exists on :latest. If not, I have explained why, in detail, in my description below.
• I have searched existing issues and could not find a match for this bug
• I'd like to contribute the fix myself (see contributing guide)

What happened? What did you expect to happen?

When users sometimes misconfigure PostgreSQL information (perhaps due to network disconnection), errors cannot be reported and workflow-controller will freeze.

time="2025-01-21T07:09:49.701Z" level=info msg="Persistence configuration enabled"
time="2025-01-21T07:09:49.706Z" level=warning msg="Non-transient error: <nil>"
time="2025-01-21T07:09:49.709Z" level=warning msg="Non-transient error: <nil>"

Version(s)

latest

Paste a minimal workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflow that uses private images.

Unreachable PostgreSQL address

Logs from the workflow controller

kubectl logs -n argo deploy/workflow-controller | grep ${workflow}

Logs from in your workflow's wait container

kubectl logs -n argo -c wait -l workflows.argoproj.io/workflow=${workflow},workflow.argoproj.io/phase!=Succeeded