Skip to content

Sensitive Information Leaked in AWX by automation job pod #16141

@fluzzykitten

Description

@fluzzykitten

Please confirm the following

  • I agree to follow this project's code of conduct.
  • I have checked the current issues for duplicates.
  • I understand that AWX is open source software provided for free and that I might not receive a timely response.

Feature type

New Feature

Feature Summary

We are deploying AWX using the AWX-operator in Kubernetes. New AWX jobs spin up automation-job pods. Those pods have logs to stdout during execution that aren’t rendered in the UI, but are seen at runtime on the pod stdout. In a K8s platform environment, it’s common to send all stdout pod logs for all pods in the cluster to a central log aggregator.

The initial "starting" stdout log includes the ENV passed to the job. Some modules in Ansible require sensitive information like passwords to be passed via ENV, so logging the ENV to stdout leaks this information, which is then sent off to the central log aggregator.

Because the kubernetes logging solutions just captures all pod stdout logs and ships them off, we end up with sensitive information in the logs that are stored with all the other pods. There doesn’t appear to be any way to prevent the automation job from logging the env at runtime. Is there a way to prevent this? If not, could we request a way to reduce logging or turn off this log?

Example startup log to stdout:

{
    "status": "starting", 
    "runner_ident": "288886", 
    "command": ["ansible-playbook", "-u", "root", "-e", "@/runner/env/tmp5lo54lwn", "-i", "/runner/inventory/hosts", "-e", "@/runner/env/extravars", "main.yml"], 
    "env": {"KUBERNETES_SERVICE_PORT_HTTPS": "443", "SUPER_SECRET_PASSWORD": "FluffyAndAdorableKittens", "KUBERNETES_SERVICE_PORT": "443", "AWX_PROD_SERVICE_PORT_80_TCP": "tcp://10.43.165.29:80", 
...

Select the relevant components

  • UI
  • API
  • Docs
  • Collection
  • CLI
  • Other

Steps to reproduce

Run any job with a sensitive credential passed through the ENV

Current results

sensitive credential is logged to pod stdout, which kubernetes will capture along with all stdout logs for all pods in the cluster.

Sugested feature result

Ability to turn off logging sensitive information at job start, so that it's not printed to stdout

Additional information

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions