Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Custom Policy Check] Plan steps success despite statut error 1 #4585

Open
tweiss-mdm opened this issue May 21, 2024 · 3 comments · May be fixed by #5156
Open

[Custom Policy Check] Plan steps success despite statut error 1 #4585

tweiss-mdm opened this issue May 21, 2024 · 3 comments · May be fixed by #5156
Labels
bug Something isn't working

Comments

@tweiss-mdm
Copy link

tweiss-mdm commented May 21, 2024

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Overview of the Issue

Weird behavior when activating custom_policy_check on a custom workflow: all the failing plan/apply are in success status.

Before custom_policy_check: true
image

After custom_policy_check: true
image

Environment details

  • Atlantis version: 0.27.3
  • Deployment method: helm
  • Gitlab 16.9

Atlantis server-side config file:

---
environment:
  ATLANTIS_ALLOW_COMMANDS: "plan,apply,unlock,approve_policies,import"
  ATLANTIS_FAIL_ON_PRE_WORKFLOW_HOOK_ERROR: true
  ATLANTIS_ENABLE_POLICY_CHECKS: true
  ATLANTIS_HIDE_UNCHANGED_PLAN_COMMENTS: true
  ATLANTIS_QUIET_POLICY_CHECKS: true
  repoConfig: |
  ---
  repos:
  - id: "/.*/"
    delete_source_branch_on_merge: true
    apply_requirements: [approved, mergeable]
    allowed_overrides: [workflow]
    repo_locking: false
    policy_check: true
    custom_policy_check: true
    workflow: terragrunt
    pre_workflow_hooks:
    - run: |
        echo "generating atlantis.yaml"
        terragrunt-atlantis-config generate --output atlantis.yaml --autoplan --automerge --parallel --create-workspace="true" --workflow="terragrunt" --ignore-dependency-blocks="true" --filter "live/staging" --filter "live/dev" --filter "live/rec" --filter "live/pp" --filter "live/prod";
  metrics:
    prometheus:
      endpoint: /metrics
  workflows:
    terragrunt:
      plan:
        steps:
        - env:
            # Reduce Terraform suggestion output
            name: TF_IN_AUTOMATION
            value: 'true'
        - run: terragrunt hclfmt --terragrunt-check
        - run:
            # Allow for targetted plans/applies as not supported for Terraform wrappers by default
            command: terragrunt plan -input=false $(printf '%s' $COMMENT_ARGS | sed 's/,/ /g' | tr -d '\\') -no-color -out $PLANFILE
            output: hide
        - run: terragrunt show $PLANFILE
      policy_check:
        steps:
          - run:
              command: terragrunt show -json $PLANFILE | awk '/^\{/ {print}' > "plan.json"
              output: hide
          - run: 
              command: |
                cat <<-EOF > ./variables.json
                {
                   "mode": "FULL"
                }
                EOF
              output: hide
          - run: |
              conftest test plan.json variables.json --combine --all-namespaces -p /atlantis-data/opa-rules/policy --quiet --no-color --strict
      apply:
        steps:
        - env:
            name: TF_IN_AUTOMATION
            value: 'true'
        - run: terragrunt apply -input=false $PLANFILE
      import:
        steps:
        - run: terragrunt import -input=false $(printf '%s' $COMMENT_ARGS | sed 's/,/ /' | tr -d '\\')
---
@tweiss-mdm tweiss-mdm added the bug Something isn't working label May 21, 2024
@Webgardener
Copy link

By refactoring our (old) rego code, I managed to use regular policy_check instead of the custom one. But Maybe this bug needs to be addressed nevertheless ?

@aston-r
Copy link

aston-r commented Aug 21, 2024

from doc:

Note: custom policy tool output is simply parsed for "fail" substrings to determine if the policy set passed.

I think it would make more sence to use exit code !=0 condition, but I doubt that it would be implemented due to lack of maintainers(

@bakuljajan
Copy link

Do you know if there is any workaround for this issue?

we also use a custom policy check and when the apply fails, the status shows success. We're running v0.31.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants