Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Atlantis crashes on atlantis plan after redis configuration #5379

Open
jgatt513 opened this issue Mar 4, 2025 · 3 comments
Open

Atlantis crashes on atlantis plan after redis configuration #5379

jgatt513 opened this issue Mar 4, 2025 · 3 comments
Labels
bug Something isn't working waiting-on-response Waiting for a response from the user

Comments

@jgatt513
Copy link

jgatt513 commented Mar 4, 2025

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Overview of the Issue

After configuring Atlantis Server to use redis as lockingdbtype the container gets created without any issues but on the attempt to run a plan it fails with the following:

runtime error: invalid memory address or nil pointer dereference runtime/panic.go:262 (0x472a98) runtime/signal_unix.go:917 (0x472a68) github.com/runatlantis/atlantis/server/events/command_runner.go:340 (0x11539da) runtime/asm_amd64.s:1700 (0x478960)

Reproduction Steps

  • add redis cluster to your kubernetes cluster and same namespace as atlantis.

  • set atlanits server args
    args:
    - server
    - --log-level={{ .Values.log.level }}
    - --repo-config=/atlantis-config/repos.yaml
    - --redis-host="atlantis-leader-headless.atlantis.svc.cluster.local"
    - --redis-password=""
    - --locking-db-type="redis"
    - --checkout-strategy=merge
    - --atlantis-url=https://{{ get (first .Values.ingress.hosts ) "host" }}
    - --default-tf-version={{ .Values.defaultTFVersion }}
    - --enable-regexp-cmd
    - --hide-unchanged-plan-comments
    - --quiet-policy-checks

  • Once the environment and pods are stable and healthy attempt to run a plan.

Logs

{"level":"info","ts":"2025-03-04T15:37:05.261Z","caller":"events/events_controller.go:692","msg":"Handling 'plan' comment","json":{"repo":"alienadmin/terragrunt","pull":4185}} │
│ {"level":"info","ts":"2025-03-04T15:37:05.261Z","caller":"events/events_controller.go:736","msg":"Running comment command 'plan' for user '712020:1ba2ddde-b573-42df-a1d4-c6a7b1a4d83b'.","json":{"repo" │
│ {"level":"error","ts":"2025-03-04T15:37:05.262Z","caller":"events/command_runner.go:555","msg":"PANIC: runtime error: invalid memory address or nil pointer dereference\nruntime/panic.go:262 (0x472a98)

Environment details

atlantis version v.0.33.0

@jgatt513 jgatt513 added the bug Something isn't working label Mar 4, 2025
Copy link

dosubot bot commented Mar 4, 2025

Hey @jgatt513! I'm here to assist you with any bugs, questions, or contributions you have for Atlantis. Let's tackle this together!

The error you're encountering, "invalid memory address or nil pointer dereference," suggests that there might be an issue with how the Redis client is being initialized or used in the command_runner.go file. Here are a few steps you can take to troubleshoot and potentially resolve the issue:

  1. Check Redis Configuration: Ensure that all required Redis configurations are correctly set. This includes --redis-host, --redis-port, and --redis-password. Even if the password is not required, ensure that the parameter is correctly handled in your setup.

  2. Redis Connection Validation: Verify that the Redis instance is accessible from the Atlantis server. You can do this by manually connecting to the Redis instance using a Redis client from the same environment where Atlantis is running.

  3. TLS and Security Settings: If you are using TLS, ensure that the --redis-tls-enabled and --redis-insecure-skip-verify settings are correctly configured. Misconfiguration here can lead to connection issues.

  4. Atlantis Version: Since you are using Atlantis version v.0.33.0, check if there are any known issues or patches related to Redis integration in this version. Although I couldn't find specific issues in the repository, it's worth checking the release notes or any related issues in the repository.

  5. Code Review: If you have access to the code, review the command_runner.go file around line 340 to see how the Redis client is being used. Look for any potential nil pointer dereferences or uninitialized variables.

  6. Logs and Debugging: Enable detailed logging if possible to get more insights into what might be causing the crash. This can help identify if the issue occurs during the connection setup or during a specific operation with Redis.

By following these steps, you should be able to identify and resolve the issue with the Redis configuration in Atlantis. If the problem persists, consider reviewing the code or configuration for any overlooked details.

To continue talking to Dosu, mention @dosu.


Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@josemrs
Copy link

josemrs commented Mar 4, 2025

Redis is accessible from the same namespace using redis-cli and the same hostname, port and no password as configured in the Atlantis pod

@X-Guardian
Copy link
Contributor

Hi @jgatt513, Atlantis is crashing here, probably because the PullStatusFetcher struct has not been correctly initialised.

status, err := c.PullStatusFetcher.GetPullStatus(pull)

This is assigned here, with the backend value, which is a locking backend type of redis or boltdb.
PullStatusFetcher: backend,

It looks therefore like the backend object is not initialising correctly for you, but not throwing an initialisation error. I tested Atlantis locally with a basic Redis container, and I could not recreate this issue.

To help investigate this further, Can you run your Atlantis container with log-level=debug set and update your issue description with the logs. Please strip out the metrics debug log lines.

@X-Guardian X-Guardian added the waiting-on-response Waiting for a response from the user label Mar 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working waiting-on-response Waiting for a response from the user
Projects
None yet
Development

No branches or pull requests

3 participants