Skip to content

Enhance Atmos Configuration with atmos.d Support and Import Functionality#1085

Merged
aknysh merged 92 commits intomainfrom
atmos-import-config
Mar 21, 2025
Merged

Enhance Atmos Configuration with atmos.d Support and Import Functionality#1085
aknysh merged 92 commits intomainfrom
atmos-import-config

Conversation

@haitham911
Copy link
Collaborator

@haitham911 haitham911 commented Feb 21, 2025

what

  • Support the atmos.d convention for atmos.yaml configuration, allowing automatic inclusion of configuration files from the atmos.d directory.
  • Made the path to atmos.d configurable within atmos.yaml, enabling users to specify custom directories for additional configurations.
  • Add support for the import key inside `atmos.yaml, allowing users to define a list of locations (local files, directories using glob patterns, and remote URLs) to import configurations from.
  • process YAML directives !env are correctly processed and stored in Viper. Key changes

why

  • atmos.d convention allows automatic inclusion of configuration files
  • Import key enables pulling configurations from multiple sources (local, remote, and glob patterns)
  • handle Atmos custom functions !env

references

Summary by CodeRabbit

  • New Features

    • Introduced modular configuration imports that allow users to load settings from remote URLs, local files, and recursive directories.
    • Added a new CLI command to run tests and enhanced default configuration settings.
    • Added new configuration sections for logging, Terraform, Helmfile, and stacks.
    • New configuration file created for Atmos CLI with a base import path set to ./.
    • Enhanced CLI commands with detailed configurations for components, stacks, and workflows.
    • New section in the Atmos CLI configuration documentation outlining the functionality of importing other configurations.
  • Refactor

    • Streamlined the configuration loading and merging process for a more efficient and maintainable experience.
    • Enhanced the test suite with a modular, parameterized testing approach for better organization and clarity.
  • Documentation

    • Updated docs with a new "Imports" section detailing functionality and best practices for configuration imports.
  • Tests

    • Expanded test coverage for configuration import scenarios with improved logging and structured test cases.
    • Introduced a comprehensive suite of unit tests for the configuration import functionality.

@haitham911 haitham911 requested review from a team as code owners February 21, 2025 02:45
@mergify mergify bot added the triage Needs triage label Feb 21, 2025
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 21, 2025

📝 Walkthrough

Walkthrough

This pull request introduces extensive changes to the Atmos configuration system. It refactors the configuration loading functions and import handling logic within the package, updates the schema structure to include an import field, and adds new tests and documentation. Additionally, new YAML configuration files and enhanced logging are provided to support CLI configuration imports. Linter configurations have also been updated to recognize the term "import" where appropriate.

Changes

File(s) Change Summary
pkg/config/README.md Added a new "Import Config" section with a Mermaid flowchart illustrating the recursive configuration import process.
pkg/config/config.go Refactored the configuration initialization; removed legacy code; added new functions (processAtmosConfigs, processStackConfigs, atmosConfigAbsolutePaths); removed processConfigFile.
pkg/config/default.go, pkg/config/imports.go, pkg/config/load.go Introduced comprehensive configuration management features: default configuration (defaultCliConfig, NotFound, mergeDefaultConfig), robust import processing (including local/remote detection and recursive handling), and a complete configuration loader (LoadConfig and associated helpers).
pkg/schema/schema.go Added a new field Import []string to the AtmosConfiguration struct to support configuration imports.
pkg/config/import_test.go Added unit tests for import processing including nested and remote import scenarios and a helper function setupTestFile.
tests/cli_test.go Enhanced sanitizeOutput with new regex rules to normalize URLs and standardize atmos-import paths.
tests/fixtures/scenarios/atmos-cli-imports/* Introduced new YAML configuration files defining CLI import scenarios, covering aspects like imports, stacks, logs, vendor settings, and Terraform configurations.
tests/fixtures/scenarios/atmos-configuration/* Added new YAML configuration files for extended Atmos configuration covering commands, logs, Helmfile, stacks, and Terraform settings.
tests/snapshots/* Updated snapshot files with detailed debug logging for configuration file retrieval failures and successful merges, including new "import" keys and platform identifier updates.
tests/test-cases/atmos-cli-imports.yaml Added new test cases for verifying proper CLI configuration import functionality with expected exit codes and snapshot validations.
.golangci.yml Updated linter settings: added "import" to the allowed constants in the revive.add-constant rule and changed the exclusion list for revive.comment-spacings.
website/docs/cli/configuration/configuration.mdx Added a new "Imports" section in the documentation explaining how to modularize configurations using imports with remote URLs, specific paths, and wildcard globs.

Possibly related PRs


📜 Recent review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7aa222f and 1c4706e.

📒 Files selected for processing (3)
  • pkg/config/config_test.go (1 hunks)
  • pkg/config/import_test.go (1 hunks)
  • tests/cli_terraform_test.go (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • pkg/config/import_test.go
🧰 Additional context used
🧬 Code Definitions (1)
pkg/config/config_test.go (1)
pkg/config/config.go (1) (1)
  • InitCliConfig (25-62)
⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: Build (windows-latest, windows)
  • GitHub Check: Summary
🔇 Additional comments (9)
tests/cli_terraform_test.go (1)

16-19: Environment cleanup enhances test reliability

Adding code to unset environment variables ensures the test runs in a clean, predictable environment. This is a solid testing practice that prevents environment variables from affecting test behavior.

pkg/config/config_test.go (8)

16-21: Clean environment setup is appropriate

Unsetting environment variables at the beginning ensures tests run in a controlled environment. Good testing practice.


45-53: Well-structured test case definition

The testCase struct provides a clean, reusable pattern for parameterized testing. This improves test maintainability and readability.


55-132: Comprehensive basic configuration tests

These test cases thoroughly validate core functionality with different file extensions and fallbacks. The validation logic is thorough and checks all important aspects of configuration loading.


147-162: Environment variable interpolation test is well-implemented

This test correctly validates that environment variables can be used in configuration files, with proper setup and cleanup.


164-177: Good test coverage for atmos.d convention

This test case verifies the new atmos.d directory convention support, addressing a key objective from the PR. The assertions properly validate the expected behavior.


178-192: Custom import functionality verified

This test ensures the new import functionality works correctly with custom paths, which is another key PR objective. The verification of the debug log level confirms the imports are working properly.


195-219: Well-structured test execution

The test runner properly sets up temporary directories, handles environment variables, and executes test-specific setup. Using t.Run() for subtests improves output organization and test isolation.


223-240: Useful helper functions

These helper functions encapsulate common setup tasks and include proper cleanup through t.Cleanup(). Using require.NoError ensures tests fail fast on setup issues.

✨ Finishing Touches
  • 📝 Generate Docstrings

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary or @auto-summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai or @auto-title anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (14)
tests/fixtures/scenarios/atmos-cli-imports/configs.d/commands.yaml (2)

6-6: Remove Trailing Spaces in Command Step.
There are trailing spaces at the end of this line, which may lead to YAML lint warnings.

Suggested diff:

-      - atmos describe config  
+      - atmos describe config
🧰 Tools
🪛 YAMLlint (1.35.1)

[error] 6-6: trailing spaces

(trailing-spaces)


7-7: Remove Excess Blank Line.
There is an extra blank line at the end of the file. Removing it will improve clarity.

🧰 Tools
🪛 YAMLlint (1.35.1)

[warning] 7-7: too many blank lines

(1 > 0) (empty-lines)

tests/fixtures/scenarios/atmos-configuration/atmos.d/commands.yaml (1)

3-4: Ensure Consistency in Command Descriptions.
Notice that the description here ("Run all tests with custom command") slightly differs from the one in the other configuration file ("Run all tests"). For a consistent CLI experience, consider aligning these descriptions if they are intended to perform the same action.

pkg/config/load.go (4)

20-26: Clear documentation block.

Your docstring is thorough. Consider adding a short usage snippet to guide new users.

🧰 Tools
🪛 GitHub Check: golangci

[failure] 26-26: [golangci] pkg/config/load.go#L26
cyclomatic: function LoadConfig has cyclomatic complexity 13 (> max enabled 10) (revive)


210-210: Consolidate repeated string literals.

You're logging "error" and "path" multiple times across the codebase. Declaring them as constants could help reduce typos and improve consistency.

- log.Debug("error process imports", "path", path, "error", err)
+ const (
+   logKeyError = "error"
+   logKeyPath  = "path"
+ )
+ ...
+ log.Debug("error process imports", logKeyPath, path, logKeyError, err)

226-226: Use wrapped static errors to provide richer context.

Instead of fmt.Errorf("atmos config directory not found path %s", dirPath), consider:

- return fmt.Errorf("atmos config directory not found path %s", dirPath)
+ return fmt.Errorf("atmos config directory not found path %s: %w", dirPath, ErrAtmosDirNotFound)
🧰 Tools
🪛 GitHub Check: golangci

[failure] 226-226: [golangci] pkg/config/load.go#L226
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("atmos config directory not found path %s", dirPath)" (err113)


230-230: Avoid wildcard paths inside filepath.Join.

Using "atmos.d/**/*" within filepath.Join can be brittle across platforms. You may directly concatenate or rely on glob utilities.

🧰 Tools
🪛 GitHub Check: golangci

[failure] 230-230: [golangci] pkg/config/load.go#L230
filepathJoin: "atmos.d/**/*" contains a path separator (gocritic)

pkg/config/imports.go (4)

36-36: Consider passing source by pointer.

source schema.AtmosConfiguration may be large, so passing it as a pointer could reduce copying costs.

🧰 Tools
🪛 GitHub Check: golangci

[failure] 36-36: [golangci] pkg/config/imports.go#L36
hugeParam: source is heavy (5776 bytes); consider passing it by pointer (gocritic)


68-68: Prefer wrapped static errors over dynamic ones.

Replace these fmt.Errorf(...) calls with well-defined error constants, wrapping them for context. This approach improves error handling consistency.

Also applies to: 71-71, 74-74, 78-78, 117-117, 129-129, 149-149, 160-160, 174-174, 283-283, 373-373

🧰 Tools
🪛 GitHub Check: golangci

[failure] 68-68: [golangci] pkg/config/imports.go#L68
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("base_Path required to process imports")" (err113)


145-145: Simplify slice length check.

Since len(Imports) is zero for nil slices, you can omit the nil check and rely on len(Imports) > 0.

🧰 Tools
🪛 golangci-lint (1.62.2)

145-145: S1009: should omit nil check; len() for []string is defined as zero

(gosimple)


250-251: Review usage of "/.yaml" and "**/.yml" in filepath.Join.**

Inline wildcards in filepath.Join can be error-prone. Using direct string concatenation or specialized file matching functions is often safer across environments.

🧰 Tools
🪛 GitHub Check: golangci

[failure] 250-250: [golangci] pkg/config/imports.go#L250
filepathJoin: "**/*.yaml" contains a path separator (gocritic)


[failure] 251-251: [golangci] pkg/config/imports.go#L251
filepathJoin: "**/*.yml" contains a path separator (gocritic)

pkg/config/import_test.go (1)

114-123: Consider refactoring HTTP handler to use switch statement.

The current if-else chain could be more readable using a switch statement.

Apply this diff to improve readability:

-		if r.URL.Path == "/config.yaml" {
-			fmt.Fprint(w, remoteContent)
-		} else if r.URL.Path == "/nested-remote.yaml" {
-			fmt.Fprint(w, nestedRemoteContent)
-		} else {
-			http.NotFound(w, r)
-		}
+		switch r.URL.Path {
+		case "/config.yaml":
+			fmt.Fprint(w, remoteContent)
+		case "/nested-remote.yaml":
+			fmt.Fprint(w, nestedRemoteContent)
+		default:
+			http.NotFound(w, r)
+		}
🧰 Tools
🪛 GitHub Check: golangci

[failure] 115-115: [golangci] pkg/config/import_test.go#L115
ifElseChain: rewrite if-else to switch statement (gocritic)

pkg/config/utils.go (1)

1-801: Consider splitting this file into smaller, focused files.

The file is 667 lines long (exceeds 500-line limit). Consider breaking it down into smaller files based on functionality:

  • Configuration file operations (search, merge)
  • Environment variable processing
  • Context and stack name handling
🧰 Tools
🪛 GitHub Check: golangci

[failure] 800-800: [golangci] pkg/config/utils.go#L800
file-length-limit: file length is 667 lines, which exceeds the limit of 500 (revive)

tests/fixtures/scenarios/atmos-configuration/atmos.yaml (1)

1-6: Minor formatting adjustment needed.

The configuration file is straightforward and meets its purpose of defining the default import path. However, static analysis has flagged trailing spaces on line 5. Please remove these trailing spaces to adhere to YAML best practices.

Proposed diff to remove trailing spaces on line 5:

🧰 Tools
🪛 YAMLlint (1.35.1)

[error] 5-5: trailing spaces

(trailing-spaces)

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 141c731 and eae3f97.

📒 Files selected for processing (29)
  • pkg/config/README.md (1 hunks)
  • pkg/config/config.go (1 hunks)
  • pkg/config/default.go (1 hunks)
  • pkg/config/import_test.go (1 hunks)
  • pkg/config/imports.go (1 hunks)
  • pkg/config/load.go (1 hunks)
  • pkg/config/utils.go (3 hunks)
  • pkg/schema/schema.go (1 hunks)
  • tests/cli_test.go (1 hunks)
  • tests/fixtures/scenarios/atmos-cli-imports/atmos.yaml (1 hunks)
  • tests/fixtures/scenarios/atmos-cli-imports/configs.d/commands.yaml (1 hunks)
  • tests/fixtures/scenarios/atmos-cli-imports/configs.d/tools/stack.yml (1 hunks)
  • tests/fixtures/scenarios/atmos-cli-imports/configs.d/tools/terraform.yaml (1 hunks)
  • tests/fixtures/scenarios/atmos-cli-imports/configs.d/vendor.yaml (1 hunks)
  • tests/fixtures/scenarios/atmos-cli-imports/logs.yaml (1 hunks)
  • tests/fixtures/scenarios/atmos-configuration/atmos.d/commands.yaml (1 hunks)
  • tests/fixtures/scenarios/atmos-configuration/atmos.d/logs.yaml (1 hunks)
  • tests/fixtures/scenarios/atmos-configuration/atmos.d/tools/helmfile.yml (1 hunks)
  • tests/fixtures/scenarios/atmos-configuration/atmos.d/tools/stack.yaml (1 hunks)
  • tests/fixtures/scenarios/atmos-configuration/atmos.d/tools/terraform.yaml (1 hunks)
  • tests/fixtures/scenarios/atmos-configuration/atmos.yaml (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_describe_config.stdout.golden (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_describe_config_-f_yaml.stdout.golden (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_describe_config_imports.stderr.golden (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_describe_config_imports.stdout.golden (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_describe_configuration.stderr.golden (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_describe_configuration.stdout.golden (1 hunks)
  • tests/test-cases/atmos-cli-imports.yaml (1 hunks)
  • tests/test-cases/env.yaml (1 hunks)
✅ Files skipped from review due to trivial changes (3)
  • tests/fixtures/scenarios/atmos-configuration/atmos.d/logs.yaml
  • tests/fixtures/scenarios/atmos-cli-imports/configs.d/vendor.yaml
  • pkg/config/README.md
🧰 Additional context used
🧠 Learnings (2)
tests/fixtures/scenarios/atmos-cli-imports/atmos.yaml (1)
Learnt from: osterman
PR: cloudposse/atmos#808
File: examples/demo-atmos-cli-imports/atmos.yaml:8-8
Timestamp: 2025-01-25T15:21:40.413Z
Learning: In Atmos, when a directory is specified for configuration loading (e.g., in the `import` section of atmos.yaml), all files within that directory should be treated as Atmos configurations. Do not suggest restricting file extensions in directory-based glob patterns.
tests/fixtures/scenarios/atmos-configuration/atmos.d/tools/helmfile.yml (1)
Learnt from: osterman
PR: cloudposse/atmos#808
File: examples/demo-atmos.d/atmos.d/tools/helmfile.yml:10-10
Timestamp: 2024-12-12T15:17:45.245Z
Learning: In `examples/demo-atmos.d/atmos.d/tools/helmfile.yml`, when suggesting changes to `kubeconfig_path`, ensure that the values use valid Go template syntax.
🪛 YAMLlint (1.35.1)
tests/fixtures/scenarios/atmos-cli-imports/configs.d/commands.yaml

[error] 6-6: trailing spaces

(trailing-spaces)


[warning] 7-7: too many blank lines

(1 > 0) (empty-lines)

tests/fixtures/scenarios/atmos-configuration/atmos.yaml

[error] 5-5: trailing spaces

(trailing-spaces)

🪛 GitHub Check: golangci
pkg/config/utils.go

[failure] 800-800: [golangci] pkg/config/utils.go#L800
file-length-limit: file length is 667 lines, which exceeds the limit of 500 (revive)

pkg/config/import_test.go

[failure] 16-16: [golangci] pkg/config/import_test.go#L16
G306: Expect WriteFile permissions to be 0600 or less (gosec)


[failure] 38-38: [golangci] pkg/config/import_test.go#L38
G306: Expect WriteFile permissions to be 0600 or less (gosec)


[failure] 42-42: [golangci] pkg/config/import_test.go#L42
G306: Expect WriteFile permissions to be 0600 or less (gosec)


[failure] 47-47: [golangci] pkg/config/import_test.go#L47
G306: Expect WriteFile permissions to be 0600 or less (gosec)


[failure] 54-54: [golangci] pkg/config/import_test.go#L54
G306: Expect WriteFile permissions to be 0600 or less (gosec)


[failure] 115-115: [golangci] pkg/config/import_test.go#L115
ifElseChain: rewrite if-else to switch statement (gocritic)

pkg/config/load.go

[failure] 242-242: [golangci] pkg/config/load.go#L242
add-constant: string literal "error" appears, at least, 4 times, create a named constant for it (revive)


[failure] 250-250: [golangci] pkg/config/load.go#L250
add-constant: string literal "path" appears, at least, 4 times, create a named constant for it (revive)


[failure] 26-26: [golangci] pkg/config/load.go#L26
cyclomatic: function LoadConfig has cyclomatic complexity 13 (> max enabled 10) (revive)


[failure] 226-226: [golangci] pkg/config/load.go#L226
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("atmos config directory not found path %s", dirPath)" (err113)


[failure] 230-230: [golangci] pkg/config/load.go#L230
filepathJoin: "atmos.d/**/*" contains a path separator (gocritic)


[failure] 239-239: [golangci] pkg/config/load.go#L239
filepathJoin: ".atmos.d/**/*" contains a path separator (gocritic)

pkg/config/imports.go

[failure] 128-128: [golangci] pkg/config/imports.go#L128
add-constant: string literal "error" appears, at least, 4 times, create a named constant for it (revive)


[failure] 148-148: [golangci] pkg/config/imports.go#L148
add-constant: string literal "import" appears, at least, 4 times, create a named constant for it (revive)


[failure] 173-173: [golangci] pkg/config/imports.go#L173
add-constant: string literal "path" appears, at least, 4 times, create a named constant for it (revive)


[failure] 68-68: [golangci] pkg/config/imports.go#L68
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("base_Path required to process imports")" (err113)


[failure] 71-71: [golangci] pkg/config/imports.go#L71
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("tempDir required to process imports")" (err113)


[failure] 74-74: [golangci] pkg/config/imports.go#L74
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("maximum import depth of %d exceeded", maxDepth)" (err113)


[failure] 78-78: [golangci] pkg/config/imports.go#L78
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("failed to resolve base path: %v", err)" (err113)


[failure] 117-117: [golangci] pkg/config/imports.go#L117
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("unsupported URL '%s': %v", importPath, err)" (err113)


[failure] 129-129: [golangci] pkg/config/imports.go#L129
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("failed to read remote config")" (err113)


[failure] 149-149: [golangci] pkg/config/imports.go#L149
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("failed to process nested imports")" (err113)


[failure] 160-160: [golangci] pkg/config/imports.go#L160
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("import_path required to process imports")" (err113)


[failure] 174-174: [golangci] pkg/config/imports.go#L174
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("failed to resolve local import path")" (err113)


[failure] 283-283: [golangci] pkg/config/imports.go#L283
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("no valid absolute paths found")" (err113)


[failure] 373-373: [golangci] pkg/config/imports.go#L373
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("no files matching patterns found")" (err113)


[failure] 250-250: [golangci] pkg/config/imports.go#L250
filepathJoin: "**/*.yaml" contains a path separator (gocritic)


[failure] 251-251: [golangci] pkg/config/imports.go#L251
filepathJoin: "**/*.yml" contains a path separator (gocritic)


[failure] 36-36: [golangci] pkg/config/imports.go#L36
hugeParam: source is heavy (5776 bytes); consider passing it by pointer (gocritic)

🪛 golangci-lint (1.62.2)
pkg/config/imports.go

145-145: S1009: should omit nil check; len() for []string is defined as zero

(gosimple)

⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: Acceptance Tests (windows-latest, windows)
  • GitHub Check: Summary
🔇 Additional comments (25)
tests/fixtures/scenarios/atmos-cli-imports/configs.d/commands.yaml (1)

1-7: Overall YAML structure looks good.
The file correctly defines the custom CLI command “test” with its description and steps.

🧰 Tools
🪛 YAMLlint (1.35.1)

[error] 6-6: trailing spaces

(trailing-spaces)


[warning] 7-7: too many blank lines

(1 > 0) (empty-lines)

tests/fixtures/scenarios/atmos-configuration/atmos.d/commands.yaml (1)

1-7: YAML structure and content are set up correctly.
This file defines the “test” command with the expected key-value pairs.

pkg/config/config.go (1)

18-18: Excellent simplification with new LoadConfig usage.

This single call effectively removes extensive boilerplate, streamlining the flow.

tests/snapshots/TestCLICommands_atmos_describe_config_-f_yaml.stdout.golden (1)

46-47: LGTM! The changes align with the PR objectives.

The update to cli_config_path and addition of the import array prepare the configuration for the new import functionality.

tests/snapshots/TestCLICommands_atmos_describe_configuration.stdout.golden (1)

59-60: LGTM! The configuration structure is comprehensive and well-organized.

The configuration includes all necessary sections and maintains consistency with the import functionality.

pkg/config/default.go (2)

13-15: LGTM! The error message is user-friendly and actionable.

The error message clearly explains the issue and provides a solution by referencing the sample configuration.


93-97:

Details

❓ Verification inconclusive

Verify the version check configuration values.

Please confirm if these default values are appropriate:

  • Timeout: 1000 (units not specified)
  • Frequency: "daily"

🌐 Web query:

What are the recommended timeout and frequency values for version checks in Go applications?

💡 Result:

When implementing version checks in Go applications, recommended practices for timeouts and frequency depend on the specific use case and network conditions. Key considerations from industry practices:

Timeout Recommendations

  1. HTTP-based version checks:

    • Default timeout: 5 seconds [6]
    • Adaptive policy example:
      policy := Adaptive(
          200*time.Millisecond, // Initial attempt
          time.Second,          // First retry after timeout
          10*time.Second        // Subsequent retries
      )
    • Maximum single attempt: 30 seconds (common web service limit)[7]
  2. Critical path operations:

    • Database transactions: 15-30 seconds[4]
    • Script execution: Set via RUNNER_SCRIPT_TIMEOUT with value less than job timeout[1]

Frequency Recommendations

  1. Automated dependency checks:

    • Daily scans for security updates[8]
    • Align with release cycles (Go releases every 6 months)[5]
  2. Application self-checks:

    • Production: Every 5-15 minutes for critical services[4]
    • Development/Staging: Every 24 hours[6]

Implementation Patterns

// Example using context with timeout
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()

req, _ := http.NewRequestWithContext(ctx, "GET", versionURL, nil)
resp, err := http.DefaultClient.Do(req)

Error Handling Considerations

  • Use error wrapping to detect timeout causes:
    if errors.Is(err, context.DeadlineExceeded) {
        // Handle timeout specifically
    }
  • Implement transaction rollback safeguards for timeout-induced state inconsistencies[4]

Testing Recommendations

  • Validate with fault injection patterns:
    // Test-specific slow response simulation
    if testingMode {
        time.Sleep(20 * time.Second) 
    }
  • Verify timeout cascade effects through dependency chain[1][4]

These values should be adjusted based on observed latency percentiles and business requirements. For public APIs/CLIs checking upstream versions, a balance between freshness (15-30 minute intervals) and reliability (retry budgets) is typically optimal[6][7].

Citations:


Review Update: Confirm Default Timeout and Frequency Configuration

The default configuration in pkg/config/default.go (lines 93‑97):

			Check: schema.VersionCheck{
				Enabled:   true,
				Timeout:   1000,
				Frequency: "daily",
			},

has raised two points for verification:

  • Timeout:
    Current value is set to 1000. If this represents milliseconds, it implies a 1‑second timeout. However, best practices for HTTP-based version checks generally recommend a default timeout closer to 5 seconds (5000 ms). Please verify whether the unit is indeed milliseconds and, if so, consider if a higher default would yield more reliable checks.

  • Frequency:
    The default of "daily" is acceptable for automated dependency checks. Nonetheless, depending on the operational context (e.g., production vs. development), more frequent checks (such as every 5–15 minutes) might be advisable. Please assess if "daily" aligns with the intended use case.

Thanks for reviewing these configuration values to ensure they meet the required operational criteria.

tests/snapshots/TestCLICommands_atmos_describe_configuration.stderr.golden (1)

1-18: LGTM! The debug logs demonstrate proper configuration loading behavior.

The logs show:

  • Expected handling of missing .atmos.d files
  • Successful merging of configuration files
  • Appropriate debug level information
tests/snapshots/TestCLICommands_atmos_describe_config.stdout.golden (1)

148-149: LGTM! Configuration structure aligns with the new import functionality.

The changes correctly reflect the new import capability by adding the import field and updating the cli_config_path to explicitly reference the YAML file.

tests/snapshots/TestCLICommands_atmos_describe_config_imports.stdout.golden (1)

162-165:

Details

✅ Verification successful

Verify the remote import URL.

The import configuration includes both remote and local paths, which is good. However, let's verify the remote URL:


🏁 Script executed:

#!/bin/bash
# Description: Verify the remote URL is accessible
curl -I https://raw.githubusercontent.com/cloudposse/atmos/refs/heads/main/atmos.yaml

Length of output: 1279


Remote URL Verified: Import Configuration is Good

The remote URL has been confirmed accessible (HTTP/2 200), and the import configuration correctly includes both remote and local paths. No further changes are needed.

pkg/config/import_test.go (2)

20-91: LGTM! Comprehensive test coverage for import processing.

The test cases effectively cover various scenarios including:

  • Remote URL imports
  • Recursive directory imports
  • Local file imports
  • Invalid paths
  • Error handling
🧰 Tools
🪛 GitHub Check: golangci

[failure] 38-38: [golangci] pkg/config/import_test.go#L38
G306: Expect WriteFile permissions to be 0600 or less (gosec)


[failure] 42-42: [golangci] pkg/config/import_test.go#L42
G306: Expect WriteFile permissions to be 0600 or less (gosec)


[failure] 47-47: [golangci] pkg/config/import_test.go#L47
G306: Expect WriteFile permissions to be 0600 or less (gosec)


[failure] 54-54: [golangci] pkg/config/import_test.go#L54
G306: Expect WriteFile permissions to be 0600 or less (gosec)


93-161: LGTM! Well-structured nested import tests.

The test cases thoroughly validate:

  • Local nested imports
  • Remote nested imports
  • Depth limit handling
  • Mixed import scenarios
🧰 Tools
🪛 GitHub Check: golangci

[failure] 115-115: [golangci] pkg/config/import_test.go#L115
ifElseChain: rewrite if-else to switch statement (gocritic)

tests/snapshots/TestCLICommands_atmos_describe_config_imports.stderr.golden (1)

1-26: LGTM! Comprehensive debug logging for import processing.

The debug logs effectively capture:

  • Failed glob matches with clear error messages
  • Successful configuration merges
  • Processing of store configurations
pkg/config/utils.go (1)

785-800:

Details

✅ Verification successful

LGTM! Clean implementation of config file merging.

The function follows Go best practices with proper error handling for both file reading and config merging operations.

Let's verify the error handling by checking if the function is used correctly in the codebase:


🏁 Script executed:

#!/bin/bash
# Search for MergeConfigFile usage to ensure errors are properly handled
rg -A 3 "MergeConfigFile\(" 

Length of output: 701


Excellent work with the MergeConfigFile function!

The implementation in pkg/config/utils.go elegantly handles file reading and configuration merging with proper error propagation. Our verification of its usage in both pkg/config/imports.go and pkg/config/load.go confirms that errors are checked and logged, ensuring robust handling.

🧰 Tools
🪛 GitHub Check: golangci

[failure] 800-800: [golangci] pkg/config/utils.go#L800
file-length-limit: file length is 667 lines, which exceeds the limit of 500 (revive)

tests/cli_test.go (1)

314-317: LGTM! Good addition to test output sanitization.

The new regex pattern effectively handles dynamic import file paths, ensuring consistent test output by standardizing paths containing random numbers.

pkg/schema/schema.go (1)

41-41: LGTM! Well-structured addition of Import field.

The new field follows the struct's tag conventions and uses an appropriate type for storing multiple import paths.

tests/fixtures/scenarios/atmos-cli-imports/logs.yaml (1)

1-4: Verify if Debug level is intended for this configuration.

While the configuration is well-structured, Debug level logging can be verbose and impact performance. Please confirm if this level is specifically needed for these test fixtures.

tests/fixtures/scenarios/atmos-configuration/atmos.d/tools/stack.yaml (1)

1-8: Clear and Consistent "stacks" Configuration

This new YAML configuration clearly defines the stacks section with all essential keys:

  • base_path is set appropriately,
  • included_paths and excluded_paths use glob patterns effectively,
  • name_pattern is defined to be "{dev}" for dynamic naming.

Everything is consistent and follows the intended structure for the PR’s configuration enhancements.

tests/fixtures/scenarios/atmos-cli-imports/configs.d/tools/stack.yml (1)

1-8: Mirrored "stacks" Configuration for Consistency

This file mirrors the configuration seen in atmos-configuration/atmos.d/tools/stack.yaml. Reusing the same structure across different scenarios enhances consistency.

tests/fixtures/scenarios/atmos-cli-imports/configs.d/tools/terraform.yaml (1)

1-8: Well-Defined Terraform Component Settings

The new Terraform configuration section under components.terraform is clear:

  • The base_path is set to "components/terraform",
  • Boolean flags for auto-approval, initialization running, reconfiguration, and backend file generation are appropriately defined.

These settings provide a structured way to manage Terraform components as per the PR objectives.

tests/test-cases/env.yaml (1)

17-19: Updated Expected Regex for atmos_cli_config_path

The expected output for atmos_cli_config_path now correctly matches a pattern ending with "atmos.yaml". This change reflects the updated configuration filename requirements and aligns with the overall PR objectives for enhanced configuration clarity.

tests/fixtures/scenarios/atmos-cli-imports/atmos.yaml (1)

1-10: Robust Import Configuration File

This new atmos.yaml file demonstrates the new import functionality by:

  • Including comments that concisely explain the purpose and behavior,
  • Defining a base_path and an import list with diverse sources: remote URL, a glob pattern for directory-based imports, and a specific file.

Notably, the glob pattern ("configs.d/**/*") does not restrict file extensions, which aligns with the guidance from long-term learnings. The file is clear, well documented, and meets the PR objectives for import functionality.

tests/test-cases/atmos-cli-imports.yaml (1)

1-29: Clear and well-structured test cases for configuration imports.

The test cases effectively validate the new import configuration functionality, ensuring that both import-specific and general configuration paths are exercised. The descriptions and snapshot settings are concise and informative.

tests/fixtures/scenarios/atmos-configuration/atmos.d/tools/helmfile.yml (1)

1-15: Well-documented Helmfile configuration.

The configuration section clearly explains each parameter with alternative methods for overriding via environment variables or command-line arguments. One point to verify: please ensure that the kubeconfig_path value ("/dev/shm") complies with valid Go template syntax if the implementation requires it.

tests/fixtures/scenarios/atmos-configuration/atmos.d/tools/terraform.yaml (1)

1-24: Comprehensive Terraform configuration section.

All configuration settings for Terraform, including defaults and optional overrides, are clearly laid out with supporting comments. Confirm that the environment variable names and default values match the conventions used elsewhere in the project.

coderabbitai[bot]
coderabbitai bot previously approved these changes Feb 21, 2025
@mergify mergify bot removed the triage Needs triage label Feb 21, 2025
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
pkg/config/config_test.go (2)

157-185: Add test cases for error handling with imports

The current tests validate successful import scenarios, but there are no tests for error cases such as invalid import paths or malformed imported configurations.

Consider adding test cases for error handling scenarios like:

  • Non-existent import path
  • Invalid YAML in imported file
  • Circular imports

218-219: Add error message to WriteFile

The error message from os.WriteFile could be enhanced to include the filename being written.

-	err := os.WriteFile(path, []byte(content), 0o644)
-	require.NoError(t, err, "Failed to create config file")
+	err := os.WriteFile(path, []byte(content), 0o644)
+	require.NoError(t, err, "Failed to create config file %s", path)
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 773e812 and 7aa222f.

📒 Files selected for processing (2)
  • pkg/config/config_test.go (1 hunks)
  • pkg/schema/schema.go (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • pkg/schema/schema.go
🧰 Additional context used
🧬 Code Definitions (1)
pkg/config/config_test.go (1)
pkg/config/config.go (1) (1)
  • InitCliConfig (25-62)
⏰ Context from checks skipped due to timeout of 90000ms (5)
  • GitHub Check: Acceptance Tests (macos-latest, macos)
  • GitHub Check: Acceptance Tests (windows-latest, windows)
  • GitHub Check: Acceptance Tests (ubuntu-latest, linux)
  • GitHub Check: [localstack] demo-localstack
  • GitHub Check: Summary
🔇 Additional comments (12)
pkg/config/config_test.go (12)

10-10: Good addition of the require package

Adding the require package from testify is a good practice. It provides better error messages and stops test execution on failure, making tests more reliable and easier to debug.


16-37: Well-structured configuration setup

Moving the configuration content to a variable at the top of the test function improves readability and maintainability. The YAML structure is clear and includes all necessary configuration sections.


38-46: Good use of test structure with testCase

The testCase struct provides a clean way to organize parameterized tests. This approach makes it easier to add new test cases and maintain existing ones.


48-72: YAML extension test looks good

The test case for the YAML extension is well-structured with clear setup and assertions. It properly validates all the important configuration properties.


73-92: YML extension test adds good coverage

Testing both YAML and YML extensions is thorough and ensures the system works with both common extension formats.


93-110: Good fallback test

The test for invalid config file names properly verifies that the system falls back to default configuration values when an invalid file is provided.


120-120: Use filepath.Join consistently

Good use of filepath.Join to create platform-independent paths. This ensures tests run correctly on all operating systems.


156-170: Good test for atmos.d support

This test validates the new atmos.d convention support mentioned in the PR objectives. It confirms that configuration files in the .atmos.d directory are correctly loaded.


171-185: Good test for import functionality

This test validates the new import functionality mentioned in the PR objectives. It confirms that configuration files can be imported from custom locations.


188-212: Well-structured test execution loop

The test execution loop is clean and handles all aspects of test setup and teardown. The use of t.Run for subtests provides better test output and organization.


215-220: Helpful utility function

The createConfigFile function is a good abstraction that makes the test code cleaner and more maintainable.


222-233: Good directory management

The changeWorkingDir function properly changes and restores the working directory, using t.Cleanup to ensure teardown happens even if tests fail.

coderabbitai[bot]
coderabbitai bot previously approved these changes Mar 21, 2025
Copy link
Member

@aknysh aknysh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @haitham911

@aknysh aknysh added no-release Do not create a new release (wait for additional code changes) and removed minor New features that do not break anything labels Mar 21, 2025
@aknysh aknysh merged commit 5b58700 into main Mar 21, 2025
69 checks passed
@aknysh aknysh deleted the atmos-import-config branch March 21, 2025 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-release Do not create a new release (wait for additional code changes)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants