Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance enhancement of GitTopLevelDir in shell pkg #2668

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

RaphSku
Copy link

@RaphSku RaphSku commented Aug 16, 2023

Description

Fixes #2344.

  • GitTopLevelDir will only run git rev-parse --show-toplevel at the beginning and then re-use the previously found git root path
  • A simple benchmark test is added in order to compare the old code vs. the new code in terms of performance

TODOs

Read the Gruntwork contribution guidelines.

  • Update the docs.
    I didn't update the docs since I am not changing any function signatures or behavior, I am just making the specified code more efficient. If I missed some document where my change chould be described or added to, please tell me.

  • Run the relevant tests successfully, including pre-commit checks.

  • Include release notes. If this PR is backward incompatible, include a migration guide.

Release Notes (draft)

GitTopLevelDir will only run git rev-parse --show-toplevel at the beginning and then re-use the previously found git root path
Added / Removed / Updated [X].

Gist

Gist: https://gist.github.com/RaphSku/8cb9aae84ea374de85d621b6cc6bf8f5
for the test output and benchmark

@RaphSku RaphSku requested a review from denis256 as a code owner August 16, 2023 10:41
@@ -53,3 +53,13 @@ func TestRunShellOutputToStderrAndStdout(t *testing.T) {
assert.True(t, strings.Contains(stderr.String(), "Terraform"), "Output directed to stderr")
assert.True(t, len(stdout.String()) == 0, "No output to stdout")
}

func BenchmarkPerformanceOfGitTopLevelDir(b *testing.B) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, will be helpful to have a functional test that will validate cache usage

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, I'll add it

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I have added a test 👍

@@ -230,23 +230,29 @@ type CmdOutput struct {
Stderr string
}

var gitTopLevelDir string = ""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this approach will work in case of multiple git repositories

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I'll consider this

…onfig for new generic cache + added tests for generic cache as well as for the GitTopLevelDir function that uses the cache
@RaphSku
Copy link
Author

RaphSku commented Aug 25, 2023

@denis256 Okay, so I have added a test for the caching. Previously, the StringCache, etc. were all in the config package. The problem with that is, that I cannot use them in the shell package due to circular import. So I have moved out all of the caches except the TerragruntConfigCache into a separate package.

Also, I have made the cache structs generic with type constraint since the implementation of the StringCache and options.IAMRoleOptions are practically identical.

This approach also supports multiple git repos now. I have also updated the gist with the new test output and new benchmark. The benchmark is almost identical to the previous one.

@RaphSku RaphSku requested a review from denis256 August 25, 2023 10:26
@joaocc
Copy link

joaocc commented Sep 16, 2023

Hi. Any updates on merging this PR? Thanks

@denis256
Copy link
Member

Hi,
should be fixed conflicts on files:

Conflicting files
config/config.go

@RaphSku
Copy link
Author

RaphSku commented Sep 20, 2023

@denis256 I've updated the branch and fixed the conflict

cache/cache.go Outdated Show resolved Hide resolved
shell/run_shell_cmd.go Outdated Show resolved Hide resolved
@denis256
Copy link
Member

This PR still misses integration tests(test directory) which will validate that git rev-parse is executed only once and cached value is returned

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

git show-toplevel should use caching
3 participants