New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
git
module: Failed to init/update submodules: warning: could not look up configuration 'remote.origin.url'
#83146
Comments
Files identified in the description: If these files are incorrect, please update the |
It looks like the initial The .git/config for the main repo looks like this before the failing call: $ cat /tmp/submod-relpath-test-mainrepo/.git/config
[core]
repositoryformatversion = 0
filemode = true
bare = false
logallrefupdates = true
[remote "github"]
url = https://github.com/wtcline-intc/submod-relpath-test-mainrepo.git
fetch = +refs/heads/*:refs/remotes/github/*
[branch "master"]
remote = github
merge = refs/heads/master But the checkout is not on the branch with I think it should work if you set
This should add the |
Oh, I see what you mean (from
While in interactive sessions I've always happened to run something like (which works):
Turns out another valid option is:
This feels more like a faulty git assumption/limitation than an Ansible bug...
I don't believe it's a feasible workaround to specify the remote's name as part of the submodule metadata in the main repo since that would affect all users of the main repo. I'm also not trying to check out the latest version of a submodule; in fact, the reason for specifying a tag and being in a detached HEAD is to explicitly lock in a given version of the main repo and its submodules. |
Ok, it makes sense setting a remote tracking branch won't work since you don't want the latest commit on the branch. I meant to configure a
I can also reproduce the issue by cloning a repo without a submodule, and then updating to a tag/commit with a submodule. I feel like the same task should work regardless of the initial state of the repo (whether cloning or recursively updating) since a Adding |
What I would expect is something like
That might work, though I wonder if it's a good idea to add an option for a corner-case that git should be handling. I'd change:
to something like It might also make sense to try and fix the configuration. A good clone sets configuration:
while a bad one does:
Since the module knows what the remote is at module execution time ( |
Setting the configuration means we'd need to parse the .gitmodules configuration ourselves, unless there's a git helper we can easily use. It's also totally valid to have a submodule relative to the CWD, so I think anything we attempt may need to be done as a fallback to the existing behavior... |
The
Well, a relative submodule URL is assumed to be relative to the default remote's URL, so if the module sets the submodule paths based off of A side-benefit to fixing the URLs is that it might allow one to change the |
I guess I wasn't clear, but I meant a utility to do the relative URL/path conversion for .gitmodules urls would help here. There was a helper before 2.34.0:
I don't see a way to do something similar on recent versions. If the Ansible module reimplements it, it can unintentionally diverge with
My point was that existing playbooks that currently work using relative paths should continue working as they do now. |
That's unfortunate.
That's probably the best that can be done with the way git currently behaves. |
Here's a possible fix that tries to resolve and configure a new url if it initially fails: devel...s-hertel:git-configure-remote-rel-url. I'm working on the tests (including adding one for the reproducer you gave), but any thoughts on the approach are welcome (or if there are any other test cases you want to throw at it, that would be awesome). |
Thanks for working on this! I still think that remote URL fix-ups should be unconditional if they exist rather than reactive to a submodule clone failure (get the repo configuration into the correct state and then pull), especially if it allows a user to change the remote (say, from HTTPS to SSH) and have the submodule URLs update across existing and new hosts without including hysteresis from previous runs. I can't review the code normally, so here are some ad hoc notes: 1045: Should that be recursive? Shouldn't 1077: Does that work in all cases?
578-587: Wouldn't it be better to return 1014: Why not
1083: Won't this always be True after running 1088: I think this is a problem; ex: pretty sure a relative path like 1100: A tempfile might be the best way to restore, but might hamper debugging. |
Thanks for looking at the POC. Once the approach is hashed out I'll clean up the implementation. These are the commands the module would run for your reproducer:
I don't see how it is at all possible to proactively fix the URLs since Git doesn't give a way to access that plumbing anymore. Am I missing something? A fix for this should not be breaking existing playbooks, and trying to circumvent the normal behavior of Your point about changing the remote seems like it's not totally incompatible with this approach, but I'm confused what the use case is. Is this something you would do manually with
The documentation didn't describe the format, so I was not sure. Thanks for the example of when it won't work.
It will traceback. Tested
I don't understand how the example you give is related to that line. Your example does start with os.path.sep. Essentially, that was trying to skip any URLs that are not a path or is a valid path, and only fix the invalid paths.
That does look like a bug, good catch. (Oh, I guess submodules of submodules could encounter this same issue with relative URLs? Haven't tested, but would assume my fix doesn't handle that). |
Summary
The
git
module fails to properly clone submodules when using:remote
version
I am able to consistently reproduce the issue with the below "Steps to reproduce". My hypothesis is that something along the following is happening:
git
module tries to clone the repo submodule, but it doesn't know what protocol to use because the git submodule URL is relativegit
module is then hardcoded to find the protocol viaremote.origin.url
, not taking into account that the remote isgithub
, and thus fails to find the correct protocol for cloning the repo submodule.git
module tries to clone with a relative path, which fails because it's supposed to use HTTPS.I don't know how the
version
field plays into this, but the module works fine when a customversion
field is not specified. It also works fine if I keep theversion
field and remove theremote
field. It's only the combination which appears to cause the issue.Issue Type
Bug Report
Component Name
git
Ansible Version
Configuration
OS / Environment
Ubuntu 22.04 LTS
Steps to Reproduce
I created two public repos that are capable of reproducing the problem:
The following playbook tries to clone the main repo with its submodule and then fails.
Expected Results
I expected the main repo to be cloned with its source (remote) named
github
and its submodule cloned as expected (though the submodule's remote would be namedorigin
); instead, the submodules were not cloned!Actual Results
Code of Conduct
The text was updated successfully, but these errors were encountered: