Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Mismatch when updating latest via webhook and after clone #10915

Closed
nadyawilliams opened this issue Nov 21, 2023 · 9 comments · May be fixed by #10927
Closed

Mismatch when updating latest via webhook and after clone #10915

nadyawilliams opened this issue Nov 21, 2023 · 9 comments · May be fixed by #10927
Labels
Bug A bug

Comments

@nadyawilliams
Copy link

Details

I am building the docs for a repo that has not been changed today. Simply making sure that builds go ok.

First successful build at the timestamp Completed Nov. 20, 2023. 11:07 p.m.
According to the console logs, the first two commands of the build were :
git clone --depth 1 https://github.com/RCIC-UCI-Public/rcs3-docs .
git fetch origin --force --prune --prune-tags --depth 50

With no changes to the repo or RTD settings, from RTD console i execute a second build about 30 min later

Second build failed at the timestamp Completed Nov. 20, 2023. 11:33 p.m.
According to the console logs the first two commands of this build were different:
git clone --depth 1 https://github.com/RCIC-UCI-Public/rcs3-docs .
git fetch origin --force --prune --prune-tags --depth 50 refs/heads/master:refs/remotes/origin/master
fatal: couldn't find remote ref refs/heads/master

Expected Result

Considering no repo or RTD configuration was touched in any way I assume it is a result of some changes in
the automated build process.

The second git command additional parameters refs/heads/master:refs/remotes/origin/master
make this git fetch command fail.

Please advise how to avoid the addition of these extraneous arguments to git fetch command during the automated build.
No such parameters are present anywhere in my repo files

Thanks
@nadyawilliams
Copy link
Author

Additional info. As of this morning with no changes to RTD or city repo the build succeeds. Here is the build history of
3 builds:
passed Build #22612469 Completed Nov. 20, 2023. 11:07 p.m.
failed Build #22612647 Completed Nov. 20, 2023. 11:33 p.m.
passed Build #22621615 Completed Nov. 21, 2023. 5 p.m.

Quite a similar situation is happening to another project/repo (no changes to either git files or rtd configuration)
failed Build #22612511 Completed Nov. 20, 2023. 11:12 p.m.
failed Build #22621623 Completed Nov. 21, 2023. 5:01 p.m.
passed Build #22623612 Completed Nov. 21, 2023. 8:12 p.m.

All successful builds have no extra "refs/heads/master:refs/remotes/origin/master" in the git fetch commands.

It's almost as if the successful builds land on a different server compare to the failed builds.
It would be nice to have a better predictability of what is going on so that one can depend on the build outcome.

@Qizhen-Xue
Copy link

Met the same fetch error when build documents on ReadtheDocs today.

The document for our project is https://nnabla.readthedocs.io/en/latest/. We use tags of the project to control the release document version. But the default git fetch command used by ReadtheDocs could not support tags now.
readthedocs_fetch_error

I tried to replace refs/heads/v1.37.0 with refs/tags/v1.37.0. Then the fetch command could run successfully.

I believe there must be many other projects would met the same error. Hope the maintainers of ReadtheDocs could handle this issue soon.

@nadyawilliams
Copy link
Author

This morning, Nov 22, both of my projects builds are failing yet again after a successful build yesterday.
No tags, branches, etc changes to the git repo. NO changes to the RTD settings.
Build #22634012
Build #22633991
The same culprit is in git fetch command on the server where the builds are running.

@Qizhen-Xue Qizhen-Xue mentioned this issue Nov 24, 2023
Closed
@stsewd
Copy link
Member

stsewd commented Nov 27, 2023

@nadyawilliams hi, did you change your project's default branch setting to be explicitly main? That should fix the problem.

Previously, what was happening is that your project's default version was being updated by the GitHub webbook (to point to main)

readthedocs.org/readthedocs/api/v2/views/integrations.py

Lines 315 to 320 in 1b38024

if self.get_integration().secret and not self.project.default_branch:
# Always check for the machine attribute, since latest can be user created.
# RTD doesn't manage those.
self.project.versions.filter(slug=LATEST, machine=True).update(
identifier=default_branch
)

And then again after triggering a build (to point to master)

readthedocs.org/readthedocs/projects/models.py

Lines 1284 to 1292 in 1b38024

def get_default_branch(self):
"""Get the version representing 'latest'."""
if self.default_branch:
return self.default_branch
if self.remote_repository and self.remote_repository.default_branch:
return self.remote_repository.default_branch
return self.vcs_class().fallback_branch

This is kind of related to #10892 and how we handle latest.

@stsewd stsewd added the Bug A bug label Nov 27, 2023
@stsewd stsewd changed the title Failed builds due to git fetch command change Mismatch when updating latest via webhook and after clone Nov 27, 2023
stsewd added a commit that referenced this issue Nov 27, 2023
@stsewd
Build: fix mismatch between webhook and version sync for latest
8ea7576 
Closes #10915
@stsewd stsewd mentioned this issue Nov 27, 2023
Open
@nadyawilliams
Copy link
Author

Yes, i did this change earlier today before i saw your response. My git repos were initially on the "main" branch and never had any changes from master to main. Webhook behavior was not changed either with the exception of creating a new one with the secret and all the rest remaining the same. The webhook was changed before i run the builds and got pass/fail behavior. What throw me for a loop was an inconsistent behavior: i would get pass or fail without any changes to the repo or RTD settings. What settings i have should either make all builds work or all builds fail but failing or working within few minutes from each other with no changes on my side indicate that some of the build servers are probably not using the same build logic path. This is an inconsistency on RTD side that probably needs to be looked at.

@stsewd
Copy link
Member

stsewd commented Nov 27, 2023

@nadyawilliams yep this is a bug in RTD, I just described the cause at #10915 (comment), there is a mismatch in how the webhook updates latest and how the sync after a clone does.

@nadyawilliams
Copy link
Author

Ok, thanks. Can I assume that the update of a default branch to main will hold on for all the builds moving forward, not just some builds.

@stsewd
Copy link
Member

stsewd commented Nov 27, 2023

@nadyawilliams yeah, explicitly defining your default branch will prevent this bug from re-appearing in your projects.

@nadyawilliams
Copy link
Author

Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug A bug
Projects
None yet
Milestone
No milestone
3 participants
@nadyawilliams @stsewd @Qizhen-Xue