Do we need a better strategy for regular conflict files?

[trexfeathers]

docs/src/whatsnew/latest.rst

When

Almost every PR involves a contribution to this file. Any two that are under the same heading will conflict.

Solution?

• I believe the current strategy is still an improvement over the previous directory of single-contribution .txt files that must be aggregated before a release, so I'd prefer not to go back to that.
• Can we tweak the structure of the file to make conflicts less frequent?
• See Possible common solutions

requirements/locks/*

When

Anyone making changes to the requirements/*.yml files will need to make a corresponding update to the lock files. Any two such PR's will conflict, which can be quite regular due to the automated weekly updates of these files.

Solution?

• Insist on just one open PR with lock file changes?
• See Possible common solutions

Possible common solutions

• Can be set some sort of etiquette norm where people can foresee conflicts? Live communication via a Discussion?
• Can we automate a check through all open PR's, to check for upcoming conflicts?
• Any What's New contributions and lock file updates are considered stale as soon as an Approve review has been made - they must be updated before merging.
  Could combine with auto-merge

This is a pretty tricky one. I don't expect the solution to be simple or without compromise.

[rcomer]

Is it possible to make a GitHub Action that updates the lockfiles within the PR? This would mean

• No need to bother creating them locally
• The lockfile change would be in its own commit, so if there's a conflict you just drop that commit and respin (if you never pulled the bot's commit, it will disappear if you force-push).

[trexfeathers]

Literally writing the same thing at the same time!

[trexfeathers]

We could probably make a tool that updates the lock files within a PR, perhaps called by a magic comment (like conda-forge PR's), which would make it less clunky than the offline updates and reduce the pain of resolving the conflict.

[rcomer]

Great minds....

You can make the action conditional on which paths were modified in the PR
https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#example-including-paths

[pp-mo]

Problems not always confined to latest.rst ??

My lazy-save PR has been a constant problem to maintain, to get running tests, because it wants to add 'distributed' into the deps.

Every time someone merged an update to the lockfiles, I needed to ...

1. mergeback-from-trunk
2. re-resolve lockfiles with mamba + conda-lock
3. re-fix the lockfiles package urls with "ssstack shareable"
   • (converts urls referring to out internal Artifactory channel proxy to plain conda-forge urls)
4. add all + merge --continue
5. push back

It's been painful, for sure !
I've lost count of how any times I've had to do this.

[pp-mo]

Some opinions that emerged on the latest.rst problem...

@pp-mo It's sad because we used to have a ~simple mechanism for this, and we ditched it.
#3769
I think at one point the structure of the "latest.rst" became such that doing without wasn't such a problem.  But now it is again, not clear why ??

@r****r I always had to look up the whatsnew file structure to figure out where I was supposed to put things.  Editing the latest.rst is much more simple and intuitive IMO

@BJL****e hilarious 😂

[rcomer]

I think at one point the structure of the "latest.rst" became such that doing without wasn't such a problem. But now it is again, not clear why ??

Maybe it's just that the repo is busier and we are merging more changes. I think when we switched to this method your team was smaller and spending a significant proportion of time on the meshgrid work.

[rcomer]

Within each whatsnew category, there is no particular order to the entries, though I guess it's roughly chronological. If we adopted the convention that they are ordered alphabetically by name of author, conflicts between authors would occur less often.

[pp-mo]

In effect, at the moment, they are in a chronological order.
The trouble is, FWIW I quite like it that way.

The problem is, obviously, that the upstream + local branches generally both add something at the same point in the list.

 * item 1
 * item 2
=== @head
 * NEW main entry
=== @local branch
 * NEW my entry
==
... other stuff

TBH this is never a big deal to fix, except when there are other conflicts it might get confused with.

On the other hand, it seems surprising in a way that this can't be automatically merged.
I wonder if there is a magic keyword that might achieve that,
or maybe if the information were structured differently somehow.

[trexfeathers]

TBH this is never a big deal to fix

I was wondering if this simple sort of conflict might in fact have an automatable solution. Not sure though.

[bouweandela]

For the whatsnew entries, we could consider using towncrier.

[trexfeathers]

Yes, I think we're trying this out with GeoVista. It would be a shame to lose the sphinx domains that we have at the moment, but the convenience might be worth it.

[trexfeathers]

I take it back. This PyTest example DOES include sphinx domains, and that seems to be working fine.

[pp-mo]

Checking this out, the features look good -- we should try it ??

possible downsides IMHO

• our "featured items" content doesn't really fit this format -- but could go somewhere else (release page for featured changes only ?)
• it makes a break with the previous record.

on the second point, BTW I've never liked that we don't have a searchable record on one page like many projects.
Wiht that in mind, I guess we could digest all the older content + use it to seed a CHANGE.rst
But, would that be manageable, or too long ?