Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

umpf: add built-in support for synchronizing topic branches #53

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

umpf: add built-in support for synchronizing topic branches #53

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion bash_completion
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ _umpf_completion()
"")
COMPREPLY=( $( compgen -W "${completion_cmds[*]} help" -- $cur ) )
;;
diff|show|tag|tig|build)
diff|show|tag|tig|build|push|pull)
local -a refs
refs=( $( compgen -W "$( git for-each-ref --format='%(refname:short)' refs/tags refs/heads refs/remotes)" -- $cur ) )
if [ ${#refs[@]} -eq 0 ]; then
Expand Down
41 changes: 41 additions & 0 deletions doc/getting-started.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -447,6 +447,47 @@ Or tell umpf to rebase onto a new *umpf-base* when creating a fresh *utag*::
# umpf-topic-range: 8bae5bbec8cb4599c141405e9755b7c0e42e064f..19cdc2b857e662a38c712b41ce610000a5ddc6ae
# umpf-end

Synchronizing umpf topic branch
-------------------------------

Due to Git's distributed nature, checked out topic branches can get
out-of-sync. To compare local topic branches against those referenced
in a *utag*, ``umpf pull`` can be used::

umpf --dry-run pull 5.0/special-customer-release/20190311-1
umpf: Using series from commit message...
* [new branch] 02fb74aa381080855a57080138b29ecc96586788 -> v5.0/topic/most-fixes
! [rejected] f0693b782dd026f2adc4d3c336d9ac6dfb352a73 -> v5.0/topic/more-fixes (non-fast-forward)

Following options are supported:

- ``--dry-run``: compare the branches, but stop short of actually updating
them
- ``--force``: reset local branches that are not checked-out to the
``umpf-hashinfo`` in the ``utag``
- ``--update``: restrict updates to only branches available locally

The counterpart to publish topic branches to a remote after creating a new
``utag`` is ``umpf push``:

umpf --dry-run --remote=downstream push 5.0/special-customer-release/20190311-1
umpf: Using series from commit message...
To ssh://downstream
* [new branch] 02fb74aa381080855a57080138b29ecc96586788 -> v5.0/topic/most-fixes
! [rejected] f0693b782dd026f2adc4d3c336d9ac6dfb352a73 -> v5.0/topic/more-fixes (non-fast-forward)
error: failed to push some refs to 'ssh:/downstream'

It supports the same options as ``umpf pull``, but instead of doing local
changes, it operates on the specified remote.

``umpf push`` is especially useful when multiple developers are creating
`utags` for the same project in parallel. Each developer will initially
only push their `utag` to the common repository. Once the changes
introduced by a `utag` are accepted, all topic branches can be force
updated on the remote to this most recent `utag`, possibly via
a server-side pull-request post-merge hook running, e.g.::

umpf --remote=downstream --force .../linux/patches/series.inc

Overview
--------
Expand Down
202 changes: 201 additions & 1 deletion umpf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ PATCH_DIR="umpf-patches"
IDENTICAL=false
STABLE=false
FORCE=false
DRYRUN=false
UPDATE=false
VERBOSE=false
VERSION_SEPARATOR=-
Expand Down Expand Up @@ -178,6 +179,8 @@ usage() {
--nix with format-patch: write patch series nix
-h, --help
-f, --force
--dry-run with push/pull: Do everything except actually send
the updates.
--flags specify/override umpf-flags
-i, --identical use exact commit hashes, not tip of branches
-s, --stable create a 'stable' tag from a branch based on an
Expand All @@ -194,6 +197,7 @@ usage() {
specified, it's interpreted as
<topic>=[<remote>/]<topic>
-u, --update with --patchdir: update existing patches in <path>
with push/pull: update only existing branches
-v, --version <version> with tag: overwrite version number [default: 1]

Commands:
Expand All @@ -218,6 +222,8 @@ usage() {
build <umpf> build an umerge from another umpf
distribute <commit-ish> push patches not yet in any topic branch
upstream
push [<umpf>] push topic branches to the given remote
pull [<umpf>] pull topic branches into the local repository

continue continue a previously interrupted umpf command
abort abort a previously started umpf command
Expand Down Expand Up @@ -245,7 +251,7 @@ setup() {
fi

o="fhilsub:n:p:r:v:"
l="auto-rerere,bb,nix,flags:,force,help,identical,stable,update,base:,name:,patchdir:,relative:,override:,remote:,local,version:"
l="auto-rerere,bb,nix,flags:,dry-run,force,help,identical,stable,update,base:,name:,patchdir:,relative:,override:,remote:,local,version:"
if ! args="$(getopt -n umpf -o "${o}" -l "${l}" -- "${@}")"; then
usage
exit 1
Expand All @@ -271,6 +277,9 @@ setup() {
-f|--force)
FORCE=true
;;
--dry-run)
DRYRUN=true
;;
--flags)
FLAGS="${1}"
shift
Expand Down Expand Up @@ -1855,6 +1864,197 @@ do_distribute() {
run_distribute
}

### namespace: push ###

push_topic() {
echo "${content}" >> "${STATE}/topic-names"
}

push_hashinfo() {
echo "${content}" >> "${STATE}/topics"
}

push_release() {
echo "${content}" >> "${STATE}/tagname"
}

push_topic_range() {
[ ! -e "${STATE}/tagname" ] && return
[ -e "${STATE}/tagrev-flat" ] && abort "more than one 'topic-range' after 'release'!"

echo "${content##*..}" > "${STATE}/tagrev-flat"
}

### command: push ###

resolve_commitish() {
${GIT} rev-parse --revs-only "$@" 2>/dev/null
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use ${..} everywhere.

}

shorten_commitish() {
resolve_commitish --short ${1}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quote arguments everywhere.

}

resolve_tag() {
local remote=$1 tag=$2 commit
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With remote="${remote:-${GIT_DIR}}" you can use ls-remote for local tags as well.

if [ -n "$remote" ]; then
# handles conflicting tags on remote
commit=$(git ls-remote -q $remote refs/tags/$tag 2>/dev/null | \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add --tags, anything else is not needed.

sed 's/\s\+.*$//')
else
commit="refs/tags/$tag"
fi

resolve_commitish "${commit}^{}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only used to compare the local and remote tags. What's the use-case for accepting different tags with the same commit?

}

update_local() {
local success=false args="${1}"
local opts
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use an array for opts.


${FORCE} && opts+="--force"

local line
while read -r line; do
local prefix="" suffix=""

eval set -- ${line}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

local a b c
read -r a b c <<< "${line}"

Use better variable names but you get the idea. This avoids other expansions.

Or use read -r -a array_var ... but I think individual variables make this more readable,

[ ${#} -eq 0 ] && continue

local refparsed=$(shorten_commitish $3)

if [ -z "${refparsed}" ]; then
prefix=" * [new branch]"
elif [[ "${1}" = *~* ]]; then
prefix=" ${1}..$(shorten_commitish ${2})"
elif $FORCE; then
prefix=" + ${refparsed}..$(shorten_commitish ${2})"
suffix=" (forced update)"
else
prefix=" ! [rejected]"
suffix=" (non-fast-forward)"
fi

printf "%-40s %s -> %s%s\n" "$prefix" $2 $3 "$suffix"
done <<< "$args"

while read -r line; do
eval set -- ${line}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here.

[ ${#} -eq 0 ] && continue

if $DRYRUN || git branch $opts $3 $2; then
success=true
fi
done <<< "$args"

$success || abort
}

do_push () {
local opts args remote
local -a branches branch_names
local -A topics

if [ -z "${GIT_REMOTE}" ]; then
info "Git remote must be specified. Cannot continue."
exit 1
fi

if [ "${GIT_REMOTE}" != "refs/heads/" ]; then
remote=${GIT_REMOTE%/}
fi

prepare_persistent push "${@}"
parse_series push "${STATE}/series"

local tagname="$(<"${STATE}/tagname")"
local tagrevf="$(<"${STATE}/tagrev-flat")"
mapfile -t branches < "${STATE}/topics"
mapfile -t branch_names < "${STATE}/topic-names"

if [ -n "${remote}" ]; then
# Needed, so git rev-parse below can check for existent branches
git fetch --quiet --no-tags ${remote} 2>/dev/null
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, use ls-remote for the branch checks instead.

That would make a --force-with-lease possible as well (I'd like to see that as well at some point but I won't insist on it for this PR).

fi

local rtagrev="$(resolve_tag "${remote}" ${tagname})"
local rtagrevf="$(resolve_commitish ${rtagrev}^)"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use resolve_tag here as well to make it less confusing.

if [ "$tagrevf" != "$rtagrevf" ]; then
if [ -z "$rtagrevf" ]; then
abort "${remote}${remote:+/}refs/tags/$tagname not found"
else
abort "${remote}${remote:+/}refs/tags/$tagname" \
"has unexpected commit-ish $rtagrev"
fi
fi
Comment on lines +1980 to +1989
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skip for emtpy remote? In that case you're comparing the same tag, right?


for i in "${!branch_names[@]}"; do
local branch=${branch_names[$i]}
local rbranchrev="$(resolve_commitish "${GIT_REMOTE}${branch}")"

[ -z "${remote}" ] && [ "${branches[$i]}" = "${rbranchrev}" ] && continue

# Don't touch local branches that are already on the correct revision.
# For remote branches, we let git push handle it.
[ -z "${remote}" ] && [ "${branches[$i]}" = "${rbranchrev}" ] && continue
$UPDATE && [ -z "${rbranchrev}" ] && continue

topics[${branch}]=$(resolve_commitish ${branches[$i]})
done

if [ -n "${remote}" ]; then
${FORCE} && opts+="--force-with-lease"
${DRYRUN} && opts+="--dry-run"

for topic in "${!topics[@]}"; do
args+="${topics[$topic]}:refs/heads/${topic} "
done

if [ -z "$args" ]; then
info "No branches to push"
cleanup
return
fi

${GIT} push $opts ${remote} -- $args
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think, if you use ${GIT_DIR} as remote for the local case, you can use git push for that as well. That would avoid a lot of complexity. So try that first, it may make some of my comments obsolete.

else # local
for topic in "${!topics[@]}"; do
local ref=$topic rev=${topics[$topic]}

# The old value being computed below is not needed to
# create the branch. We compute a suitable one anyway,
# so we can show how a ref's commit-ish has changed in
# the pull case like we do in the push case.
if git merge-base --is-ancestor $ref $rev &>/dev/null; then
local ancestors=$(git rev-list $rev ^$ref --count)
args+="$(shorten_commitish "$rev")~$ancestors"
elif $FORCE; then
args+="$(shorten_commitish "$ref")"
else
args+='""'
fi
args+=" $rev $ref"
args+=$'\n'
done

if [ -z "$args" ]; then
info "No branches to push"
cleanup
return
fi

update_local "$args"
fi

cleanup
}

### command: pull ###

do_pull () {
GIT_REMOTE=refs/heads/ do_push "$@"
}

### command: continue ###

do_continue() {
Expand Down
Loading