yaml file traversal by bash tab completion

[shakibamoshiri]

Please describe your feature request.
It is about 3 months heavily I am using yq to parse, modify, check yaml files
I have to check many yaml files locally or on some servers (after ssh-ing into them)
Since I code in bash and create their tab-completion as well, I could create a tab-completion script and fulfill me needs.
My need was checking each object's value rather than cat those files and scroll the terminal

But yq has no option to select a file (e.g docker-compose -f file-name.yaml) explicitly

So I wish yq had these two features or at least the first one

1. An option to select a file , a yaml file to parse
2. Support tab-completion script to have this traversal feature ( I can contribute)

yq -f file.yml <TAB>

Describe the solution you'd like
Solution are

1. Add -f or any other available letter in order the explicitly select a file (I solved this using an other script)
2. Refactor your tab-completion (it is ready)

As an input

kind: database
type: source
project: project-name
salt: my-salt
encrypt:
  object:
    - .company.people.data
    - .company.employee.data
  file:
    - .company.people
    - .company.employee
decrypt:
  object:
    - .company.people.data
    - .company.employee.data
  file:
    - .company.people
    - .company.employee

And we run a command:

yq -f sample.yml .<TAB>

it could output

.decrypt.file.0    .decrypt.object.1  .encrypt.object.0  .project           
.decrypt.file.1    .encrypt.file.0    .encrypt.object.1  .salt              
.decrypt.object.0  .encrypt.file.1    .kind              .type

My solution for -f option

#!/bin/bash

yq $2 $1 ${@:3}

My solution for object traversal

cmd_name='yqi';   # $1
current_arg='';   # $2
previous_arg='';  # $3


yqi_path=$(which yqi);
yqi_short_opt=($(perl -lne 'print $& if /-[a-zA-Z0-9]\b/' <($yqi_path --help) | sort | uniq))
yqi_long_opt=($(perl -lne 'print $& if  /--[a-zA-Z0-9-]+\b/' <($yqi_path --help 2>&1) | sort | uniq));


_comp_yqi () {
    current_arg=${COMP_WORDS[$COMP_CWORD]};
    previous_arg=${COMP_WORDS[$COMP_CWORD-1]};

    case ${previous_arg} in
        yqi )
            files=($(echo *.yml *.json *.xml));
            COMPREPLY=(${files[@]});
        ;;

    esac

    case ${current_arg} in
        .* )
            keys=($(yq -o props ${COMP_WORDS[1]} | perl -lpe 's/^/./; s/ =.*$//g'));
            COMPREPLY=( $(egrep -o "${current_arg//./\\.}[^ ]+" <<< ${keys[@]}) );
        ;;
        [a-zA-Z0-9]* )
            COMPREPLY=( $(egrep -o "${current_arg//./\\.}[^ ]+" <<< ${files[@]}) );
        ;;
    esac
}

complete -o bashdefault -o default -o nospace -F _comp_yqi yqi

This tab-completion script is pretty short , but smart enough to give us list of keys to check their value.
Actually by any TAB hit - it parses the yaml file using yq -o props and generates a new list .

how to test and use

1. save the script as yqi and give the right permission
2. source the second script to activate Tab-Completion for first script yqi

here is a sceenshot of it

[mikefarah]

This is really neat! I'm not going to merge it into yq at this stage as it requires perl to be installed, and it does not support complex expressions. Nonetheless, I'm sure it's useful for many 👍🏼 👍🏼

[shakibamoshiri]

This version is not that portable as it could be, I am more comfortable with perl so I pick it up first, but it can be replaced with grep or even grep -E or grep -P ; not to mention that perl on no Linux machine I worked with, was needed to be installed. Anyway, I love this tool and it deserves more stars. Thank you and all contributors.

[shakibamoshiri]

+ Where is the script? On the page #1271 See this part "My solution for object traversal" + Is it a normal bash completion file? Yes but in a more smart way completes It is more dynamic than usual bash script *I don't think I have come across a system that does not have perl.* *Yes by default all Linux distro have perl* *I could not find this reply on the page, where have you written your reply*

…

On Sat, Dec 3, 2022 at 1:45 AM Jason Gomez ***@***.***> wrote: Where is this script? Is it a normal bash completion file? I don't think I have come across a system that does not have perl. — Reply to this email directly, view it on GitHub <#1271 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AENWERHHUVLMYUIJMIJMM4TWLJYHLANCNFSM53NLKUAA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[planeiii-te]

awk and sed could easily replace perl here:

cmd_name='yqi';   # $1
current_arg='';   # $2
previous_arg='';  # $3


yqi_path=$(which yqi);
yqi_short_opt=($(awk 'match($0, / -[a-zA-Z0-9][ ,]/) { print substr($0, RSTART + 1, RLENGTH - 2) }' <($yqi_path --help) | sort | uniq))
yqi_long_opt=($(awk 'match($0, / --[^ "]+/) { print substr($0, RSTART + 1, RLENGTH - 1) }' <($yqi_path --help 2>&1) | sort | uniq));


_comp_yqi () {
    current_arg=${COMP_WORDS[$COMP_CWORD]};
    previous_arg=${COMP_WORDS[$COMP_CWORD-1]};

    case ${previous_arg} in
        yqi )
            files=($(echo *.yml *.json *.xml));
            COMPREPLY=(${files[@]});
        ;;

    esac

    case ${current_arg} in
        .* )
            keys=($(yq --output-format props ${COMP_WORDS[1]} | sed 's/^/./; s/ =.*$//g'));
            COMPREPLY=( $(egrep -o "${current_arg//./\\.}[^ ]+" <<< ${keys[@]}) );
        ;;
        [a-zA-Z0-9]* )
            COMPREPLY=( $(egrep -o "${current_arg//./\\.}[^ ]+" <<< ${files[@]}) );
        ;;
    esac
}

complete -o bashdefault -o default -o nospace -F _comp_yqi yqi

Here's the diff

--- yqi.original	2023-03-24 12:13:05
+++ yqi	2023-03-24 12:12:02
@@ -4,8 +4,8 @@
 
 
 yqi_path=$(which yqi);
-yqi_short_opt=($(perl -lne 'print $& if /-[a-zA-Z0-9]\b/' <($yqi_path --help) | sort | uniq))
-yqi_long_opt=($(perl -lne 'print $& if  /--[a-zA-Z0-9-]+\b/' <($yqi_path --help 2>&1) | sort | uniq));
+yqi_short_opt=($(awk 'match($0, / -[a-zA-Z0-9][ ,]/) { print substr($0, RSTART + 1, RLENGTH - 2) }' <($yqi_path --help) | sort | uniq))
+yqi_long_opt=($(awk 'match($0, / --[^ "]+/) { print substr($0, RSTART + 1, RLENGTH - 1) }' <($yqi_path --help 2>&1) | sort | uniq));
 
 
 _comp_yqi () {
@@ -22,7 +22,7 @@
 
     case ${current_arg} in
         .* )
-            keys=($(yq -o props ${COMP_WORDS[1]} | perl -lpe 's/^/./; s/ =.*$//g'));
+            keys=($(yq --output-format props ${COMP_WORDS[1]} | sed 's/^/./; s/ =.*$//g'));
             COMPREPLY=( $(egrep -o "${current_arg//./\\.}[^ ]+" <<< ${keys[@]}) );
         ;;
         [a-zA-Z0-9]* )

Note: The process substitution seems unnecessary, but I left it in to shrink the diff

awk 'match($0, / -[a-zA-Z0-9][ ,]/) { print substr($0, RSTART + 1, RLENGTH - 2) }' <($yqi_path --help) | sort | uniq

Could just as easily be

$yqi_path --help | awk 'match($0, / -[a-zA-Z0-9][ ,]/) { print substr($0, RSTART + 1, RLENGTH - 2) }' | sort | uniq

This was tested on MacOS with the absolute worst version of awk known to mankind.

[shakibamoshiri]

Thank you for sharing this.
I am more comfortable with perl but of course someone knows awk and sed can make it work.