Skip to content

Commit

Permalink
#2 #3 #7 #8 #9 #11 - refactored and changed the function names
Browse files Browse the repository at this point in the history
  • Loading branch information
wwalker committed Dec 7, 2019
1 parent 47cf8b3 commit 53167ac
Show file tree
Hide file tree
Showing 2 changed files with 198 additions and 60 deletions.
50 changes: 25 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ Simple usage example:
Runs the program `barman` and creates `~/logs/barman-2019-12-05T23:54:43` and logs all the program output to the log file.

```
justlog barman -v check-backup mtv-pg-01 --run 20191127T045147
log-plain barman -v check-backup mtv-pg-01 --run 20191127T045147
```

View the newly created log file in less
```
lesslog barman
log-less barman
```

# Install
Expand Down Expand Up @@ -50,21 +50,21 @@ That is great when I want to catch an interactive session.

However, when I want to log the output of a command (script or binary), I either want the exact output, or I want the output with timestamps in front of each line. So I wrote these some bash functions:

* `justlog <command and its arguments>` - run the command passed in as arguments, automatically writing stdout and stderr to a new unique log file ( ~/logs/<program-name>-YYYY-MM-DD-HH:MM:SS )
* `timelog <command and its arguments>` - just like justlog, except it prefixes everyline with a timestamp ( YYYY-MM-DD-HH:MM:SS.micros )
* `newestlog <program-name>` - returns the name the newest log file for program-name
* `lesslog <program-name>` - opens the newest log file for program-name using less
* `viewlog <program-name>` - opens the newest log file for program-name using view
* `greplog <program-name> <grep arguments and regular expression>` - runs grep on the newest log file for program-name
* `log-plain <command and its arguments>` - run the command passed in as arguments, automatically writing stdout and stderr to a new unique log file ( ~/logs/<program-name>-YYYY-MM-DD-HH:MM:SS )
* `log-time <command and its arguments>` - just like log-plain, except it prefixes everyline with a timestamp ( YYYY-MM-DD-HH:MM:SS.micros )
* `log-newest <program-name>` - returns the name the newest log file for program-name
* `log-less <program-name>` - opens the newest log file for program-name using less
* `log-view <program-name>` - opens the newest log file for program-name using view
* `log-grep <program-name> <grep arguments and regular expression>` - runs grep on the newest log file for program-name

Just run your command like you normally would prefixing it with either `justlog` or `timelog`:
Just run your command like you normally would prefixing it with either `log-plain` or `log-time`:

```
justlog barman -v check-backup mtv-pg-01 --run 20191127T045147
log-plain barman -v check-backup mtv-pg-01 --run 20191127T045147
```
or
```
timelog barman -v check-backup mtv-pg-01 --run 20191127T045147
log-time barman -v check-backup mtv-pg-01 --run 20191127T045147
```

This will painlessly create 2 files like this:
Expand All @@ -77,51 +77,51 @@ wwalker@polonium:~ ✓ $ ls -l ~/logs/barman*

But, Wait there's more!!

I didn't want to have to do an ls to find the name of the log file that justlog or timelog just created for me.
I didn't want to have to do an ls to find the name of the log file that log-plain or log-time just created for me.

So, you run your command with justlog (or timelog), and then you just use lesslog or viewlog (I'll probably create an emacs log for Those people):
So, you run your command with log-plain (or log-time), and then you just use log-less or log-view (I'll probably create an emacs log for Those people):

```
justlog barman -v check-backup mtv-pg-01 --run 20191127T045147
lesslog barman
log-plain barman -v check-backup mtv-pg-01 --run 20191127T045147
log-less barman
```

That's it, no `ls -lrt ~/tmp`, no tab completion games to find the file name. Just run lesslog (or viewlog if you like using vim to look at logs).
That's it, no `ls -lrt ~/tmp`, no tab completion games to find the file name. Just run log-less (or log-view if you like using vim to look at logs).

But, Wait there's more!!

`lesslog`, `viewlog`, and `newestlog` only look at the first argument; so, you can be lazy:
`log-less`, `log-view`, and `log-newest` only look at the first argument; so, you can be lazy:

```
justlog make -D prefix=/opt
log-plain make -D prefix=/opt
```

Now, you just up arrow and change "just" to "less", and your looking at the recent log file:

```
lesslog make -D prefix=/opt
log-less make -D prefix=/opt
```

`lesslog` (and `viewlog` and `newestlog`) just ignore the extra arguments.
`log-less` (and `log-view` and `log-newest`) just ignore the extra arguments.

But, wait! There's more!

You get programmable completion (tab completion) for whatever commands you put after `justlog` and `timelog`.
You get programmable completion (tab completion) for whatever commands you put after `log-plain` and `log-time`.

But, wait! There's even more!

"I use grep all the time on my log files" - And the answer is, you guessed it, `greplog`
"I use grep all the time on my log files" - And the answer is, you guessed it, `log-grep`

First, get the text from all 800 servers' /etc/cron.d/atop files that are broken:

```
justlog fordcrun 'grep -F "0 0 * * root" /etc/cron.d/atop'
log-plain fordcrun 'grep -F "0 0 * * root" /etc/cron.d/atop'
```

Then get the hostnames (on the line above the output in the file :-) ) with greplog (without having to look up the log file name):
Then get the hostnames (on the line above the output in the file :-) ) with log-grep (without having to look up the log file name):

```
wwalker@polonium:~ ✓ $ greplog fordcrun -B1 -F "0 0 * * root"
wwalker@polonium:~ ✓ $ log-grep fordcrun -B1 -F "0 0 * * root"
int-salt-01:
0 0 * * root systemctl restart atop
--
Expand Down
208 changes: 173 additions & 35 deletions quick-log.bash
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
# https://github.com/wwalker/quick-log
# Original Author Wayne Walker, [email protected]
#
# Most up to date README.md: https://github.com/wwalker/quick-log/blob/master/README.md
#
# Once (on each machine):
# mkdir ~/logs
#
Expand All @@ -8,74 +13,207 @@
#
# This will run "barman -a -b -c -d asdf qwer" add time stamps to each
# line of stdout or stderr into ~/logs/barman-2019-11-12T21:53:34:
# timelog barman -a -b -c -d asdf qwer
# log-time barman -a -b -c -d asdf qwer
#
# This will do the same with no added timestamps:
# justlog barman -a -b -c -d asdf qwer
# log-plain barman -a -b -c -d asdf qwer
#
# Instead of having to ls or play completion tabbing game, you can look
# at, in view or in less, without having to know the name of the file:
#
# viewlog and lesslog ignore 3everything after the first arg (barman)
# log-view and log-less ignore everything after the first arg (barman)
#
# viewlog barman
# viewlog barman -a -b -c -d asdf qwer
# viewlog !!:1
# log-view barman
# log-view barman -a -b -c -d asdf qwer
# log-view !!:1
#
# or in less:
# lesslog barman
# lesslog barman -a -b -c -d asdf qwer
# log-less barman
# log-less barman -a -b -c -d asdf qwer
#
# You can grep out of the file without looking up the filename:
# greplog barman -B1 -F "FAILED"
# log-grep barman -B1 -F "FAILED"
#
# You can get the filename of the newest log
# ls -l $(newestlog barman)
# ls -l $(log-newest barman)

# Set up programmable completion :-)
complete -F _command timelog
complete -F _command justlog
complete -F _command lesslog
complete -F _command viewlog
complete -F _command greplog
complete -F _command newestlog
complete -F _command log-time
complete -F _command log-plain
complete -F _command log-less
complete -F _command log-view
complete -F _grep log-grep
complete -F _command log-newest

alias iso8601="date +%Y-%m-%dT%H:%M:%S"

alias scr='script ~/tmp/typescript-$(iso8601)'
alias scr='script ~/logs/typescript-$(iso8601)'

justlog(){
name=$(basename "$1")
log=~/logs/${name}-$(iso8601)
"$@" > "$log" 2>&1
_ql_args=()

_ql_init_vars(){
_ql_opt_header=0
_ql_opt_tee=0
_ql_opt_ts=0
_ql_opt_usage=
_ql_opt_directory=$HOME/logs/
_ql_opt_process_name=
_ql_abort=0
}

timelog(){
name=$(basename "$1")
log=~/logs/${name}-$(iso8601)
# You could replace ilts with ts if it is installed
# /usr/bin/ts %FT%H:%M:%.S
"$@" |& ilts -S -E > "$log" 2>&1
_ql_usage(){
printf "Unrecognized options %s\n" "$_ql_opt_usage"
printf "Usage message goes here\n"
return 1
}

_ql_parse_args(){
_ql_done=0
while [[ $_ql_done = 0 ]]
do
key="$1"

case $key in
-h|--header)
_ql_opt_header=1
shift
;;
--ts)
_ql_opt_ts=1
shift
;;
-t|--tee)
_ql_opt_tee=1
shift
;;
-d|--directory)
_ql_opt_directory=$2
shift
shift
;;
-p|--process-name)
_ql_opt_process_name=$2
shift
shift
;;
-*) # unknown option
_ql_opt_usage+=" $1"
shift
;;
*)
_ql_done=1
;;
esac
done

if [[ -n "$_ql_opt_usage" ]]
then
_ql_usage "$_ql_opt_usage"
return 1
fi

_ql_args=()
while [[ $# -gt 0 ]]
do
_ql_args+=("$1")
shift
done
}

_ql_logfile_name(){
if [[ -z "$_ql_opt_process_name" ]]
then
_ql_opt_process_name=$(basename "$1")
fi
}

_ql_timestamp(){
echo
}
_ql_logfile_path(){
_ql_logfile_name "$1"
_ql_filepath=$(printf "%s/%s-%s" "$_ql_opt_directory" "$_ql_opt_process_name" "$(date +%Y-%m-%dT%H:%M:%S)")
}

_ql_ts_cmd(){
local cmd_path
if cmd_path=$(command -v ilts)
then
_ql_append+=" | $cmd_path -S -E "
return
fi
if cmd_path=$(command -v ts)
then
_ql_append+=" | $cmd_path "
return
fi
_ql_append+=" | awk '{ print strftime(\"%Y-%m-%d %H:%M:%S\"), \$0; fflush(); }' "
}

viewlog(){
log-plain(){
_ql_init_vars
if ! _ql_parse_args "$@"
then
# parsing arguments failed
return 1
fi

set -- "${_ql_args[@]}"
# debug line...
# set | grep '^_ql_opt_[a-z]*='
#
# Compute the path to the log file based on the command and the options
# Which it puts in _ql_filepath
_ql_logfile_path "$1"
_ql_append=" cat "
if [[ "1" = "$_ql_opt_ts" ]]
then
# if using time stamped lines (_ql_opt_ts, from log-time),
# call _ql_opt_ts which figure out whether to use ilts, ts, or awk
# which it then appends to _ql_append
_ql_ts_cmd
fi

if [[ "$_ql_opt_tee" = "1" ]]
then
_ql_append+=" | tee $(tty)"
fi

# printf "%s\n" "$_ql_append"

if [[ "$_ql_opt_header" = "1" ]]
then
# printf "%s\n" "( echo \"$@\"; \"$@\" ) | bash -c \"$_ql_append\" > \"$_ql_filepath\" 2>&1"
echo "_ql_append"
( echo "$@"; "$@" ) | bash -c "$_ql_append" > "$_ql_filepath" 2>&1
else
"$@" | bash -c "$_ql_append" > "$_ql_filepath" 2>&1
fi
}

log-time(){
# log-time is just like log-plain except for the timestamping
# So, we just set the time stamp option and call log-plain
log-plain --ts "$@"
}

log-view(){
name=$(basename "$1")
view $( newestlog "$name" )
view "$( log-newest "$name" )"
}

lesslog(){
log-less(){
name=$(basename "$1")
less $( newestlog "$name" )
less "$( log-newest "$name" )"
}

greplog(){
log-grep(){
name=$(basename "$1")
shift
grep "$@" $( newestlog "$name" )
grep "$@" "$( log-newest "$name" )"
}

newestlog(){
log-newest(){
name=$(basename "$1")
ls ~/logs/"${name}"* | tail -1
}

0 comments on commit 53167ac

Please sign in to comment.