updated readme

Author: husixu1

README.md

@@ -2,31 +2,33 @@
 
 BPT - **B**ash **P**ure **T**emplate (Engine)
 
+<p align="center"><img src="res/bpt.svg"></img></div>
+
 **Features:**
 
-- **Pure:** The rendering process is pure, ensuring that the rendered result is fully predictable and consistent across multiple runs.
-    - This is the core feature and also the purpose of this template engine. This feature makes `bpt` suitable for generating a large number of small configuration files without unexpected changes due to an environment variable change or an external file change when re-generating config files.
-    - A fingerprinting function is provided to digest all the inputs, including files, variables, and the engine itself.
+- **Pure:** The rendering process resembles a [pure function][pure_function], ensuring that the rendered results are fully predictable and consistent across multiple runs given the same inputs. It also produces no side-effects other than the generated script or the render result.
+    - This feature makes `bpt` suitable for generating a large number of small configuration files without unexpected changes due to an environment variable change or an external file change when re-generating config files.
+    - A fingerprinting function is provided to digest all the inputs, including template files, variables, and the engine itself.
     - Functionalities are also provided to collect all the variables and includes used in a template.
-- **Minimal dependency:** `bpt` requires only minimal external programs in addition to bash itself. All the external programs required are within coreutils, which is most likely bundled with the Linux installation.
+- **Minimal dependency:** `bpt` requires only a minimal set of external programs in addition to bash. All the external programs required are in coreutils, which is most likely bundled with your Linux installation.
     - `bash >= 5`
-    - `md5sum`, `sort`, `uniq`, and `cat`.
+    - coreutils: `md5sum`, `sort`, `uniq`, `cat`, `mktemp`, ...
 - **Single bash script:** The `bpt.sh` script contains everything you need.
     - It can be called directly or used as a library by sourcing it in another bash script.
     - It works out of the box.
-- **A little bit of extra functionalities:** While a simple template engine that only replaces `{{var}}`s can also be easily made pure, the functionalities is somewhat limited. BPT provides a little bit extra that makes life easier.
+- **A little bit of extra functionalities:** While a simple template engine that only replaces `{{var}}`s can also be easily made pure, their functionalities are somewhat limited. BPT provides a little bit extra that makes life easier.
     - Delimiters can be replaced.
-    - Branching, looping, and basic boolean operations are supported.
+    - Branching, list iteration, and basic boolean operations are supported.
     - Recursive inclusion of templates is also supported.
     - Refer to [Writing Templates](#writing-templates) for more details.
 
 **Deficiencies:**
 
-- **Slow:** Although optimization has been made, a LALR(1) parser implemented in bash is still quite slow. It is not recommended to use `bpt` for large and complex templates such as an HTML templating engine.
+- **Slow:** Although optimizations have been made, a LALR(1) parser implemented in bash is still quite slow. It is not recommended to use `bpt` for large and complex templates.
     - The startup overhead is around 16ms.
     - Rendering a template consisting of 100 variables takes around 75ms.
     - Rendering a template consisting of 1000 variables takes around 900ms, with most of the time spent on reduction.
-    - Refer to [#Benchmarking](#benchmarking) for more details.
+    - Refer to [#Benchmarks](#benchmarks) for more details.
 
 # Installation & Usages
 
@@ -38,7 +40,7 @@ All you need is `bpt.sh`. Place it anywhere you'd like, then `chmod +x bpt.sh`.
 var1=a var2=b ./bpt.sh ge template.tpl     # Render the template
 var1=a var2=b ./bpt.sh f template.tpl      # Fingerprint the template
 ./bpt.sh g template.tpl > out.sh           # Generate reusable script
-./bpt.sh g -l '<<' -r '>>' ge template.tpl # Customize delimiters
+./bpt.sh g -l '<<' -r '>>' template.tpl    # Customize delimiters
 ./bpt.sh cv template.tpl                   # Collect variables
 ./bpt.sh ci template.tpl                   # Collect includes
 ./bpt.sh -h                                # Print help
@@ -54,15 +56,16 @@ mapfile -t vars <<<"$(bpt.main cv template.tpl)"
 mapfile -t includes <<<"$(bpt.main ci template.tpl)"
 script="$(bpt.main g template.tpl)"
 render_result="$(bpt.main ge template.tpl)"
+fingerprint="$(bpt.main f template.tpl)"
 ```
 
 # Writing Templates
 
-BPT employs a straightforward grammar whereby any content beyond the top-level delimiters is treated as strings, whereas anything enclosed by delimiters is considered either a string or a keyword. Notably, any string located within these delimiters must be enclosed in single or double quotes.
+BPT employs a straightforward grammar whereby any content beyond the top-level delimiters is treated as strings, whereas anything enclosed by delimiters is considered either a string, an identifier, or a keyword. Notably, any string located within these delimiters must be enclosed in single or double quotes.
 
 ## Examples
 
-### Basic Variable Replacement
+### Variable Replacement
 
 ```
 {{var}}
@@ -72,9 +75,17 @@ BPT employs a straightforward grammar whereby any content beyond the top-level d
 {{var and {{var2}}}}  # Use {{var2}} if var is not empty
 ```
 
+Bash internal variables can also be used:
+
+```
+{{RANDOM}}, {{BASH_VERSION}}, {{HOSTNAME}}, {{PWD}}, ...
+```
+
+**Note:** It's obvious that using non-deterministic variables such as `{{RANDOM}}` or `{{SRANDOM}}` results in impure fingerprinting/rendering processes. `$RANDOM` can produce varying results in two subsequent evaluations. It is thus recommended to use `rand=$RANDOM ./bpt ge ...` and use `{{rand}}` in templates to let the `fingerprint` command correctly capture the non-determinism of `$RANDOM`.
+
 ### Include Another Template
 
-All include paths are relative to where the script is called from (`$PWD`).
+All include paths are relative to where the script is called from (`$PWD`). Includes are processed recursively.
 
 ```
 {{include: "another.tpl"}}
@@ -103,13 +114,20 @@ All include paths are relative to where the script is called from (`$PWD`).
 {{if "abc" == "def": "yes"}}
 {{if "abc" != "def": "no"}}
 {{if ! "abc" == "def": "yes"}}
+{{if "abc" =~ "a.*": "yes"}}
 
 # Combined comparison
 {{if "a" or "": "yes"}}
 {{if ("1" -lt "2" or "") and (("a" < "b") or "abc"): "yes"}}
+
+# Shorthands
+{{""}}                    # 'false'
+{{"a"}}                   # 'true'
+{{"a": "yes"}}            # 'yes'
+{{{{var}}: "yes": "no"}}  # 'yes' if {{var}} is not empty, otherwise 'no'
 ```
 
-### Looping
+### Iterate a List
 
 ```
 {{for var in "a" "b" "c": {{var}}","}}
@@ -118,13 +136,23 @@ All include paths are relative to where the script is called from (`$PWD`).
 
 ### Builtin Functions
 
-Currently we only support `seq`, `len` and `quote`. The `seq` builtin is a wrapper of the `seq` executable in coreutils. The other two are implemented in pure bash.
+Currently we only support `seq`, `len`, `quote`, `cat` and `split`. The `seq` builtin is a wrapper of the `seq` executable in coreutils. The others are implemented in pure bash.
+
+```
+{{len: "abc"}}                                  # 3
+{{seq: "3"}}                                    # 1 2 3
+{{seq: "1" "3" "10"}}                           # 1 4 7 10
+```
 
 ```
-{{len: "abc"}}              # 3
-{{seq: "3"}}                # 1 2 3
-{{seq: "1" "3" "10"}}'      # 1 4 7 10
-{{quote: {{seq: "3"}}}}     # 123
+{{quote: "2 3" "3" }}                           # 2 33
+{{for i in {{quote: "2 3" "3" }}: {{i}}","}}    # 2 33,
+
+{{cat: "2 3" "3" }}                             # 2 33
+{{for i in {{cat: "2 3" "3" }}: {{i}}","}}      # 2,33,
+
+{{split: "2 3" "3" }}                           # 2 3 3
+{{for i in {{split: "2 3" "3" }}: {{i}}","}}    # 2,3,3,
 ```
 
 ### Mixture of the above
@@ -148,7 +176,8 @@ obj1=Pen obj2=Apple obj3=Pineapple ./bpt.sh ge <<-EOF
 }}
 
 EOF
-
+```
+```
 I have a Pen, I have a Apple,
 Uh! Apple-Pen,
 I have a Pen, I have Pineapple,
@@ -166,8 +195,11 @@ For reference, the complete BNF of bpt is as follows:
 ```
 DOC     -> DOC STMT
          | .
-STMT    -> IF | FORIN | INCLUDE | BUILTIN | VAR | STR .
-IF      -> ld if BOOLS cl DOC ELIF ELSE rd .
+STMT    -> IF | FORIN | INCLUDE | BUILTIN | QUOTE | VAR | STR .
+IF      -> ld if BOOLS cl DOC ELIF ELSE rd
+         | ld BOOLS cl DOC cl DOC rd
+         | ld BOOLS cl DOC rd
+         | ld BOOLS rd .
 ELIF    -> ELIF elif BOOLS cl DOC
          | .
 ELSE    -> else cl DOC
@@ -188,20 +220,22 @@ BOOL    -> ARGS BOP ARGS
 FORIN   -> ld for ID in ARGS cl DOC rd .
 INCLUDE -> ld include cl STR rd .
 BUILTIN -> ld ID cl ARGS rd .
+QUOTE   -> ld quote cl ARGS rd .
 ARGS    -> ARGS STMT
          | STMT .
 VAR     -> ld ID rd
          | ld ID or VAR rd
          | ld ID or STR rd
          | ld ID and VAR rd
          | ld ID and STR rd .
-BOP     -> ne | eq | gt | lt | ge | le | strgt | strlt | streq | strne .
+BOP     -> ne     | eq    | gt    | lt    | ge    | le
+         | strne  | streq | strgt | strlt | strcm .
 UOP     -> ex .
 ID      -> id .
 STR     -> str .
 ```
 
-And token mappings:
+Token mappings:
 
 ```
 ld    -> left delimiter, default {{
@@ -213,15 +247,13 @@ id    -> [[:alpha:]_][[:alnum:]_]*
 str   -> Everything outside the toplevel ld...rd or inside "..." or '...' in ld...rd
 ```
 
-# Benchmarking
+# Benchmarks
 
 Comparisons were made among some pure-bash template engines that work out of the box in two different settings:
 
 - Rendering a simple plain text "aaa": This measures the startup overhead of the engine.
 - Rendering 1000 variables `${var}`/`{{var}}`/`<%var%>` whose content is set to `a`: This measures the variable render performance.
 
-We have set the plain bash processing/variable replacement as the baseline. Each test is run 12 times, with the first 2 times as warmups. You can find more details in the benchmark.sh file. If you want to run the benchmarks yourself, you'll need to install [hyperfine][hyperfine].
-
 The plain bash processing/variable replacement was set as the baseline. Each test was run 12 times, with the first 2 times as warmups. More details can be found in the [benchmark.sh](benchmark/benchmark.sh) file. If you want to run the benchmarks yourself, [hyperfine][hyperfine] needs to be installed.
 
 The following results were obtained with Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz:
@@ -248,7 +280,7 @@ The following results were obtained with Intel(R) Core(TM) i9-9900K CPU @ 3.60GH
 | `var=a ./mo 1000vars.tpl` | 3200.0 ± 92.6 | 3106.5 | 3405.4 | 1885.37 ± 1740.78 |
 | `var=a ./bpt.sh ge 1000vars.tpl` | 895.1 ± 14.8 | 886.6 | 935.6 | 527.36 ± 486.76 |
 
-Engines that do complete parsing ([mo][mo], [bash-tpl][bash-tpl] and [bpt][bpt]) are slower by orders of magitude than engines that only do pattern matching and replacement (plain bash, [shtpl][shtpl] and [renderest][renderest]). A simple profiling of bpt shows that most of its time are spent on parsing (compared to lexing and evaluating the output).
+Engines that do complete parsing ([mo][mo], [bash-tpl][bash-tpl] and [bpt][bpt]) are slower by orders of magitude than engines that only do pattern matching and replacement (plain bash, [shtpl][shtpl] and [renderest][renderest]). A simple profiling of bpt shows that most of its time are spent on the parsing part (time spent on lexing and evaluating the output is relatively negligible).
 
 # Contributing
 
@@ -267,6 +299,7 @@ Adding functionalities shall not conflict with or break existing features.
 MIT. See the [LICENSE](LICENSE) file.
 
 <!-- References --->
+[pure_function]:https://en.wikipedia.org/wiki/Pure_function
 [bash_unit]:https://github.com/pgrange/bash_unit
 [grammophone]:https://github.com/mdaines/grammophone
 [hyperfine]:https://github.com/sharkdp/hyperfine