(feat) Support Preserving Variable Values (#115)

Updates the variable assignment command to support preserving the value
of a variable if one is already declared. The new syntax adds an
optional `?` preceding the `=` to indicate that the assignment should
only occur if the variable does not have a value.

docs/SYNTAX.md

@@ -3,34 +3,42 @@
 This guide will walk you through the template language used to generate dynamic prompts. It covers various features such as variants, wildcards, variables, and parameterized templates.
 
 ## Table of contents
-   * [Variants](#variants)
-      * [Basic Syntax](#basic-syntax)
-      * [Weighting Options](#weighting-options)
-      * [Choosing Multiple Values](#choosing-multiple-values)
-      * [Custom Separator](#custom-separator)
-      * [Range of Options](#range-of-options)
-         * [Omitting Bounds](#omitting-bounds)
-         * [Limitations](#limitations)
-   * [Wildcards](#wildcards)
-      * [Basic Syntax](#basic-syntax-1)
-      * [Wildcards in Variants](#wildcards-in-variants)
-      * [Variants in Wildcards](#variants-in-wildcards)
-      * [Nested Wildcards](#nested-wildcards)
-      * [Resolving Wildcards with Globbing](#resolving-wildcards-with-globbing)
-         * [Basic Syntax](#basic-syntax-2)
-         * [Example](#example)
-      * [File formats](#file-formats)
-         * [Text files](#text-files)
-         * [YAML files](#yaml-files)
-         * [JSON files](#json-files)
-   * [Variables](#variables)
-      * [Immediate Evaluation](#immediate-evaluation)
-      * [Non-immediate Evaluation](#non-immediate-evaluation)
-   * [Parameterized Templates](#parameterized-templates)
-      * [Basic Syntax](#basic-syntax-3)
-      * [Default values](#default-values)
-   * [Whitespace and comments](#whitespace-and-comments)
-   * [Samplers](#samplers)
+- [Syntax Guide](#syntax-guide)
+  - [Table of contents](#table-of-contents)
+  - [Variants](#variants)
+    - [Basic Syntax](#basic-syntax)
+    - [Weighting Options](#weighting-options)
+    - [Choosing Multiple Values](#choosing-multiple-values)
+    - [Custom Separator](#custom-separator)
+    - [Range of Options](#range-of-options)
+      - [Omitting Bounds](#omitting-bounds)
+      - [Limitations](#limitations)
+  - [Wildcards](#wildcards)
+    - [Basic Syntax](#basic-syntax-1)
+    - [Wildcards in Variants](#wildcards-in-variants)
+    - [Variants in Wildcards](#variants-in-wildcards)
+    - [Nested Wildcards](#nested-wildcards)
+    - [Resolving Wildcards with Globbing](#resolving-wildcards-with-globbing)
+      - [Basic Syntax](#basic-syntax-2)
+      - [Example](#example)
+      - [Recursive globbing](#recursive-globbing)
+    - [File formats](#file-formats)
+      - [Text files](#text-files)
+      - [YAML files](#yaml-files)
+        - [Weighted options in YAML](#weighted-options-in-yaml)
+      - [JSON files](#json-files)
+  - [Variables](#variables)
+    - [Immediate Evaluation](#immediate-evaluation)
+    - [Non-immediate Evaluation](#non-immediate-evaluation)
+  - [Parameterized Templates](#parameterized-templates)
+    - [Basic Syntax](#basic-syntax-3)
+    - [Default values](#default-values)
+    - [Preserving Existing Values](#preserving-existing-values)
+  - [Whitespace and comments](#whitespace-and-comments)
+  - [Samplers](#samplers)
+    - [Random Sampler](#random-sampler)
+    - [Combinatorial Sampler](#combinatorial-sampler)
+    - [Cyclical Sampler](#cyclical-sampler)
 
 
 ## Variants
@@ -444,7 +452,7 @@ __season_clothes(season=winter)__
 Note - for now you can only pass a literal string into the template rather than an expression. This syntax will also work
 
 ```
-${season={summer|autumn|winter|spring} __season_clothes__
+${season={summer|autumn|winter|spring}} __season_clothes__
 ```
 
 ### Default values
@@ -457,6 +465,37 @@ In ${season:summer}, I wear ${season:summer} shirts and ${season:summer} trouser
 
 Now if you forget to create the season variable, the prompt will be `In summer, I wear summer shirts and summer trousers`
 
+### Preserving Existing Values
+
+Within a parameterized template, there may be cases where you want to assign a variable instead of providing a default value in order to achieve better consistency across nested parameterized templates. When assigning a variable a value, you can indicate that you do not want to overwrite an existing value  by placing a `?` before the `=` in the assignment.
+
+For instance, given the following example parameterized templates with variables:
+
+```yaml
+examples:
+  prompt:
+    - '${subject?=!{man|woman}} ${weather?=!{sun|rain}} ${drink?=!{__examples/drink__}} a ${subject} standing in the ${weather} drinking ${drink}'
+  drink:
+    - coffee
+    - tea
+  winter:
+    - '${weather=snow} ${drink=hot chocolate} __examples/prompt__'
+```
+
+Then the following prompts would produce results similar to the following:
+
+> `${subject=boy} __examples/prompt__`
+> a boy standing in the rain drinking tea
+
+> `${subject=cowboy} ${weather=sun} ${drink=sasparilla} __examples/prompt__`
+> a cowboy standing in the sun drinking sasparilla
+
+> `__examples/winter__`
+> a woman standing in the snow drinking hot chocolate
+
+> `${subject=boy} ${weather=rain} ${drink=iced tea} __vartest/winter__`
+> a boy standing in the snow drinking hot chocolate
+
 ## Whitespace and comments
 
 As your prompts become more complex, the become harder to read. To prevent creating unreadable and unmaintainable prompts you can use whitespace such as newlines, which will be ignored by the parser. Python-style comments are also supported so that you can annotate your prompt.

src/dynamicprompts/commands/variable_commands.py

@@ -10,6 +10,7 @@ class VariableAssignmentCommand(Command):
     name: str
     value: Command
     immediate: bool
+    overwrite: bool = True
     sampling_method = None
 
 

src/dynamicprompts/parser/parse.py

@@ -240,6 +240,7 @@ def _configure_variable_assignment(
         + OPT_WS
         + var_name("name")
         + OPT_WS
+        + pp.Opt(pp.Literal("?"))("preserve_existing_value")
         + pp.Literal("=")
         + pp.Opt(pp.Literal("!"))("immediate")
         + OPT_WS
@@ -401,6 +402,7 @@ def _parse_variable_assignment_command(
     return VariableAssignmentCommand(
         name=parts["name"],
         value=parts["value"],
+        overwrite=("preserve_existing_value" not in parts),
         immediate=("immediate" in parts),
     )
 

src/dynamicprompts/sampling_context.py

@@ -162,6 +162,8 @@ def process_variable_assignment(
         self,
         command: VariableAssignmentCommand,
     ) -> Command:
+        if not command.overwrite and command.name in self.variables:
+            return self.variables[command.name]
         if command.immediate:
             if isinstance(command.value, LiteralCommand):
                 # Optimization: if the variable assignment is a literal, just use that

tests/parser/test_parser.py

@@ -390,9 +390,15 @@ def test_alternative_wildcard_wrap(self, wildcard_wrap: str, template: str):
         assert variant.values[1].literal == "B"
         assert variant.values[2].wildcard == "some/wildcard"
 
-    @pytest.mark.parametrize("immediate", (False, True))
-    def test_variable_commands(self, immediate: bool):
-        op = "=!" if immediate else "="
+    @pytest.mark.parametrize("immediate", (False, True), ids=("delayed", "immediate"))
+    @pytest.mark.parametrize(
+        "overwrite",
+        (False, True),
+        ids=("preserve existing value", "overwrite existing value"),
+    )
+    def test_variable_commands(self, immediate: bool, overwrite: bool):
+        op = "?" if not overwrite else ""
+        op += "=!" if immediate else "="
         sequence = cast(
             SequenceCommand,
             parse(f"${{animal {op} cat}} the animal is ${{animal:dog}}"),
@@ -402,6 +408,7 @@ def test_variable_commands(self, immediate: bool):
         assert isinstance(ass, VariableAssignmentCommand)
         assert ass.name == "animal"
         assert ass.value == LiteralCommand("cat")
+        assert ass.overwrite == overwrite
         assert ass.immediate == immediate
         acc = sequence[2]
         assert isinstance(acc, VariableAccessCommand)

tests/samplers/test_common.py

@@ -758,6 +758,43 @@ def test_immediate_literal_variable(self, random_sampling_context: SamplingConte
         cmd = parse("${a =! foo}${a}")
         assert str(next(random_sampling_context.generator_from_command(cmd))) == "foo"
 
+    @pytest.mark.parametrize(
+        "prompt, possible_results",
+        [
+            (
+                "${season=summer} ${temp=cold} ${location=north}__drink/beverage__",
+                ("a glass of iced tea", "a glass of iced pop"),
+            ),
+            (
+                "${season=summer} ${temp=cold} ${location=south}__drink/winter/beverage__",
+                ("a mug of hot coffee"),
+            ),
+            (
+                "${season=summer} ${temp=cold}__drink/winter/beverage__",
+                ("a mug of hot tea"),
+            ),
+            (
+                "__drink/summer/beverage__",
+                ("a glass of iced sweet tea", "a glass of iced soda"),
+            ),
+            (
+                "${location=north}__drink/summer/beverage__",
+                ("a glass of iced tea", "a glass of iced pop"),
+            ),
+        ],
+    )
+    def test_preserve_variable(
+        self,
+        random_sampling_context: SamplingContext,
+        prompt: str,
+        possible_results: list[str],
+    ):
+        cmd = parse(prompt)
+        resolved_value = str(
+            next(random_sampling_context.generator_from_command(cmd)),
+        ).strip()
+        assert resolved_value in possible_results
+
     def test_unknown_variable(self, wildcard_manager: WildcardManager):
         ctx1 = SamplingContext(
             default_sampling_method=SamplingMethod.RANDOM,

tests/test_data/wildcards/drink.yaml

@@ -0,0 +1,33 @@
+drink:
+  beverage:
+    - '${season?=!{winter|summer}} ${temp?=!{hot|cold}} ${location?=!{north|south}} a __drink/container/${temp}__ of __drink/temp/${temp}__ __drink/${season}/${location}__'
+  winter:
+    beverage:
+      - '${temp=hot} ${season=winter} ${location?=north} __drink/beverage__'
+
+    north:
+      - tea
+    south:
+      - coffee
+
+  summer:
+    beverage:
+      - '${temp=cold} ${season=summer} ${location?=south} __drink/beverage__'
+    north:
+      - tea
+      - pop
+    south:
+      - sweet tea
+      - soda
+
+  container:
+    hot:
+      - mug
+    cold:
+      - glass
+
+  temp:
+    hot:
+      - hot
+    cold:
+      - iced

tests/wildcard/test_wildcardmanager.py

@@ -145,6 +145,17 @@ def test_hierarchy(wildcard_manager: WildcardManager):
         "clothing",
         "colors-cold",
         "colors-warm",
+        "drink/beverage",
+        "drink/container/cold",
+        "drink/container/hot",
+        "drink/summer/beverage",
+        "drink/summer/north",
+        "drink/summer/south",
+        "drink/winter/beverage",
+        "drink/winter/north",
+        "drink/winter/south",
+        "drink/temp/cold",
+        "drink/temp/hot",
         "flavors/bitter",
         "flavors/sour",
         "flavors/sweet",