Stacks: Values reference other units

[cgetzen]

The problem I am currently facing with the new Stacks RFC is that it is not possible to pass the output of one unit through to the values of another in the terragrunt.stack.hcl file.

While it is possible to do this in the unit definition, this is not extensible enough because, with stacks, the units will be reused with different inputs.

Proposal

Expand the terragrunt.values.hcl file to use dependencies:

# terragrunt.stack.hcl

unit "dep" {
  source = "modules/imadep"
  path   = "imadep"
}

unit "test" {
  source = "modules/test"
  path   = "test"
  values = {
    a = "a"
    b = unit.dep.output_b
  }
}


# .terragrunt-stack/test/terragrunt.values.hcl (auto-generated)

values {
    a = "a"
    b = dependency.dep.outputs.output_b
}

dependency "dep" {
    config_path = "../imadep"
}

There is a POC of this here: #4056

POC Q&A

Recommendation to use a new terragrunt.dependencies.hcl file
I'd be happy to POC this, but I'm curious how you see the relationship between the dependencies file, values file, and terragrunt file.

My current view is that consolodating all dynamic inputs to units in the terragrunt.values.hcl is optimal:

values {
    a = "a"
    b = dependency.dep.outputs.output_b
}

dependency "dep" {
    config_path = "../imadep"
}

Yes, it is more complex than just putting (static) key-value pairs. However:

1. the complexity has to go somewhere
2. Putting this inside of the values file may be the least complex

Why is this the least complex?

• I think it can clearly define interfaces and boundaries between components:terragrunt.values.hcl can be processed (i.e. dependencies resolved) and only export a single values interface. Processing would be self-contained to the terragrunt.values.hcl file, and terragrunt.hcl would only ever see the values object from it.
• I suspect there may be other constructs that users may want to add in the future. Splitting files by block types may lead to many files with different kinds of relationships between them.

What if two dependencies are named the same thing across these different files? What's the behavior? Can the terragrunt.hcl file make reference to them?
I think the correct choice is to make the terragrunt.values.hcl file hide it's internals and only expose only an interface for retrieving the values. This helps with composability (If there is a dependency in the values file with the same name as a dependency in the terragrunt file, they will not interfere).

Are there any alternative designs that you think would work just as well?
The constraint to keep unit definitions backwards compatible (which I agree with!) reduces the solution space. I haven't been able to come up with another design.

How are mock outputs handled?
I touch on one option for this at the bottom of this comment #3313 (comment), but there are benefits to not implementing this: it trades a simple interface for flexibility. However, with additional changes it may be possible to get the flexibility of mocks with the benefits of a simple interface, and it boils down to where the mocks are defined. Instead of defining them at the "dependency" block, mock outputs could be defined at either or both of (1) the referenced units (the unit would define it's own mock outputs), and (2) in a new mock_inputs block, which is decoupled from the dependency block. I think it makes sense for a module to define mock_outputs, because the author should know what format the output takes. At the same time, I can see the value of defining mock_inputs for certain use-cases (if the dependency's output is dynamic enough where a static mock_output isn't enough).

[yhakbar]

I'm curious how you see the relationship between the dependencies file, values file, and terragrunt file.

The way that we've approached extending the terragrunt.hcl file via the availability of the terragrunt.values.hcl file is that the values file is a simple data store, and a way to conveniently introduce dynamicity into units (in a way that is 100% compatible with everything users use to define units without stacks). Users can, for example chose to use zero terragrunt.stack.hcl files, and place terragrunt.values.hcl files next to terragrunt.hcl files, and they'll be respected regardless (although for now, we're gating them both behind the stacks experiment to make sure we can make breaking adjustments prior to general availability).

there are benefits to not implementing this: it trades a simple interface for flexibility.

Unfortunately, as OpenTofu works today, this is a bit of a non-starter. Mock outputs aren't just a convenience, but a requirement for the ability to run a plan against units with unapplied dependencies. OpenTofu has no way of allowing certain inputs to be unknown at the root module (values can only be unknown within state between resources), so we have to retain the ability to mock outputs. We've discussed the possibility of introducing an outputs block to move the responsibility for defining outputs on the unit exporting them, but that is likely too much of a shift to make prior to 1.0.

I think the correct choice is to make the terragrunt.values.hcl file hide it's internals and only expose only an interface for retrieving the values.

Dependencies aren't just a data fetching mechanism, but also a scheduling mechanism. One unit depending on another will result in scheduling changes in the DAG, so Terragrunt needs a holistic view of all dependencies when it's looking to schedule the run for a unit. It needs to be unambiguous what all the dependencies are for a unit, and conflict resolution needs to be resolved when there are conflicts between different dependencies named the same thing.

---

Generally, the design we've been going for with the new terragrunt.stack.hcl file is to approach from the perspective of the assumption that users will want to use different parts of the design for their projects. That's generally what we see in the wild with Terragrunt: People have different parts that they like, and adopt the features that matter the most to them.

Given that, would having the dependency block in the terragrunt.values.hcl file be the best design, if taken in isolation? Assume that someone manually wrote up a terragrunt.values.hcl file with no usage of a terragrunt.stack.hcl file. Would there be any benefit to configuring the dependency in the terragrunt.values.hcl file instead of directly in the terragrunt.hcl file? If there was any benefit, would users get the most benefit out of having it be in the terragrunt.values.hcl file?

[cgetzen]

@yhakbar Thank you for your engagement -- the terragrunt team is unique in how quickly it responds to the community.

BLUF: I have updated the PR so that mock_inputs can be defined, as well as accounting for the dependency graph. I have left the dependencies in the values file for now.

Mock outputs aren't just a convenience, but a requirement for the ability to run a plan against units with unapplied dependencies.

Got it. Here is the adjusted spec to accommodate mock outputs:

# terragrunt.stack.hcl

unit "dep" {
  source = "modules/imadep"
  path   = "imadep"
  mock_outputs = {
    output_b = "mock_b"
  }
}

unit "test" {
  source = "modules/test"
  path   = "test"
  values = {
    a = "a"
    b = unit.dep.output_b
  }
}


# .terragrunt-stack/test/terragrunt.values.hcl (auto-generated)

values {
    a = "a"
    b = dependency.dep.outputs.output_b
}

dependency "dep" {
    config_path = "../imadep"
    mock_outputs = {
      output_b = "mock_b"
    }
}

I have adjusted the PR to accommodate this.

Dependencies aren't just a data fetching mechanism, but also a scheduling mechanism. One unit depending on another will result in scheduling changes in the DAG, so Terragrunt needs a holistic view of all dependencies when it's looking to schedule the run for a unit.

Thank you for bearing with my naivety about terragrunt. This makes a lot of sense - for now, I have adjusted the PR so that any dependency across terragrunt.hcl or terragrunt.values.hcl is respected in the graph.

conflict resolution needs to be resolved when there are conflicts between different dependencies named the same thing.

I'm thinking about this a bit differently. If we continue to assume that dependencies in terragrunt.hcl cannot be used in terragrunt.values.hcl (and vice-versa), it's OK to have dependencies with the same name across these files.

Do you think this is a reasonable way to approach this?

I would be happy to change the implementation, but I currently believe the best interface is to have a single file contain the dynamic / injected data that the unit uses. Why?

1. There is a clear boundary between terragrunt.values.hcl and terragrunt.hcl: the file. There is a clear interface: the values object. Nothing else is shared from a configuration language perspective (even though the dependency graph covers both files).
2. Adding another file to specify just the dependencies may confuse users: does it apply to terragrunt.hcl? terragrunt.values.hcl? both? What is the benefit of having this file apart from the values file?
3. I (weakly) believe that keeping all of these configurations in the single values file will result in future-proofing, to allow for other kinds of blocks to be added.

Assume that someone manually wrote up a terragrunt.values.hcl file with no usage of a terragrunt.stack.hcl file. Would there be any benefit to configuring the dependency in the terragrunt.values.hcl file instead of directly in the terragrunt.hcl file?

I think the answer is the same to the question "Would there be any benefit to configuring the values in the terragrunt.values.hcl file instead of directly in the terragrunt.hcl file's inputs block?": There would be very little benefit, if any. However, adding dependency blocks to this file doesn't hurt users from creating their own values file (with or without dependencies).

[yhakbar]

Hey @cgetzen ,

Someone asked about this RFC in Discord, and I wanted to make sure I am explicit with where this RFC is:

I think it's a productive conversation, and I want to directly answer your questions above, but based on initial conversations, it does seem to me to be a feature that can wait until after general availability of Stacks as is.

If you haven't had a chance, I recommend reading through this blog post if you haven't to see how we're approaching general availability of Terragrunt Stacks:
https://www.gruntwork.io/blog/the-road-to-1-0-terragrunt-stacks-feature-complete

Note that there are some example repositories in there for best practice Terragrunt configurations (which might change if we adopt something like this), and that all the stack examples contain dependencies, which might be worth looking at:
https://github.com/gruntwork-io/terragrunt-infrastructure-catalog-example/tree/main/examples/terragrunt/stacks

Any change we make as a consequence of this RFC will aim to make configurations like those easier to manage.

Regarding timeline, you can expect a full response to your questions above after Stacks is generally available, but before Terragrunt 1.0 (hopefully well before).

If more members of the community engage with this RFC, it might make it easier to reason about, and the design more robust.

Thank you for your patience.

[yhakbar]

Hey @cgetzen !

More folks have been asking about functionality like this, but it doesn't seem to have risen to the level where it's a must-have for Terragrunt 1.0, so as it currently stands, we're still looking to prioritize this as a feature that becomes available post-1.0 (as nothing we've discussed so far would require a breaking change in how Terragrunt operates).

To respond to some of your responses:

I'm thinking about this a bit differently. If we continue to assume that dependencies in terragrunt.hcl cannot be used in terragrunt.values.hcl (and vice-versa), it's OK to have dependencies with the same name across these files.

I'm not following how this would play into DAG resolution. When trying to determine the correct order of unit runs, how would Terragrunt know which units need to go at a given time without merging the two definitions of dependencies together?

If you had something like this:

# .terragrunt-stack/test/terragrunt.values.hcl (auto-generated)

values {
    a = "a"
    b = dependency.dep.outputs.output_b
}

dependency "dep" {
    config_path = "../imadep"
    mock_outputs = {
      output_b = "mock_b"
    }
}

# .terragrunt-stack/test/terragrunt.hcl (generated from unit definition)

dependency "dep" {
    config_path = "../imadifferentdep"
    mock_outputs = {
      output_b = "mock_b"
    }
}

inputs = {
  b = dependency.dep.outputs.output_b
}

It's important that Terragrunt is able to determine that imadep and imadifferentdep must run before test is able to run (for plans and applies). How would Terragrunt track that these requirements are satisfied otherwise? Furthermore, wouldn't it be confusing to users if the same variable (dependency.dep.outputs.output_b) could mean two different things in the same unit?

There is a clear boundary between terragrunt.values.hcl and terragrunt.hcl

I'm not sure this is true. If you can define a dependency block in either the terragrunt.values.hcl or the terragrunt.hcl file, and they almost work the same way, that feels like a fuzzy boundary. How would you explain to a user how Terragrunt behaves when there is a dependency block in the terragrunt.hcl file and when it is in the terragrunt.values.hcl file?

Note that there's a difference in functionality here, as well. You typically can only use dependency outputs in inputs for terragrunt.hcl files, but exposing them as values would mean that you can use dependency outputs for anything, including locals. This has massive performance implications. Calling tofu output -json is slow, compared to most Terragrunt configuration parsing. Minimizing the number of times dependency blocks have to be evaluated is important for maintaining Terragrunt performance.

Would there be any benefit to configuring the values in the terragrunt.values.hcl file instead of directly in the terragrunt.hcl file's inputs block?

There's a big difference between these two. You can define more than values for inputs in the terragrunt.values.hcl file. The values in that file can be used for the source attribute of the terraform block, it can be used to define the path to units in the dependency block in the terragrunt.hcl file, etc. It can be used to define values for any variable referenced in unit configurations.

[cgetzen]

Thanks for the information! I suspect I'm still operating from a limited viewpoint, and that a new terragrunt.dependency.hcl file is necessary. But for the sake of exploration, I'll go over all the options I see available.

It's nice to see that the community wants this feature! As a quick aside, my ideal state of terragrunt (2.0?) is "no configuration required": We can shard state across modules without any code changes (stacks are a huge step, but there would need to be a solution for defaulting mock outputs, for-loops over units to match terraform functionality, etc). I believe this would substantially lower the barrier to entry for adopting terragrunt. I'm mentioning this because it influences my perspective on the developer interface exposed with each option.

OK, here are the options I see to get stack X-unit references:

A new terragrunt.dependency.hcl file

This is the option you were guiding me towards. Can we flesh this out a bit more? If we assume that this just renders "normal" dependency blocks that can be references in terragrunt.hcl, I'm not sure that it accomplishes dynamic X-unit references.

The underlying problem is how we go from the auto-generated dependency, to being referenced in terragrunt.hcl. If it's a direct reference (no semantic changes to the dependency block being inside of the terragrunt.hcl file), a strong coupling between stack and unit form:

• Does each input that we want to pull from a dynamic dependency need to reference a uniquely named dependency, in order to prevent coupling? This means that we may have duplicate generated dependencies with different names (which is probably fine, although a bit confusing).
• Within the terragrunt.dependency.hcl file, the name of the dependency needs to match the reference to it from terragrunt.hcl. This means that either:
  • The upstream terragrunt.stack.hcl needs to match the exact dependency name (which would have issues if the unit dynamic dependencies are all uniquely named: you could serve at most one X-unit reference per unit)
  • Some translation configuration needs to exist. Maybe something like

# terragrunt.stack.hcl
...

unit "test" {
  source = "modules/test"
  path   = "test"
  values = {
    a = "a"
  }
  dependencies = {
    test_dep_one = unit.dep.output_b
  }
}

# .terragrunt-stack/test/terragrunt.dependency.hcl (auto-generated)
dependency "test_dep_one" {
    config_path = "../imadep"
    mock_outputs = {
      output_b = "mock_b"
    }
}

My biggest issue with this option is that it forces the module/unit author to pre-determine how the inputs are going to be used (the inputs of another module, vs static). It prevents someone from changing a dependency input to a value input. My main argument here is that the stack author, not the unit author, should make this decision.

A secondary issue is that it surfaces what I believe should be hidden implementation details (the name "test_dep_one").

I suspect I am off-base on a lot of this! Let me know where I veered off course here :)

JIT Dependency Graph

Since values can be used inside the path, some dependencies would have to be evaluated while the graph is being created. I could see this causing performance issues with terragrunt graph (although you could add flags to create a partial graph). This option would add a fair amount of complexity, and seems to provide little value (who needs an infrastructure Abstract Factory?).

Limit the use of values using dependencies

This option would prevent dependency-based values from being used anywhere except for the inputs block. This was the initial intention of this RFC, and seems like a reasonable limitation given that terraform often has the same limitations (variables module paths, etc).

How would you explain to a user how Terragrunt behaves when there is a dependency block in the terragrunt.hcl file and when it is in the terragrunt.values.hcl file?

Dependency blocks are scoped to the file that defines them. A terragrunt.values.hcl can define dependency blocks that are referenced in value definitions, like so:

values {
    a = "a"
    b = dependency.dep.outputs.output_b
}

dependency "dep" {
    config_path = "../imadep"
    mock_outputs = {
      output_b = "mock_b"
    }
}

In order to use them in terragrunt.hcl, reference the value that they were assigned to, like so:

inputs = {
  vpc_name = values.b
}

[riccardomassullo]

A similar/extended RFC where also stacks values referencing is taken in account would be very helpful (at least from my point of view)

e.g. I want to pass an output value of a unit to other units, stacks or i want to pass the output of a stack into another stack

# terragrunt.stack.hcl

unit "vpc" {
  source = "modules/vpc"
  path   = "vpc"
}

unit "vpc_endpoints" {
  source = "modules/vpc_endpoints"
  path   = "vpc_pendpoints"
  values = {
    vpc_id = unit.vpc.vpc_id
  }
}

stack "bastion" {
  source = "stacks/bastion"
  path   = "bastion"
  values = {
    vpc_id = unit.vpc.vpc_id
    ...
  }
}

stack "app" {
  source = "stacks/app"
  path   = "app"
  values = {
    vpc_id = unit.vpc.vpc_id
    sg_bastion_id = stack.bastion.sg_id
  }
}

Is this possible in any way with the new units/stack feature? Using only traditional terragrunt units, defining stacks as folders containing units and referencing the units directly works as i wish.
Obviously (or at least in my case), using the output of the entire stack instead of hundreds of small units referenced between them, in a lot of cases, would lead to a much cleaner infra.

[talpa-robin]

I don't know if that RFC already covers that (or should cover that at all) but one scenario that I haven't seen been mentioned is where a single standalone unit (maybe from times before Stacks) needs an output from a unit in a stack. In our current example we've build an EKS stack that includes a couple of units to bootstrap a fully working EKS cluster and now we have an older module which needs the OIDC provider ARN of the EKS cluster as an input.

But independent from "older / newer" code, IMO, there will always be scenarios where you don't want to wrap everything in a stack but need the output from one and there should be a "native" way to reference the outputs of units within a stack.

[anocendi]

I would love to see this capability implemented in terragrunt stacks as well!

Our existing setup mimics units and stacks in a very hacky way. If the capability in this issue is implemented, we should be able to migrate with some small-sized work to become more terragrunt proper.

[andromeda306]

Is there a clear guide on how to implement dependencies in their current state? I'm finding it difficult to differentiate what is a real example vs. what are people proposing with a POC vs. what is in fact a working example but symbolised using generic terminology.

[yhakbar]

@andromeda306
Take a look at these examples.