terraform output not display the outputs

[scott-doyland-burrows]

Terraform Version

Terraform v1.8.2
on linux_amd64
+ provider registry.terraform.io/cloudflare/cloudflare v4.30.0
+ provider registry.terraform.io/hashicorp/aws v5.47.0
+ provider registry.terraform.io/hashicorp/external v2.3.3

Terraform Configuration Files

Relevant parts are:

In a root module:

output "a" {
  value = module.terraform-module-ou.a
}

output "b" {
  value = "from root module"
}

In a submodule:

output "a" {
  value = "from module"
}

Debug Output

I run terraform apply and then a terraform output and see this:

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

a = "from module"
b = "from root module"
doylands@HGRSDELTCG30106:~$ terraform output
╷
│ Warning: No outputs found
│
│ The state file either has no outputs defined, or all the defined outputs are empty. Please define an output in your configuration with the `output` keyword and run `terraform refresh` for it to become
│ available. If you are using interpolation, please verify the interpolated value is not empty. You can use the `terraform console` command to assist.

Expected Behavior

The outputs should be displayed

Actual Behavior

terraform says there are no outputs.

Steps to Reproduce

terraform apply
terraform output

Additional Context

I can see the state file has the outputs:

{
  "version": 4,
  "terraform_version": "1.8.2",
  "serial": 56,
  "lineage": "1cf6b725-b0a4-0b5f-915f-963481c8eca4",
  "outputs": {
    "a": {
      "value": "from module",
      "type": "string"
    },
    "b": {
      "value": "from root module",
      "type": "string"
    }
  },

References

No response

[jbardin]

Hi @scott-doyland-burrows,

Thanks for filing the issue. I'm not able to replicate the behavior using v1.8 or the current main branch, so there must be something unique to your setup

% terraform output
a = "from module"
b = "from root module"

Can you maybe supply a more complete example of the steps you are using? Are you possibly using a different state location for different operations?

Thanks!

[scott-doyland-burrows]

Well, this is so weird.

1. I pulled down the state file from Terraform Cloud and stored it in a local directory, and terraform output worked OK against the local state file.

2. So I tested against another state file that was on Terraform Cloud - and I had the same issue (worked locally, failed in TFC).

3. So I thought it was maybe a Terraform Cloud issue - so setup a new workspace on TFC - and had the same issue.

4. So now I was convinced it was when the state files was on TFC that this was a problem - although this was literally working a few hours before.

5. Here is the really weird thing, look at the screen shot below, two images as the screen had some unrelated text on it so I cut it into two:

6. It literally just started to work - one minute it didn't, the next it did.

7. The ./terra you can see in the command is just a wrapper script, and we have used this for years, and it always works fine, so I know this is not the cause.

8. I tested against my other state file (that I used in number 2 above) and it is also working.

I literally altered nothing. It worked this morning, stopped working, and started again. I know that sounds like I must have altered something, but 100% I did not. I ran the command, it failed, I ran again, it worked.

Never ever had an issue with this until today - where it lasted for maybe 3hrs, then worked again. Any possibility that the state being read from TFC was somehow causing issues for those few hours?

[jbardin]

Thanks for the follow up. I'm not hearing about anything unusual from the TFC side, so it's hard to say what might have happened here. I'll keep an eye out for anything unusual, and let us know if you do find a way to replicate the issue!

Thanks!

[scott-doyland-burrows]

The issue is happening again.

Exactly the same as yesterday.

I am running applies and then terraform output --json to see the output (as it is sensitive) and it is working, and then it just stops working. Literally no difference at all in what I was doing. I tried adding a non-sensitive output as a test and this also fails:

output "a" {
  value = "a"
}

The statefile shows this (cropped the output as it just contains a list of IPs which we class as sensitive (company owned IPs)):

As before, I can see the outputs in the statefile, and I can "pull" the statefile and run terraform output against the local copy and it works fine.

Definitely something wrong here - to me it seems some very weird issue when the statefile is sitting in terraform cloud (or at least running from a statefile on my local machine it works fine).

I switched on TF_LOG=TRACE but it didn't show anything much - I can send the output if needed, just wasn't sure if there was anything in it that would need redacting.

[jbardin]

If you are storing the state in the cloud storage, then there is a different interface used to fetch the root module outputs which does not pull the entire state. I don't see any way the CLI could be dropping these values, but perhaps there's a missed error path somewhere in the client code.

We can look into the CLI portion some more here, but TFC support may be better able to troubleshoot if there is any server-side problem causing this.

[scott-doyland-burrows]

Thanks. I will find out how to raise this as a TFC issue.

[crw]

To report an issue with HCP Terraform or Terraform Enterprise: please email [email protected] or open a new request.

[scott-doyland-burrows]

I have opened a ticket with HCP Terraform.

[brandonc]

@scott-doyland-burrows The underlying issue in HCP Terraform was that the current outputs are empty for an appreciable amount of time between when the latest state is saved and ready to be used by the next terraform plan, and when the latest state is processed and outputs extracted and saved in association with it.

This was patched yesterday in HCP Terraform in a way that is backwards compatible with terraform. Now, during this interim period, the platform will detect that the current state is still being processed and will return a 5XX error which forces terraform to retry the request. The error message is slightly misleading (see screenshot below) because it mentions potential data loss, but nevertheless can be interrupted without consequence. Future versions of terraform will display a more accurate and graceful error message.

[scott-doyland-burrows]

Thanks for the update.

It was more like minutes, sometimes hours of delay, so I do wonder if the issue I reported was the same as the one that has been fixed.

I will be running terraform a few times over the next few days, so will see.