Cost Estimation Regression in Azure Terraform IAC: Prices Default to $0 from v0.10.32 Onwards

[seksham]

Description

After updating Infracost to version v0.10.32 or above, the cost estimates for all Azure Terraform resources are being reported as $0, despite having valid cost estimations in previous versions.

Steps to Reproduce

1. Run infracost breakdown --path . --show-skipped --fields all --terraform-var 'prefix=abcd' on the Azure Terraform project viya4-iac-azure.
2. Observe that prior to version v0.10.32, cost estimations are reported correctly.
3. After updating to version v0.10.32 or above, the cost estimations for the same resources are reported as $0.

Expected Behavior

The cost estimation tool should provide a breakdown of costs associated with the Azure resources defined in the Terraform configuration.

Actual Behavior

All resources are being reported with a cost of $0.00 from Infracost version v0.10.32 onwards.

Additional Context

Here is a comparison of outputs:

• Price before v0.10.32:

infracost breakdown --path . --show-skipped --fields all --terraform-var 'prefix=abcd'
2024-03-19T23:45:27+05:30 INF Enabled policies V2
2024-03-19T23:45:27+05:30 INF Enabled tag policies
Evaluating Terraform directory at .
✔ Downloading Terraform modules
✔ Evaluating Terraform directory
⠋ Retrieving cloud prices to calculate costs 2024-03-19T23:45:29+05:30 WRN No products found for module.nfs[0].azurerm_managed_disk.vm_data_disk[2] Disk operations, using 0.00
2024-03-19T23:45:29+05:30 WRN No products found for module.nfs[0].azurerm_managed_disk.vm_data_disk[0] Disk operations, using 0.00
2024-03-19T23:45:29+05:30 WRN No products found for os_disk Disk operations, using 0.00
2024-03-19T23:45:29+05:30 WRN No products found for module.nfs[0].azurerm_managed_disk.vm_data_disk[3] Disk operations, using 0.00
2024-03-19T23:45:29+05:30 WRN No products found for module.nfs[0].azurerm_managed_disk.vm_data_disk[1] Disk operations, using 0.00
2024-03-19T23:45:29+05:30 WRN No products found for os_disk Disk operations, using 0.00
✔ Retrieving cloud prices to calculate costs

2024-03-19T23:45:29+05:30 INF Hiding cost with no usage: module.node_pools["cas"].azurerm_kubernetes_cluster_node_pool.autoscale_node_pool[0] 'Instance usage (Linux, pay as you go, Standard_E16s_v3)'
2024-03-19T23:45:29+05:30 INF Hiding cost with no usage: module.node_pools["cas"].azurerm_kubernetes_cluster_node_pool.autoscale_node_pool[0].os_disk 'Storage (P15, LRS)'
2024-03-19T23:45:29+05:30 INF Hiding resource with no usage: module.node_pools["cas"].azurerm_kubernetes_cluster_node_pool.autoscale_node_pool[0].os_disk
2024-03-19T23:45:29+05:30 INF Hiding resource with no usage: module.node_pools["cas"].azurerm_kubernetes_cluster_node_pool.autoscale_node_pool[0]
2024-03-19T23:45:29+05:30 INF Hiding cost with no usage: module.node_pools["stateful"].azurerm_kubernetes_cluster_node_pool.autoscale_node_pool[0] 'Instance usage (Linux, pay as you go, Standard_D8s_v3)'
2024-03-19T23:45:29+05:30 INF Hiding cost with no usage: module.node_pools["stateful"].azurerm_kubernetes_cluster_node_pool.autoscale_node_pool[0].os_disk 'Storage (P15, LRS)'
2024-03-19T23:45:29+05:30 INF Hiding resource with no usage: module.node_pools["stateful"].azurerm_kubernetes_cluster_node_pool.autoscale_node_pool[0].os_disk
2024-03-19T23:45:29+05:30 INF Hiding resource with no usage: module.node_pools["stateful"].azurerm_kubernetes_cluster_node_pool.autoscale_node_pool[0]
2024-03-19T23:45:29+05:30 INF Hiding cost with no usage: module.node_pools["stateless"].azurerm_kubernetes_cluster_node_pool.autoscale_node_pool[0] 'Instance usage (Linux, pay as you go, Standard_D16s_v3)'
2024-03-19T23:45:29+05:30 INF Hiding cost with no usage: module.node_pools["stateless"].azurerm_kubernetes_cluster_node_pool.autoscale_node_pool[0].os_disk 'Storage (P15, LRS)'
2024-03-19T23:45:29+05:30 INF Hiding resource with no usage: module.node_pools["stateless"].azurerm_kubernetes_cluster_node_pool.autoscale_node_pool[0].os_disk
2024-03-19T23:45:29+05:30 INF Hiding resource with no usage: module.node_pools["stateless"].azurerm_kubernetes_cluster_node_pool.autoscale_node_pool[0]
Project: project_672bf4gt

Name Price Monthly Qty Unit Hourly Cost Monthly Cost

module.aks.azurerm_kubernetes_cluster.aks
├─ default_node_pool
│ ├─ Instance usage (Linux, pay as you go, Standard_D8s_v4) $0.38 730 hours $0.38 $280.32
│ └─ os_disk
│ └─ Storage (P10, LRS) $19.71 1 months $0.03 $19.71
└─ Load Balancer
└─ Data processed Monthly cost depends on usage: $0.005 per GB

module.jump[0].azurerm_linux_virtual_machine.vm
├─ Instance usage (Linux, pay as you go, Standard_B2s) $0.0416 730 hours $0.04 $30.37
└─ os_disk
├─ Storage (S6, LRS) $3.01 1 months $0.00 $3.01
└─ Disk operations Monthly cost depends on usage: $0.00 per 10k operations

module.jump[0].azurerm_public_ip.vm_ip[0]
└─ IP address (static) $0.0036 730 hours $0.00 $2.63

module.nfs[0].azurerm_linux_virtual_machine.vm
├─ Instance usage (Linux, pay as you go, Standard_D8s_v4) $0.38 730 hours $0.38 $280.32
└─ os_disk
├─ Storage (S6, LRS) $3.01 1 months $0.00 $3.01
└─ Disk operations Monthly cost depends on usage: $0.00 per 10k operations

module.nfs[0].azurerm_managed_disk.vm_data_disk[0]
├─ Storage (S10, LRS) $5.89 1 months $0.01 $5.89
└─ Disk operations Monthly cost depends on usage: $0.00 per 10k operations

module.nfs[0].azurerm_managed_disk.vm_data_disk[1]
├─ Storage (S10, LRS) $5.89 1 months $0.01 $5.89
└─ Disk operations Monthly cost depends on usage: $0.00 per 10k operations

module.nfs[0].azurerm_managed_disk.vm_data_disk[2]
├─ Storage (S10, LRS) $5.89 1 months $0.01 $5.89
└─ Disk operations Monthly cost depends on usage: $0.00 per 10k operations

module.nfs[0].azurerm_managed_disk.vm_data_disk[3]
├─ Storage (S10, LRS) $5.89 1 months $0.01 $5.89
└─ Disk operations Monthly cost depends on usage: $0.00 per 10k operations

module.node_pools["compute"].azurerm_kubernetes_cluster_node_pool.autoscale_node_pool[0]
├─ Instance usage (Linux, pay as you go, Standard_E16s_v3) $1.01 730 hours $1.01 $735.84
└─ os_disk
└─ Storage (P15, LRS) $38.01 1 months $0.05 $38.01

OVERALL TOTAL $1,416.77
──────────────────────────────────
26 cloud resources were detected:
∙ 12 were estimated, 11 of which include usage-based costs, see https://infracost.io/usage-file
∙ 14 were free:
∙ 4 x azurerm_virtual_machine_data_disk_attachment
∙ 2 x azurerm_network_interface
∙ 2 x azurerm_network_interface_security_group_association
∙ 2 x azurerm_subnet
∙ 1 x azurerm_network_security_group
∙ 1 x azurerm_resource_group
∙ 1 x azurerm_user_assigned_identity
∙ 1 x azurerm_virtual_network

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Project ┃ Monthly cost ┃
┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━┫
┃ project_672bf4gt ┃ $1,417 ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━┛

• Price v0.10.32 onwards:

infracost2 breakdown --path . --show-skipped --fields all --terraform-var 'prefix=abcd'
Evaluating Terraform directory at .
✔ Downloading Terraform modules
✔ Evaluating Terraform directory
Warning: Input values were not provided for following Terraform variables: "variable.subscription_id", "variable.tenant_id". Use --terraform-var-file or --terraform-var to specify them.
✔ Retrieving cloud prices to calculate costs

Project: project_raae67x4

Name Price Monthly Qty Unit Hourly Cost Monthly Cost

module.aks.azurerm_kubernetes_cluster.aks
├─ default_node_pool
│ ├─ Instance usage (Linux, pay as you go, Standard_D8s_v4) $0.00 1,460 hours $0.00 $0.00
│ └─ os_disk
│ └─ Storage (P10, LRS) $0.00 2 months $0.00 $0.00
└─ Load Balancer
└─ Data processed Monthly cost depends on usage: $0.005 per GB

module.flex_postgresql["default"].azurerm_postgresql_flexible_server.flexpsql
├─ Compute (GP_Standard_D16s_v3) $0.00 730 hours $0.00 $0.00
├─ Storage $0.00 64 GB $0.00 $0.00
└─ Additional backup storage Monthly cost depends on usage: $0.00 per GB

module.jump[0].azurerm_linux_virtual_machine.vm
├─ Instance usage (Linux, pay as you go, Standard_B2s) $0.00 730 hours $0.00 $0.00
└─ os_disk
├─ Storage (S6, LRS) $0.00 1 months $0.00 $0.00
└─ Disk operations Monthly cost depends on usage: $0.00 per 10k operations

module.jump[0].azurerm_public_ip.vm_ip[0]
└─ IP address (static) $0.00 730 hours $0.00 $0.00

module.nfs[0].azurerm_linux_virtual_machine.vm
├─ Instance usage (Linux, pay as you go, Standard_D8s_v4) $0.00 730 hours $0.00 $0.00
└─ os_disk
├─ Storage (S6, LRS) $0.00 1 months $0.00 $0.00
└─ Disk operations Monthly cost depends on usage: $0.00 per 10k operations

module.nfs[0].azurerm_managed_disk.vm_data_disk[0]
├─ Storage (S10, LRS) $0.00 1 months $0.00 $0.00
└─ Disk operations Monthly cost depends on usage: $0.00 per 10k operations

module.nfs[0].azurerm_managed_disk.vm_data_disk[1]
├─ Storage (S10, LRS) $0.00 1 months $0.00 $0.00
└─ Disk operations Monthly cost depends on usage: $0.00 per 10k operations

module.nfs[0].azurerm_managed_disk.vm_data_disk[2]
├─ Storage (S10, LRS) $0.00 1 months $0.00 $0.00
└─ Disk operations Monthly cost depends on usage: $0.00 per 10k operations

module.nfs[0].azurerm_managed_disk.vm_data_disk[3]
├─ Storage (S10, LRS) $0.00 1 months $0.00 $0.00
└─ Disk operations Monthly cost depends on usage: $0.00 per 10k operations

module.node_pools["cas"].azurerm_kubernetes_cluster_node_pool.static_node_pool[0]
├─ Instance usage (Linux, pay as you go, Standard_E16s_v3) $0.00 730 hours $0.00 $0.00
└─ os_disk
└─ Storage (P15, LRS) $0.00 1 months $0.00 $0.00

module.node_pools["compute"].azurerm_kubernetes_cluster_node_pool.static_node_pool[0]
├─ Instance usage (Linux, pay as you go, Standard_E16s_v3) $0.00 730 hours $0.00 $0.00
└─ os_disk
└─ Storage (P15, LRS) $0.00 1 months $0.00 $0.00

module.node_pools["stateful"].azurerm_kubernetes_cluster_node_pool.autoscale_node_pool[0]
├─ Instance usage (Linux, pay as you go, Standard_D8s_v3) $0.00 730 hours $0.00 $0.00
└─ os_disk
└─ Storage (P15, LRS) $0.00 1 months $0.00 $0.00

module.node_pools["stateless"].azurerm_kubernetes_cluster_node_pool.autoscale_node_pool[0]
├─ Instance usage (Linux, pay as you go, Standard_D16s_v3) $0.00 730 hours $0.00 $0.00
└─ os_disk
└─ Storage (P15, LRS) $0.00 1 months $0.00 $0.00

OVERALL TOTAL $0.00
──────────────────────────────────
24 cloud resources were detected:
∙ 13 were estimated, 12 of which include usage-based costs, see https://infracost.io/usage-file
∙ 11 were free:
∙ 4 x azurerm_virtual_machine_data_disk_attachment
∙ 2 x azurerm_network_interface
∙ 2 x azurerm_network_interface_security_group_association
∙ 1 x azurerm_postgresql_flexible_server_configuration
∙ 1 x azurerm_postgresql_flexible_server_firewall_rule
∙ 1 x azurerm_proximity_placement_group

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Project ┃ Monthly cost ┃
┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━┫
┃ project_raae67x4 ┃ $0.00

The output clearly shows that, starting with version v0.10.32, the cost estimates for resources that previously had valid costs are now being reported as zero.

Environment

• Infracost version: v0.10.32 and above
• Terraform version: v1.5.7
• OS: MAC

---

Please let me know if you need any more details or logs to troubleshoot this issue.

[aliscott]

@seksham thanks for reporting 🙏 I need to discuss more with the team what we do to resolve this. It seems to be something that changed with how we autodetect variable files. In the meantime adding a --terraform-var 'location=eastus' flag should resolve this for you.

[seksham]

Thanks, @aliscott , for your input. After specifying location=eastus, I indeed began receiving pricing data. However, the calculated prices appear to be inaccurate. It includes costs for resources that shouldn't incur charges, notably module.node_pools["cas"].azurerm_kubernetes_cluster_node_pool.static_node_pool[0]. According to the Terraform plan, node_count for this resource is 0, indicating it shouldn't be charged. Versions prior to 10.32 displayed the correct pricing.

Here's the output I received:

infracost2 breakdown --path . --show-skipped --fields all --terraform-var 'prefix=abcd' --terraform-var 'location=eastus'
Evaluating Terraform directory at .
✔ Downloading Terraform modules
✔ Evaluating Terraform directory
✔ Retrieving cloud prices to calculate costs

Project: project_672bf4gt

Name Price Monthly Qty Unit Hourly Cost Monthly Cost

module.aks.azurerm_kubernetes_cluster.aks
├─ default_node_pool
│ ├─ Instance usage (Linux, pay as you go, Standard_D8s_v4) $0.38 1,460 hours $0.77 $560.64
│ └─ os_disk
│ └─ Storage (P10, LRS) $19.71 2 months $0.05 $39.42
└─ Load Balancer
└─ Data processed Monthly cost depends on usage: $0.005 per GB

module.flex_postgresql["default"].azurerm_postgresql_flexible_server.flexpsql
├─ Compute (GP_Standard_D16s_v3) $1.37 730 hours $1.37 $998.64
├─ Storage $0.12 64 GB $0.01 $7.36
└─ Additional backup storage Monthly cost depends on usage: $0.095 per GB

module.jump[0].azurerm_linux_virtual_machine.vm
├─ Instance usage (Linux, pay as you go, Standard_B2s) $0.0416 730 hours $0.04 $30.37
└─ os_disk
├─ Storage (S6, LRS) $3.01 1 months $0.00 $3.01
└─ Disk operations Monthly cost depends on usage: $0.0005 per 10k operations

module.jump[0].azurerm_public_ip.vm_ip[0]
└─ IP address (static) $0.0036 730 hours $0.00 $2.63

module.nfs[0].azurerm_linux_virtual_machine.vm
├─ Instance usage (Linux, pay as you go, Standard_D8s_v4) $0.38 730 hours $0.38 $280.32
└─ os_disk
├─ Storage (S6, LRS) $3.01 1 months $0.00 $3.01
└─ Disk operations Monthly cost depends on usage: $0.0005 per 10k operations

module.nfs[0].azurerm_managed_disk.vm_data_disk[0]
├─ Storage (S10, LRS) $5.89 1 months $0.01 $5.89
└─ Disk operations Monthly cost depends on usage: $0.0005 per 10k operations

module.nfs[0].azurerm_managed_disk.vm_data_disk[1]
├─ Storage (S10, LRS) $5.89 1 months $0.01 $5.89
└─ Disk operations Monthly cost depends on usage: $0.0005 per 10k operations

module.nfs[0].azurerm_managed_disk.vm_data_disk[2]
├─ Storage (S10, LRS) $5.89 1 months $0.01 $5.89
└─ Disk operations Monthly cost depends on usage: $0.0005 per 10k operations

module.nfs[0].azurerm_managed_disk.vm_data_disk[3]
├─ Storage (S10, LRS) $5.89 1 months $0.01 $5.89
└─ Disk operations Monthly cost depends on usage: $0.0005 per 10k operations

module.node_pools["cas"].azurerm_kubernetes_cluster_node_pool.static_node_pool[0]
├─ Instance usage (Linux, pay as you go, Standard_E16s_v3) $1.01 730 hours $1.01 $735.84
└─ os_disk
└─ Storage (P15, LRS) $38.01 1 months $0.05 $38.01

module.node_pools["compute"].azurerm_kubernetes_cluster_node_pool.static_node_pool[0]
├─ Instance usage (Linux, pay as you go, Standard_E16s_v3) $1.01 730 hours $1.01 $735.84
└─ os_disk
└─ Storage (P15, LRS) $38.01 1 months $0.05 $38.01

module.node_pools["stateful"].azurerm_kubernetes_cluster_node_pool.autoscale_node_pool[0]
├─ Instance usage (Linux, pay as you go, Standard_D8s_v3) $0.38 730 hours $0.38 $280.32
└─ os_disk
└─ Storage (P15, LRS) $38.01 1 months $0.05 $38.01

module.node_pools["stateless"].azurerm_kubernetes_cluster_node_pool.autoscale_node_pool[0]
├─ Instance usage (Linux, pay as you go, Standard_D16s_v3) $0.77 730 hours $0.77 $560.64
└─ os_disk
└─ Storage (P15, LRS) $38.01 1 months $0.05 $38.01

OVERALL TOTAL $4,413.63
──────────────────────────────────
24 cloud resources were detected:
∙ 13 were estimated, 12 of which include usage-based costs, see https://infracost.io/usage-file
∙ 11 were free:
∙ 4 x azurerm_virtual_machine_data_disk_attachment
∙ 2 x azurerm_network_interface
∙ 2 x azurerm_network_interface_security_group_association
∙ 1 x azurerm_postgresql_flexible_server_configuration
∙ 1 x azurerm_postgresql_flexible_server_firewall_rule
∙ 1 x azurerm_proximity_placement_group

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Project ┃ Monthly cost ┃
┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━┫
┃ project_672bf4gt ┃ $4,414 ┃

This includes pricing for resources that, due to their configuration, should not contribute to the overall cost, contradicting expectations set by the Terraform plan.
Correct total cost should be - $1,417

[aliscott]

@seksham this looks like the same issue. The problem is we're detecting variable values from the tfvar files in the repo and some of them have min_node = 1.

I can work round this by manually specifying which variables to use for the project, e.g:

infracost breakdown --terraform-var-file=examples/sample-input-minimal.tfvars --terraform-var="location=eastus"

We will work out what we do about this case.

[seksham]

@aliscott
I ran the above command, but I still see the price for node_pools with min_nodes = 0. The file sample-input-minimal.tfvars has 2 node pools - cas and generic, both with min_nodes set to 0, yet I still see the price for both.

I tested a simple independent Terraform configuration using just the azurerm_kubernetes_cluster_node_pool resource, with no tfvars in the project. Even for those, we are seeing the price for a 0 node_count.

provider "azurerm" {
  features {}
}

resource "azurerm_kubernetes_cluster_node_pool" "autoscale_node_pool" {
  name                  = "cas"
  kubernetes_cluster_id = "" # Make sure to replace this with your AKS cluster's ID
  vm_size               = "Standard_E16s_v3"
  enable_auto_scaling   = true
  min_count             = 0
  max_count             = 5
  node_count            = 0
  os_disk_size_gb       = 200
  os_disk_type          = "Managed"
  os_type               = "Linux"
  max_pods              = 110
  mode                  = "User"
  orchestrator_version  = "1.27"
  priority              = "Regular"
  scale_down_mode       = "Delete"
  spot_max_price        = -1
  ultra_ssd_enabled     = false
  zones                 = ["1"]

  node_labels = {
    "workload.sas.com/class" = "cas"
  }

  node_taints = [
    "workload.sas.com/class=cas:NoSchedule",
  ]
}

# Note: Replace var.kubernetes_cluster_id with your actual AKS cluster ID.
# You can define it in your variables file (variables.tf) or pass it directly.

Output -

Evaluating Terraform directory at .
✔ Downloading Terraform modules
✔ Evaluating Terraform directory
✔ Retrieving cloud prices to calculate costs

Project: project_zzipd67f

Name Price Monthly Qty Unit Hourly Cost Monthly Cost

azurerm_kubernetes_cluster_node_pool.autoscale_node_pool
├─ Instance usage (Linux, pay as you go, Standard_E16s_v3) $1.01 730 hours $1.01 $735.84
└─ os_disk
└─ Storage (P15, LRS) $38.01 1 months $0.05 $38.01

OVERALL TOTAL $773.85
──────────────────────────────────
1 cloud resource was detected:
∙ 1 was estimated, it includes usage-based costs, see https://infracost.io/usage-file

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Project ┃ Monthly cost ┃
┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━┫
┃ project_zzipd67f ┃ $774 ┃