Terraform for_each set includes values that cannot be determined until apply

We wrote a module that creates an ACM certificate and its DNS validation records in Route 53. It worked when we added it to an existing environment where the certificate already existed. In a brand new environment, terraform plan failed:

╷
│ Error: Invalid for_each argument
│
│   on modules/cert/main.tf line 14, in resource "aws_route53_record" "validation":
│   14:   for_each = toset([for o in aws_acm_certificate.this.domain_validation_options : o.resource_record_name])
│     ├────────────────
│     │ aws_acm_certificate.this.domain_validation_options is a set of object, known only after apply
│
│ The "for_each" set includes values derived from resource attributes that cannot be determined
│ until apply, and so Terraform cannot determine the full set of keys that will identify the
│ instances of this resource.
│
│ When working with unknown values in for_each, it's better to define the map keys statically in
│ your configuration and place apply-time results only in the map values.
│
│ Alternatively, you could use the -target planning option to first apply only the resources that
│ the for_each value depends on, and then apply a second time to fully converge.
╵

Reproducing it

Any configuration that derives for_each keys from attributes of a resource that does not exist yet. A second example in the same codebase:

resource "aws_instance" "worker" { count = 3 ... }

resource "aws_eip" "worker" {
  for_each = toset(aws_instance.worker[*].id)     # ids unknown until instances exist
  instance = each.value
}

What we ruled out

• A Terraform or provider bug. The error message explains the rule exactly.
• Ordering with depends_on. Dependencies control apply order; they cannot make unknown values known at plan time.

Root cause

Terraform must know the keys of every for_each instance during planning, because the keys are part of each resource's address (aws_route53_record.validation["_abc.example.com"]). The validation record names are generated by ACM when the certificate is created, so on a fresh environment they are unknown at plan time. The values may be unknown; the keys may not.

What is happening under the hood

During plan, Terraform builds a graph of resource instances. For count, it needs the number. For for_each, it needs the set of keys. Values of attributes can stay unknown (shown as (known after apply)), but instance identity cannot, otherwise Terraform cannot say which instances will be created, updated or destroyed.

The fix pattern is always the same: build for_each from something known at plan time, such as input variables, literals, or attributes that are themselves known from configuration, and put unknown, apply-time values in the map values.

For the ACM case, the domain names are known from our configuration even though the validation record names are not. Keying by domain name works.

Environment

Minimal reproduction

resource "aws_acm_certificate" "this" {
  domain_name               = "api.example.com"
  subject_alternative_names = ["www.api.example.com"]
  validation_method         = "DNS"
}

resource "aws_route53_record" "validation" {
  for_each = toset([for o in aws_acm_certificate.this.domain_validation_options : o.resource_record_name])
  # ...
}

The fix

Key the records by domain name, which comes from input variables and is known at plan time:

variable "domain_name" { type = string }
variable "alternative_names" {
  type    = list(string)
  default = []
}

locals {
  all_domains = toset(concat([var.domain_name], var.alternative_names))
}

resource "aws_acm_certificate" "this" {
  domain_name               = var.domain_name
  subject_alternative_names = var.alternative_names
  validation_method         = "DNS"

  lifecycle { create_before_destroy = true }
}

resource "aws_route53_record" "validation" {
  for_each = local.all_domains                          # keys known at plan time

  zone_id = var.zone_id
  name    = one([for o in aws_acm_certificate.this.domain_validation_options : o.resource_record_name if o.domain_name == each.key])
  type    = one([for o in aws_acm_certificate.this.domain_validation_options : o.resource_record_type if o.domain_name == each.key])
  records = [one([for o in aws_acm_certificate.this.domain_validation_options : o.resource_record_value if o.domain_name == each.key])]
  ttl     = 60
  allow_overwrite = true
}

resource "aws_acm_certificate_validation" "this" {
  certificate_arn         = aws_acm_certificate.this.arn
  validation_record_fqdns = [for r in aws_route53_record.validation : r.fqdn]
}

The values are still unknown at plan time and appear as (known after apply), which is fine.

For the Elastic IP example, the fix is the same idea: key by a static name, not by an instance id.

locals { workers = toset(["a", "b", "c"]) }

resource "aws_instance" "worker" {
  for_each = local.workers
  # ...
}

resource "aws_eip" "worker" {
  for_each = local.workers
  instance = aws_instance.worker[each.key].id      # unknown value, known key
}

allow_overwrite on the validation records handles the case where a wildcard and its base domain share one validation record. With overlapping domains, keying by domain can create two records with the same name; allow_overwrite makes that idempotent.

Alternatives and trade-offs

• -target in two applies. Terraform's own suggestion. Works as a one-off for bootstrapping, but it makes environments require special first-run steps, and -target is easy to misuse later.
• Split into two root modules or stacks. Create the certificate in one, validate in another. Clean separation, more state files.
• Deferred changes. Newer Terraform versions have been experimenting with letting plans defer resources with unknown for_each; we did not rely on experimental features in production.

How to test the fix

cd envs/new-region
terraform init
terraform plan -out=tfplan        # succeeds on an empty state
terraform apply tfplan            # one pass: cert, records, validation
terraform plan -detailed-exitcode # exit code 0: no changes

We also ran terraform plan against existing environments to make sure the change of keys did not plan to destroy and recreate validation records. It did, because the old keys were record names. We used moved blocks to map old addresses to new ones:

moved {
  from = aws_route53_record.validation["_a1b2c3.api.example.com."]
  to   = aws_route53_record.validation["api.example.com"]
}

Confirming in production

• The new regional environment was created in one apply, including a validated certificate.
• Existing environments showed only moves, no replacements, in the plan after adding the moved blocks.

Regression guard

• Module tests with terraform test run a plan against an empty state for every module, which catches unknown for_each keys before they reach a new environment.
• A review guideline: for_each keys come from variables or literals, never from resource attributes.

Security implications

None directly. Certificate validation records are public DNS records by design. The moved blocks were important for safety in a different way: without them, the plan would have deleted and recreated validation records in production, which can briefly break certificate renewal if timing is unlucky.

Prevention checklist

• Build for_each keys from configuration, not from resource outputs.
• Put apply-time values in map values.
• Test every module with a plan against an empty state.
• Use moved blocks when changing keys of existing resources.
• Avoid -target as a routine workflow.

References

• Terraform for_each meta-argument — https://developer.hashicorp.com/terraform/language/meta-arguments/for_each
• Terraform refactoring with moved blocks — https://developer.hashicorp.com/terraform/language/modules/develop/refactoring
• aws_acm_certificate_validation resource — https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/acm_certificate_validation
• terraform test command — https://developer.hashicorp.com/terraform/cli/commands/test