terraform_codegen

Ingest existing Terraform (.tf) files from a git repo, modify or regenerate them, and generate brand-new Terraform code — all from Python.

It parses HCL with python-hcl2 and ships a faithful HCL writer that turns the parse tree back into valid Terraform, so you get true round-tripping (parse → modify → re-emit).

Install

pip install python-hcl2        # required (tested on 8.1.2)
pip install GitPython          # optional; CLI git is used as a fallback

Drop the terraform_codegen/ package next to your script (or put it on your PYTHONPATH).

The three things it does

from terraform_codegen import TerraformRepo, Document, ref, obj, block, lit

repo = TerraformRepo.clone("git@github.com:you/infra.git", "./infra", branch="main")

1. Ingest + regenerate an existing file

regenerate parses the file, hands you the tree, and rewrites whatever you return.

def transform(tree):
    body = tree["resource"][0]['"aws_instance"']['"web"']
    body["ami"]           = lit("ami-2024new")        # quoted string literal
    body["instance_type"] = ref("var.instance_type")  # unquoted expression
    body["monitoring"]    = True
    body.setdefault("tags", {})["Env"] = lit("prod")
    return tree

repo.regenerate("main.tf", transform)

2. Generate a new file from scratch

doc = Document()
doc.add_terraform(required_version=">= 1.5.0")
doc.add_variable("instance_type", type=ref("string"), default="t3.small")
doc.add_resource(
    "aws_security_group", "web",
    name="web-sg",                       # attribute names never clash with the API
    vpc_id=ref("aws_vpc.main.id"),
    ingress=[block(from_port=443, to_port=443, protocol="tcp",
                   cidr_blocks=["0.0.0.0/0"])],
    tags=obj(Name="web-sg", ManagedBy="python"),
)
doc.add_output("sg_id", value=ref("aws_security_group.web.id"))
repo.write("security.tf", doc)

3. Commit (and push) back to the repo

repo.new_branch("automated/codegen")            # optional
repo.commit_changes("codegen update", push=True)

Value rules (important)

Terraform distinguishes string literals from expressions. This is the one thing to get right:

You want	In the Document builder	Editing a parsed tree directly
"t3.small" (quoted string)	"t3.small"	lit("t3.small")
var.x (expression/reference)	ref("var.x")	ref("var.x")
42, true, null	42, True, None	42, True, None
{ Name = "x" } (map)	obj(Name="x")	a plain dict
ingress { ... } (block)	block(...)	dict with __is_block__: True

In the builder, a plain Python str is always a quoted literal; wrap expressions in ref(). When you edit a parsed tree in place, string values are in parser form already ('"t3.small"'), so use lit() / ref() helpers to produce the right form.

Files

File	Purpose
terraform_codegen/hcl_writer.py	Serialize an hcl2 parse tree back to HCL
terraform_codegen/hcl_builder.py	Document + ref/lit/obj/block helpers
terraform_codegen/repo_manager.py	git clone/pull, parse, write, commit/push
example_usage.py	Runnable end-to-end demo (python example_usage.py)

Notes & limitations

• Run terraform fmt on output for canonical spacing. The writer emits valid HCL but doesn't align = signs or insert blank lines the way fmt does. (You can call it from Python: subprocess.run(["terraform", "fmt", path]).)
• Comments are not preserved — python-hcl2 discards them during parsing. If comment preservation matters, keep generated and hand-written code in separate files, or use .tf.json (Terraform reads it natively) instead.
• Blocks of the same type are grouped together on output (all resource blocks, then all variable blocks, etc.). This is cosmetic; ordering is not semantically meaningful to Terraform.
• Validated by parse → write → re-parse equality across resources, variables, providers, locals, outputs, nested/repeated blocks, object lists, heredocs, and references.

tf_codegen