feat(relationships): fill related data from ?include= and make all related resources accessible

AGENTS.md

@@ -57,6 +57,8 @@ A handful of conventions are pervasive enough that you'll regret breaking them.
 
 2. **JSON:API attribute names go through `Field(alias="...")`.** The API sends `created-at`, Python uses `created_at`. Pair with `model_config = ConfigDict(populate_by_name=True, validate_by_name=True)` on the model. ([MODELS.md](docs/MODELS.md))
 
+2b. **Top-level response/resource models inherit `TFEModel`, not `BaseModel`.** It's config-light (you keep your own `model_config`) and adds the lossless `.relationships`/`.included`/`.related()`/`.has_*` accessors. Wire the resource's parser to call `attach_jsonapi(model, data, included)` so they're populated. Options/sub-object/enum models stay on `BaseModel`. ([MODELS.md](docs/MODELS.md) — TFEModel vs BaseModel)
+
 3. **`model_dump(by_alias=True, exclude_none=True)` for write payloads.** Without `by_alias=True` you'll send snake_case to the API and it will silently drop the fields. Add `mode="json"` if the options contain enums.
 
 4. **For new public APIs, prefer typed `TFEError` subclasses.** The error hierarchy in `errors.py` is part of the public API, and downstream consumers often `except TFEError:` once. Existing methods still expose many `ValueError` paths; do not change those established exceptions unless the breaking-change impact is explicitly accepted.

CHANGELOG.md

@@ -1,5 +1,24 @@
 # Unreleased
 
+## Enhancements
+
+### Relationships
+Related data is now a complete, first-class part of every response. Before, `?include=` often did not actually fill the related fields it returned, and anything the SDK did not model as a typed field was dropped on the floor. Now the full set of related resources the API hands back is always available to you: typed where pytfe models it, raw where it does not. The practical win is that **you are no longer limited to the relationships pytfe has added typed support for.** You can read any related resource in a response without dropping to manual HTTP or waiting for a new SDK release.
+
+* `?include=` now fills in related data. When the SDK models a relation as a typed field (for example `workspace.outputs`, `policy_set.current_version`, `organization_membership.user`, `run_event.actor`), passing `?include=<relation>` fills that field with the real record instead of an id-only stub.
+* Relations the SDK does **not** model are no longer lost. Every top-level resource model now derives from a new `pytfe.models.TFEModel` base and gains read-only accessors for the raw JSON:API data the API returned: `model.relationships`, `model.included`, `model.related(name)`, `model.included_by(type, id)`, and the `model.has_relationships` and `model.has_included` flags. So when a relation has no typed field of its own (for example an organization's `subscription`, or a workspace `readme`), `?include=` still returns it and you reach it with `model.related("subscription")` or `model.included_by(type, id)`. These accessors are read-only extras that never appear in `model_dump()` or affect equality, so this is additive and non-breaking. List endpoints expose the relationship refs but do not yet fill `included`. See [docs/related-resources.md](docs/related-resources.md) for the per-resource table and a "typed field vs raw accessor" guide.
+* Added `?include=` support to three reads that previously had no include option, matching the HCP Terraform API:
+  * `teams.read(team_id, TeamReadOptions(include=[...]))`: `users`, `organization-memberships`.
+  * `task_stages.read(task_stage_id, TaskStageReadOptions(include=[...]))`: `run`, `run.workspace`, `task-results`, `policy-evaluations`.
+  * `organizations.read(name, OrganizationReadOptions(include=[...]))`: `subscription`. The new `options` argument is optional, so existing calls are unchanged.
+
+## Bug Fixes
+
+### Relationships
+* Fixed `workspaces.read*(include=[WorkspaceIncludeOpt.OUTPUTS])` returning outputs with `None` name, value, and type. Workspace `outputs` is now filled from the `included` data. [#134](https://github.com/hashicorp/python-tfe/issues/134)
+* Fixed `policy_set.read*(include=[current_version | newest_version])` returning an id-only stub. `PolicySetVersion` is now exported from `pytfe.models` and fully resolved, so the version's `source`, `created_at`, and `status` are populated.
+* Fixed `variable_set.read` inventing placeholder values (such as `name="workspace-<id>"` or `key="var-<id>"`) for `workspaces`, `projects`, and `vars`. These are now id-only stubs by default and fill from `included` when requested.
+
 # Released
 # v1.1.0
 

README.md

@@ -164,7 +164,7 @@ and upstream HCP Terraform API docs.
 | Need | Start here |
 |---|---|
 | Configure the SDK | [Authentication](./docs/authentication.md), [Pagination](./docs/pagination.md), [Logging](./docs/LOGGING.md) |
-| API guides | [API index](./docs/api/index.md), [API coverage](./docs/api-coverage.md), [Workspaces](./docs/api/workspaces.md), [Runs/plans/applies](./docs/api/runs-plans-applies.md), [State versions](./docs/api/state-versions.md) |
+| API guides | [API index](./docs/api/index.md), [API coverage](./docs/api-coverage.md), [Related resources (`include`)](./docs/related-resources.md), [Workspaces](./docs/api/workspaces.md), [Runs/plans/applies](./docs/api/runs-plans-applies.md), [State versions](./docs/api/state-versions.md) |
 | Scenario guides | [API-driven run](./docs/scenarios/api-driven-run.md), [State management](./docs/scenarios/state-management.md), [Migrate workspaces and state](./docs/scenarios/migrate-workspaces-and-state.md), [Team access onboarding](./docs/scenarios/team-access-onboarding.md), [No-code provisioning](./docs/scenarios/no-code-provisioning.md), [TFE identity bootstrap](./docs/scenarios/tfe-identity-bootstrap.md), [TFE admin bootstrap](./docs/scenarios/tfe-admin-bootstrap.md), [OIDC dynamic credentials](./docs/scenarios/oidc-dynamic-credentials.md) |
 | Operations guides | [Troubleshooting](./docs/troubleshooting.md), [Errors](./docs/errors.md), [Terraform Enterprise](./docs/terraform-enterprise.md) |
 | Contribute to the SDK | [CONTRIBUTING](./docs/CONTRIBUTING.md), [ITERATORS](./docs/ITERATORS.md), [MODELS](./docs/MODELS.md), [RESOURCE](./docs/RESOURCE.md) |

docs/MODELS.md

@@ -37,6 +37,42 @@ class Foo(BaseModel):
     ...
 ```
 
+## Base class: `TFEModel` vs `BaseModel`
+
+| Inherit from | For |
+|---|---|
+| **`TFEModel`** (`pytfe.models`, defined in `models/_base.py`) | **Top-level resource models**: anything returned from a `read`/`list`/`create`/`update` that corresponds to a JSON:API *resource object* (`Workspace`, `Run`, `Project`, `Policy`, `AdminRun`, …). |
+| **`BaseModel`** | Everything else: `*CreateOptions` / `*UpdateOptions` / `*ListOptions`, nested attribute sub-objects (`WorkspacePermissions`, `VCSRepo`, …), enums, and `*List` envelopes. |
+
+`TFEModel` is **config-light**: it adds no `model_config`, so you still set your own (`extra="allow"`, etc.) exactly as above. What it adds is the lossless related-resource escape hatch: `.relationships`, `.included`, `.included_by(type, id)`, `.related(name)`, and the `.has_relationships` / `.has_included` presence flags. These are private attributes, so they never touch `model_dump()` and add no public fields, and `TFEModel` also overrides `__eq__` to ignore them, so equality stays identical to a plain `BaseModel`. Inheriting it is additive and non-breaking.
+
+For the accessors to be *populated* (not just present-and-empty), the resource's parser must hand the raw JSON:API resource dict (and any document `included`) to `attach_jsonapi`:
+
+```python
+from .._jsonapi import attach_jsonapi, parse_relationships   # in a resources/*.py
+
+def _foo_from(data, included=None):
+    attr = dict(data.get("attributes") or {})
+    attr["id"] = data.get("id")
+    attr.update(parse_relationships(data.get("relationships"), _FOO_REL_MAP, included=included))
+    return attach_jsonapi(Foo.model_validate(attr), data, included)
+```
+
+`attach_jsonapi(obj, data, included)` is the one line that captures both raw blocks; pass `included=payload.get("included")` from any `read`/`list` that supports `?include=`. See [related-resources.md](related-resources.md) for the consumer-facing view.
+
+### Why not put it on every model
+
+`TFEModel` is scoped to resource objects on purpose. The escape hatch is only meaningful for things parsed from a JSON:API *resource object* (a thing with a `relationships` block, returned from `read`/`list`/`create`/`update`). Putting it on `*Options`, sub-objects, and `*List` envelopes would cost more than it gives:
+
+* **It does nothing on its own.** The accessors stay empty until a parser calls `attach_jsonapi(...)`. A request/options model is never parsed from a response, so its accessors would be permanently empty and pointless.
+* **It adds a confusing, response-only surface to request objects.** A `WorkspaceReadOptions` is something you *build and send*. Exposing `.relationships`, `.included`, and `.related(...)` on it (always empty) is misleading in code, in docs, and in editor autocomplete.
+* **It reserves names.** `TFEModel` claims `relationships`, `included`, `related`, `included_by`, `has_relationships`, and `has_included`. If a model later needs a field with one of those names, Pydantic warns (`Field name "..." shadows an attribute in parent "TFEModel"`) and the field silently wins, quietly breaking the escape hatch on that model. Keeping non-resource models on `BaseModel` avoids reserving those names where they are not needed.
+* **It changes the public class hierarchy and equality** for many public, downstream-facing models. `TFEModel`'s `__eq__` is intentionally equivalent to `BaseModel`'s for public fields, but there is no reason to swap a custom `__eq__` onto the hundreds of options and sub-object models that behave like plain Pydantic today.
+
+What it does **not** break: `frozen=True` models stay hashable, because Pydantic regenerates `__hash__` for a frozen subclass. The only real failure mode is the name collision above.
+
+Rule of thumb: inherit `TFEModel` when the model is a JSON:API resource you parse *from* a response; use `BaseModel` for everything you *build* (options), *nest* (sub-objects), or *wrap* (`*List` envelopes).
+
 ## Field aliases: JSON:API hyphens → Python snake_case
 
 HCP Terraform speaks JSON:API, which uses hyphenated attribute names (`created-at`, `auto-apply`, `state-versions`). Python uses snake_case. Bridge with `Field(alias=...)`:

docs/api-coverage.md

@@ -75,7 +75,9 @@ partially covered are listed at the bottom of this page.
 | | Azure OIDC configurations | `client.azure_oidc_configurations` | ✅ |
 | | GCP OIDC configurations | `client.gcp_oidc_configurations` | ✅ |
 | | Vault OIDC configurations | `client.vault_oidc_configurations` | ✅ |
-| Admin | SAML / SCIM / SMTP / token-TTL settings | `client.admin` | ✅ |
+| Admin (TFE site-admin) | Organizations, users, runs, workspaces | `client.admin.organizations` / `.users` / `.runs` / `.workspaces` | ✅ |
+| | Terraform / OPA / Sentinel versions | `client.admin.terraform_versions` / `.opa_versions` / `.sentinel_versions` | ✅ |
+| | SAML / SCIM / SMTP settings + SCIM tokens | `client.admin.saml_settings` / `.scim_settings` / `.scim_tokens` / `.smtp_settings` | ✅ |
 
 ## Partial coverage
 
@@ -117,8 +119,6 @@ Public HCP Terraform API resources that do not yet have a pytfe client namespace
 | User tokens | Personal (user) API tokens. |
 | VCS events | — |
 
-### Terraform Enterprise only (separate admin API)
-
-| Resource | Notes |
-|---|---|
-| Site-admin | TFE site-admin endpoints (admin organizations, users, runs, workspaces, Terraform / OPA / Sentinel versions). Not part of the public HCP Terraform API. |
+> Note: the TFE site-admin API (`/api/v2/admin/*`, TFE-only — not part of the
+> public HCP Terraform API) **is** implemented under `client.admin` (see the
+> Admin rows above).