Skip to content

dbt osi-to-msi silently corrupts composite primary and unique keys #311

Description

@mattfaltyn

Problem

The dbt OSIToMSIConverter flattens every Ossie key group into a set of columns. For a composite primary key such as [tenant_id, order_id], it emits two independent MetricFlow PRIMARY entities. The reverse conversion then keeps only the final primary entity, silently changing the key to [order_id]. Composite unique keys are likewise converted into independent singleton unique keys.

Reproduction

On main at 88e0011148283302c9a04cd0287e00e0b9d87354, convert a valid dataset containing:

primary_key: [tenant_id, order_id]

Observed after Ossie -> MSI:

entities=[(tenant_id, primary), (order_id, primary)]

Observed after Ossie -> MSI -> Ossie:

primary_key: [order_id]

A singleton primary key round-trips unchanged.

Root cause

converters/dbt/src/ossie_dbt/osi_to_msi.py::_build_key_sets converts primary_key to a set and flattens all unique_keys groups. _classify_field then treats every member as independently primary or unique. MSIToOSIConverter._extract_keys has no grouping information to reconstruct.

Impact

Composite keys are common for tenant-scoped and line-item datasets. Emitting each component as independently unique can produce incorrect join cardinality or aggregations, while round trips silently lose key components.

Proposed fix

Until MSI can represent and round-trip grouped keys losslessly, fail Ossie -> MSI conversion clearly when a dataset contains a composite primary or unique key. Keep singleton key behavior unchanged and add regression tests for both composite key types. This prevents generating a plausible but semantically incorrect manifest.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions