RecordPreparer Bug: Context-Unaware __typename Addition

[marcdaniels-toast]

Summary

The RecordPreparer adds __typename to a type everywhere if that type is indexed in any union/interface index, even when the type is embedded in a concrete-typed field where __typename shouldn't be in the mapping. This causes indexing failures.

Example: When NamedInventor interface gets an index, Person (which implements it) gets indexed in the union inventors index. The RecordPreparer then adds __typename to ALL Person objects, including when Person is embedded in Manufacturer.ceo. But Manufacturer.ceo is typed as concrete Person, so its mapping doesn't include __typename, causing the count_accumulator to fail when it encounters the key.

Root Cause

In elasticgraph-indexer/lib/elastic_graph/indexer/record_preparer.rb, lines 155-157:

if @types_requiring_typename.include?(type_name) && !value.key?("__typename")
  prepared_fields["__typename"] = type_name
end

The @types_requiring_typename set is computed globally (lines 98-100) and includes any type that:

1. Has requires_typename in its JSON schema metadata, OR
2. Has __typename in ANY of its index mappings

This means if a type like Person implements an interface that is indexed (e.g., NamedInventor with t.index "inventors"), it will have __typename added everywhere - including when used as an embedded object in a concrete-typed field like Manufacturer.ceo.

How to Reproduce

Apply this diff to main branch and run bundle exec rake boot_locally:

diff --git a/config/schema/widgets.rb b/config/schema/widgets.rb
index 7df5223f..54063773 100644
--- a/config/schema/widgets.rb
+++ b/config/schema/widgets.rb
@@ -31,12 +31,14 @@ ElasticGraph.define_schema do |schema|
 
   schema.object_type "Person" do |t|
     t.implements "NamedInventor"
+    t.field "id", "ID!"
     t.field "name", "String"
     t.field "nationality", "String"
   end
 
   schema.object_type "Company" do |t|
     t.implements "NamedInventor"
+    t.field "id", "ID!"
     t.field "name", "String"
     t.field "stock_ticker", "String"
   end
@@ -47,7 +49,9 @@ ElasticGraph.define_schema do |schema|
 
   # Embedded interface type.
   schema.interface_type "NamedInventor" do |t|
+    t.field "id", "ID!"
     t.field "name", "String"
+    t.index "inventors"
   end
 
   # Indexed interfae type.
@@ -338,6 +342,7 @@ ElasticGraph.define_schema do |schema|
     t.field "created_at", "DateTime!"
     t.relates_to_many "manufactured_parts", "Part", via: "manufacturer_id", dir: :in, singular: "manufactured_part"
     t.relates_to_one "address", "Address", via: "manufacturer_id", dir: :in
+    t.field "ceo", "Person"
 
     t.index "manufacturers" do |i|
       i.default_sort "created_at", :desc
diff --git a/config/settings/development.yaml b/config/settings/development.yaml
index cbc914cd..3a306fd7 100644
--- a/config/settings/development.yaml
+++ b/config/settings/development.yaml
@@ -18,6 +18,7 @@ datastore:
       setting_overrides_by_timestamp: {}
     components: *main_index_settings
     electrical_parts: *main_index_settings
+    inventors: *main_index_settings
     manufacturers: *main_index_settings
     mechanical_parts: *main_index_settings
     teams: *main_index_settings
diff --git a/spec_support/lib/elastic_graph/spec_support/factories/widgets.rb b/spec_support/lib/elastic_graph/spec_support/factories/widgets.rb
index f618add2..14e397ba 100644
--- a/spec_support/lib/elastic_graph/spec_support/factories/widgets.rb
+++ b/spec_support/lib/elastic_graph/spec_support/factories/widgets.rb
@@ -140,6 +140,15 @@ FactoryBot.define do
     __typename { "Manufacturer" }
     name { Faker::Company.name }
     created_at { Faker::Time.between(from: recent_date - 30, to: recent_date).utc.iso8601 }
+    ceo do
+      if has_ceo
+        { id: Faker::Alphanumeric.alpha(number: 20), name: Faker::Name.name, nationality: Faker::Nation.nationality }
+      end
+    end
+
+    transient do
+      has_ceo { Faker::Boolean.boolean(true_ratio: 0.5) }
+    end
   end
 
   factory :address, parent: :indexed_type do

Error: KeyError: key not found: "__typename" at /elastic_graph/indexer/operation/count_accumulator.rb:160

Expected vs Actual Behavior

Expected:

• Person objects indexed directly into inventors should have __typename: "Person" (because it's a union index for NamedInventor interface)
• Person objects embedded in Manufacturer.ceo should NOT have __typename (because the field is typed as concrete Person)

Actual:

• RecordPreparer adds __typename to ALL Person objects regardless of context
• The manufacturers index mapping for ceo field does NOT include __typename property
• CountAccumulator tries to traverse __typename key but can't find it in the mapping properties
• Indexing fails

Index Mapping Context

inventors index (union index for NamedInventor interface):

properties:
  id: { type: keyword }
  name: { type: keyword }
  nationality: { type: keyword }
  stock_ticker: { type: keyword }
  __typename: { type: keyword }  # ✓ Present

manufacturers index (concrete field):

properties:
  id: { type: keyword }
  name: { type: keyword }
  ceo:
    properties:
      id: { type: keyword }
      name: { type: keyword }
      nationality: { type: keyword }
      # ✗ __typename is NOT present

Impact

This bug prevents using any type that is indexed in a union/interface index as an embedded object in other types. The schema definition API allows this pattern without error. The failure only occurs at indexing time, not at schema definition time.

Proposed Fix

The fix requires making RecordPreparer.prepare_for_index context-aware. Instead of globally tracking @types_requiring_typename, it should:

1. Track which specific field paths require __typename based on their index mappings
2. Only add __typename when preparing data for a field context that requires it
3. Pass field path context through the recursive prepare_value_for_indexing calls

Example approach:

def prepare_for_index(type_name, value, field_path = [])
  # ...
  prepared_fields = value.filter_map do |field_name, field_value|
    if field_name == "__typename"
      # Check if __typename is in THIS field's mapping, not globally
      [field_name, field_value] if field_requires_typename?(field_path + [field_name])
    # ...
  end

  # Only add __typename if the current field context requires it
  if field_requires_typename?(field_path) && !value.key?("__typename")
    prepared_fields["__typename"] = type_name
  end
  # ...
end

Workaround

For now, avoid using types that are indexed in union/interface indexes as embedded objects in concrete-typed fields. Use separate types for indexing vs embedding if needed.

Related Files

• elasticgraph-indexer/lib/elastic_graph/indexer/record_preparer.rb (lines 92-101, 148-157)
• elasticgraph-indexer/lib/elastic_graph/indexer/operation/count_accumulator.rb (line 160)
• Test schema: config/schema/widgets.rb (Manufacturer + OnlineStore example)
• Factory: spec_support/lib/elastic_graph/spec_support/factories/widgets.rb

[myronmarston]

Thanks for the thorough diagnosis! I agree this is a bug we should fix. More generally, I don't think our test schema has any types which both are root index types and also are embedded in another index. It'd be good to cover that case, confirming it works end-to-end, as part of fixing this.