One possible problem with the run start document is that the “sample” key can accept either an object or a string. However, fields in elastic search cannot accept both of these datatypes. I have mapped this to be a text field, but if an object were to be in this field in a specific document then this entire document would be rejected. Even setting "index.mapping.ignore_malformed": True, will not fix this problem. As explained in the elasticsearch documentation:
Limits for JSON Objects
You can’t use ignore_malformed with the following data types:
Nested data type
Object data type
Range data types
You also can’t use ignore_malformed to ignore JSON objects submitted to fields of the wrong data type. A JSON object is any data surrounded by curly brackets "{}" and includes data mapped to the nested, object, and range data types.
If you submit a JSON object to an unsupported field, Elasticsearch will return an error and reject the entire document regardless of the ignore_malformed setting.
Source: https://www.elastic.co/guide/en/elasticsearch/reference/current/ignore-malformed.html#json-object-limits
I personally think the best solution to this is to implement scripting on an index that will transform incoming objects to strings. Scripting explained here: https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-scripting.html
One possible problem with the run start document is that the “sample” key can accept either an object or a string. However, fields in elastic search cannot accept both of these datatypes. I have mapped this to be a text field, but if an object were to be in this field in a specific document then this entire document would be rejected. Even setting "index.mapping.ignore_malformed": True, will not fix this problem. As explained in the elasticsearch documentation:
Source: https://www.elastic.co/guide/en/elasticsearch/reference/current/ignore-malformed.html#json-object-limits
I personally think the best solution to this is to implement scripting on an index that will transform incoming objects to strings. Scripting explained here: https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-scripting.html