It's designed for usage when you have a small number of malformed events. When every event is malformed, the searches performed by that method can overwhelm the cluster.
It's ok to just fail on a malformed event. That method is used purely as an optimization, to ignore the malformed event when it's safe to do so, and log a message like:
Ignoring 1 malformed event(s) because they have been superseded by corrected events targeting the same id: Type:id@v123.
We should improve this workflow so that it's only used when a small number of events are malformed. It could be a percent that defaults to 10% or something. The config option could be called something like detect_superseded_malformed_events_percentage_threshold.
It's designed for usage when you have a small number of malformed events. When every event is malformed, the searches performed by that method can overwhelm the cluster.
It's ok to just fail on a malformed event. That method is used purely as an optimization, to ignore the malformed event when it's safe to do so, and log a message like:
We should improve this workflow so that it's only used when a small number of events are malformed. It could be a percent that defaults to 10% or something. The config option could be called something like
detect_superseded_malformed_events_percentage_threshold.