Skip to content

[BUG] starts_with/ends_with logs an ERROR with a stack trace on every evaluation when the property is absent #2018

Description

@scottt732

Observed behavior

When a starts_with / ends_with targeting rule references a context property that is absent, flagd logs at ERROR — with a full stack trace — on every evaluation, while the evaluation itself reports success.

An absent optional context attribute is a completely ordinary condition (browser clients that don't send a client_version, for example), but it is treated identically to a genuine type error.

Two things compound it:

  1. ERROR implies a stack trace. zap.Config.Build() attaches stack traces at error level, so each occurrence emits ~40 log lines rather than one.
  2. It fires per evaluation. A single rule that ORs several version prefixes multiplies accordingly — one of ours has 8 starts_with calls in one rule, so a single bulk OFREP evaluation of that flag produced ~320 log lines.

This took down log ingestion for our whole organisation. A rule matching on client_version was evaluated by the browser-facing slice, where that attribute is usually absent, producing ~4M log lines/minute across 8 replicas. It exhausted our daily log-index quota in about 9 minutes, after which no service in the org could index logs until the quota reset. The flag itself behaved fine — the cost was entirely in log volume.

Expected behavior

An absent property should not be an ERROR. I'd expect either a silent false, or a Debug-level message.

A property that is present but not a string is a genuine configuration mistake and is reasonable to log — but not at ERROR severity on every evaluation, since the volume is driven by request rate rather than by the number of bad rules.

Steps to reproduce

flags.json:

{
  "flags": {
    "sw_flag": {
      "state": "ENABLED",
      "defaultVariant": "off",
      "variants": { "on": true, "off": false },
      "targeting": {
        "if": [{ "starts_with": [{ "var": "email" }, "cosmos"] }, "on", "off"]
      }
    }
  }
}
flagd start --sources='[{"uri":"flags.json","provider":"file"}]'

# context does NOT contain "email"
curl -s -X POST localhost:8016/ofrep/v1/evaluate/flags \
  -H 'content-type: application/json' \
  -d '{"context":{"targetingKey":"u"}}' | jq -c '.flags[]'

Response:

{"value":false,"key":"sw_flag","reason":"TARGETING_MATCH","variant":"off","metadata":{}}

Log, per evaluation, followed by ~40 stack frames:

error  evaluator/string_comparison.go:46  parse starts_with evaluation data:
       [start/end]s_with evaluation: property did not resolve to a string value

Passing "email": 42 (present, wrong type) produces byte-identical output.

Tested against core/v0.16.1.

Root cause

parseStringComparisonEvaluationData does not distinguish absent from present-but-wrong-type. jsonlogic resolves a missing var to nil, so parsed[0].(string) fails and both paths hit the same sce.Logger.Error(...).

Secondary observation

The evaluation reports reason: TARGETING_MATCH with no errorCode, even though the operator errored and the if fell through to its else-branch. The response is indistinguishable from a legitimate match on the else-branch.

That matters beyond cosmetics: we run a save-time validator that pushes each candidate flag document to a private flagd and rejects it if any flag comes back with an errorCode. It could not catch this, because there is no field in the response that differs from a healthy evaluation — the only evidence exists in the logs, which the validator doesn't read. It's also invisible to any SDK consumer.

I appreciate that changing an existing reason is a behaviour break and may warrant its own discussion; I've kept it separate from the logging problem above, which seems straightforwardly fixable on its own.

I'm happy to open a PR for the logging fix (distinguishing absent from wrong-type, and lowering the severity) if that direction sounds right — and separately for the reason/errorCode question if maintainers want to go there.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions