Skip to content

remove duplicates on leaderboard(s) #268

Description

@houtanb

the reworked LLM forecaster causes a model to appear twice on the leaderboard, once with the zero shot prompt and once with the zero-shot with freeze values prompt.

For the Tournament and Preliminary leaderboards, we should drop the:

  • zero shot prompt models when the model does not have other tools enabled (E.g. web search)
  • zero shot with freeze values prompt when the model has tools enabled

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingleaderboardanything to do with leaderboards

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions