Skip to content

Add ScoringRules.jl as scoring backend #32

Description

@jc-macdonald

Summary

The scoring module is being developed as a standalone package at jcm-sci/ScoringRules.jl since proper scoring rules have broad utility beyond model criticism.

TradeStudy.jl should depend on ScoringRules.jl and wire it into the AbstractScorer dispatch system from the Protocols module (#25).

Proposed Design

struct ProperScoringRuleScorer <: AbstractScorer
    metric::Symbol  # :crps, :energy, :wis, :brier, etc.
end

function score(s::ProperScoringRuleScorer, obs, pred)
    ScoringRules.score(s.metric, obs, pred)
end

Depends On

Acceptance Criteria

  • ScoringRules added to [deps] in Project.toml
  • ProperScoringRuleScorer exported and documented
  • Integration test using CRPS on synthetic data

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions