Skip to content

Implement CRPS (Continuous Ranked Probability Score) #1

Description

@jc-macdonald

Summary

The CRPS is the most widely used proper scoring rule for continuous probabilistic forecasts. Implement both ensemble-based and closed-form (distributional) variants.

Functions to Implement

Function Description
crps_ensemble(obs, fct_samples) CRPS from ensemble/sample forecasts
crps_normal(obs, μ, σ) Closed-form CRPS for normal distribution

Algorithm

The ensemble CRPS decomposes as:

$$ \text{CRPS}(F, y) = \mathbb{E}|X - y| - \frac{1}{2}\mathbb{E}|X - X'| $$

where $X, X' \sim F$ are independent draws. The NRG (energy) representation avoids sorting:

crps = mean(abs.(fct .- obs)) - 0.5 * mean(abs.(fct .- fct'))

For the normal distribution, the closed-form uses erf and the standard normal PDF/CDF.

References

  • Gneiting, T., & Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation. JASA, 102(477), 359-378.
  • Jordan, A., Krüger, F., & Lerch, S. (2019). Evaluating probabilistic forecasts with scoringRules. JSS, 90(12).

Acceptance Criteria

  • crps_ensemble matches scoringrules (Python) on reference vectors
  • crps_normal matches analytical formula
  • Handles edge cases: single ensemble member, zero variance
  • Docstrings with mathematical formulas

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions