epibench score treats a forecast unit as a unique combination of model, reference_date, target_end_date, horizon, location (one forecast unit will likely have multiple quantiles, making up an entry). Should we, within a single model, fail or filter (or do nothing) if we find a duplicate entry (i.e., a forecast unit with two or more of the same quantile)? @jcblemai
Presently, the code does nothing and will execute without failure (does not search for duplicate values). To me, duplicates make scores much less valid (in the same way that missing forecast units do).
epibench scoretreats a forecast unit as a unique combination ofmodel,reference_date,target_end_date,horizon,location(one forecast unit will likely have multiple quantiles, making up an entry). Should we, within a single model, fail or filter (or do nothing) if we find a duplicate entry (i.e., a forecast unit with two or more of the same quantile)? @jcblemaiPresently, the code does nothing and will execute without failure (does not search for duplicate values). To me, duplicates make scores much less valid (in the same way that missing forecast units do).