Skip to content

BUG: PriorDict silently drops invalid values added after initialization #1111

Description

@asb5468

Note: the bug report below was prepared by AI under my supervision.

Describe the bug

PriorDict validates entries supplied during initialization, but apparently does not apply the same validation when entries are subsequently added using item assignment.

A zero-dimensional NumPy array is rejected when passed to the PriorDict constructor:

priors = PriorDict({
    "x": Uniform(0, 1, name="x"),
    "Om0": np.array(0.30966),
})

This raises a clear TypeError.

However, the same value can be added after the PriorDict has been instantiated:

priors = PriorDict({"x": Uniform(0, 1, name="x")})
priors["Om0"] = np.array(0.30966)

The invalid entry then appears in the prior dictionary, but is silently omitted by PriorDict.sample().

This creates inconsistent behavior depending on whether an entry is supplied during initialization or assigned afterward. More importantly, the post-initialization path fails silently: the parameter remains visible in priors.keys() but is absent from the sampled parameter dictionary without any error or warning.

To Reproduce

import numpy as np
from bilby.core.prior import PriorDict, Uniform

First, supplying a zero-dimensional NumPy array during initialization raises an error:

priors = PriorDict({
    "x": Uniform(0, 1, name="x"),
    "H0": np.float64(67.66),
    "Om0": np.array(0.30966),
    "w0": np.array(-1.0),
})

This produces:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/andreasb/.conda/envs/gwpop_jax_gpu/lib/python3.11/site-packages/bilby/core/prior/dict.py", line 37, in __init__
    self.from_dictionary(dictionary)
  File "/home/andreasb/.conda/envs/gwpop_jax_gpu/lib/python3.11/site-packages/bilby/core/prior/dict.py", line 305, in from_dictionary
    raise TypeError(
TypeError: Unable to parse prior, bad entry: Om0 = 0.30966 of type <class 'numpy.ndarray'>

However, adding the same entry after initialization does not raise an error:

priors = PriorDict({
    "x": Uniform(0, 1, name="x"),
})

priors["Om0"] = np.array(0.30966)

print(priors)
print(priors.keys())

The output is:

{
    'x': Uniform(
        minimum=0,
        maximum=1,
        name='x',
        latex_label='x',
        unit=None,
        boundary=None
    ),
    'Om0': array(0.30966)
}

dict_keys(['x', 'Om0'])

Calling sample() then silently omits Om0:

sample = priors.sample()

print(sample)
print(sample.keys())

For example:

{'x': 0.4084391218257899}

dict_keys(['x'])

Thus, NumPy scalar values such as np.float64 are retained, while zero-dimensional numpy.ndarray values are silently omitted.

Error message

When the zero-dimensional array is supplied during initialization, the full error is:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/andreasb/.conda/envs/gwpop_jax_gpu/lib/python3.11/site-packages/bilby/core/prior/dict.py", line 37, in __init__
    self.from_dictionary(dictionary)
  File "/home/andreasb/.conda/envs/gwpop_jax_gpu/lib/python3.11/site-packages/bilby/core/prior/dict.py", line 305, in from_dictionary
    raise TypeError(
TypeError: Unable to parse prior, bad entry: Om0 = 0.30966 of type <class 'numpy.ndarray'>

When the same entry is added after initialization using:

priors["Om0"] = np.array(0.30966)

there is no error message or warning. The entry is accepted into the dictionary and is later silently omitted by PriorDict.sample().

Expected behavior

The behavior should be consistent regardless of whether an entry is provided to the constructor or assigned after initialization.

There are at least two reasonable behaviors:

  1. Zero-dimensional NumPy arrays are considered invalid fixed prior values. In that case, assigning one after initialization should raise the same TypeError as supplying one to the constructor.

  2. Zero-dimensional NumPy arrays are considered scalar fixed values. In that case, Bilby should retain them during sampling, perhaps after converting them to NumPy or Python scalars using .item().

Under no circumstances should a key remain visible in the PriorDict but then be silently absent from the output of PriorDict.sample().

At minimum, PriorDict.sample() should raise an error or warning when it encounters an unsupported entry type.

Suggested solution (optional)

Apply the same validation used by PriorDict.from_dictionary() whenever a new value is assigned through PriorDict.__setitem__().

For example, assignment such as:

priors["Om0"] = np.array(0.30966)

could raise the same error as constructor initialization:

TypeError: Unable to parse prior, bad entry: Om0 = 0.30966 of type <class 'numpy.ndarray'>

Alternatively, Bilby could explicitly support zero-dimensional NumPy arrays as fixed scalar values:

if isinstance(value, np.ndarray) and value.ndim == 0:
    value = value.item()

It may also be useful for PriorDict.sample() to check that every entry has either been sampled or copied into the output. If an entry is skipped because its type is unsupported, it should raise an informative error rather than silently dropping the parameter.

A user-side workaround is:

priors["Om0"] = np.asarray(Om0).item()
priors["w0"] = np.asarray(w0).item()

but the current behavior is difficult to detect because assignment succeeds and the key remains present in the prior dictionary.

Environment (please complete the following information):

  • Bilby version: [2.8.0]
  • Installation method [e.g. conda, pip, source]: [conda]
  • Python version: 3.11

Additional context

This arose in a larger prior dictionary containing fixed cosmological parameters:

{
    ...
    "H0": np.float64(67.66),
    "Om0": np.array(0.30966),
    "w0": np.array(-1.0),
}

The entries were added after the PriorDict had already been instantiated. All three keys appeared in fast_priors.keys(), but only H0 appeared in the dictionary returned by fast_priors.sample().

The relevant types and shapes are:

type(fast_priors["H0"])
# numpy.float64

type(fast_priors["Om0"])
# numpy.ndarray

fast_priors["Om0"].shape
# ()

type(fast_priors["w0"])
# numpy.ndarray

fast_priors["w0"].shape
# ()

Therefore, the primary issue is not simply that zero-dimensional arrays are unsupported. The issue is that they are rejected during construction but accepted through later assignment, after which they are silently discarded during sampling.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions