OpenIIR-core

The headless simulation engine behind OpenIIR, an open platform for running multi-agent information-retrieval experiments. A small shared core (agent runtime, world-model store, retrieval, claim extractor, persona library) and a typed interface let the same primitives drive a deliberation, a feed-curation study, and a producer–detector simulation. A new kind of study is a small plug-in over the core, not a new codebase.

This repository is the engine and its reference runs only. It carries no web UI and no API server — everything here runs from the command line or as a library, offline, with no API key.

Artefact	Link
Live platform & demo	https://openiir.com
Pre-print (arXiv)	https://arxiv.org/abs/2605.09321
Hosted runs	https://openiir.com/simulation/

Three-layer architecture: a type-agnostic core, scenario-type plug-ins, and a local-first substrate. See the arXiv pre-print for the figure and details.

Repository layout

openiir-core/
├── openiir_core/
│   ├── core/                # the shared core
│   │   ├── scenario.py      # the four-method ScenarioType interface + registry
│   │   ├── llm.py           # minimal OpenAI-compatible client (Ollama by default)
│   │   ├── argument_graph.py# claim/edge data model + append-only JSONL store
│   │   └── claim_extractor.py# LLM-driven claim + typed-edge extraction
│   ├── types/
│   │   └── curated_feed.py  # reference scenario type (LLM-free, ~120 lines)
│   └── cli.py               # `openiir` command-line entry point
├── configs/                 # reference run configurations
├── eval/claim_extractor/    # extraction-quality benchmark + annotation tooling
├── docs/                    # writing a type, reproducibility
└── pyproject.toml

Install

pip install -e .            # core (numpy only)
pip install -e ".[llm]"     # add LLM-backed types (Panel claim extraction)

Reproducing results

Nothing below needs a network connection or an API key.

Curated-Feed (paper Figure 2). Deterministic for a fixed seed:

openiir run curated-feed --out runs/curated_feed.dat

Reproduces the reference run (200 users, 12 weeks, k=12, eta=0.20): popularity homogenises opinion (variance −53%), belief-aligned personalisation polarises it (+19%), the engagement-aligned ranker sits between. The .dat is the table the paper's plot is drawn from.

Inspect an argument graph. The four real ECIR Search Futures panels are bundled under eval/claim_extractor/exports/:

openiir types                                              # list scenario types
openiir inspect eval/claim_extractor/exports/breakout1.json

Output is written to the path you pass with --out; argument-graph snapshots are plain JSON ({nodes, edges}).

Extraction-quality benchmark

The Panel claim extractor's stance accuracy is reproducible from bundled human annotations of the four SF3 panels:

cd eval/claim_extractor
python score_annotations.py            # extractor vs. human, all four panels
python score_second_annotator.py --kind stance \
    --ann1 annotations/claims_to_label_annotated.csv \
    --ann2 annotations/reannotated_claims_download.csv   # inter-annotator agreement

See eval/claim_extractor/README.md for the full protocol and ANNOTATOR_GUIDE.md for the labeling instructions.

Writing a new scenario type

Implement four methods over the shared core; see docs/writing-a-type.md and the worked example in openiir_core/types/curated_feed.py.

Citation

@misc{zerhoudi2026openiir,
  author        = {Zerhoudi, Saber},
  title         = {{OpenIIR}: An Open Simulation Platform for Information Retrieval Research},
  year          = {2026},
  eprint        = {2605.09321},
  archivePrefix = {arXiv},
  primaryClass  = {cs.IR},
  url           = {https://arxiv.org/abs/2605.09321},
}

License

Released under the MIT License. Bundled ECIR Search Futures run data is published from the public hosted runs; third-party dependencies retain their own upstream licenses.