Labrador-RetrievR: Proof of Concept

Labrador is a proof-of-concept implementation that operationalizes the RetrievR virtual linked stream discovery and access framework. It acts as a mediator between stream reasoning agents and VoCaLS organizations, providing virtual stream provisioning as well as contract negotiation. Labrador is intentionally RSP-engine-agnostic and focuses on stream discovery, stream orchestration, and virtual stream lifecycle management.

Getting started

Run Labrador and a subset of the streams from CityBench using Docker Compose:

docker compose up -d

You can verify that the streams are running by opening up playground/index.html in a browser.

Paste one of the websocket URLs and start listening to one of the "raw" streams:

Stream	WebSocket URL
parking-all	ws://localhost:8081/
pollution158324	ws://localhost:8082/
pollution158355	ws://localhost:8083/
traffic158324	ws://localhost:8084/
traffic158355	ws://localhost:8085/
traffic158386	ws://localhost:8086/
weather	ws://localhost:8087/

Register a new virtual stream endpoint that conforms to a predetermined shape by run playground/run.sh in a terminal. The first two request will fail and return suggestions for alternative contracts. The final request will return a websocket URL populated by labrador according to the accepted contract.

You can add a listener to the virtual stream using:

docker compose run --rm stream-console <stream URL>

To shut down Labrador:

docker compose down

Labrador🦮 ❤️ 🐦Kolibrie

Kolibrie is a high-performance Rust-based RSP engine that supports continuous RDF stream processing using RSP-QL style windows, while also providing a broader query and reasoning pipeline for SPARQL, rules, and machine learning operators. Together with a Labrador-Kolibrie RetrievR adapter, Kolibrie can benefit from Labrador's virtual streams.

Note: Running Kolibrie natively requires Rust version 1.60 or higher.

To start Kolibrie, run:

cargo run -p kolibrie-http-server

Kolibrie should be available at http://127.0.0.1:8080. We can now execute the live query by running a Python script:

python3 scripts/retriever_sidecar.py execute \
  --query-file ../retrieve_multiple_window.rspql \
  --shape-file ../labrador/playground/shape.ttl \
  --labrador-url http://127.0.0.1:8181 \
  --kolibrie-url http://127.0.0.1:8080 \
  --ws-connect-host 127.0.0.1 \
  --ws-connect-port 8181 \
  --max-events 5 \
  --print-before-after

The expected output includes:

Pushed event 1 to Kolibrie at t=0
Kolibrie result: {"congestion":"OK", ...}

Those Kolibrie result: lines are the query results emitted by Kolibrie over /rsp/events/{session_id}.

Command Summary

resolve

Resolve RetrieveR stream variables and print the rewritten query.

python3 scripts/retriever_sidecar.py resolve \
  --query-file ../retrieve_multiple_window.rspql \
  --shape-file ../labrador/playground/shape.ttl \
  --labrador-url http://127.0.0.1:8181 \
  --strip-retrieve \
  --print-before-after

Use this when you only want to see the Labrador-generated stream URI and the Kolibrie-ready query.

register

Resolve the query and register it with Kolibrie, but do not consume Labrador events.

python3 scripts/retriever_sidecar.py register \
  --query-file ../retrieve_multiple_window.rspql \
  --shape-file ../labrador/playground/shape.ttl \
  --labrador-url http://127.0.0.1:8181 \
  --kolibrie-url http://127.0.0.1:8080 \
  --print-before-after

This returns a Kolibrie session_id.

execute

Resolve, register, follow the Labrador WebSocket stream, push events into Kolibrie, and print results.

Useful options:

• --max-events 5: stop after five Labrador events.
• --drain-seconds 3: wait for Kolibrie SSE results before exiting after --max-events.
• --ws-connect-host 127.0.0.1: connect to Labrador through the host port mapping.
• --ws-connect-port 8181: connect to Labrador through Docker's exposed port.
• --print-before-after: show the original RetrieveR query and the final Kolibrie query.

serve

Run the sidecar as an HTTP service:

python3 scripts/retriever_sidecar.py serve \
  --host 127.0.0.1 \
  --port 8090 \
  --kolibrie-url http://127.0.0.1:8080 \
  --default-labrador-url http://127.0.0.1:8181

The sidecar endpoint is POST http://127.0.0.1:8090/retrieve-rsp/register

Docker Hostname Note

Labrador may return stream URLs like ws://labrador:8080/stream/... That name works inside Docker, but from PowerShell or a Linux shell on the host machine you usually need to connect via ws://127.0.0.1:8181/stream/... The sidecar keeps Kolibrie's stream IRI as ws://labrador:8080/..., but uses --ws-connect-host 127.0.0.1 --ws-connect-port 8181 for the actual Python WebSocket connection.

Improving retriever_sidecar.py --help

In retriever_sidecar.py, make the top-level parser use a raw epilog:

parser = argparse.ArgumentParser(
    description="RetrieveR sidecar for Kolibrie RSP-QL",
    epilog=manual,
    formatter_class=argparse.RawDescriptionHelpFormatter,
)

Where manual is a string that lists the resolve, register, execute, and serve commands with examples. Then:

python3 scripts/retriever_sidecar.py --help

will show the command overview, and:

python3 scripts/retriever_sidecar.py execute --help

will show every option for execute.