rms-metadata-tools

Introduction

rms-metadata-tools (the importable package metadata_tools) generates PDS3 index, geometry, and cumulative metadata tables, and their PDS3 labels, for planetary science data collections. Each row of a table holds the metadata for a single data product, such as one image.

The Planetary Data System (PDS) distributes data as collections of volumes. Alongside the data, each collection ships flat ASCII metadata tables that summarize every product: observation times, instrument settings, and the geometry of what was observed. These tables feed the OPUS search service and are downloaded directly by PDS users. Producing them by hand is tedious and error-prone; rms-metadata-tools generates them, and their validated labels, for a whole volume tree from a small per-collection configuration.

rms-metadata-tools is a product of the PDS Ring-Moon Systems Node at the SETI Institute.

Features

• Three table types. Generates supplemental index tables (extra columns from PDS3 labels), geometry tables (body, ring, sky, and Sun quantities computed from SPICE through oops), and cumulative tables that span a whole collection.
• PDS3 labels included. Writes a validated .lbl label for every table from reusable templates.
• Engine plus per-host configuration. A host-agnostic engine does the work; each collection plugs in through a small configuration package, so adding a new collection means writing config, not engine code.
• Local or cloud. Run on one machine, or distribute per-volume work across Google Cloud with the included rms-cloud-tasks workers.
• Local and remote storage. Reads and writes through rms-filecache, so local paths and gs:// / s3:// URIs are interchangeable.

Installation

rms-metadata-tools requires Python 3.11 or later and is tested on Python 3.11, 3.12, and 3.13. Generating geometry tables additionally requires the SPICE kernels for your collection, installed where oops can find them.

Install from PyPI:

pip install rms-metadata-tools

To work from a checkout (recommended when adding or modifying a host configuration, since host configuration lives in the source tree):

git clone https://github.com/SETI/rms-metadata-tools.git
cd rms-metadata-tools
python -m venv venv
source venv/bin/activate
pip install -e ".[dev]"

The optional extras are dev (linting, type-checking, tests, docs), docs (Sphinx), and cloud (the GCP workers).

Quick Start

Each supported collection ("host") has its own configuration under src/metadata_tools/hosts/<HOST>/, generated through three console-script entry points that take the host id as their first argument. They can be run from any directory. The three stages, for the Galileo SSI host, are:

# 1. Supplemental index tables (extra columns from the PDS3 labels)
metadata-index GO_0xxx "$RMS_VOLUMES/GO_0xxx/" "$RMS_METADATA/GO_0xxx/" \
    "$RMS_METADATA_TEST/GO_0xxx/"

# 2. Geometry tables (body/ring/sky/Sun quantities from SPICE)
metadata-geometry GO_0xxx "$RMS_METADATA/GO_0xxx/" "$RMS_METADATA_TEST/GO_0xxx/"

# 3. Cumulative tables across the whole collection
metadata-cumulative GO_0xxx "$RMS_METADATA_TEST/GO_0xxx/GO_0999/"

Path arguments are expanded for environment variables. Restrict a run to one or more volumes with --volumes GO_0017, or to a few images with --first 5. See the user guide for the full list of programs and options.

Documentation

Full documentation is hosted on ReadTheDocs. It includes a user guide (installation, configuration, and a reference for every command-line program) and a developer guide (architecture, per-subsystem internals, how to add a new collection or geometry column, and the API reference).

To build the documentation locally:

scripts/read-docs.sh

Contributing

Information on contributing to this package can be found in the Contributing Guide.

Links

• Documentation
• Repository
• Issue tracker
• PyPI

License

This code is licensed under the Apache License v2.0.