diff --git a/.githooks/pre-push b/.githooks/pre-push index 59dc517a..c9de9206 100755 --- a/.githooks/pre-push +++ b/.githooks/pre-push @@ -50,7 +50,7 @@ do # Check test examples set -e - doc-generation/check-examples.bash + data-model/tools/check-examples.bash set +e fi diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml index 605d8a66..df5a9f32 100644 --- a/.github/workflows/pages.yml +++ b/.github/workflows/pages.yml @@ -60,7 +60,7 @@ jobs: - name: Run tests run: | - doc-generation/check-examples.bash + data-model/tools/check-examples.bash documents: needs: @@ -90,17 +90,18 @@ jobs: - name: Install dependencies run: poetry install --no-interaction --all-extras - - name: Generate MarkDown documentation + - name: Generate all artifacts run: | - doc-generation/generate-documentation.bash + data-model/tools/generate-all.bash - - name: Share generated documentation with later jobs + - name: Share generated artifacts with later jobs uses: actions/upload-artifact@v4 with: - name: generated-documentation + name: generated-artifacts path: | system-design/specification/applications/application-description.md system-design/specification/margo-management-interface/desired-state.md + system-design/specification/margo-management-interface/workload-management-api-1.0.0.yaml pages: needs: @@ -135,12 +136,16 @@ jobs: - name: Install the project dependencies run: poetry install - - name: Get generated documentation from previous jobs + - name: Get generated artifacts from previous jobs uses: actions/download-artifact@v4 with: - name: generated-documentation + name: generated-artifacts path: system-design/specification + - name: Generate merged documentation tree + run: | + data-model/tools/generate-docs.bash + - name: Build Pages run: poetry run -- mkdocs build diff --git a/.gitignore b/.gitignore index d687d211..bdd9a5fc 100644 --- a/.gitignore +++ b/.gitignore @@ -10,3 +10,5 @@ system-design/specification/applications/application-description.md src/specification/applications/docs src/specification/margo-management-interface/docs *.code-workspace +generated +merged diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..d73d4a9e --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,318 @@ +# AGENTS.md — Margo Specification Contributor Guide + +## Project Overview + +The Margo Specification defines open standards for workload fleet management on edge compute devices. This repository contains the normative specification documents, some authored manually in MarkDown and others generated from a [LinkML](https://linkml.io/) data model using Jinja2 templates. + +The final HTML documentation is built with [MkDocs](https://www.mkdocs.org/) using the [Material theme](https://squidfunk.github.io/mkdocs-material/). + +## Repository Layout + +``` +. +├── src/specification/ # Legacy per-resource LinkML schemas + Jinja2 templates + examples +│ ├── applications/ +│ │ ├── application-description.linkml.yaml +│ │ └── resources/ +│ │ ├── class.md.jinja2 +│ │ ├── index.md.jinja2 +│ │ └── examples/{valid,invalid}/ +│ └── margo-management-interface/ +│ ├── desired-state.linkml.yaml +│ └── resources/ +│ ├── index.md.jinja2 +│ └── examples/{valid,invalid}/ +├── data-model/ # Aggregate data model (current source of truth) +│ ├── margo-data-model.linkml.yaml # Top-level schema aggregating all sub-schemas +│ ├── application-description.linkml.yaml +│ ├── application-deployment.linkml.yaml +│ ├── desired-state-manifest.linkml.yaml +│ ├── device-capabilities.linkml.yaml +│ ├── deployment-status.linkml.yaml +│ ├── margo-resources.linkml.yaml +│ ├── margo-deployments.linkml.yaml +│ ├── tools/ # Generation & validation scripts +│ │ ├── generate-all.bash # Runs all generators in sequence +│ │ ├── generate-docs.bash # Generates MarkDown from LinkML +│ │ ├── generate-json-schemas.bash # Generates JSON-Schema artifacts +│ │ ├── generate-openapi.bash # Generates OpenAPI spec +│ │ ├── generate-class-diagram.bash # Generates PlantUML class diagrams +│ │ ├── check-examples.bash # Validates schemas and examples +│ │ └── configurations/ # Per-spec JSON configs +│ │ ├── application-deployment.json +│ │ ├── application-description.json +│ │ ├── deployment-status.json +│ │ ├── desired-state-manifest.json +│ │ └── device-capabilities.json +│ └── resources/ +│ ├── examples/{valid,invalid}/ # Valid and invalid example files +│ ├── markdown-templates/ # Templates for the aggregate data-model docs +│ └── markdown-templates_main-classes/ # Templates for per-resource docs +├── doc-generation/ # Legacy scripts (superseded by data-model/tools/) +├── system-design/ # Manually-authored + generated MarkDown (copied into merged/ by generate-docs.bash) +│ └── specification/ # ← generated .md files land here +├── generated/ # Additional generated artifacts (diagrams, OpenAPI, JSON-Schema, etc.) +├── merged/ # Merged MarkDown tree used by mkdocs build +├── mkdocs.yml # MkDocs configuration +└── pyproject.toml # Python dependencies (linkml>=1.11.0, mkdocs, etc.) +``` + +## Setup + +### Option A: Development Container + +The repository includes a dev-container with all dependencies pre-installed. Open the repo in VS Code and accept the dev-container prompt. + +### Option B: Poetry (recommended for local development) + +```bash +poetry install +``` + +### Option C: pip + +```bash +pip install ./pyproject.toml +``` + +## Key Commands + +### Validate LinkML schemas and examples + +```bash +data-model/tools/check-examples.bash +``` + +This script: +- Reads each config from `data-model/tools/configurations/*.json` +- Validates the corresponding LinkML schema +- Validates valid examples in `data-model/resources/examples/valid/` against the schema +- Validates that invalid examples in `data-model/resources/examples/invalid/` are correctly rejected +- Exits non-zero if any check fails + +### Generate all artifacts + +```bash +data-model/tools/generate-all.bash +``` + +This runs all generators in sequence: class diagrams, JSON-Schemas, OpenAPI, and MarkDown docs. + +### Generate MarkDown from LinkML + +```bash +data-model/tools/generate-docs.bash +``` + +This script: +- Generates per-resource MarkDown using `data-model/resources/markdown-templates_main-classes/` +- Generates the full data model MarkDown using `data-model/resources/markdown-templates/` +- Copies generated diagrams and OpenAPI spec into the merged tree +- Copies everything from `system-design/` and `generated/` into `merged/` (which is what `mkdocs build` reads) + +### Generate JSON-Schemas + +```bash +data-model/tools/generate-json-schemas.bash +``` + +Generates one `.schema.json` file per resource schema into `generated/json-schemas/`. + +### Generate OpenAPI spec + +```bash +data-model/tools/generate-openapi.bash +``` + +Generates `workload-management-api-1.0.0.openapi.yaml` into `generated/openapi/`. + +### Generate class diagrams + +```bash +data-model/tools/generate-class-diagram.bash +``` + +Generates SVG/PNG class diagrams via PlantUML into `generated/diagrams/`. + +### Build HTML documentation + +```bash +mkdocs build # one-time build +mkdocs serve # live-reloading local server +``` + +## Generation Pipeline + +Understanding how artifacts flow through the pipeline is essential when modifying schemas or templates. + +### Artifact flow + +``` +data-model/*.linkml.yaml (source of truth — LinkML schemas) + │ + ├──► data-model/tools/check-examples.bash + │ uses: data-model/tools/configurations/*.json + │ validates: data-model/resources/examples/{valid,invalid}/ + │ + ├──► data-model/tools/generate-docs.bash (orchestrator) + │ │ + │ ├──► linkml generate doc (per-resource, using markdown-templates_main-classes/) + │ │ → generated/markdown_main-classes/.md + │ │ → moved into merged/specification/{applications,margo-management-interface}/ + │ │ + │ ├──► linkml generate doc (aggregate, using markdown-templates/) + │ │ → generated/markdown/* + │ │ → moved into merged/data-model/ + │ │ + │ ├──► generate-class-diagram.bash + │ │ → generated/diagrams/DataModel-ClassDiagram.{svg,png} + │ │ → copied into merged/figures/ + │ │ + │ ├──► generate-openapi.bash + │ │ → generated/openapi/workload-management-api-1.0.0.openapi.yaml + │ │ → moved into merged/specification/margo-management-interface/ + │ │ + │ └──► JSON schemas copied into merged/json-schemas/ + │ + └──► data-model/tools/generate-json-schemas.bash (standalone) + → generated/json-schemas/*.schema.json + +mkdocs build reads from merged/ → site/ +``` + +Key points: +- `mkdocs build` reads from `merged/`, not directly from `system-design/` or `generated/`. The `generate-docs.bash` script copies everything into `merged/` first. +- The per-resource Markdown generation and the aggregate data-model generation use **different template directories** and produce output in **different locations**. +- `check-examples.bash` reads the list of schemas from `data-model/tools/configurations/*.json`, but `generate-docs.bash` and `generate-json-schemas.bash` have **hardcoded** schema lists. When adding a new resource, you must update all three. + +### OpenAPI generation + +The OpenAPI spec is generated by `data-model/tools/openapigen.py`, which: +- Reads an OpenAPI template (`data-model/tools/workload-management-api-1.0.0.openapi.yaml`) that defines endpoints, security schemes, and request/response structure. +- Fills in `components/schemas` from the LinkML model using the JSON-Schema generator. +- Only classes referenced by the endpoints in the template are included in the output. + +When adding a new resource to the API, you must update both the LinkML schema **and** the OpenAPI template (add new endpoints that reference the new class). The template follows standard OpenAPI 3.0.3 structure — add a new entry under `paths` with request/response schemas that use `$ref: "#/components/schemas/"`. + +## Legacy Code + +The `src/specification/` directory contains the original per-resource LinkML schemas, templates, and examples. These are **legacy** and no longer the source of truth. The current source of truth is `data-model/`. Do not modify files under `src/specification/` unless specifically instructed. + +The `doc-generation/` directory contains the original generation and validation scripts. These have been superseded by `data-model/tools/`. Do not use the old scripts. + +## How to Modify the LinkML Data Model + +### Step 1: Edit the schema + +The **source of truth** for LinkML-specified resources lives under `data-model/`. Each resource has its own `.linkml.yaml` file. The aggregate schema `data-model/margo-data-model.linkml.yaml` imports all sub-schemas. + +When making changes: + +1. Edit the relevant `.linkml.yaml` file to add/modify classes, attributes, enums, slots, or types. +2. Add or update valid examples in `data-model/resources/examples/valid/` to cover the new or changed model elements. +3. Add invalid counter-examples in `data-model/resources/examples/invalid/` if new validation rules are introduced. +4. If the change affects rendering, update the Jinja2 templates: + - `data-model/resources/markdown-templates_main-classes/` — for per-resource specification pages (attribute tables, examples, JSON-Schema links). + - `data-model/resources/markdown-templates/` — for the aggregate data-model documentation (class hierarchy, diagrams). +5. If the change adds new API endpoints, update the OpenAPI template `data-model/tools/workload-management-api-1.0.0.openapi.yaml`. + +### Step 2: Validate + +```bash +data-model/tools/check-examples.bash +``` + +Fix any validation errors before proceeding. + +### Step 3: Regenerate artifacts + +```bash +data-model/tools/generate-all.bash +``` + +This runs all generators in sequence. Alternatively, run individual generators if you only need to update one artifact type. + +The generated `.md` files land under `system-design/specification/` and `system-design/data-model/`. Generated JSON-Schemas, OpenAPI specs, and diagrams land under `generated/`. Verify the output looks correct. + +### Step 4: Preview the site + +```bash +mkdocs serve +``` + +Open http://127.0.0.1:8000 and navigate to the relevant specification page to visually verify. + +### Step 5: Commit + +Include the modified LinkML schema, updated examples, and all regenerated artifacts (`system-design/`, `generated/`) in your commit. These directories are tracked in git and must reflect the generated output. + +## Adding a New LinkML-Specified Resource + +1. Add the LinkML schema: `data-model/.linkml.yaml`. +2. Add valid examples: `data-model/resources/examples/valid/-NNN.{yaml,json}`. +3. Add invalid counter-examples: `data-model/resources/examples/invalid/-NNN.{yaml,json}`. +4. Add a configuration file in `data-model/tools/configurations/.json` with: + ```json + { + "root": "data-model", + "targetclass": "", + "schemafile": ".linkml.yaml", + "markdowndoc": ".md" + } + ``` +5. Add an import in `data-model/margo-data-model.linkml.yaml`. +6. Add the new MarkDown file to `mkdocs.yml` under the `nav` section. +7. Add the schema name to the hardcoded lists in `generate-docs.bash` (line 41) and `generate-json-schemas.bash` (line 24). +8. Run validation and generation as described above. + +## Jinja2 Templates + +The generation uses `linkml generate doc`, which provides the following variables and objects in the template context: + +- `schema` — the parsed LinkML schema object +- `schemaview` — a `SchemaView` instance for querying classes, slots, enums, etc. +- `gen` — the generator instance with helper methods like `all_class_objects()`, `get_direct_slots()`, `link()`, `mermaid_diagram()` + +Common patterns in existing templates: + +- Iterate over class slots: `{% for slot in schemaview.class_slots("ClassName")|sort(attribute='rank') %}` +- Get slot details: `schemaview.get_slot(slot_name).range`, `.required`, `.description` +- Include example files: `{% include 'examples/valid/FileName.yaml' %}` +- Format ranges with inline macros for multivalued/inlined slots (see `index.md.jinja2` in each template directory) + +There are two separate sets of templates that serve different purposes: + +| Template directory | Used by | Produces | Output location | +| --- | --- | --- | --- | +| `data-model/resources/markdown-templates_main-classes/` | `generate-docs.bash` (per-resource loop) | One `.md` per schema with attribute tables, examples, JSON-Schema links | `merged/specification/{applications,margo-management-interface}/` | +| `data-model/resources/markdown-templates/` | `generate-docs.bash` (aggregate step) | Full data model overview with class hierarchy, diagrams, all-class listing | `merged/data-model/` | + +Each directory contains an `index.md.jinja2` (the main page) and may contain `class.md.jinja2` (individual class detail pages). When modifying rendering, determine which template directory to edit based on the table above. + +**When do templates need updating?** Most schema changes (adding/removing attributes, changing types, adding enums) do **not** require template changes — the templates iterate dynamically over class slots. Templates need updating only when: +- Changing the **structure** of the rendered page (e.g., adding a new section, changing table columns) +- Changing how **multivalued/inlined ranges** are displayed (the `format_range` macro in `index.md.jinja2`) +- Adding support for a **new example format** (e.g., rendering `.json` examples alongside `.yaml`) + +## CI + +The GitHub Actions pipeline (`.github/workflows/pages.yml`) runs: + +1. **Quality checks** — validates `pyproject.toml` and `poetry.lock` consistency +2. **Validation** — runs `data-model/tools/check-examples.bash` +3. **Document generation** — runs `data-model/tools/generate-docs.bash` +4. **Pages build & deploy** — deploys to GitHub Pages on the `pre-draft` branch + +PR checks (`.github/workflows/pr-checks.yml`) verify that commits are signed off. + +## Conventions + +- Sign off all commits (`git commit -s`) +- All contributions require CLA compliance (EasyCLA) +- One logical change per commit; the tree must build and work after each commit +- Base PRs on the `pre-draft` branch +- Example files follow the naming convention `-NNN.{yaml,json}` (e.g., `ApplicationDescription-001.yaml`, `DeploymentStatusManifest-001.json`) +- LinkML schemas use the `.linkml.yaml` extension +- The YAML language server schema annotation `# yaml-language-server: $schema=...` should be kept at the top of each schema file for IDE support +- Do **not** add `default_range: string` to schemas that define slots using `any_of` — it causes the JSON-Schema generator to emit `type: string` at the top level, overriding the `anyOf` union. See [linkml/linkml#1483](https://github.com/linkml/linkml/issues/1483) +- Omit explicit `required: false` on optional attributes — the LinkML default is already `false` diff --git a/data-model/application-deployment.linkml.yaml b/data-model/application-deployment.linkml.yaml new file mode 100644 index 00000000..5fb9ccf2 --- /dev/null +++ b/data-model/application-deployment.linkml.yaml @@ -0,0 +1,128 @@ +# yaml-language-server: $schema=https://linkml.io/linkml-model/linkml_model/jsonschema/meta.schema.json +id: https://specification.margo.org/application_deployment_schema +name: ApplicationDeployment +description: >- + Each workload is represented as an `ApplicationDeployment` YAML file that specifies its components, configuration, and parameters. + This resource is delivered via the Desired State API and referenced by `id` in the Deployment Status API. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#applicationdeployment-yaml-definition +version: 1.0.0 #Arne: update later +prefixes: + linkml: https://w3id.org/linkml/ + margo: https://specification.margo.org/ +imports: + - linkml:types + - margo-deployments.linkml + +default_prefix: margo +default_range: string + +classes: + ApplicationDeployment: + description: >- + A class representing the desired state of an entity. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#applicationdeployment-yaml-definition + attributes: + apiVersion: + description: Identifier of the version of the API the object definition follows. + required: true + range: string + kind: + description: Must be `ApplicationDeployment`. + equals_string: "ApplicationDeployment" + required: true + range: string + designates_type: true + metadata: + description: >- + Metadata element specifying characteristics about the application deployment. + See the [Metadata Attributes](#metadata-attributes) section below. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#metadata-attributes + range: DeploymentMetadata + required: true + spec: + description: >- + Spec element that defines deployment profile and parameters associated with the application deployment. + See the [Spec Attributes](#spec-attributes) section below. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#spec-attributes + range: Spec + required: true + + DeploymentMetadata: + description: >- + Metadata associated with the desired state. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#metadata-attributes + attributes: + annotations: + description: >- + Defines the application ID and unique identifier associated to the deployment specification. + Needs to be assigned by the Workload Orchestration Software. + See the [Annotation Attributes](#annotations-attributes) section below. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#annotations-attributes + range: DeploymentAnnotations + required: true + name: + description: >- + When deploying to Kubernetes, the manifests name. + The name is chosen by the workload orchestration vendor and is not displayed anywhere. + required: true + range: string + namespace: + description: When deploying to Kubernetes, the namespace the manifest is added under. The namespace is chosen by the workload orchestration solution vendor. + required: true + range: string + + DeploymentAnnotations: + description: >- + A class representing annotations. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#annotations-attributes + attributes: + applicationId: + description: >- + An identifier for the application. + The id is used to help create unique identifiers where required, such as namespaces. + The id must be lower case letters and numbers and MAY contain dashes. + Uppercase letters, underscores and periods MUST NOT be used. + The id MUST NOT be more than 200 characters. + The applicationId MUST match the associated application package Metadata "id" attribute. + range: string + required: true + pattern: "^[-a-z0-9]{1,200}$" + id: + description: >- + The unique identifier UUID of the deployment specification. + Needs to be assigned by the Workload Orchestration Software. + range: string + required: true + pattern: "^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$" + + Spec: + description: >- + Specification details of the desired state. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#spec-attributes + attributes: + deploymentProfile: + description: >- + Section that defines deployment details including type and components. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#deploymentprofile-attributes + range: DeploymentProfile + required: true + parameters: + description: >- + Describes the configured parameters applied via the end-user. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#parameter-attributes + range: Parameter + required: true + multivalued: true + inlined: true + inlined_as_list: false + diff --git a/data-model/application-description.linkml.yaml b/data-model/application-description.linkml.yaml new file mode 100644 index 00000000..a7c0a9fa --- /dev/null +++ b/data-model/application-description.linkml.yaml @@ -0,0 +1,387 @@ +# yaml-language-server: $schema=https://linkml.io/linkml-model/linkml_model/jsonschema/meta.schema.json +id: https://specification.margo.org/application-schema +name: ApplicationDescription +title: Application Description +description: >- + The purpose of the Application Description is to enable an application's discovery, configuration, and deployment on edge devices. + To deploy an application the end user specifies values for the [parameters](#defining-configurable-application-parameters) given in + an Application Description (e.g., through a UI of the WFM) to instantiate an `ApplicationDeployment`, + which defines the [desired state](../margo-management-interface/desired-state.md) for an application. + + The structure of the ApplicationDescription object to be provided in the YAML document + specified in this page. + Some examples are provided also [at the bottom of this page](#examples). + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/applications/application-registry.md + for the application registry specification. +version: 1.0.0 +prefixes: + linkml: https://w3id.org/linkml/ + margo: https://specification.margo.org/ +imports: + - linkml:types + - margo-resources.linkml + - margo-deployments.linkml + +default_prefix: margo +# default_range: string # https://github.com/linkml/linkml/issues/1483 + + +# Class Definitions +classes: + ApplicationDescription: + description: Root class for an application description. + attributes: + apiVersion: + description: Identifier of the version of the API the object definition follows. + required: true + range: string + kind: + description: Specifies the object type; must be `ApplicationDescription`. + range: string + required: true + equals_string: "ApplicationDescription" + designates_type: true + metadata: + description: >- + Metadata element specifying characteristics about the application deployment. + See the [Metadata Attributes](#metadata-attributes) section below. + range: ApplicationMetadata + required: true + deploymentProfiles: + description: >- + Deployment profiles element specifying the types of deployments the application supports. + See the [Deployment](#deploymentprofile-attributes) section below. + range: DeploymentProfileDescription + multivalued: true + inlined: true + inlined_as_list: true + required: true + parameters: + description: >- + Parameters element specifying the configurable parameters to use when installing, or updating, the application. + See the [Parameter](#parameter-attributes) section below. + range: Parameter + multivalued: true + inlined: true + inlined_as_list: false + configuration: + description: >- + Configuration element specifying how parameters should be displayed to the user for setting the value + as well as the rules to use to validate the user's input. + See the [Configuration](#configuration-attributes) section below. + range: Configuration + + ApplicationMetadata: + description: Metadata about the application. + attributes: + id: + description: >- + An identifier for the application. The id is used to help create unique identifiers where required, + such as namespaces. The id must be lower case letters and numbers and MAY contain dashes. + Uppercase letters, underscores and periods MUST NOT be used. The id MUST NOT be more than 200 characters. + range: string + required: true + pattern: ^[a-z0-9-]{1,200}$ + name: + description: >- + The application's official name. + This name is for display purposes only and can container whitespace and special characters. + range: string + required: true + description: + range: string + version: + description: The application's version. + range: string + required: true + catalog: + description: >- + Catalog element specifying the application's metadata for enabling its discovery. + See the [Catalog](#catalog-attributes) section below. + range: Catalog + required: true + + Catalog: + description: Catalog metadata for displaying the application. + attributes: + application: + description: >- + Application element specifying the application specific metadata. + See the [Application Metadata](#applicationmetadata-attributes) section below. + range: CatalogApplicationMetadata + author: + description: >- + Author element specifying metadata about the application's author. + See the [Author Metadata](#author-attributes) section below. + range: Author + multivalued: true + inlined: true + inlined_as_list: true + organization: + description: >- + Organization element specifying metadata about the organization/company providing the application. + See the [Organization Metadata](#organization-attributes) section below. + range: Organization + multivalued: true + required: true + inlined: true + inlined_as_list: true + + CatalogApplicationMetadata: + description: Metadata specific to the application. + attributes: + descriptionFile: + description: Link to the file containing the application's full description. The file should be a markdown file. + range: string + icon: + description: Link to the icon file (e.g., in PNG format). + range: string + licenseFile: + description: Link to the file that details the application's license. The file should either be a plain text, markdown or PDF file. + range: string + releaseNotes: + description: Statement about the changes for this application's release. The file should either be a markdown or PDF file. + range: string + site: + description: Link to the application's website. + range: string + tagline: + description: The application's slogan. + range: string + tags: + description: An array of strings that can be used to provide additional context for the application in a user interface to assist with task such as categorizing, searching, etc. + range: string + multivalued: true + inlined: true + inlined_as_list: true + + Author: + description: Information about the application's author. + attributes: + name: + description: The name of the application's creator. + range: string + email: + description: Email address of the application's creator. + range: string + pattern: .*@[a-z0-9.-]* + + Organization: + description: Information about the providing organization. + attributes: + name: + description: Organization responsible for the application's development and distribution. + range: string + required: true + site: + description: Link to the organization's website. + range: string + + DeploymentProfileDescription: + description: Represents a deployment configuration for the application. + is_a: DeploymentProfile + attributes: + id: + description: >- + An identifier for the deployment profile, given by the application developer, used to uniquely identify this deployment profile from others within this application description's scope. + range: string + required: true + description: + description: >- + This human-readable description of a deployment profile allows for providing additional context about the deployment profile. E.g., the application developer can use this to describe the deployment profile's purpose, + such as the intended use case. Additionally, the application developer can use this to provide further details about the resources, peripherals, and interfaces required to run the application. + range: string + requiredResources: + description: >- + Required resources element specifying the resources required to install the application. + See the [Required Resources](#requiredresources-attributes) section below. + The consequences (e.g., aborting / blocking the installation or execution of the application) of not meeting these required resources are not defined (yet) by margo. + range: Resources + + HelmDeploymentProfileDescription: + is_a: DeploymentProfileDescription + slot_usage: + type: + equals_string: "helm.v3" + components: + range: HelmComponent + + ComposeDeploymentProfileDescription: + is_a: DeploymentProfileDescription + slot_usage: + type: + equals_string: "compose" + components: + range: ComposeComponent + + Configuration: + description: Configuration layout and validation rules. + attributes: + sections: + description: >- + Sections are used to group related parameters together, + so it is possible to present a user interface with a logical grouping of the parameters in each section. + See the [Section](#section-attributes) section below. + range: Section + multivalued: true + inlined: true + inlined_as_list: true + required: true + schema: + description: >- + Schema is used to provide details about how to validate each parameter value. + At a minimum, the parameter value must be validated to match the schema's data type. + The schema indicates additional rules the provided value must satisfy to be considered valid input. + See the [Schema](#schema-attributes) section below. + range: Schema + multivalued: true + inlined: true + inlined_as_list: true + required: true + + Section: + description: Named sections within the configuration layout. + attributes: + name: + description: >- + The name of the section. This may be used in the user interface to show the grouping of the associated parameters within the section. + range: string + required: true + settings: + description: >- + Settings are used to provide instructions to the workload orchestration software vendor for displaying parameters to the user. + A user MUST be able to provide values for all settings. + See the [Setting](#setting-attributes) section below. + range: Setting + multivalued: true + inlined: true + inlined_as_list: true + required: true + + Setting: + description: Individual configuration settings. + attributes: + parameter: + description: The name of the [parameter](#parameter-attributes) the setting is associated with. + range: string + required: true + name: + description: The parameter's display name to show in the user interface. + range: string + required: true + description: + description: The parameters's short description to provide additional context to the user in the user interface about what the parameter is for. + range: string + immutable: + description: If true, indicates the parameter value MUST not be changed once it has been set and used to install the application. Default is false if not provided. + range: boolean + schema: + description: The name of the schema definition to use to validate the parameter's value. See the [Schema](#schema-attributes) section below. + range: Schema + inlined: false + required: true + + Schema: + description: Defines data type and rules for validating user provided parameter values. Subclasses (see below) define for each data type their own set of validation rules that can be used. The value MUST be validated against all rules defined in the schema. + attributes: + name: + description: The name of the schema rule. This used in the [setting](#setting-attributes) to link the setting to the schema rule. + range: string + required: true + identifier: true + dataType: + description: >- + Indicates the expected data type for the user provided value. + Accepted values are string, integer, double, boolean, array[string], array[integer], array[double], array[boolean]. + At a minimum, the provided parameter value MUST match the schema's data type if no other validation rules are provided. + range: string + required: true + #validationRule: + # description: >- + # Defines the validation rules to use to validate the user provided parameter value. + # The rules are based on the schema's data type and are listed below. + # The value MUST be validated against any validation rules defined in the schema. + # range: ValidationRule + # required: false + + TextValidationSchema: + is_a: Schema + description: Extends schema to define a string/text-specific set of validation rules that can be used. + attributes: + allowEmpty: + description: >- + If true, indicates a value must be provided. Default is false if not provided. + range: boolean + minLength: + description: If set, indicates the minimum number of characters the value must have to be considered valid. + range: integer + maxLength: + description: If set, indicates the maximum number of characters the value must have to be considered valid. + range: integer + regexMatch: + description: If set, indicates a regular expression to use to validate the value. + range: string + + BooleanValidationSchema: + is_a: Schema + description: Extends schema to define a boolean-specific set of validation rules that can be used. + attributes: + allowEmpty: + description: >- + If true, indicates a value must be provided. Default is false if not provided. + range: boolean + + NumericIntegerValidationSchema: + is_a: Schema + description: Extends schema to define a integer-specific set of validation rules that can be used. + attributes: + allowEmpty: + description: >- + If true, indicates a value must be provided. Default is false if not provided. + range: boolean + minValue: + description: If set, indicates the minimum allowed integer value the value must have to be considered valid. + range: integer + maxValue: + description: If set, indicates the maximum allowed integer value the value must have to be considered valid. + range: integer + + NumericDoubleValidationSchema: + is_a: Schema + description: Extends schema to define a double-specific set of validation rules that can be used. + attributes: + allowEmpty: + description: >- + If true, indicates a value must be provided. Default is false if not provided. + range: boolean + minValue: + description: If set, indicates the minimum value to be considered valid. + range: float + maxValue: + description: If set, indicates the maximum value to be considered valid. + range: float + minPrecision: + description: If set, indicates the minimum level of precision the value must have to be considered valid. + range: integer + maxPrecision: + description: If set, indicates the maximum level of precision the value must have to be considered valid. + range: integer + + SelectValidationSchema: + is_a: Schema + description: Extends schema to define a specific set of validation rules that can be used for select options. + attributes: + allowEmpty: + description: >- + If true, indicates a value must be provided. Default is false if not provided. + range: boolean + multiselect: + description: If true, indicates multiple values can be selected. If multiple values can be selected the resulting value is an array of the selected values. The default is false if not provided. + range: boolean + options: + description: This provides the list of acceptable options the user can select from. The data type for each option must match the parameter setting’s data type. + range: string + multivalued: true + required: true + diff --git a/data-model/deployment-status.linkml.yaml b/data-model/deployment-status.linkml.yaml new file mode 100644 index 00000000..7227fad1 --- /dev/null +++ b/data-model/deployment-status.linkml.yaml @@ -0,0 +1,117 @@ +# yaml-language-server: $schema=https://linkml.io/linkml-model/linkml_model/jsonschema/meta.schema.json +id: https://specification.margo.org/deployment-status +name: DeploymentStatus +description: >- + Schema for reporting the deployment status of workloads from a device to the WFM. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/deployment-status.md +version: 1.0.0 +prefixes: + linkml: https://w3id.org/linkml/ + margo: https://specification.margo.org/ +imports: + - linkml:types + +default_prefix: margo +default_range: string + + +classes: + DeploymentStatusManifest: + description: >- + Manifest sent by the device client to report the deployment status of a workload. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/deployment-status.md#request-body-attributes + attributes: + apiVersion: + range: string + required: true + kind: + range: string + required: true + equals_string: "DeploymentStatusManifest" + designates_type: true + deploymentId: + description: >- + The unique identifier of the deployment whose status is being reported. + range: string + required: true + status: + description: >- + Overall status of the deployment. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/deployment-status.md#status-attributes + range: Status + required: true + components: + description: >- + Per-component status list. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/deployment-status.md#component-attributes + range: ComponentStatus + required: true + multivalued: true + + Status: + description: >- + Overall deployment state and optional error details. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/deployment-status.md#status-attributes + slots: + - state + - error + + Error: + description: >- + Error details associated with a failed deployment state. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/deployment-status.md#error-attributes + attributes: + code: + range: string + message: + range: string + + ComponentStatus: + is_a: Status + description: >- + Status of a component deployment. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/deployment-status.md#component-attributes + attributes: + name: + range: string + required: true + + +slots: + state: + description: >- + Current deployment state. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/deployment-status.md#status-attributes + range: State + required: true + + error: + description: >- + Optional error details when the state is `failed`. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/deployment-status.md#error-attributes + range: Error + + +enums: + State: + description: >- + Permissible deployment states. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/deployment-status.md#status-attributes + permissible_values: + pending: + installing: + installed: + failed: + removing: + removed: + diff --git a/data-model/desired-state-manifest.linkml.yaml b/data-model/desired-state-manifest.linkml.yaml new file mode 100644 index 00000000..874d766a --- /dev/null +++ b/data-model/desired-state-manifest.linkml.yaml @@ -0,0 +1,148 @@ +# yaml-language-server: $schema=https://linkml.io/linkml-model/linkml_model/jsonschema/meta.schema.json +id: https://specification.margo.org/desired-state-manifest-schema +name: DesiredStateManifest +description: >- + Each workload is represented as an `ApplicationDeployment` YAML file that specifies its components, configuration, and parameters. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md +version: 1.0.0 +prefixes: + linkml: https://w3id.org/linkml/ + margo: https://specification.margo.org/ +imports: + - linkml:types + +default_prefix: margo +default_range: string + +classes: + DesiredStateManifest: + description: >- + Manifest from the Workload Fleet Manager, representing the complete desired workload configuration assigned to the device. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#endpoints---state-manifest + attributes: + manifestVersion: + description: >- + Monotonically increasing unsigned 64-bit integer in the inclusive range [1, 2^64-1]. Prevents rollback attacks. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#response-body-attributes + range: integer + minimum_value: 1 + required: true + bundle: + description: >- + Package optimization containing multiple ApplicationDeployment YAMLs. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#response-body-attributes + range: Bundle + required: true + deployments: + description: >- + List of deployment objects describing each workload. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#response-body-attributes + multivalued: true + inlined: true + inlined_as_list: true + range: Deployment + required: true + + Bundle: + description: >- + Describes an archive containing all referenced ApplicationDeployment YAMLs. + If there are zero deployments (i.e., the deployments array is empty), this field MUST be present with the value null. + An empty archive MUST NOT be served. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#endpoints---deployment-bundle + attributes: + mediaType: + description: >- + MUST be application/vnd.margo.bundle.v1+tar+gzip, + which denotes a gzip-compressed tar archive (commonly delivered as a .tar.gz) whose root contains one or more ApplicationDeployment YAML files. + Servers MUST set the HTTP Content-Type to this media type. + The archive MUST contain exactly the set of YAML files referenced by deployments. + range: string + equals_string: "application/vnd.margo.bundle.v1+tar+gzip" + required: true + digest: + description: >- + Digest of the bundle archive. + MUST equal the digest computed over the exact sequence of bytes in the bundle endpoint's HTTP 200 OK response body. + See Protocol - Digest for further details. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#protocol---digest + range: DigestType + required: true + sizeBytes: + description: >- + Optional unsigned 64-bit advisory estimate of the decoded payload length in bytes for the bundle archive. + Provided for bandwidth estimation and update planning. + MUST NOT be used for integrity verification. + range: SizeBytesType + url: + description: >- + Content-addressable retrieval endpoint for the bundle of the form /api/v1/clients/{clientId}/bundles/{digest} where {digest} equals bundle.digest. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#endpoints---deployment-bundle + range: UrlType + required: true + + Deployment: + description: >- + Reference to an individual ApplicationDeployment within the desired state manifest. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#response-body-attributes + attributes: + deploymentId: + description: >- + The UUID of the deployment. + MUST equal metadata.annotations.id in the ApplicationDeployment. + range: string + required: true + digest: + description: >- + Digest of the corresponding ApplicationDeployment YAML file. + MUST equal the digest computed over the exact sequence of bytes in the individual deployment endpoint's HTTP 200 OK response body. + See Protocol - Digest for further details. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#protocol---digest + range: DigestType + required: true + sizeBytes: + description: >- + Optional unsigned 64-bit advisory estimate of the decoded payload length in bytes for the ApplicationDeployment YAML. + Provided for bandwidth estimation and update planning. + MUST NOT be used for integrity verification. + range: SizeBytesType + url: + description: >- + Content-addressable retrieval endpoint for the ApplicationDeployment YAML of the form /api/v1/clients/{clientId}/deployments/{deploymentId}/{digest} where {digest} equals deployments[].digest. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#endpoints---individual-deployment-yaml + range: UrlType + required: true + +types: + DigestType: + description: >- + Hash that identifies an element by its content. + Digests have to follow the pattern: ":". + For example: "sha256:8a9b07...". + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#protocol---digest + uri: xsd:string + base: string + repr: str + pattern: "^[a-zA-Z0-9_-]+:[0-9a-fA-F]+$" + SizeBytesType: + description: Size of an element in bytes. + uri: xsd:nonNegativeInteger + base: integer + minimum_value: 0 + UrlType: + description: Endpoint (URL without schema and domain) associated to an element. + uri: xsd:anyURI + base: URI + repr: str + pattern: "^/[a-zA-Z0-9-._~:/?#\\[\\]@!$&'()*+,;=]+$" diff --git a/data-model/device-capabilities.linkml.yaml b/data-model/device-capabilities.linkml.yaml new file mode 100644 index 00000000..108907c6 --- /dev/null +++ b/data-model/device-capabilities.linkml.yaml @@ -0,0 +1,139 @@ +# yaml-language-server: $schema=https://linkml.io/linkml-model/linkml_model/jsonschema/meta.schema.json +id: https://specification.margo.org/device-capabilities +name: DeviceCapabilities +title: Device Capabilities +description: >- + Schema for defining device capabilities. + The purpose of the Device Capabilities is that WFMs can match application deployments with the capabilities of the target devices. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/device-capabilities.md +version: 1.0.0 +prefixes: + linkml: https://w3id.org/linkml/ + margo: https://specification.margo.org/ +imports: + - linkml:types + - margo-resources.linkml + +default_prefix: margo +# default_range: string # https://github.com/linkml/linkml/issues/1483 + + +# Class Definitions +classes: + DeviceCapabilitiesManifest: + description: >- + Capabilities of a device on which applications can be deployed. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/device-capabilities.md#request-body-attributes + attributes: + apiVersion: + description: Identifier of the version the API resource follows. + range: string + required: true + kind: + description: Must be `DeviceCapabilitiesManifest`. + range: string + required: true + equals_string: "DeviceCapabilitiesManifest" + designates_type: true + properties: + description: >- + Element that defines characteristics about the device. + See the [Properties Attributes](#properties-attributes) section below. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/device-capabilities.md#properties-attributes + required: true + range: Properties + + Properties: + description: >- + Device properties reported to the WFM. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/device-capabilities.md#properties-attributes + attributes: + id: + description: Unique deviceID assigned to the device via the Device Owner. + range: string + required: true + vendor: + description: Defines the device vendor. + range: string + required: true + modelNumber: + description: Defines the model number of the device. + range: string + required: true + serialNumber: + description: Defines the serial number of the device. + range: string + required: true + roles: + description: >- + Element that defines the device role it can provide to the Margo environment. + MUST be one of the following: Standalone Cluster, Cluster Leader, or Standalone Device. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/device-capabilities.md#enumerations + required: true + multivalued: true + inlined: false + range: DeviceRole + resources: + description: >- + Element that defines the device's resources available to the application deployed on the device. + See the [Resources Attributes](#resources-attributes) section below. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/device-capabilities.md#resources-attributes + required: true + range: Resources + + # Override Resources for device capabilities (availability offering): + # all fields are required here, whereas in application descriptions (resource claiming) + # they are optional. Include the slots list so the OpenAPI generator emits the full schema. + Resources: + slots: + - cpu + - memory + - storage + - peripherals + - interfaces + slot_usage: + cpu: + required: true + memory: + required: true + storage: + required: true + peripherals: + required: true + interfaces: + required: true + + +# Enumeration Definitions +enums: + DeviceRole: + description: >- + Role a device can provide to the Margo environment. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/device-capabilities.md#enumerations + permissible_values: + Standalone Cluster: + description: >- + Select this role to run Helm applications. + See [Edge compute devices](../../concepts/edge-compute-devices/devices#standalone-cluster-role-details) for more information. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/concepts/edge-compute-devices/devices.md#standalone-cluster-role-details + Standalone Device: + description: >- + Select this role to run Compose applications. + See [Edge compute devices](../../concepts/edge-compute-devices/devices#standalone-device-role-details) for more information. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/concepts/edge-compute-devices/devices.md#standalone-device-role-details + Cluster Leader: + description: >- + Select this role for the leader node of a multi-node Helm cluster. + See [Edge compute devices](../../concepts/edge-compute-devices/devices#cluster-leader-role-details) for more information. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/concepts/edge-compute-devices/devices.md#cluster-leader-role-details + diff --git a/data-model/margo-data-model.linkml.yaml b/data-model/margo-data-model.linkml.yaml new file mode 100644 index 00000000..1174c5aa --- /dev/null +++ b/data-model/margo-data-model.linkml.yaml @@ -0,0 +1,40 @@ +# yaml-language-server: $schema=https://linkml.io/linkml-model/linkml_model/jsonschema/meta.schema.json +id: https://specification.margo.org/data-model +name: DataModel +title: Margo Data Model +description: >- + Margo specification involves complex, interrelated data structures that are used for multiple, different purposes. + + This data model documents the mentioned data structures and their relationships. + + ??? info "General information about the Margo data model" + + The Margo data model has been modelled with **[LinkML](https://linkml.io)** + a **modelling language** that does not only offer very poserfull modelling capabilities, + but also very useful conversion and validation tooling. + + All **examples** shown on the documentation of the classes (see + [ApplicationDescription](ApplicationDescription.md#examples) for an example) are being automatically validated + against the model before a new version of the specification gets published. + + The whole [**HTML documentation** of the data model](.) is also automatically generated from the model. + This documentation provides some graphics that help visualicing the relationship between classes. + + Also following **OpenAPI** v3.0.3 YAML specifications are being automatically generated from the model: + + - [workload-management-api-1.0.0.openapi.yaml](../specification/margo-management-interface/management-interface-swagger/) +version: 1.0.0 +prefixes: + margo: https://specification.margo.org/ +imports: + - application-deployment.linkml + - application-description.linkml + - desired-state-manifest.linkml + - device-capabilities.linkml + - deployment-status.linkml + +default_prefix: margo + +subsets: + api_resources: + title: API Resources diff --git a/data-model/margo-deployments.linkml.yaml b/data-model/margo-deployments.linkml.yaml new file mode 100644 index 00000000..0226cbd4 --- /dev/null +++ b/data-model/margo-deployments.linkml.yaml @@ -0,0 +1,181 @@ +# yaml-language-server: $schema=https://linkml.io/linkml-model/linkml_model/jsonschema/meta.schema.json +id: https://specification.margo.org/deployments +name: Deployments +description: >- + Shared deployment profile, component, parameter, and target types used by both + ApplicationDescription and ApplicationDeployment schemas. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#deploymentprofile-attributes +version: 1.0.0 +prefixes: + linkml: https://w3id.org/linkml/ + margo: https://specification.margo.org/ +imports: + - linkml:types + +default_prefix: margo +default_range: string + + +classes: + DeploymentProfile: + description: >- + Represents a deployment configuration for the application. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#deploymentprofile-attributes + slots: + - type + - components + + HelmDeploymentProfile: + is_a: DeploymentProfile + slot_usage: + type: + equals_string: "helm.v3" + components: + range: HelmComponent + + ComposeDeploymentProfile: + is_a: DeploymentProfile + slot_usage: + type: + equals_string: "compose" + components: + range: ComposeComponent + + Component: + description: >- + A class representing a component of a deployment profile. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#component-attributes + attributes: + name: + description: >- + A unique name used to identify the component package. For helm installations the name will be used as the chart name. + The name must be lower case letters and numbers and MAY contain dashes. + Uppercase letters, underscores and periods MUST NOT be used. + required: true + range: string + properties: + description: >- + A dictionary element specifying the component packages's deployment details. + See the [Component Properties](#componentproperties-attributes) section below. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#component-attributes + range: ComponentProperties + required: true + + HelmComponent: + is_a: Component + + ComposeComponent: + is_a: Component + + ComponentProperties: + description: >- + Properties dictionary for component deployment details. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#component-attributes + attributes: + repository: + description: Repository location for the component. + range: string + revision: + description: Revision version for the component. + range: string + wait: + description: If True, indicates the device waits for the component installation to complete. + range: boolean + timeout: + description: Time to wait for component installation to complete, formatted as "##m##s". + range: string + packageLocation: + description: URL indicating the Compose package's location. + range: string + keyLocation: + description: URL for the public key used to validate a digitally signed package. + range: string + + Parameter: + description: >- + Defines a configurable parameter for the application. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#parameter-attributes + attributes: + name: + description: Name of the parameter. + identifier: true + required: true + range: string + value: + description: >- + The parameter's default value. + Accepted data types are string, integer, double, boolean, array[string], array[integer], array[double], array[boolean]. + any_of: #support for arrays still TBD + - range: boolean + - range: integer + - range: double + - range: string + targets: + description: >- + Used to indicate which component the value should be applied to when installing, or updating, the application. + See the [Target](#target-attributes) section below. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#target-attributes + range: Target + required: true + multivalued: true + inlined: true + inlined_as_list: true + + Target: + description: >- + Specifies where the parameter applies in the deployment. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#target-attributes + attributes: + pointer: + description: >- + The name of the parameter in the deployment configuration. + For Helm deployments, this is the dot notation for the matching element in the `values.yaml` file. This follows the same naming convention you would use with the `--set` command line argument with the `helm install` command. + For compose deployments, this is the name of the environment variable to set. + range: string + required: true + components: + description: >- + Indicates which deployment profile [component](#component-attributes the parameter target applies to. + The component name specified here MUST match a component name in the [deployment profiles](#deploymentprofile-attributes) section. + range: string + multivalued: true + required: true + + +slots: + type: + description: >- + Defines the type of this deployment configuration for the application. + The allowed values are `helm.v3`, to indicate the deployment profile's format is Helm version 3, + and `compose` to indicate the deployment profile's format is a Compose file. + When installing the application on a device supporting the Kubernetes platform, all `helm.v3` components, + and only `helm.v3` components, will be provided to the device in same order they are listed in the application description file. + When installing the application on a device supporting Compose, all `compose` components, + and only `compose` components, will be provided to the device in the same order they are listed in the application description file. + The device will install the components in the same order they are listed in the application description file. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#deploymentprofile-attributes + range: string + required: true + pattern: ^(helm\.v3|compose)$ + + components: + description: >- + Component element indicating the components to deploy when installing the application. + See the [Component](#component-attributes) section below. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/desired-state.md#component-attributes + range: Component + multivalued: true + required: true + inlined: true + inlined_as_list: true + diff --git a/data-model/margo-resources.linkml.yaml b/data-model/margo-resources.linkml.yaml new file mode 100644 index 00000000..37a86af7 --- /dev/null +++ b/data-model/margo-resources.linkml.yaml @@ -0,0 +1,198 @@ +# yaml-language-server: $schema=https://linkml.io/linkml-model/linkml_model/jsonschema/meta.schema.json +id: https://specification.margo.org/device-resources +name: DeviceResources +description: >- + Shared resource types used by both device capabilities and application descriptions + to describe hardware resources (CPU, memory, storage, peripherals, communication interfaces). + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/device-capabilities.md#resources-attributes +version: 1.0.0 +prefixes: + linkml: https://w3id.org/linkml/ + margo: https://specification.margo.org/ +imports: + - linkml:types + +default_prefix: margo +default_range: string + +classes: + Resources: + description: >- + Required resources element specifying the resources required to install the application. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/device-capabilities.md#resources-attributes + slots: + - cpu + - memory + - storage + - peripherals + - interfaces + + CPU: + description: >- + CPU element specifying the CPU requirements for the application. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/device-capabilities.md#cpu-attributes + attributes: + cores: + description: The required amount of CPU cores the application must use to run in its full functionality. + Specified as decimal units of CPU cores (e.g., `0.5` is half a core). + This is defined by the application developer. + After deployment of the application, the device MUST provide this number of CPU cores for the application. + rank: 10 + range: double + required: true + architectures: + description: The CPU architectures supported by the application. This can be e.g. amd64, x86_64, arm64, arm. + See the [CpuArchitectureType](#cpuarchitecturetype) definition for all permissible values. + Multiple arcitecture types can be specified, as the deployment profile may support multiple CPU architectures. + rank: 20 + range: CpuArchitectureType + multivalued: true + inlined: false + + Peripheral: + description: >- + Peripheral hardware of a device. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/device-capabilities.md#peripheral-attributes + attributes: + type: + description: The type of peripheral. This can be e.g. GPU, display, camera, microphone, speaker. + See the [PeriperalType](#peripheraltype) definition for all permissible values. + rank: 20 + range: PeripheralType + required: true + manufacturer: + description: The name of the manufacturer. If `manufacturer` is specified as a requirement here, it may be difficult to find devices that can host the application. Please use these requirements with caution. + rank: 30 + range: string + model: + description: The model of the peripheral. If `model` is specified as a requirement here, it may be difficult to find devices that can host the application. Please use these requirements with caution. + rank: 40 + range: string + + CommunicationInterface: + description: >- + Communication interface of a device. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/device-capabilities.md#communicationinterface-attributes + attributes: + type: + description: The type of a communication interface. This can be e.g. Ethernet, WiFi, Cellular, Bluetooth, USB, CANBus, RS232. + See the [CommunicationInterfaceType](#communicationinterfacetype) definition for all permissible values. + rank: 30 + range: CommunicationInterfaceType + required: true + +slots: + cpu: + description: >- + CPU element specifying the CPU requirements for the application. + See the [CPU](#cpu-attributes) section below. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/device-capabilities.md#cpu-attributes + range: CPU + + memory: + description: The minimum amount of memory required. + The value is given in binary units (`Ki` = Kibibytes, `Mi` = Mebibytes, `Gi` = Gibibytes). + This is defined by the application developer. + After deployment of the application, the device MUST provide this amount of memory for the application. + range: string + pattern: ^[0-9]+(Mi|Gi|Ki)$ + + storage: + description: The amount of storage required for the application to run. This encompasses the installed application and the data it needs to store. + The value is given in binary units (`Ki` = Kibibytes, `Mi` = Mebibytes, `Gi` = Gibibytes, `Ti` Tebibytes, `Pi` = Pebibytes, `Ei` = Exbibytes). + This is defined by the application developer. + After deployment of the application, the device MUST provide this amount of storage for the application + range: string + pattern: ^[0-9]+(Mi|Gi|Ki|Ti|Pi|Ei)$ + + peripherals: + description: >- + Peripherals element specifying the peripherals required to run the application. + See the [Peripheral](#peripheral-attributes) section below. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/device-capabilities.md#peripheral-attributes + range: Peripheral + multivalued: true + inlined: true + inlined_as_list: true + + interfaces: + description: >- + Interfaces element specifying the communication interfaces required to run the application. + See the [Communication Interfaces](#communicationinterface-attributes) section below. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/device-capabilities.md#communicationinterface-attributes + range: CommunicationInterface + multivalued: true + inlined: true + inlined_as_list: true + +enums: + CpuArchitectureType: + description: >- + Permissible CPU architecture values. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/device-capabilities.md#cpuarchitecturetype + permissible_values: + amd64: + description: AMD 64-bit architecture. + x86_64: + description: x86 64-bit architecture. + arm64: + description: ARM 64-bit architecture. + arm: + description: ARM 32-bit architecture. + riscv64: + description: RISC-V 64-bit architecture. + other: + description: >- + Any other CPU architecture not listed here. The application developer MUST provide a description of the architecture in the deployment profile's `description` field. + + CommunicationInterfaceType: + description: >- + Permissible communication interface types. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/device-capabilities.md#communicationinterfacetype + permissible_values: + ethernet: + description: This type stands for an Ethernet interface. + wifi: + description: This type stands for an WiFi interface. + cellular: + description: This type stands for cellular communication technologies such as 5G, LTE, 3G, 2G, .... + bluetooth: + description: This type stands for a Bluetooth or Bluetooth Low-Energy (BLE) interface. + usb: + description: This type stands for a USB interface. + canbus: + description: This type stands for a CANBus interface. + rs232: + description: This type stands for a RS232 interface. + other: + description: This type stands for any other communication interface not listed here. The application developer MUST provide a description of the interface in the deployment profile's `description` field. + + PeripheralType: + description: >- + Permissible peripheral types. + + Pre-data-model reference https://github.com/margo/specification/blob/3519aa7be4b565b2e2395134b99778fdc0379ace/system-design/specification/margo-management-interface/device-capabilities.md#peripheraltype + permissible_values: + gpu: + description: This type stands for a Graphics Processing Unit (GPU) peripheral. + display: + description: This type stands for a display peripheral. + camera: + description: This type stands for a camera peripheral. + microphone: + description: This type stands for a microphone peripheral. + speaker: + description: This type stands for a speaker peripheral. + other: + description: This type stands for any other peripheral not listed here. The application developer MUST provide a description of the peripheral in the deployment profile's `description` field. + diff --git a/data-model/resources/diagrams/all-classes.png b/data-model/resources/diagrams/all-classes.png new file mode 100644 index 00000000..a6db6586 Binary files /dev/null and b/data-model/resources/diagrams/all-classes.png differ diff --git a/data-model/resources/diagrams/application-deployment.png b/data-model/resources/diagrams/application-deployment.png new file mode 100644 index 00000000..551bbb69 Binary files /dev/null and b/data-model/resources/diagrams/application-deployment.png differ diff --git a/data-model/resources/diagrams/application-description.png b/data-model/resources/diagrams/application-description.png new file mode 100644 index 00000000..a1367140 Binary files /dev/null and b/data-model/resources/diagrams/application-description.png differ diff --git a/data-model/resources/diagrams/desired-state-manifest.png b/data-model/resources/diagrams/desired-state-manifest.png new file mode 100644 index 00000000..68525a50 Binary files /dev/null and b/data-model/resources/diagrams/desired-state-manifest.png differ diff --git a/data-model/resources/diagrams/device-capabilities.png b/data-model/resources/diagrams/device-capabilities.png new file mode 100644 index 00000000..4a79d152 Binary files /dev/null and b/data-model/resources/diagrams/device-capabilities.png differ diff --git a/data-model/resources/diagrams/margo-data-model.png b/data-model/resources/diagrams/margo-data-model.png new file mode 100644 index 00000000..2911969e Binary files /dev/null and b/data-model/resources/diagrams/margo-data-model.png differ diff --git a/data-model/resources/examples/invalid/ApplicationDeployment-001.yaml b/data-model/resources/examples/invalid/ApplicationDeployment-001.yaml new file mode 100644 index 00000000..e3438545 --- /dev/null +++ b/data-model/resources/examples/invalid/ApplicationDeployment-001.yaml @@ -0,0 +1,92 @@ +# Demonstrates validation of metadata.annotations.id +# Invalid kind of Kubernetes custom resource. +apiVersion: application.margo.org/v1alpha1 +# `kind` != `ApplicationDeployment` +kind: SomethingErroneous +metadata: + annotations: + applicationId: com-northstartida-digitron-orchestrator + id: a3e2f5dc-912e-494f-8395-52cf3769bc06 + name: com-northstartida-digitron-orchestrator-deployment + namespace: margo-poc +spec: + deploymentProfile: + type: helm.v3 + components: + - name: database-services + properties: + repository: oci://quay.io/charts/realtime-database-services + revision: 2.3.7 + timeout: 8m30s + wait: true + - name: digitron-orchestrator + properties: + repository: oci://northstarida.azurecr.io/charts/northstarida-digitron-orchestrator + revision: 1.0.9 + wait: true + parameters: + adminName: + value: Some One + targets: + - pointer: administrator.name + components: + - digitron-orchestrator + adminPrincipalName: + value: someone@somewhere.com + targets: + - pointer: administrator.userPrincipalName + components: + - digitron-orchestrator + cpuLimit: + value: "4" + targets: + - pointer: settings.limits.cpu + components: + - digitron-orchestrator + idpClientId: + value: 123-ABC + targets: + - pointer: idp.clientId + components: + - digitron-orchestrator + idpName: + value: Azure AD + targets: + - pointer: idp.name + components: + - digitron-orchestrator + idpProvider: + value: aad + targets: + - pointer: idp.provider + components: + - digitron-orchestrator + idpUrl: + value: https://123-abc.com + targets: + - pointer: idp.providerUrl + components: + - digitron-orchestrator + - pointer: idp.providerMetadata + components: + - digitron-orchestrator + memoryLimit: + value: "16384" + targets: + - pointer: settings.limits.memory + components: + - digitron-orchestrator + pollFrequency: + value: "120" + targets: + - pointer: settings.pollFrequency + components: + - digitron-orchestrator + - database-services + siteId: + value: SID-123-ABC + targets: + - pointer: settings.siteId + components: + - digitron-orchestrator + - database-services diff --git a/data-model/resources/examples/invalid/ApplicationDeployment-002.yaml b/data-model/resources/examples/invalid/ApplicationDeployment-002.yaml new file mode 100644 index 00000000..8876caa5 --- /dev/null +++ b/data-model/resources/examples/invalid/ApplicationDeployment-002.yaml @@ -0,0 +1,92 @@ +# Demonstrates validation of metadata.annotations.applicationId +# Invalid characters being used. +apiVersion: application.margo.org/v1alpha1 +kind: ApplicationDeployment +metadata: + annotations: + # id contains invalid characters: upper-case letters ('A' and 'Z'), special characters ('.' and '_') + applicationId: com_northstartida.Digitron-Orchestrator + id: a3e2f5dc-912e-494f-8395-52cf3769bc06 + name: com-northstartida-digitron-orchestrator-deployment + namespace: margo-poc +spec: + deploymentProfile: + type: helm.v3 + components: + - name: database-services + properties: + repository: oci://quay.io/charts/realtime-database-services + revision: 2.3.7 + timeout: 8m30s + wait: true + - name: digitron-orchestrator + properties: + repository: oci://northstarida.azurecr.io/charts/northstarida-digitron-orchestrator + revision: 1.0.9 + wait: true + parameters: + adminName: + value: Some One + targets: + - pointer: administrator.name + components: + - digitron-orchestrator + adminPrincipalName: + value: someone@somewhere.com + targets: + - pointer: administrator.userPrincipalName + components: + - digitron-orchestrator + cpuLimit: + value: "4" + targets: + - pointer: settings.limits.cpu + components: + - digitron-orchestrator + idpClientId: + value: 123-ABC + targets: + - pointer: idp.clientId + components: + - digitron-orchestrator + idpName: + value: Azure AD + targets: + - pointer: idp.name + components: + - digitron-orchestrator + idpProvider: + value: aad + targets: + - pointer: idp.provider + components: + - digitron-orchestrator + idpUrl: + value: https://123-abc.com + targets: + - pointer: idp.providerUrl + components: + - digitron-orchestrator + - pointer: idp.providerMetadata + components: + - digitron-orchestrator + memoryLimit: + value: "16384" + targets: + - pointer: settings.limits.memory + components: + - digitron-orchestrator + pollFrequency: + value: "120" + targets: + - pointer: settings.pollFrequency + components: + - digitron-orchestrator + - database-services + siteId: + value: SID-123-ABC + targets: + - pointer: settings.siteId + components: + - digitron-orchestrator + - database-services diff --git a/data-model/resources/examples/invalid/ApplicationDeployment-003.yaml b/data-model/resources/examples/invalid/ApplicationDeployment-003.yaml new file mode 100644 index 00000000..62513516 --- /dev/null +++ b/data-model/resources/examples/invalid/ApplicationDeployment-003.yaml @@ -0,0 +1,92 @@ +# Demonstrates validation of metadata.annotations.applicationId +# Too long. +apiVersion: application.margo.org/v1alpha1 +kind: ApplicationDeployment +metadata: + annotations: + # id is over 200 characters long (it is 201 characters long) + applicationId: "123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901" + id: a3e2f5dc-912e-494f-8395-52cf3769bc06 + name: com-northstartida-digitron-orchestrator-deployment + namespace: margo-poc +spec: + deploymentProfile: + type: helm.v3 + components: + - name: database-services + properties: + repository: oci://quay.io/charts/realtime-database-services + revision: 2.3.7 + timeout: 8m30s + wait: true + - name: digitron-orchestrator + properties: + repository: oci://northstarida.azurecr.io/charts/northstarida-digitron-orchestrator + revision: 1.0.9 + wait: true + parameters: + adminName: + value: Some One + targets: + - pointer: administrator.name + components: + - digitron-orchestrator + adminPrincipalName: + value: someone@somewhere.com + targets: + - pointer: administrator.userPrincipalName + components: + - digitron-orchestrator + cpuLimit: + value: "4" + targets: + - pointer: settings.limits.cpu + components: + - digitron-orchestrator + idpClientId: + value: 123-ABC + targets: + - pointer: idp.clientId + components: + - digitron-orchestrator + idpName: + value: Azure AD + targets: + - pointer: idp.name + components: + - digitron-orchestrator + idpProvider: + value: aad + targets: + - pointer: idp.provider + components: + - digitron-orchestrator + idpUrl: + value: https://123-abc.com + targets: + - pointer: idp.providerUrl + components: + - digitron-orchestrator + - pointer: idp.providerMetadata + components: + - digitron-orchestrator + memoryLimit: + value: "16384" + targets: + - pointer: settings.limits.memory + components: + - digitron-orchestrator + pollFrequency: + value: "120" + targets: + - pointer: settings.pollFrequency + components: + - digitron-orchestrator + - database-services + siteId: + value: SID-123-ABC + targets: + - pointer: settings.siteId + components: + - digitron-orchestrator + - database-services diff --git a/data-model/resources/examples/invalid/ApplicationDeployment-004.yaml b/data-model/resources/examples/invalid/ApplicationDeployment-004.yaml new file mode 100644 index 00000000..ec985bec --- /dev/null +++ b/data-model/resources/examples/invalid/ApplicationDeployment-004.yaml @@ -0,0 +1,92 @@ +# Demonstrates validation of metadata.annotations.id +# Not a valid UUID. +apiVersion: application.margo.org/v1alpha1 +kind: ApplicationDeployment +metadata: + annotations: + applicationId: com-northstartida-digitron-orchestrator + # id is not a valid UUID + id: this-is-definitively-not-a-uuid + name: com-northstartida-digitron-orchestrator-deployment + namespace: margo-poc +spec: + deploymentProfile: + type: helm.v3 + components: + - name: database-services + properties: + repository: oci://quay.io/charts/realtime-database-services + revision: 2.3.7 + timeout: 8m30s + wait: true + - name: digitron-orchestrator + properties: + repository: oci://northstarida.azurecr.io/charts/northstarida-digitron-orchestrator + revision: 1.0.9 + wait: true + parameters: + adminName: + value: Some One + targets: + - pointer: administrator.name + components: + - digitron-orchestrator + adminPrincipalName: + value: someone@somewhere.com + targets: + - pointer: administrator.userPrincipalName + components: + - digitron-orchestrator + cpuLimit: + value: "4" + targets: + - pointer: settings.limits.cpu + components: + - digitron-orchestrator + idpClientId: + value: 123-ABC + targets: + - pointer: idp.clientId + components: + - digitron-orchestrator + idpName: + value: Azure AD + targets: + - pointer: idp.name + components: + - digitron-orchestrator + idpProvider: + value: aad + targets: + - pointer: idp.provider + components: + - digitron-orchestrator + idpUrl: + value: https://123-abc.com + targets: + - pointer: idp.providerUrl + components: + - digitron-orchestrator + - pointer: idp.providerMetadata + components: + - digitron-orchestrator + memoryLimit: + value: "16384" + targets: + - pointer: settings.limits.memory + components: + - digitron-orchestrator + pollFrequency: + value: "120" + targets: + - pointer: settings.pollFrequency + components: + - digitron-orchestrator + - database-services + siteId: + value: SID-123-ABC + targets: + - pointer: settings.siteId + components: + - digitron-orchestrator + - database-services diff --git a/data-model/resources/examples/invalid/ApplicationDescription-001.yaml b/data-model/resources/examples/invalid/ApplicationDescription-001.yaml new file mode 100644 index 00000000..02c3ae6d --- /dev/null +++ b/data-model/resources/examples/invalid/ApplicationDescription-001.yaml @@ -0,0 +1,59 @@ +apiVersion: margo.org/v1-alpha1 +kind: ApplicationSpecification +metadata: + id: com-northstartida-hello-world + name: Hello World + description: A basic hello world application + version: "1.0" + catalog: + application: + icon: ./resources/hw-logo.png + tagline: Northstar Industrial Application's hello world application. + descriptionFile: ./resources/description.md + releaseNotes: ./resources/release-notes.md + licenseFile: ./resources/license.pdf + site: http://www.northstar-ida.com + tags: ["monitoring"] + author: + - name: Roger Wilkershank + email: rpwilkershank@northstar-ida.com + organization: + - name: Northstar Industrial Applications + site: http://northstar-ida.com +deploymentProfiles: + - type: helm.v3 + id: com-northstartida-hello-world-helm.v3-a + components: + - name: hello-world + properties: + repository: oci://northstarida.azurecr.io/charts/hello-world + revision: 1.0.1 + wait: true +parameters: + greeting: + value: Hello + targets: + - pointer: global.config.appGreeting + components: ["hello-world"] + greetingAddressee: + value: World + targets: + - pointer: global.config.appGreetingAddressee + components: ["hello-world"] +configuration: + sections: + - name: General Settings + settings: + - parameter: greeting + name: Greeting + description: The greeting to use. + schema: requireText + - parameter: greetingAddressee + name: Greeting Addressee + description: The person, or group, the greeting addresses. + schema: requireText + schema: + - name: requireText + dataType: string + maxLength: 45 + allowEmpty: false diff --git a/data-model/resources/examples/invalid/DeploymentStatusManifest-001.json b/data-model/resources/examples/invalid/DeploymentStatusManifest-001.json new file mode 100644 index 00000000..92dcd322 --- /dev/null +++ b/data-model/resources/examples/invalid/DeploymentStatusManifest-001.json @@ -0,0 +1,14 @@ +{ + "apiVersion": "v1", + "kind": "WrongKind", + "deploymentId": "a3e2f5dc-912e-494f-8395-52cf3769bc06", + "status": { + "state": "installed" + }, + "components": [ + { + "name": "frontend", + "state": "installed" + } + ] +} diff --git a/data-model/resources/examples/invalid/DeploymentStatusManifest-002.json b/data-model/resources/examples/invalid/DeploymentStatusManifest-002.json new file mode 100644 index 00000000..031d997f --- /dev/null +++ b/data-model/resources/examples/invalid/DeploymentStatusManifest-002.json @@ -0,0 +1,14 @@ +{ + "apiVersion": "v1", + "kind": "DeploymentStatusManifest", + "deploymentId": "a3e2f5dc-912e-494f-8395-52cf3769bc06", + "status": { + "state": "running" + }, + "components": [ + { + "name": "frontend", + "state": "installed" + } + ] +} diff --git a/data-model/resources/examples/invalid/DeploymentStatusManifest-003.json b/data-model/resources/examples/invalid/DeploymentStatusManifest-003.json new file mode 100644 index 00000000..672f3fc6 --- /dev/null +++ b/data-model/resources/examples/invalid/DeploymentStatusManifest-003.json @@ -0,0 +1,13 @@ +{ + "apiVersion": "v1", + "kind": "DeploymentStatusManifest", + "status": { + "state": "installed" + }, + "components": [ + { + "name": "frontend", + "state": "installed" + } + ] +} diff --git a/data-model/resources/examples/invalid/DesiredStateManifest-001.json b/data-model/resources/examples/invalid/DesiredStateManifest-001.json new file mode 100644 index 00000000..c0488575 --- /dev/null +++ b/data-model/resources/examples/invalid/DesiredStateManifest-001.json @@ -0,0 +1,15 @@ +{ + "manifestVersion": 0, + "bundle": { + "mediaType": "application/vnd.margo.bundle.v1+tar+gzip", + "digest": "sha256:01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b", + "url": "/api/v1/clients/1234/bundles/sha256:01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b" + }, + "deployments": [ + { + "deploymentId": "a3e2f5dc-912e-494f-8395-52cf3769bc06", + "digest": "sha256:01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b", + "url": "/api/v1/clients/1234/deployments/a3e2f5dc-912e-494f-8395-52cf3769bc06/sha256:01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b" + } + ] +} diff --git a/data-model/resources/examples/invalid/DesiredStateManifest-002.json b/data-model/resources/examples/invalid/DesiredStateManifest-002.json new file mode 100644 index 00000000..431dd775 --- /dev/null +++ b/data-model/resources/examples/invalid/DesiredStateManifest-002.json @@ -0,0 +1,15 @@ +{ + "manifestVersion": 101, + "bundle": { + "mediaType": "application/json", + "digest": "sha256:01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b", + "url": "/api/v1/clients/1234/bundles/sha256:01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b" + }, + "deployments": [ + { + "deploymentId": "a3e2f5dc-912e-494f-8395-52cf3769bc06", + "digest": "sha256:01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b", + "url": "/api/v1/clients/1234/deployments/a3e2f5dc-912e-494f-8395-52cf3769bc06/sha256:01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b" + } + ] +} diff --git a/data-model/resources/examples/invalid/DesiredStateManifest-003.json b/data-model/resources/examples/invalid/DesiredStateManifest-003.json new file mode 100644 index 00000000..16bac6aa --- /dev/null +++ b/data-model/resources/examples/invalid/DesiredStateManifest-003.json @@ -0,0 +1,15 @@ +{ + "manifestVersion": 101, + "bundle": { + "mediaType": "application/vnd.margo.bundle.v1+tar+gzip", + "digest": "invalid-digest", + "url": "/api/v1/clients/1234/bundles/invalid-digest" + }, + "deployments": [ + { + "deploymentId": "a3e2f5dc-912e-494f-8395-52cf3769bc06", + "digest": "sha256:01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b", + "url": "/api/v1/clients/1234/deployments/a3e2f5dc-912e-494f-8395-52cf3769bc06/sha256:01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b" + } + ] +} diff --git a/data-model/resources/examples/invalid/DesiredStateManifest-004.json b/data-model/resources/examples/invalid/DesiredStateManifest-004.json new file mode 100644 index 00000000..a53980ac --- /dev/null +++ b/data-model/resources/examples/invalid/DesiredStateManifest-004.json @@ -0,0 +1,10 @@ +{ + "manifestVersion": 101, + "deployments": [ + { + "deploymentId": "a3e2f5dc-912e-494f-8395-52cf3769bc06", + "digest": "sha256:01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b", + "url": "/api/v1/clients/1234/deployments/a3e2f5dc-912e-494f-8395-52cf3769bc06/sha256:01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b" + } + ] +} diff --git a/data-model/resources/examples/invalid/DeviceCapabilitiesManifest-001.json b/data-model/resources/examples/invalid/DeviceCapabilitiesManifest-001.json new file mode 100644 index 00000000..0388d106 --- /dev/null +++ b/data-model/resources/examples/invalid/DeviceCapabilitiesManifest-001.json @@ -0,0 +1,37 @@ +{ + "apiVersion": "device.margo.org/v1alpha1", + "kind": "DeviceCapabilitiesManifest", + "properties": { + "id": "northstarida.xtapro.k8s.edge", + "vendor": "Northstar Industrial devices", + "modelNumber": "332ANZE1-N1", + "serialNumber": "PF45343-AA", + "roles": [ + "Non existing role" + ], + "resources": { + "cpu": { + "cores": 24, + "architectures": [ + "x86_64" + ] + }, + "memory": "59Gi", + "storage": "1862Gi", + "peripherals": [ + { + "type": "gpu", + "manufacturer": "NVIDIA" + } + ], + "interfaces": [ + { + "type": "ethernet" + }, + { + "type": "wifi" + } + ] + } + } +} diff --git a/data-model/resources/examples/invalid/DeviceCapabilitiesManifest-002.json b/data-model/resources/examples/invalid/DeviceCapabilitiesManifest-002.json new file mode 100644 index 00000000..4c7afc02 --- /dev/null +++ b/data-model/resources/examples/invalid/DeviceCapabilitiesManifest-002.json @@ -0,0 +1,36 @@ +{ + "apiVersion": "device.margo.org/v1alpha1", + "kind": "DeviceCapabilitiesManifest", + "properties": { + "id": "northstarida.xtapro.k8s.edge", + "modelNumber": "332ANZE1-N1", + "serialNumber": "PF45343-AA", + "roles": [ + "Standalone Device" + ], + "resources": { + "cpu": { + "cores": 24, + "architectures": [ + "x86_64" + ] + }, + "memory": "59Gi", + "storage": "1862Gi", + "peripherals": [ + { + "type": "gpu", + "manufacturer": "NVIDIA" + } + ], + "interfaces": [ + { + "type": "ethernet" + }, + { + "type": "wifi" + } + ] + } + } +} diff --git a/data-model/resources/examples/invalid/DeviceCapabilitiesManifest-003.json b/data-model/resources/examples/invalid/DeviceCapabilitiesManifest-003.json new file mode 100644 index 00000000..f0a16de2 --- /dev/null +++ b/data-model/resources/examples/invalid/DeviceCapabilitiesManifest-003.json @@ -0,0 +1,37 @@ +{ + "apiVersion": "device.margo.org/v1alpha1", + "kind": "DeviceCapabilitiesManifest", + "properties": { + "id": "northstarida.xtapro.k8s.edge", + "vendor": "Northstar Industrial devices", + "modelNumber": "332ANZE1-N1", + "serialNumber": "PF45343-AA", + "roles": [ + "Standalone Cluster" + ], + "resources": { + "cpu": { + "cores": 24, + "architectures": [ + "riscv" + ] + }, + "memory": "59Gi", + "storage": "1862Gi", + "peripherals": [ + { + "type": "gpu", + "manufacturer": "NVIDIA" + } + ], + "interfaces": [ + { + "type": "ethernet" + }, + { + "type": "wifi" + } + ] + } + } +} diff --git a/data-model/resources/examples/valid/ApplicationDeployment-compose.yaml b/data-model/resources/examples/valid/ApplicationDeployment-compose.yaml new file mode 100644 index 00000000..e48606ec --- /dev/null +++ b/data-model/resources/examples/valid/ApplicationDeployment-compose.yaml @@ -0,0 +1,65 @@ +apiVersion: application.margo.org/v1alpha1 +kind: ApplicationDeployment +metadata: + annotations: + applicationId: com-northstartida-digitron-orchestrator + id: ad9b614e-8912-45f4-a523-372358765def + name: com-northstartida-digitron-orchestrator-deployment + namespace: margo-poc +spec: + deploymentProfile: + type: compose + components: + - name: digitron-orchestrator-docker + properties: + keyLocation: https://northsitarida.com/digitron/docker/public-key.asc + packageLocation: https://northsitarida.com/digitron/docker/digitron-orchestrator.tar.gz + parameters: + adminName: + value: Some One + targets: + - pointer: ENV.ADMIN_NAME + components: + - digitron-orchestrator-docker + adminPrincipalName: + value: someone@somewhere.com + targets: + - pointer: ENV.ADMIN_PRINCIPALNAME + components: + - digitron-orchestrator-docker + idpClientId: + value: 123-ABC + targets: + - pointer: ENV.IDP_CLIENT_ID + components: + - digitron-orchestrator-docker + idpName: + value: Azure AD + targets: + - pointer: ENV.IDP_NAME + components: + - digitron-orchestrator-docker + idpProvider: + value: aad + targets: + - pointer: ENV.IDP_PROVIDER + components: + - digitron-orchestrator-docker + idpUrl: + value: https://123-abc.com + targets: + - pointer: ENV.IDP_URL + components: + - digitron-orchestrator-docker + pollFrequency: + value: "120" + targets: + - pointer: ENV.POLL_FREQUENCY + components: + - digitron-orchestrator-docker + siteId: + value: SID-123-ABC + targets: + - pointer: ENV.SITE_ID + components: + - digitron-orchestrator-docker diff --git a/data-model/resources/examples/valid/ApplicationDeployment-helm.yaml b/data-model/resources/examples/valid/ApplicationDeployment-helm.yaml new file mode 100644 index 00000000..9baa6032 --- /dev/null +++ b/data-model/resources/examples/valid/ApplicationDeployment-helm.yaml @@ -0,0 +1,89 @@ +apiVersion: application.margo.org/v1alpha1 +kind: ApplicationDeployment +metadata: + annotations: + applicationId: com-northstartida-digitron-orchestrator + id: a3e2f5dc-912e-494f-8395-52cf3769bc06 + name: com-northstartida-digitron-orchestrator-deployment + namespace: margo-poc +spec: + deploymentProfile: + type: helm.v3 + components: + - name: database-services + properties: + repository: oci://quay.io/charts/realtime-database-services + revision: 2.3.7 + timeout: 8m30s + wait: true + - name: digitron-orchestrator + properties: + repository: oci://northstarida.azurecr.io/charts/northstarida-digitron-orchestrator + revision: 1.0.9 + wait: true + parameters: + adminName: + value: Some One + targets: + - pointer: administrator.name + components: + - digitron-orchestrator + adminPrincipalName: + value: someone@somewhere.com + targets: + - pointer: administrator.userPrincipalName + components: + - digitron-orchestrator + cpuLimit: + value: "4" + targets: + - pointer: settings.limits.cpu + components: + - digitron-orchestrator + idpClientId: + value: 123-ABC + targets: + - pointer: idp.clientId + components: + - digitron-orchestrator + idpName: + value: Azure AD + targets: + - pointer: idp.name + components: + - digitron-orchestrator + idpProvider: + value: aad + targets: + - pointer: idp.provider + components: + - digitron-orchestrator + idpUrl: + value: https://123-abc.com + targets: + - pointer: idp.providerUrl + components: + - digitron-orchestrator + - pointer: idp.providerMetadata + components: + - digitron-orchestrator + memoryLimit: + value: "16384" + targets: + - pointer: settings.limits.memory + components: + - digitron-orchestrator + pollFrequency: + value: "120" + targets: + - pointer: settings.pollFrequency + components: + - digitron-orchestrator + - database-services + siteId: + value: SID-123-ABC + targets: + - pointer: settings.siteId + components: + - digitron-orchestrator + - database-services diff --git a/data-model/resources/examples/valid/ApplicationDescription-helm_and_compose.yaml b/data-model/resources/examples/valid/ApplicationDescription-helm_and_compose.yaml new file mode 100644 index 00000000..2d76d63d --- /dev/null +++ b/data-model/resources/examples/valid/ApplicationDescription-helm_and_compose.yaml @@ -0,0 +1,208 @@ +apiVersion: margo.org/v1-alpha1 +kind: ApplicationDescription +metadata: + id: com-northstartida-digitron-orchestrator + name: Digitron orchestrator + description: The Digitron orchestrator application + version: 1.2.1 + catalog: + application: + icon: ./resources/ndo-logo.png + tagline: Northstar Industrial Application's next-gen, AI driven, Digitron instrument orchestrator. + descriptionFile: ./resources/description.md + releaseNotes: ./resources/release-notes.md + licenseFile: ./resources/license.pdf + site: http://www.northstar-ida.com + tags: ["optimization", "instrumentation"] + author: + - name: Roger Wilkershank + email: rpwilkershank@northstar-ida.com + organization: + - name: Northstar Industrial Applications + site: http://northstar-ida.com +deploymentProfiles: + - type: helm.v3 + id: com-northstartida-digitron-orchestrator-helm.v3-a + description: This allows to install / run the application as a Helm chart deployment. + The device where this application is installed needs to have a screen and a keyboard (as indicated in the required peripherals). + components: + - name: database-services + properties: + repository: oci://quay.io/charts/realtime-database-services + revision: 2.3.7 + wait: true + timeout: 8m30s + - name: digitron-orchestrator + properties: + repository: oci://northstarida.azurecr.io/charts/northstarida-digitron-orchestrator + revision: 1.0.9 + wait: true + requiredResources: + cpu: + cores: 1.5 + architectures: + - amd64 + - x86_64 + memory: 1024Mi + storage: 10Gi + peripherals: + - type: gpu + manufacturer: NVIDIA + - type: display + interfaces: + - type: ethernet + - type: bluetooth + - type: compose + id: com-northstartida-digitron-orchestrator-compose-a + components: + - name: digitron-orchestrator-docker + properties: + packageLocation: https://northsitarida.com/digitron/docker/digitron-orchestrator.tar.gz + keyLocation: https://northsitarida.com/digitron/docker/public-key.asc +parameters: + idpName: + targets: + - pointer: idp.name + components: ["digitron-orchestrator"] + - pointer: ENV.IDP_NAME + components: ["digitron-orchestrator-docker"] + idpProvider: + targets: + - pointer: idp.provider + components: ["digitron-orchestrator"] + - pointer: ENV.IDP_PROVIDER + components: ["digitron-orchestrator-docker"] + idpClientId: + targets: + - pointer: idp.clientId + components: ["digitron-orchestrator"] + - pointer: ENV.IDP_CLIENT_ID + components: ["digitron-orchestrator-docker"] + idpUrl: + targets: + - pointer: idp.providerUrl + components: ["digitron-orchestrator"] + - pointer: idp.providerMetadata + components: ["digitron-orchestrator"] + - pointer: ENV.IDP_URL + components: ["digitron-orchestrator-docker"] + adminName: + targets: + - pointer: administrator.name + components: ["digitron-orchestrator"] + - pointer: ENV.ADMIN_NAME + components: ["digitron-orchestrator-docker"] + adminPrincipalName: + targets: + - pointer: administrator.userPrincipalName + components: ["digitron-orchestrator"] + - pointer: ENV.ADMIN_PRINCIPALNAME + components: ["digitron-orchestrator-docker"] + pollFrequency: + value: 30 + targets: + - pointer: settings.pollFrequency + components: ["digitron-orchestrator", "database-services"] + - pointer: ENV.POLL_FREQUENCY + components: ["digitron-orchestrator-docker"] + siteId: + targets: + - pointer: settings.siteId + components: ["digitron-orchestrator", "database-services"] + - pointer: ENV.SITE_ID + components: ["digitron-orchestrator-docker"] + cpuLimit: + value: 1 + targets: + - pointer: settings.limits.cpu + components: ["digitron-orchestrator"] + memoryLimit: + value: 16384 + targets: + - pointer: settings.limits.memory + components: ["digitron-orchestrator"] +configuration: + sections: + - name: General + settings: + - parameter: pollFrequency + name: Poll Frequency + description: How often the service polls for updated data in seconds + schema: pollRange + - parameter: siteId + name: Site Id + description: Special identifier for the site (optional) + schema: optionalText + - name: Identity Provider + settings: + - parameter: idpName + name: Name + description: The name of the Identity Provider to use + immutable: true + schema: requiredText + - parameter: idpProvider + name: Provider + description: Provider something something + immutable: true + schema: requiredText + - parameter: idpClientId + name: Client ID + description: The client id + immutable: true + schema: requiredText + - parameter: idpUrl + name: Provider URL + description: The url of the Identity Provider + immutable: true + schema: url + - name: Administrator + settings: + - parameter: adminName + name: Presentation Name + description: The presentation name of the administrator + schema: requiredText + - parameter: adminPrincipalName + name: Principal Name + description: The principal name of the administrator + schema: email + - name: Resource Limits + settings: + - parameter: cpuLimit + name: CPU Limit + description: Maximum number of CPU cores to allow the application to consume + schema: cpuRange + - parameter: memoryLimit + name: Memory Limit + description: Maximum number of memory to allow the application to consume + schema: memoryRange + schema: + - name: requiredText + dataType: string + maxLength: 45 + allowEmpty: false + - name: email + dataType: string + allowEmpty: false + regexMatch: .*@[a-z0-9.-]* + - name: url + dataType: string + allowEmpty: false + regexMatch: ^(http(s):\/\/.)[-a-zA-Z0-9@:%._\+~#=]{2,256}\.[a-z]{2,6}\b([-a-zA-Z0-9@:%_\+.~#?&//=]*)$ + - name: pollRange + dataType: integer + minValue: 30 + maxValue: 360 + allowEmpty: false + - name: optionalText + dataType: string + minLength: 5 + allowEmpty: true + - name: cpuRange + dataType: double + minValue: 0.5 + maxPrecision: 1 + allowEmpty: false + - name: memoryRange + dataType: integer + minValue: 16384 + allowEmpty: false diff --git a/data-model/resources/examples/valid/ApplicationDescription-helm_only.yaml b/data-model/resources/examples/valid/ApplicationDescription-helm_only.yaml new file mode 100644 index 00000000..158abb88 --- /dev/null +++ b/data-model/resources/examples/valid/ApplicationDescription-helm_only.yaml @@ -0,0 +1,59 @@ +apiVersion: margo.org/v1-alpha1 +kind: ApplicationDescription +metadata: + id: com-northstartida-hello-world + name: Hello World + description: A basic hello world application + version: "1.0" + catalog: + application: + icon: ./resources/hw-logo.png + tagline: Northstar Industrial Application's hello world application. + descriptionFile: ./resources/description.md + releaseNotes: ./resources/release-notes.md + licenseFile: ./resources/license.pdf + site: http://www.northstar-ida.com + tags: ["monitoring"] + author: + - name: Roger Wilkershank + email: rpwilkershank@northstar-ida.com + organization: + - name: Northstar Industrial Applications + site: http://northstar-ida.com +deploymentProfiles: + - type: helm.v3 + id: com-northstartida-hello-world-helm.v3-a + components: + - name: hello-world + properties: + repository: oci://northstarida.azurecr.io/charts/hello-world + revision: 1.0.1 + wait: true +parameters: + greeting: + value: Hello + targets: + - pointer: global.config.appGreeting + components: ["hello-world"] + greetingAddressee: + value: World + targets: + - pointer: global.config.appGreetingAddressee + components: ["hello-world"] +configuration: + sections: + - name: General Settings + settings: + - parameter: greeting + name: Greeting + description: The greeting to use. + schema: requireText + - parameter: greetingAddressee + name: Greeting Addressee + description: The person, or group, the greeting addresses. + schema: requireText + schema: + - name: requireText + dataType: string + maxLength: 45 + allowEmpty: false diff --git a/data-model/resources/examples/valid/DeploymentStatusManifest-001.json b/data-model/resources/examples/valid/DeploymentStatusManifest-001.json new file mode 100644 index 00000000..01153e65 --- /dev/null +++ b/data-model/resources/examples/valid/DeploymentStatusManifest-001.json @@ -0,0 +1,22 @@ +{ + "apiVersion": "v1", + "kind": "DeploymentStatusManifest", + "deploymentId": "a3e2f5dc-912e-494f-8395-52cf3769bc06", + "status": { + "state": "installed" + }, + "components": [ + { + "name": "frontend", + "state": "installed" + }, + { + "name": "backend", + "state": "failed", + "error": { + "code": "ERR_IMAGE_PULL", + "message": "Failed to pull image: access denied" + } + } + ] +} diff --git a/data-model/resources/examples/valid/DesiredStateManifest-001.json b/data-model/resources/examples/valid/DesiredStateManifest-001.json new file mode 100644 index 00000000..3fb1b073 --- /dev/null +++ b/data-model/resources/examples/valid/DesiredStateManifest-001.json @@ -0,0 +1,15 @@ +{ + "manifestVersion": 101, + "bundle": { + "mediaType": "application/vnd.margo.bundle.v1+tar+gzip", + "digest": "sha256:01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b", + "url": "/api/v1/clients/1234/bundles/sha256:01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b" + }, + "deployments": [ + { + "deploymentId": "a3e2f5dc-912e-494f-8395-52cf3769bc06", + "digest": "sha256:01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b", + "url": "/api/v1/clients/1234/deployments/a3e2f5dc-912e-494f-8395-52cf3769bc06/sha256:01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b" + } + ] +} diff --git a/data-model/resources/examples/valid/DeviceCapabilitiesManifest-001.json b/data-model/resources/examples/valid/DeviceCapabilitiesManifest-001.json new file mode 100644 index 00000000..502bd828 --- /dev/null +++ b/data-model/resources/examples/valid/DeviceCapabilitiesManifest-001.json @@ -0,0 +1,37 @@ +{ + "apiVersion": "device.margo.org/v1alpha1", + "kind": "DeviceCapabilitiesManifest", + "properties": { + "id": "northstarida.xtapro.k8s.edge", + "vendor": "Northstar Industrial devices", + "modelNumber": "332ANZE1-N1", + "serialNumber": "PF45343-AA", + "roles": [ + "Standalone Cluster" + ], + "resources": { + "cpu": { + "cores": 24, + "architectures": [ + "x86_64" + ] + }, + "memory": "59Gi", + "storage": "1862Gi", + "peripherals": [ + { + "type": "gpu", + "manufacturer": "NVIDIA" + } + ], + "interfaces": [ + { + "type": "ethernet" + }, + { + "type": "wifi" + } + ] + } + } +} diff --git a/data-model/resources/markdown-templates/class.md.jinja2 b/data-model/resources/markdown-templates/class.md.jinja2 new file mode 100644 index 00000000..ce23fbc6 --- /dev/null +++ b/data-model/resources/markdown-templates/class.md.jinja2 @@ -0,0 +1,288 @@ +{%- if element.title %} + {%- set title = element.title ~ ' (' ~ element.name ~ ')' -%} +{%- else %} + {%- if gen.use_class_uris -%} + {%- set title = element.name -%} + {%- else -%} + {%- set title = gen.name(element) -%} + {%- endif -%} +{%- endif -%} + +{% macro compute_range(slot) -%} + {%- if slot.any_of or slot.exactly_one_of -%} + {%- for subslot_range in schemaview.slot_range_as_union(slot) -%} + {{ gen.link(subslot_range) }} + {%- if not loop.last -%} +  or 
+ {%- endif -%} + {%- endfor -%} + {%- else -%} + {{ gen.link(slot.range) }} + {%- endif -%} +{% endmacro %} + +# Class: {{ title }} {% if element.deprecated %} (DEPRECATED) {% endif %} + +{%- if header -%} +{{ header }} +{%- endif -%} + +{% if element.description %} +{% set element_description_lines = element.description.split('\n') %} +{% for element_description_line in element_description_lines %} +_{{ element_description_line }}_ +{% endfor %} +{% endif %} + +{% if element.abstract %} +* __NOTE__: this is an abstract class and should not be instantiated directly +{% endif %} + +{# +URI: {{ gen.uri_link(element) }} +#} + +{% if diagram_type == "er_diagram" %} +```{{ gen.mermaid_directive() }} +{{ gen.mermaid_diagram([element.name]) }} +``` +{% elif diagram_type == "plantuml_class_diagram" %} +```puml +{{ gen.mermaid_diagram([element.name]) }} +``` +{% else %} +{% include "class_diagram.md.jinja2" %} +{% endif %} + +{% if schemaview.class_parents(element.name) or schemaview.class_children(element.name, mixins=False) %} + +## Inheritance +{{ gen.inheritance_tree(element, mixins=True) }} +{% else %} + +{% endif %} + +{%- set has_class_props = element.class_uri or element.tree_root or element.mixin + or element.subclass_of or element.union_of or element.disjoint_with + or element.slot_names_unique or element.represents_relationship + or element.children_are_mutually_disjoint %} + +{%- if has_class_props %} +## Class Properties + +| Property | Value | +| --- | --- | +{%- if element.class_uri %} +| Class URI | {{ gen.uri_link(element.class_uri) }} | +{%- endif %} +{%- if element.mixin %} +| Mixin | Yes | +{%- endif %} +{%- if element.tree_root %} +| Tree Root | Yes | +{%- endif %} +{%- if element.slot_names_unique %} +| Slot Names Unique | Yes | +{%- endif %} +{%- if element.represents_relationship %} +| Represents Relationship | Yes | +{%- endif %} +{%- if element.subclass_of %} +| Subclass Of | {{ gen.links(element.subclass_of) | join(', ') }} | +{%- endif %} +{%- if element.union_of %} +| Union Of | {{ gen.links(element.union_of) | join(', ') }} | +{%- endif %} +{%- if element.disjoint_with %} +| Disjoint With | {{ gen.links(element.disjoint_with) | join(', ') }} | +{%- endif %} +{%- if element.children_are_mutually_disjoint %} +| Children Are Mutually Disjoint | Yes | +{%- endif %} + +{% endif %} +## Attributes + +| Name | Cardinality and Range | Description | Inheritance | +| --- | --- | --- | --- | +{% if gen.get_direct_slots(element)|length > 0 %} +{%- for slot in gen.get_direct_slots(element) -%} +| {{ gen.link(slot) }} | {{ gen.cardinality(slot) }}
{{ compute_range(slot) }} | {{ slot.description|enshorten }} | direct | +{% endfor -%} +{% endif -%} +{% if gen.get_indirect_slots(element)|length > 0 %} +{%- for slot in gen.get_indirect_slots(element) -%} +| {{ gen.link(slot) }} | {{ gen.cardinality(slot) }}
{{ compute_range(slot) }} | {{ slot.description|enshorten }} | {{ gen.links(gen.get_slot_inherited_from(element.name, slot.name))|join(', ') }} | +{% endfor -%} +{% endif %} + +{%- if element.unique_keys %} +## Unique Keys + +{% for uk in element.unique_keys.values() %} +### {{ uk.unique_key_name }} + +**Unique key slots:** {{ uk.unique_key_slots | join(', ') }} +{%- if uk.consider_nulls_inequal %} + +Considers null values as inequal +{%- endif %} +{% endfor %} +{% endif %} + +{%- if element.defining_slots %} +## Defining Slots + +This class is defined by the following slots: + +{% for slot_name in element.defining_slots %} +* {{ gen.link(slot_name) }} +{%- endfor %} +{% endif %} + +{%- set has_expressions = element.any_of or element.all_of or element.exactly_one_of + or element.none_of or element.slot_conditions %} + +{%- if has_expressions %} +
+Expressions & Logic + +{%- if element.any_of %} +#### Any Of + +The class must satisfy at least one of: + +{%- for expr in element.any_of %} +- {{ expr }} +{%- endfor %} +{%- endif %} + +{%- if element.all_of %} +#### All Of + +The class must satisfy all of: + +{%- for expr in element.all_of %} +- {{ expr }} +{%- endfor %} +{%- endif %} + +{%- if element.exactly_one_of %} +#### Exactly One Of + +The class must satisfy exactly one of: + +{%- for expr in element.exactly_one_of %} +- {{ expr }} +{%- endfor %} +{%- endif %} + +{%- if element.none_of %} +#### None Of + +The class must not satisfy any of: + +{%- for expr in element.none_of %} +- {{ expr }} +{%- endfor %} +{%- endif %} + +{%- if element.slot_conditions %} +#### Slot Conditions + +{%- for slot_name, conditions in element.slot_conditions.items() %} +- **{{ gen.link(slot_name) }}**: {{ conditions }} +{%- endfor %} +{%- endif %} + +
+{% endif %} + +{% if schemaview.is_mixin(element.name) %} +## Mixin Usage + +| mixed into | description | +| --- | --- | +{% for c in schemaview.class_children(element.name, is_a=False) -%} +| {{ gen.link(c) }} | {{ schemaview.get_class(c).description|enshorten }} | +{% endfor %} +{% endif %} + +{% if schemaview.usage_index().get(element.name) %} +## Usages + +| used by | used in | type | used | +| --- | --- | --- | --- | +{% for usage in schemaview.usage_index().get(element.name) -%} +| {{ gen.link(usage.used_by) }} | {{ gen.link(usage.slot) }} | {{ usage.metaslot }} | {{ gen.link(usage.used) }} | +{% endfor %} +{% endif %} + +{% if element.rules %} +## Rules + +{% for rule in gen.classrule_to_dict_view(element) %} +### {{ rule.title }} + +| Rule Applied | Preconditions | Postconditions | Elseconditions | +|--------------|---------------|----------------|----------------| +{% for key in rule.preconditions -%} +| {{ key }} | +{%- if rule.preconditions[key] is defined -%} +```{{ rule.preconditions[key] }}``` +{%- else -%} +{% endif %} | +{%- if rule.postconditions and rule.postconditions[key] is defined -%} +```{{ rule.postconditions[key] }}``` +{%- else -%} +{% endif %} | +{%- if rule.elseconditions and rule.elseconditions[key] is defined -%} +```{{ rule.elseconditions[key] }}``` +{%- else -%} +{% endif %} | +{% endfor %} + +{% endfor %} +{% endif %} + +{% include "common_metadata.md.jinja2" %} + +{% if gen.example_object_blobs(element.name) -%} +## Examples + +Following examples have been automatically validated against the schema of this class. + +{% for name, blob in gen.example_object_blobs(element.name) -%} +??? example "{{ name }}" + + ```yaml + {{ blob | indent(4) }} + ``` +{% endfor %} +{% endif %} + +--- + +??? note "This section is only relevant for contributors of the specification" + + ## LinkML Source + + + + ### Direct + + ??? note "Details" + ```yaml +{{ gen.yaml(element) | indent(8, first=True) }} + ``` + + ### Induced + + ??? note "Details" + ```yaml +{{ gen.yaml(element, inferred=True) | indent(8, first=True) }} + ``` + +{%- if footer -%} +{{ footer }} +{%- endif -%} diff --git a/data-model/resources/markdown-templates/class_diagram.md.jinja2 b/data-model/resources/markdown-templates/class_diagram.md.jinja2 new file mode 100644 index 00000000..95c2e980 --- /dev/null +++ b/data-model/resources/markdown-templates/class_diagram.md.jinja2 @@ -0,0 +1,76 @@ +{% macro slot_relationship(element, slot) %} + {% if slot.range is not none %} + {% set range_element = gen.name(schemaview.get_element(slot.range)) %} + {% set relation_label = gen.name(slot) %} + {{ gen.name(element) }} --> "{{ gen.cardinality(slot) }}" {{ range_element }} : {{ relation_label }} + click {{ range_element }} href "{{ gen.link_mermaid(schemaview.get_element(slot.range)) }}" + {% endif %} +{% endmacro %} + +{% if schemaview.class_parents(element.name) and schemaview.class_children(element.name) %} +```{{ gen.mermaid_directive() }} + classDiagram + class {{ gen.name(element) }} + click {{ gen.name(element) }} href "{{ gen.link_mermaid(element) }}" + {% for s in schemaview.class_parents(element.name)|sort(attribute='name') -%} + {{ gen.name(schemaview.get_element(s)) }} <|-- {{ gen.name(element) }} + click {{ gen.name(schemaview.get_element(s)) }} href "{{ gen.link_mermaid(schemaview.get_element(s)) }}" + {% endfor %} + + {% for s in schemaview.class_children(element.name)|sort(attribute='name') -%} + {{ gen.name(element) }} <|-- {{ gen.name(schemaview.get_element(s)) }} + click {{ gen.name(schemaview.get_element(s)) }} href "{{ gen.link_mermaid(schemaview.get_element(s)) }}" + {% endfor %} + + {% for s in schemaview.class_induced_slots(element.name)|sort(attribute='name') -%} + {{ gen.name(element) }} : {{ gen.name(s) }} + {% if s.range is not none and s.range not in gen.all_type_object_names() %} + {{ slot_relationship(element, s) }} + {% endif %} + {% endfor %} +``` +{% elif schemaview.class_parents(element.name) %} +```{{ gen.mermaid_directive() }} + classDiagram + class {{ gen.name(element) }} + click {{ gen.name(element) }} href "{{ gen.link_mermaid(element) }}" + {% for s in schemaview.class_parents(element.name)|sort(attribute='name') -%} + {{ gen.name(schemaview.get_element(s)) }} <|-- {{ gen.name(element) }} + click {{ gen.name(schemaview.get_element(s)) }} href "{{ gen.link_mermaid(schemaview.get_element(s)) }}" + {% endfor %} + {% for s in schemaview.class_induced_slots(element.name)|sort(attribute='name') -%} + {{ gen.name(element) }} : {{ gen.name(s) }} + {% if s.range is not none and s.range not in gen.all_type_object_names() %} + {{ slot_relationship(element, s) }} + {% endif %} + {% endfor %} +``` +{% elif schemaview.class_children(element.name) %} +```{{ gen.mermaid_directive() }} + classDiagram + class {{ gen.name(element) }} + click {{ gen.name(element) }} href "{{ gen.link_mermaid(element) }}" + {% for s in schemaview.class_children(element.name)|sort(attribute='name') -%} + {{ gen.name(element) }} <|-- {{ gen.name(schemaview.get_element(s)) }} + click {{ gen.name(schemaview.get_element(s)) }} href "{{ gen.link_mermaid(schemaview.get_element(s)) }}" + {% endfor %} + {% for s in schemaview.class_induced_slots(element.name)|sort(attribute='name') -%} + {{ gen.name(element) }} : {{ gen.name(s) }} + {% if s.range is not none and s.range not in gen.all_type_object_names() %} + {{ slot_relationship(element, s) }} + {% endif %} + {% endfor %} +``` +{% else %} +```{{ gen.mermaid_directive() }} + classDiagram + class {{ gen.name(element) }} + click {{ gen.name(element) }} href "{{ gen.link_mermaid(element) }}" + {% for s in schemaview.class_induced_slots(element.name)|sort(attribute='name') -%} + {{ gen.name(element) }} : {{ gen.name(s) }} + {% if s.range is not none and s.range not in gen.all_type_object_names() %} + {{ slot_relationship(element, s) }} + {% endif %} + {% endfor %} +``` +{% endif %} diff --git a/data-model/resources/markdown-templates/common_metadata.md.jinja2 b/data-model/resources/markdown-templates/common_metadata.md.jinja2 new file mode 100644 index 00000000..9fb2c181 --- /dev/null +++ b/data-model/resources/markdown-templates/common_metadata.md.jinja2 @@ -0,0 +1,144 @@ +{% if element.categories %} +## Categories + +{% for cat in element.categories %} +* {{ cat }} +{%- endfor %} + +{% endif %} +{% if element.keywords %} +## Keywords + +{% for kw in element.keywords %} +* {{ kw }} +{%- endfor %} + +{% endif %} +{% if element.in_subset %} +## In Subsets + +{% for subset in element.in_subset %} +* {{ gen.link(subset) }} +{%- endfor %} + +{% endif %} +{% if element.aliases %} +## Aliases + +{% for alias in element.aliases %} +* {{ alias }} +{%- endfor %} +{% endif %} + +{% if element.examples %} +## Examples + +| Value | +| --- | +{% for x in element.examples -%} +| {{ x.value }} | +{% endfor %} +{% endif -%} + +{% if element.comments -%} +## Comments + +{% for x in element.comments -%} +* {{ x }} +{% endfor %} +{% endif -%} + +{% if element.todos -%} +## TODOs + +{% for x in element.todos -%} +* {{ x }} +{% endfor %} +{% endif -%} + +{% if element.see_also -%} +## See Also + +{% for x in element.see_also -%} +* {{ gen.uri_link(x) }} +{% endfor %} +{% endif -%} + +{% if element.notes -%} +## Notes + +{% for note in element.notes -%} +* {{ note }} +{% endfor %} +{% endif -%} + +{% if element.alt_descriptions %} +## Alternative Descriptions + +{% for source, alt_desc in element.alt_descriptions.items() %} +* **{{ source }}**: {{ alt_desc.description }} +{%- endfor %} +{% endif %} + +{# +## Identifier and Mapping Information + +{%- set has_admin_metadata = element.status or (element.rank is not none and element.rank != 1000) %} + +{%- if has_admin_metadata %} +### Administrative Metadata + +{% if element.status -%} +**Status:** {{ element.status }} +{% endif -%} +{% if element.rank is not none and element.rank != 1000 -%} +**Rank:** {{ element.rank }} +{% endif -%} + +{% endif %} +{% if element.id_prefixes %} +### Valid ID Prefixes + +Instances of this class *should* have identifiers with one of the following prefixes: +{% for p in element.id_prefixes %} +* {{ p }} +{% endfor %} + +{% endif %} + +{% if element.annotations %} +### Annotations + +| property | value | +| --- | --- | +{% for a in element.annotations -%} +{%- if a|string|first != '_' -%} +| {{ a }} | {{ element.annotations[a].value }} | +{% endif -%} +{% endfor %} +{% endif %} + +{% if element.from_schema or element.imported_from %} +### Schema Source + +{% if element.from_schema %} +* from schema: {{ element.from_schema }} +{% endif %} +{% if element.imported_from %} +* imported from: {{ element.imported_from }} +{% endif %} +{% endif %} + +{% if schemaview.get_mappings(element.name).items() -%} +## Mappings + +| Mapping Type | Mapped Value | +| --- | --- | +{% for m, mt in schemaview.get_mappings(element.name).items() -%} +{% if mt|length > 0 -%} +| {{ m }} | {{ mt|join(', ') }} | +{% endif -%} +{% endfor %} + +{% endif -%} +#} diff --git a/data-model/resources/markdown-templates/enum.md.jinja2 b/data-model/resources/markdown-templates/enum.md.jinja2 new file mode 100644 index 00000000..e840fae2 --- /dev/null +++ b/data-model/resources/markdown-templates/enum.md.jinja2 @@ -0,0 +1,139 @@ +{%- if element.title and element.title != element.name %} + {%- set title = element.title ~ ' (' ~ element.name ~ ')' -%} +{%- else %} + {%- set title = gen.name(element) -%} +{%- endif -%} + +# Enum: {{ title }} {% if element.deprecated %} (DEPRECATED) {% endif %} + +{% if element.description %} +{% set element_description_lines = element.description.split('\n') %} +{% for element_description_line in element_description_lines %} +_{{ element_description_line }}_ +{% endfor %} +{% endif %} + +URI: {{ gen.uri_link(element) }} + +{%- if element.enum_uri %} + +**Enum URI:** {{ gen.uri_link(element.enum_uri) }} +{% endif %} + +{%- set has_enum_source = element.code_set or element.pv_formula or element.reachable_from + or element.matches or element.concepts %} + +{%- if has_enum_source %} +## Enumeration Source + +{%- if element.code_set %} +**Code Set:** {{ gen.uri_link(element.code_set) }} + +{%- if element.code_set_tag %} +- **Tag:** {{ element.code_set_tag }} +{%- endif %} +{%- if element.code_set_version %} +- **Version:** {{ element.code_set_version }} +{%- endif %} +{%- endif %} + +{%- if element.pv_formula %} +**Permissible Value Formula:** {{ element.pv_formula }} +{%- endif %} + +{%- if element.reachable_from %} +**Reachable From:** + +{%- if element.reachable_from.source_ontology %} +- **Source:** {{ gen.link(element.reachable_from.source_ontology) }} +{%- endif %} +{%- if element.reachable_from.source_nodes %} +- **Nodes:** {{ element.reachable_from.source_nodes | join(', ') }} +{%- endif %} +{%- if element.reachable_from.relationship_types %} +- **Via:** {{ element.reachable_from.relationship_types | join(', ') }} +{%- endif %} +{%- endif %} + +{%- if element.matches %} +**Matches:** + +- **Expression:** `{{ element.matches.string_expression }}` +{%- endif %} + +{%- if element.concepts %} +**Concepts:** {{ gen.uri_links(element.concepts) | join(', ') }} +{%- endif %} + +{% endif %} + +{% if element.permissible_values -%} +{%- set has_pv_extras = namespace(found=false) -%} +{%- for pv in element.permissible_values.values() -%} + {%- if pv.title or pv.is_a or pv.mixins or pv.deprecated -%} + {%- set has_pv_extras.found = true -%} + {%- endif -%} +{%- endfor -%} +## Permissible Values + +{%- if has_pv_extras.found %} +| Value | Meaning | Description | Additional Info | +| --- | --- | --- | --- | +{% for pv in element.permissible_values.values() -%} +| {{ pv.text }} | {{ pv.meaning }} | {{ pv.description|enshorten }} | +{%- if pv.title %} Title: {{ pv.title }}
{% endif -%} +{%- if pv.is_a %} Is-A: {{ gen.link(pv.is_a) }}
{% endif -%} +{%- if pv.mixins %} Mixins: {{ gen.links(pv.mixins) | join(', ') }}
{% endif -%} +{%- if pv.deprecated %} **DEPRECATED**{% if pv.deprecated_element_has_exact_replacement %} (use {{ gen.link(pv.deprecated_element_has_exact_replacement) }}){% endif %}{% endif -%} + | +{% endfor %} +{%- else %} +| Value | Meaning | Description | +| --- | --- | --- | +{% for pv in element.permissible_values.values() -%} +| {{ pv.text }} | {{ pv.meaning }} | {{ pv.description|enshorten }} | +{% endfor %} +{%- endif %} +{% else %} +_This is a dynamic enum_ +{% endif %} + +{%- set has_enum_ops = element.inherits or element.include or element.minus %} + +{%- if has_enum_ops %} +## Enumeration Operations + +{%- if element.inherits %} +**Inherits From:** {{ gen.links(element.inherits) | join(', ') }} +{%- endif %} + +{%- if element.include %} +**Includes:** {{ gen.links(element.include) | join(', ') }} +{%- endif %} + +{%- if element.minus %} +**Excludes:** {{ gen.links(element.minus) | join(', ') }} +{%- endif %} + +{% endif %} + +{% set slots_for_enum = schemaview.get_slots_by_enum(element.name) %} +{% if slots_for_enum is defined and slots_for_enum|length > 0 -%} +## Slots + +| Name | Description | +| --- | --- | +{% for s in schemaview.get_slots_by_enum(element.name) -%} +| {{ gen.link(s) }} | {{ s.description|enshorten }} | +{% endfor %} +{% endif %} + +{% include "common_metadata.md.jinja2" %} + +## LinkML Source + +
+```yaml +{{ gen.yaml(element) }} +``` +
diff --git a/data-model/resources/markdown-templates/index.md.jinja2 b/data-model/resources/markdown-templates/index.md.jinja2 new file mode 100644 index 00000000..d02bed2b --- /dev/null +++ b/data-model/resources/markdown-templates/index.md.jinja2 @@ -0,0 +1,94 @@ + +# {% if schema.title %}{{ schema.title }}{% else %}{{ schema.name }}{% endif %} + +{% if schema.description %}{{ schema.description }}{% endif %} + +{# +URI: {{ schema.id }} + +Name: {{ schema.name }} +#} + +{% if include_top_level_diagram %} + +## Schema Diagram + +```{{ gen.mermaid_directive() }} +{{ gen.mermaid_diagram() }} +``` +{% endif %} + +## Interfacing Classes + +These are the classes directly participating in Margo interfaces (e.g. APIs): + +| Class | Description | +| --- | --- | +{% for cn in ["ApplicationDescription", "ApplicationDeployment", "DeviceCapabilitiesManifest", "DesiredStateManifest", "DeploymentStatusManifest"] -%} +| {{ gen.link(schemaview.get_class(cn), True) }} | {{ schemaview.get_class(cn).description|enshorten }} | +{% endfor %} + +??? note "Complete Margo data model" + + ## Detailed Class Diagram + + This is a class diagram showing all the classes involved in the Margo data model. + Use the mouse wheel to zoom and click-drag to pan. The controls in the bottom-right corner allow zooming in, resetting, and zooming out. + +
+
Scroll to zoom · Drag to pan · Use controls at bottom-right to zoom in/out/reset
+ + ## All Classes + + These are all the classes involved in the Margo data model: + + | Class | Description | + | --- | --- | + {% if gen.hierarchical_class_view -%} + {% for u, v in gen.class_hierarchy_as_tuples() -%} + | {{ " "|safe*u*8 }}{{ gen.link(schemaview.get_class(v), True) }} | {{ schemaview.get_class(v).description|enshorten }} | + {% endfor %} + {% else -%} + {% for c in gen.all_class_objects()|sort(attribute=sort_by) -%} + | {{ gen.link(c, True) }} | {{ c.description|enshorten }} | + {% endfor %} + {% endif %} + +{# +## Slots + +| Slot | Description | +| --- | --- | +{% for s in gen.all_slot_objects()|sort(attribute=sort_by) -%} +| {{ gen.link(s, True) }} | {{ s.description|enshorten }} | +{% endfor %} + +## Enumerations + +| Enumeration | Description | +| --- | --- | +{% for e in gen.all_enum_objects()|sort(attribute=sort_by) -%} +| {{ gen.link(e, True) }} | {{ e.description|enshorten }} | +{% endfor %} + +## Types + +| Type | Description | +| --- | --- | +{% for t in gen.all_type_objects()|sort(attribute=sort_by) -%} +| {{ gen.link(t, True) }} | {{ t.description|enshorten }} | +{% endfor %} + +## Subsets + +| Subset | Description | +| --- | --- | +{% for ss in schemaview.all_subsets().values()|sort(attribute='name') -%} +| {{ gen.link(ss, True) }} | {{ ss.description|enshorten }} | +{% endfor %} +#} diff --git a/data-model/resources/markdown-templates/slot.md.jinja2 b/data-model/resources/markdown-templates/slot.md.jinja2 new file mode 100644 index 00000000..75344cf5 --- /dev/null +++ b/data-model/resources/markdown-templates/slot.md.jinja2 @@ -0,0 +1,454 @@ +{%- if element.title %} + {%- set title = element.title ~ ' (' ~ element.name ~ ')' -%} +{%- else %} + {%- if gen.use_slot_uris -%} + {%- set title = element.name -%} + {%- else -%} + {%- set title = gen.name(element) -%} + {%- endif -%} +{%- endif -%} + +{% macro compute_range(slot) -%} + {%- if slot.any_of or slot.exactly_one_of -%} + {%- for subslot_range in schemaview.slot_range_as_union(slot) -%} + {{ gen.link(subslot_range) }} + {%- if not loop.last -%} +  or 
+ {%- endif -%} + {%- endfor -%} + {%- else -%} + {{ gen.link(slot.range) }} + {%- endif -%} +{% endmacro %} + +# Slot: {{ title }} {% if element.deprecated %} (DEPRECATED) {% endif %} + +{%- if header -%} +{{ header }} +{%- endif -%} + +{% if element.description %} +{% set element_description_lines = element.description.split('\n') %} +{% for element_description_line in element_description_lines %} +_{{ element_description_line }}_ +{% endfor %} +{% endif %} + +{% if element.abstract %} +* __NOTE__: this is an abstract slot and should not be populated directly +{% endif %} + +URI: {{ gen.uri_link(element) }} + +{%- if element.alias %} +Alias: {{ element.alias }} +{% endif -%} + +{% if schemaview.slot_parents(element.name) or schemaview.slot_children(element.name, mixins=False) %} + +## Inheritance + +{{ gen.inheritance_tree(element, mixins=True) }} +{% else %} + +{% endif %} + +{% set classes_by_slot = schemaview.get_classes_by_slot(element, include_induced=True) %} +{% if classes_by_slot %} + +## Applicable Classes + +| Name | Description | Modifies Slot | +| --- | --- | --- | +{% for c in classes_by_slot -%} +| {{ gen.link(c) }} | {{ schemaview.get_class(c).description|enshorten }} | {% if c in schemaview.get_classes_modifying_slot(element) %} yes {% else %} no {% endif %} | +{% endfor %} + +{% endif %} + +{% if schemaview.is_mixin(element.name) %} +## Mixin Usage + +| mixed into | description | range | domain | +| --- | --- | --- | --- | +{% for s in schemaview.slot_children(element.name, is_a=False) -%} +| {{ gen.link(s) }} | {{ schemaview.get_slot(s).description|enshorten }} | {{ schemaview.get_slot(s).range }} | {{ schemaview.get_classes_by_slot(schemaview.get_slot(s))|join(', ') }} | +{% endfor %} +{% endif %} + +## Properties + +### Type and Range + +| Property | Value | +| --- | --- | +| Range | {{ compute_range(element) }} | +{%- if element.domain %} +| Domain | {{ gen.link(element.domain) }} | +{%- endif %} +{%- if element.domain_of %} +| Domain Of | {{ gen.links(element.domain_of) | join(', ') }} | +{%- endif %} +{%- if element.slot_uri %} +| Slot URI | {{ gen.uri_link(element.slot_uri) }} | +{%- endif %} +{%- if element.slot_group %} +| Slot Group | {{ gen.link(element.slot_group) }} | +{%- endif %} +{%- if element.is_grouping_slot %} +| Is Grouping Slot | Yes | +{%- endif %} + +### Cardinality and Requirements + +| Property | Value | +| --- | --- | +{%- if element.required %} +| Required | Yes | +{%- elif element.recommended %} +| Recommended | Yes | +{%- endif %} +{%- if element.multivalued %} +| Multivalued | Yes | +{%- endif %} +{%- if element.minimum_cardinality is not none %} +| Minimum Cardinality | {{ element.minimum_cardinality }} | +{%- endif %} +{%- if element.maximum_cardinality is not none %} +| Maximum Cardinality | {{ element.maximum_cardinality }} | +{%- endif %} +{%- if element.exact_cardinality is not none %} +| Exact Cardinality | {{ element.exact_cardinality }} | +{%- endif %} + +{%- if element.multivalued and (element.list_elements_unique is not none or element.list_elements_ordered is not none) %} +### List/Collection Properties + +| Property | Value | +| --- | --- | +{%- if element.list_elements_unique is not none %} +| Elements Must Be Unique | {% if element.list_elements_unique %}Yes{% else %}No{% endif %} | +{%- endif %} +{%- if element.list_elements_ordered is not none %} +| Elements Are Ordered | {% if element.list_elements_ordered %}Yes{% else %}No{% endif %} | +{%- endif %} + +{% endif %} +{%- set has_slot_chars = element.key or element.identifier or element.designates_type + or element.inherited or element.readonly or element.ifabsent or element.owner + or element.shared or element.is_class_field or element.is_usage_slot + or element.usage_slot_name or element.singular_name or schemaview.is_mixin(element.name) %} + +{%- if has_slot_chars %} +### Slot Characteristics + +| Property | Value | +| --- | --- | +{%- if element.singular_name %} +| Singular Name | {{ element.singular_name }} | +{%- endif %} +{%- if element.key %} +| Key | Yes | +{%- endif %} +{%- if element.identifier %} +| Identifier | Yes | +{%- endif %} +{%- if element.designates_type %} +| Designates Type | Yes | +{%- endif %} +{%- if element.inherited %} +| Inherited | Yes | +{%- endif %} +{%- if element.readonly %} +| Readonly | Yes | +{%- endif %} +{%- if element.ifabsent %} +| If Absent | `{{ element.ifabsent }}` | +{%- endif %} +{%- if element.owner %} +| Owner | {{ gen.link(element.owner) }} | +{%- endif %} +{%- if element.shared %} +| Shared | Yes | +{%- endif %} +{%- if element.is_class_field %} +| Is Class Field | Yes | +{%- endif %} +{%- if element.is_usage_slot %} +| Is Usage Slot | Yes | +{%- endif %} +{%- if element.usage_slot_name %} +| Usage Slot Name | {{ element.usage_slot_name }} | +{%- endif %} +{%- if schemaview.is_mixin(element.name) %} +| Mixin | Yes | +{%- endif %} + +{% endif %} +{%- set has_basic_constraints = element.minimum_value is not none or element.maximum_value is not none or element.pattern %} + +{%- if has_basic_constraints %} +### Value Constraints + +| Property | Value | +| --- | --- | +{%- if element.minimum_value is not none %} +| Minimum Value | {{ element.minimum_value|int }} | +{%- endif %} +{%- if element.maximum_value is not none %} +| Maximum Value | {{ element.maximum_value|int }} | +{%- endif %} +{%- if element.pattern %} +| Regex Pattern | `{{ element.pattern }}` | +{%- endif %} + +{% endif %} +{%- set has_advanced_constraints = element.structured_pattern or element.equals_string + or element.equals_string_in or element.equals_number or element.enum_range + or element.unit or element.implicit_prefix %} + +{%- if has_advanced_constraints %} +
+Additional Constraints + +{%- if element.structured_pattern %} +**Structured Pattern:** + +- **Syntax:** `{{ element.structured_pattern.syntax }}` +- **Interpolated:** {{ element.structured_pattern.interpolated }} +{%- if element.structured_pattern.partial_match %} +- **Partial Match:** Yes +{%- endif %} +{%- endif %} + +{%- if element.equals_string %} +**Must Equal:** `{{ element.equals_string }}` +{%- endif %} + +{%- if element.equals_string_in %} +**Must Be One Of:** {{ element.equals_string_in | join(', ') }} +{%- endif %} + +{%- if element.equals_number %} +**Must Equal:** {{ element.equals_number }} +{%- endif %} + +{%- if element.enum_range %} +**Enumeration Range:** {{ gen.link(element.enum_range) }} +{%- endif %} + +{%- if element.unit %} +**Unit:** {{ gen.uri_link(element.unit) }} +{%- endif %} + +{%- if element.implicit_prefix %} +**Implicit Prefix:** {{ element.implicit_prefix }} +{%- endif %} + +
+{% endif %} + +{%- set has_rel_props = element.symmetric or element.asymmetric or element.reflexive + or element.locally_reflexive or element.irreflexive or element.transitive + or element.inverse or element.transitive_form_of or element.reflexive_transitive_form_of + or element.role or element.relational_role %} + +{%- if has_rel_props %} +
+Relationship Properties + +| Property | Value | +| --- | --- | +{%- if element.symmetric %} +| Symmetric | Yes | +{%- endif %} +{%- if element.asymmetric %} +| Asymmetric | Yes | +{%- endif %} +{%- if element.reflexive %} +| Reflexive | Yes | +{%- endif %} +{%- if element.locally_reflexive %} +| Locally Reflexive | Yes | +{%- endif %} +{%- if element.irreflexive %} +| Irreflexive | Yes | +{%- endif %} +{%- if element.transitive %} +| Transitive | Yes | +{%- endif %} +{%- if element.inverse %} +| Inverse | {{ gen.link(element.inverse) }} | +{%- endif %} +{%- if element.transitive_form_of %} +| Transitive Form Of | {{ gen.link(element.transitive_form_of) }} | +{%- endif %} +{%- if element.reflexive_transitive_form_of %} +| Reflexive Transitive Form Of | {{ gen.link(element.reflexive_transitive_form_of) }} | +{%- endif %} +{%- if element.role %} +| Role | {{ element.role }} | +{%- endif %} +{%- if element.relational_role %} +| Relational Role | {{ element.relational_role }} | +{%- endif %} + +
+{% endif %} + +{%- set has_advanced = element.path_rule or element.disjoint_with + or element.children_are_mutually_disjoint or element.subproperty_of + or element.array or element.bindings or element.type_mappings + or element.value_presence or element.range_expression %} + +{%- if has_advanced %} +
+Advanced Properties + +{%- if element.subproperty_of %} +**Subproperty Of:** {{ gen.link(element.subproperty_of) }} +{%- endif %} + +{%- if element.path_rule %} +**Path Rule:** + +``` +{{ element.path_rule }} +``` +{%- endif %} + +{%- if element.disjoint_with %} +**Disjoint With:** {{ gen.links(element.disjoint_with) | join(', ') }} +{%- endif %} + +{%- if element.children_are_mutually_disjoint %} +**Children Are Mutually Disjoint:** Yes +{%- endif %} + +{%- if element.array %} +**Array Configuration:** + +- **Dimensions:** {{ element.array.dimensions | join(' x ') }} +{%- if element.array.exact_number_dimensions %} +- **Exact Dimensions Required:** Yes +{%- endif %} +{%- endif %} + +{%- if element.range_expression %} +**Range Expression:** {{ element.range_expression }} +{%- endif %} + +{%- if element.value_presence %} +**Value Presence:** {{ element.value_presence }} +{%- endif %} + +{%- if element.bindings %} +**Term Bindings:** + +{%- for binding in element.bindings %} +- {{ binding }} +{%- endfor %} +{%- endif %} + +{%- if element.type_mappings %} +**Type Mappings:** + +{%- for tm in element.type_mappings %} +- **Framework:** {{ tm.framework }}, **Mapping:** {{ tm.mapping }} +{%- endfor %} +{%- endif %} + +
+{% endif %} + +{%- set has_expressions = element.any_of or element.all_of or element.exactly_one_of + or element.none_of or element.equals_expression or element.has_member or element.all_members %} + +{%- if has_expressions %} +
+Expressions & Logic + +{%- if element.any_of %} +#### Any Of + +Value must satisfy at least one of: + +{%- for expr in element.any_of %} +- {{ expr }} +{%- endfor %} +{%- endif %} + +{%- if element.all_of %} +#### All Of + +Value must satisfy all of: + +{%- for expr in element.all_of %} +- {{ expr }} +{%- endfor %} +{%- endif %} + +{%- if element.exactly_one_of %} +#### Exactly One Of + +Value must satisfy exactly one of: + +{%- for expr in element.exactly_one_of %} +- {{ expr }} +{%- endfor %} +{%- endif %} + +{%- if element.none_of %} +#### None Of + +Value must not satisfy any of: + +{%- for expr in element.none_of %} +- {{ expr }} +{%- endfor %} +{%- endif %} + +{%- if element.equals_expression %} +#### Equals Expression + +`{{ element.equals_expression }}` +{%- endif %} + +{%- if element.has_member %} +#### Has Member + +{{ element.has_member }} +{%- endif %} + +{%- if element.all_members %} +#### All Members + +{{ element.all_members }} +{%- endif %} + +
+{% endif %} + +{% if schemaview.usage_index().get(element.name) %} +## Usages + +| used by | used in | type | used | +| --- | --- | --- | --- | +{% for usage in schemaview.usage_index().get(element.name) -%} +| {{ gen.link(usage.used_by) }} | {{ gen.link(usage.slot) }} | {{ usage.metaslot }} | {{ gen.link(usage.used) }} | +{% endfor %} +{% endif %} + +{% include "common_metadata.md.jinja2" %} + +## LinkML Source + +
+```yaml +{{ gen.yaml(element) }} +``` +
+ +{%- if footer -%} +{{ footer }} +{%- endif -%} diff --git a/data-model/resources/markdown-templates/subset.md.jinja2 b/data-model/resources/markdown-templates/subset.md.jinja2 new file mode 100644 index 00000000..91e75e2b --- /dev/null +++ b/data-model/resources/markdown-templates/subset.md.jinja2 @@ -0,0 +1,108 @@ +{# +# Subset: {{ gen.name(element) }} {% if element.deprecated %} (DEPRECATED) {% endif %} +#} +# Subset: {{ element.title }} {% if element.deprecated %} (DEPRECATED) {% endif %} + +{%- if header -%} +{{ header }} +{%- endif -%} + +{% if element.description %} +{% set element_description_lines = element.description.split('\n') %} +{% for element_description_line in element_description_lines %} +_{{ element_description_line }}_ +{% endfor %} +{% endif %} + +{# +URI: {{ gen.link(element) }} +#} + +{% include "common_metadata.md.jinja2" %} + +{% set classes_in_subset = [] %} +{% set slots_in_subset = [] %} +{% set enums_in_subset = [] %} + +{# Collect classes, slots, and enumerations in subset #} +{% for c in gen.all_class_objects()|sort(attribute=sort_by) %} + {%- if element.name in c.in_subset %} + {% set _ = classes_in_subset.append(c) %} + {%- endif %} +{% endfor %} + +{% for s in gen.all_slot_objects()|sort(attribute=sort_by) %} + {%- if element.name in s.in_subset %} + {% set _ = slots_in_subset.append(s) %} + {%- endif %} +{% endfor %} + +{% for e in schemaview.all_enums().values() %} + {%- if element.name in e.in_subset %} + {% set _ = enums_in_subset.append(e) %} + {%- endif %} +{% endfor %} + +{% if classes_in_subset %} +## Classes in subset + +| Class | Description | +| --- | --- | +{% for c in classes_in_subset -%} +{%- if element.name in c.in_subset -%} +| {{ gen.link(c) }} | {{ c.description|enshorten }} | +{% endif -%} +{% endfor %} + +{% for c in classes_in_subset -%} +{%- if element.name in c.in_subset -%} + +{% set induced_slots = gen.class_induced_slots(c.name)|sort(attribute=sort_by) %} + +{%- set filtered_slots = [] -%} +{%- for s in induced_slots|sort(attribute=sort_by) -%} + {%- if element.name in s.in_subset or element.name in schemaview.get_slot(s.name).in_subset -%} + {% set _ = filtered_slots.append(s) %} + {%- endif -%} +{%- endfor %} + +{%- if filtered_slots|length > 0 -%} +### Slots from {{ gen.link(c) }} also in _{{ element.name }}_ + +| Name | Cardinality and Range | Description | +| --- | --- | --- | +{% for s in filtered_slots -%} +| {{ gen.link(s) }} | {{ gen.cardinality(s) }}
{{ gen.link(s.range) }} | {{ s.description|enshorten }} {% if s.identifier %}**identifier**{% endif %} | +{% endfor %} +{%- endif %} + +{%- endif %} +{% endfor %} + +{%- endif %} + +{% if slots_in_subset %} +## Slots in subset + +| Slot | Description | +| --- | --- | +{% for s in slots_in_subset|sort(attribute=sort_by) -%} +{%- if element.name in s.in_subset -%} +| {{ gen.link(s) }} | {{ s.description|enshorten }} | +{%- endif %} +{% endfor %} + +{%- endif %} + +{% if enums_in_subset %} +## Enumerations in subset + +| Enumeration | Description | +| --- | --- | +{% for e in enums_in_subset|sort(attribute='name') -%} +{% if element.name in e.in_subset -%} +| {{ gen.link(e) }} | {{ e.description|enshorten }} | +{%- endif %} +{% endfor %} + +{%- endif %} diff --git a/data-model/resources/markdown-templates/type.md.jinja2 b/data-model/resources/markdown-templates/type.md.jinja2 new file mode 100644 index 00000000..e109b47a --- /dev/null +++ b/data-model/resources/markdown-templates/type.md.jinja2 @@ -0,0 +1,134 @@ +{%- if element.title and element.title != element.name %} + {%- set title = element.title ~ ' (' ~ element.name ~ ')' -%} +{%- else %} + {%- set title = gen.name(element) -%} +{%- endif -%} + +# Type: {{ title }} {% if element.deprecated %} (DEPRECATED) {% endif %} + +{% if element.description %} +{% set element_description_lines = element.description.split('\n') %} +{% for element_description_line in element_description_lines %} +_{{ element_description_line }}_ +{% endfor %} +{% endif %} + +URI: {{ gen.uri_link(element) }} + +## Type Properties + +| Property | Value | +| --- | --- | +{%- if element.typeof %} +| Type Of | {{ gen.link(element.typeof) }} | +{%- endif %} +{%- if element.base %} +| Base | `{{ element.base }}` | +{%- endif %} +{%- if element.uri %} +| Type URI | {{ gen.uri_link(element.uri) }} | +{%- endif %} +{%- if element.repr %} +| Representation | `{{ element.repr }}` | +{%- endif %} +{%- if element.union_of %} +| Union Of | {{ gen.links(element.union_of) | join(', ') }} | +{%- endif %} + +{%- set has_basic_constraints = element.minimum_value is not none or element.maximum_value is not none or element.pattern %} + +{%- if has_basic_constraints %} +## Value Constraints + +| Property | Value | +| --- | --- | +{%- if element.minimum_value is not none or element.maximum_value is not none %} +| Numeric Range | {{ gen.number_value_range(element) }} | +{%- endif %} +{%- if element.pattern %} +| Regex Pattern | `{{ element.pattern }}` | +{%- endif %} + +{% endif %} +{%- set has_advanced_constraints = element.structured_pattern or element.equals_string + or element.equals_string_in or element.equals_number or element.unit or element.implicit_prefix %} + +{%- if has_advanced_constraints %} +
+Additional Constraints + +{%- if element.structured_pattern %} +**Structured Pattern:** + +- **Syntax:** `{{ element.structured_pattern.syntax }}` +- **Interpolated:** {{ element.structured_pattern.interpolated }} +{%- if element.structured_pattern.partial_match %} +- **Partial Match:** Yes +{%- endif %} +{%- endif %} + +{%- if element.equals_string %} +**Must Equal:** `{{ element.equals_string }}` +{%- endif %} + +{%- if element.equals_string_in %} +**Must Be One Of:** {{ element.equals_string_in | join(', ') }} +{%- endif %} + +{%- if element.equals_number %} +**Must Equal:** {{ element.equals_number }} +{%- endif %} + +{%- if element.unit %} +**Unit:** {{ gen.uri_link(element.unit) }} +{%- endif %} + +{%- if element.implicit_prefix %} +**Implicit Prefix:** {{ element.implicit_prefix }} +{%- endif %} + +
+{% endif %} + +{%- set has_expressions = element.any_of or element.all_of or element.exactly_one_of or element.none_of %} + +{%- if has_expressions %} +
+Type Expressions + +{%- if element.any_of %} +**Any Of:** Value must satisfy at least one of these expressions + +{%- for expr in element.any_of %} +- {{ expr }} +{%- endfor %} +{%- endif %} + +{%- if element.all_of %} +**All Of:** Value must satisfy all of these expressions + +{%- for expr in element.all_of %} +- {{ expr }} +{%- endfor %} +{%- endif %} + +{%- if element.exactly_one_of %} +**Exactly One Of:** Value must satisfy exactly one of these expressions + +{%- for expr in element.exactly_one_of %} +- {{ expr }} +{%- endfor %} +{%- endif %} + +{%- if element.none_of %} +**None Of:** Value must not satisfy any of these expressions + +{%- for expr in element.none_of %} +- {{ expr }} +{%- endfor %} +{%- endif %} + +
+{% endif %} + +{% include "common_metadata.md.jinja2" %} diff --git a/data-model/resources/markdown-templates_main-classes/class.md.jinja2 b/data-model/resources/markdown-templates_main-classes/class.md.jinja2 new file mode 100644 index 00000000..e69de29b diff --git a/data-model/resources/markdown-templates_main-classes/index.md.jinja2 b/data-model/resources/markdown-templates_main-classes/index.md.jinja2 new file mode 100644 index 00000000..5ab9f8f9 --- /dev/null +++ b/data-model/resources/markdown-templates_main-classes/index.md.jinja2 @@ -0,0 +1,66 @@ + +{%- macro format_range(slot) -%} + {%- if slot.multivalued -%} + {%- if slot.inlined -%} + {%- if slot.inlined_as_list -%} + []{{ slot.range }} + {%- else -%} + {%- if slot.name == "properties" -%} + map[string][string] + {%- else -%} + map[string][{{ slot.range }}] + {%- endif -%} + {%- endif -%} + {%- else -%} + []string + {%- endif -%} + {%- else -%} + {{ slot.range }} + {%- endif -%} +{%- endmacro -%} + +{%- set schema_basename = schema.source_file.split('/')[-1].replace('.linkml.yaml', '') -%} + +# {%- if schema.title %}{{ schema.title }}{% else %}{{ schema.name }}{% endif %} + +{% if schema.description %}{{ schema.description }}{% endif %} + +## Object structure + +{% for cls in schemaview.all_classes() -%} +{%- if not cls.startswith("ComponentProperties") and not cls.startswith("Helm") and not cls.startswith("Compose") +%} +### {{ cls }} Attributes + +| Attribute | Type | Required? | Description | +| --- | --- | --- | --- | +{% for slot in schemaview.class_slots(cls)|sort(attribute='rank') -%} +| {{ slot }} | {{ format_range(schemaview.get_slot(slot)) }} | {% if schemaview.get_slot(slot).required == True %} Y {% else %} N {% endif %} | {{ schemaview.get_slot(slot).description }}| +{% endfor %} + +Detailed class view: [{{ cls }}](../../data-model/{{ cls }}.md) +{% endif -%} +{%- endfor -%} + +{% if gen.example_object_blobs(schema.name) -%} +## Examples + +Following examples have been automatically validated against the schema of this class. + +{% for name, blob in gen.example_object_blobs(schema.name) -%} +??? example "{{ name }}" + + ```yaml + {{ blob | indent(4) }} + ``` +{% endfor %} +{% endif %} + +## JSON Schema + +

View JSON Schema in new tab or download

+ diff --git a/data-model/tools/check-examples.bash b/data-model/tools/check-examples.bash new file mode 100755 index 00000000..443defd6 --- /dev/null +++ b/data-model/tools/check-examples.bash @@ -0,0 +1,130 @@ +#!/usr/bin/env bash + +set -eu + +THIS_SCRIPT="$(readlink -f "${0}")" +THIS_DIR="$(dirname "${THIS_SCRIPT}")" +DOCS_GEN="${THIS_DIR}" +CONFIGS="${DOCS_GEN}/configurations" + +ROOT_DIR="$(dirname "$(dirname "${THIS_DIR}")")" + +VERBOSITY="${1:-info}" + +debug() { + if [[ "$VERBOSITY" == "debug" ]]; then + echo "$1" + fi +} + +trace() { + if [[ "$VERBOSITY" == "debug" ]] || + [[ "$VERBOSITY" == "trace" ]]; then + echo "$1" + fi +} + +info() { + if [[ "$VERBOSITY" == "debug" ]] || + [[ "$VERBOSITY" == "trace" ]] || + [[ "$VERBOSITY" == "info" ]]; then + echo "$1" + fi +} + +if command -v poetry 2>&1 >/dev/null; then + RUN="poetry run" +else + if ! command -v linkml 2>&1 >/dev/null; then + echo "The command 'linkml' is missing" + exit 1 + fi + if ! command -v mkdocs 2>&1 >/dev/null; then + echo "The command 'mkdocs' is missing" + exit 1 + fi + RUN="" +fi + +check_spec() { + trace "********************************" + SPEC_ROOT="${ROOT_DIR}/$(jq -r '.root' "${CONFIGS}/$1")" + trace "Spec root folder: ${SPEC_ROOT}" + + if [[ ! -d "${SPEC_ROOT}" ]]; then + echo "🚨 Spec root folder does not exist: ${SPEC_ROOT}" + return 1 + fi + + TARGET_CLASS="$(jq -r '.targetclass' "${CONFIGS}/$1")" + info "Target class: ${TARGET_CLASS}" + + SCHEMA_FILE="$(jq -r '.schemafile' "${CONFIGS}/$1")" + trace "Schema file: ${SCHEMA_FILE}" + + EXAMPLES_DIR="${SPEC_ROOT}/resources/examples/valid" + trace "Examples folder: ${EXAMPLES_DIR}" + FOUND_VALID=0 + for EXAMPLE in $(ls "${EXAMPLES_DIR}"/${TARGET_CLASS}-*.{yaml,json} 2>/dev/null); do + FOUND_VALID=1 + if result=$(${RUN} linkml validate \ + --schema "${SPEC_ROOT}/${SCHEMA_FILE}" \ + --target-class "${TARGET_CLASS}" \ + "${EXAMPLE}") && + [[ "${result}" == "No issues found" ]]; then + echo "✅ Valid example (${EXAMPLE})" + else + echo "🚨 Not valid example expected to be valid! (${EXAMPLE})" + echo " ERROR: $result" + return 1 + fi + done + + if [[ "${FOUND_VALID}" -eq 0 ]]; then + echo "⚠️ No valid examples found for ${TARGET_CLASS} in ${EXAMPLES_DIR}" + return 1 + fi + + COUNTEREXAMPLES_DIR="${SPEC_ROOT}/resources/examples/invalid" + FOUND_INVALID=0 + for COUNTEREXAMPLE in $(ls "${COUNTEREXAMPLES_DIR}"/${TARGET_CLASS}-*.{yaml,json} 2>/dev/null); do + FOUND_INVALID=1 + if ! result=$(${RUN} linkml validate \ + --schema "${SPEC_ROOT}/${SCHEMA_FILE}" \ + --target-class "${TARGET_CLASS}" \ + "${COUNTEREXAMPLE}"); then + echo "✅ Validation of invalid example failed, as expected (${COUNTEREXAMPLE})" + debug "${result}" + else + echo "🚨 Validation of invalid example '${COUNTEREXAMPLE}' was expected to fail, but has succeeded." + echo "${result}" + return 1 + fi + done + + if [[ "${FOUND_INVALID}" -eq 0 ]]; then + echo "⚠️ No invalid examples found for ${TARGET_CLASS} in ${COUNTEREXAMPLES_DIR}" + return 1 + fi + + if false; then + # does not work due to following LinkML bugs: + # https://github.com/linkml/linkml/issues/2423 + # https://github.com/linkml/linkml/issues/2425 + ${RUN} linkml examples \ + --schema "${SPEC_ROOT}/application-description.linkml.yaml" \ + --input-directory "${SPEC_ROOT}/resources/examples/valid" \ + --counter-example-input-directory "${SPEC_ROOT}/resources/examples/invalid" \ + --output-directory "${SPEC_ROOT}/output" + fi +} + +OVERALL_RESULT=0 + +for spec in $(ls "${CONFIGS}"); do + if ! check_spec "${spec}"; then + OVERALL_RESULT=1 + fi +done + +exit "${OVERALL_RESULT}" diff --git a/data-model/tools/configurations/application-deployment.json b/data-model/tools/configurations/application-deployment.json new file mode 100644 index 00000000..fdf209d6 --- /dev/null +++ b/data-model/tools/configurations/application-deployment.json @@ -0,0 +1,6 @@ +{ + "root": "data-model", + "targetclass": "ApplicationDeployment", + "schemafile": "application-deployment.linkml.yaml", + "markdowndoc": "application-deployment.md" +} diff --git a/data-model/tools/configurations/application-description.json b/data-model/tools/configurations/application-description.json new file mode 100644 index 00000000..02517e29 --- /dev/null +++ b/data-model/tools/configurations/application-description.json @@ -0,0 +1,6 @@ +{ + "root": "data-model", + "targetclass": "ApplicationDescription", + "schemafile": "application-description.linkml.yaml", + "markdowndoc": "application-description.md" +} diff --git a/data-model/tools/configurations/deployment-status.json b/data-model/tools/configurations/deployment-status.json new file mode 100644 index 00000000..723601b3 --- /dev/null +++ b/data-model/tools/configurations/deployment-status.json @@ -0,0 +1,6 @@ +{ + "root": "data-model", + "targetclass": "DeploymentStatusManifest", + "schemafile": "deployment-status.linkml.yaml", + "markdowndoc": "deployment-status.md" +} diff --git a/data-model/tools/configurations/desired-state-manifest.json b/data-model/tools/configurations/desired-state-manifest.json new file mode 100644 index 00000000..82d68b74 --- /dev/null +++ b/data-model/tools/configurations/desired-state-manifest.json @@ -0,0 +1,6 @@ +{ + "root": "data-model", + "targetclass": "DesiredStateManifest", + "schemafile": "desired-state-manifest.linkml.yaml", + "markdowndoc": "desired-state-manifest.md" +} diff --git a/data-model/tools/configurations/device-capabilities.json b/data-model/tools/configurations/device-capabilities.json new file mode 100644 index 00000000..6e074251 --- /dev/null +++ b/data-model/tools/configurations/device-capabilities.json @@ -0,0 +1,6 @@ +{ + "root": "data-model", + "targetclass": "DeviceCapabilitiesManifest", + "schemafile": "device-capabilities.linkml.yaml", + "markdowndoc": "device-capabilities.md" +} diff --git a/data-model/tools/generate-all.bash b/data-model/tools/generate-all.bash new file mode 100755 index 00000000..60b79f33 --- /dev/null +++ b/data-model/tools/generate-all.bash @@ -0,0 +1,11 @@ +#!/usr/bin/env bash + +set -eu + +THIS_SCRIPT="$(readlink -f "${0}")" +THIS_DIR="$(dirname "${THIS_SCRIPT}")" + +"${THIS_DIR}/generate-class-diagram.bash" +"${THIS_DIR}/generate-json-schemas.bash" +"${THIS_DIR}/generate-openapi.bash" +"${THIS_DIR}/generate-docs.bash" diff --git a/data-model/tools/generate-class-diagram.bash b/data-model/tools/generate-class-diagram.bash new file mode 100755 index 00000000..440a6f42 --- /dev/null +++ b/data-model/tools/generate-class-diagram.bash @@ -0,0 +1,108 @@ +#!/usr/bin/env bash + +TMP_PLANTUML_FILE=$(mktemp) + +cleanup() { + rm "${TMP_PLANTUML_FILE}" +} + +trap cleanup EXIT + +set -eu + +THIS_SCRIPT="$(readlink -f "${0}")" +THIS_DIR="$(dirname "${THIS_SCRIPT}")" + +ROOT_DIR="$(dirname "$(dirname "${THIS_DIR}")")" + +TGT_DIR="${ROOT_DIR}/generated/diagrams" + +if command -v poetry &>/dev/null; then + RUN="poetry run" +else + if ! command -v linkml &>/dev/null; then + echo "The command 'linkml' is missing" + exit 1 + fi + if ! command -v curl &>/dev/null; then + echo "The command 'curl' is missing" + exit 1 + fi + RUN="" +fi + +mkdir -p "${TGT_DIR}" + +# Holistic class diagram +${RUN} linkml generate plantuml "${ROOT_DIR}/data-model/margo-data-model.linkml.yaml" | + sed "s/@enduml/DeploymentAnnotations ..> ApplicationDescription\n@enduml/" | + sed "s/@enduml/DeploymentStatusManifest ..> ApplicationDeployment\n@enduml/" | + sed "s/@enduml/DesiredStateManifest ..> ApplicationDeployment\n@enduml/" >"${TMP_PLANTUML_FILE}" + +curl -H "Content-Type: test/plain" --data-binary @"${TMP_PLANTUML_FILE}" https://kroki.io/plantuml/svg -o "${TGT_DIR}/DataModel-ClassDiagram.svg" +curl -H "Content-Type: test/plain" --data-binary @"${TMP_PLANTUML_FILE}" https://kroki.io/plantuml/png -o "${TGT_DIR}/DataModel-ClassDiagram.png" + +if [ "$#" -lt 1 ]; then + exit +fi + +# Class diagram focused on ApplicationDescription +${RUN} linkml generate plantuml \ + --classes ApplicationDescription \ + --classes ApplicationMetadata \ + --classes Parameter \ + --classes Configuration \ + --classes DeploymentProfileDescription \ + "${ROOT_DIR}/data-model/application-description.linkml.yaml" | + sed "/Component.*{$/,/}/d" \ + >"${TMP_PLANTUML_FILE}" + +curl -H "Content-Type: test/plain" --data-binary @"${TMP_PLANTUML_FILE}" https://kroki.io/plantuml/svg -o "${TGT_DIR}/ApplicationDescription-ClassDiagram.svg" + +# Class diagram focused on DeploymentStatusManifest +${RUN} linkml generate plantuml \ + --classes DeploymentStatusManifest \ + --classes ApplicationDeployment \ + --classes DeploymentProfile \ + "${ROOT_DIR}/data-model/margo-data-model.linkml.yaml" | + sed "s/@enduml/DeploymentStatusManifest ..> ApplicationDeployment\n@enduml/" | + sed "s/@enduml/ComponentStatus ..> Component\n@enduml/" \ + >"${TMP_PLANTUML_FILE}" + +curl -H "Content-Type: test/plain" --data-binary @"${TMP_PLANTUML_FILE}" https://kroki.io/plantuml/svg -o "${TGT_DIR}/DeploymentStatusManifest-ClassDiagram.svg" + +# Class diagram focused on DesiredStateManifest +${RUN} linkml generate plantuml \ + --classes DesiredStateManifest \ + "${ROOT_DIR}/data-model/margo-data-model.linkml.yaml" | + sed "s/@enduml/DesiredStateManifest ..> ApplicationDeployment\n@enduml/" \ + >"${TMP_PLANTUML_FILE}" + +curl -H "Content-Type: test/plain" --data-binary @"${TMP_PLANTUML_FILE}" https://kroki.io/plantuml/svg -o "${TGT_DIR}/DesiredStateManifest-ClassDiagram.svg" + +# Class diagram focused on DeviceCapabilities +${RUN} linkml generate plantuml \ + --classes DeviceCapabilitiesManifest \ + --classes Properties \ + --classes Resources \ + "${ROOT_DIR}/data-model/margo-data-model.linkml.yaml" \ + >"${TMP_PLANTUML_FILE}" + +curl -H "Content-Type: test/plain" --data-binary @"${TMP_PLANTUML_FILE}" https://kroki.io/plantuml/svg -o "${TGT_DIR}/DeviceCapabilities-ClassDiagram.svg" + +# Class diagram focused on ApplicationDeployment +${RUN} linkml generate plantuml \ + --classes ApplicationDeployment \ + --classes DeploymentMetadata \ + --classes DeploymentAnnotations \ + --classes Spec \ + --classes Parameter \ + --classes DeploymentProfile \ + --classes ComposeDeploymentProfile \ + --classes HelmDeploymentProfile \ + --classes Component \ + "${ROOT_DIR}/data-model/application-deployment.linkml.yaml" | + sed "s/@enduml/DeploymentAnnotations ..> ApplicationDescription\n@enduml/" \ + >"${TMP_PLANTUML_FILE}" + +curl -H "Content-Type: test/plain" --data-binary @"${TMP_PLANTUML_FILE}" https://kroki.io/plantuml/svg -o "${TGT_DIR}/ApplicationDeployment-ClassDiagram.svg" diff --git a/data-model/tools/generate-docs.bash b/data-model/tools/generate-docs.bash new file mode 100755 index 00000000..651d97e6 --- /dev/null +++ b/data-model/tools/generate-docs.bash @@ -0,0 +1,97 @@ +#!/usr/bin/env bash + +set -eu + +THIS_SCRIPT="$(readlink -f "${0}")" +THIS_DIR="$(dirname "${THIS_SCRIPT}")" + +ROOT_DIR="$(dirname "$(dirname "${THIS_DIR}")")" + +TMP_DIR="$(mktemp -d)" + +cleanup() { + rm -r "${TMP_DIR}" +} + +trap cleanup EXIT + +if command -v poetry &>/dev/null; then + RUN="poetry run" +else + if ! command -v linkml &>/dev/null; then + echo "The command 'linkml' is missing" + exit 1 + fi + if ! command -v mkdocs &>/dev/null; then + echo "The command 'mkdocs' is missing" + exit 1 + fi + RUN="" +fi + +TGT_DIR="${ROOT_DIR}/generated/markdown_main-classes" +MERGED_DIR="${ROOT_DIR}/merged" + +mkdir -p "${TGT_DIR}" "${MERGED_DIR}" + +cp -R -H "${ROOT_DIR}/system-design/"* "${MERGED_DIR}/" + +# Main classes + +for schema_name in "application-description" "application-deployment" "deployment-status" "desired-state-manifest" "device-capabilities"; do + ${RUN} linkml generate doc \ + --directory="${TMP_DIR}" \ + --template-directory="${ROOT_DIR}/data-model/resources/markdown-templates_main-classes" \ + --preserve-names \ + --stacktrace \ + --example-directory="${ROOT_DIR}/data-model/resources/examples/valid" \ + "${ROOT_DIR}/data-model/${schema_name}.linkml.yaml" + + mv "${TMP_DIR}/index.md" "${TGT_DIR}/${schema_name}.md" + rm -rf "${TMP_DIR:?}/*" +done + +mv "${TGT_DIR}/deployment-status.md" "${MERGED_DIR}/specification/margo-management-interface/" +mv "${TGT_DIR}/device-capabilities.md" "${MERGED_DIR}/specification/margo-management-interface/" +mv "${TGT_DIR}/application-description.md" "${MERGED_DIR}/specification/applications/" + +# Whole Data Model + +TGT_DIR="${ROOT_DIR}/generated/markdown" + +mkdir -p "${TGT_DIR}" + +${RUN} linkml generate doc \ + --directory="${TGT_DIR}" \ + --template-directory="${ROOT_DIR}/data-model/resources/markdown-templates" \ + --preserve-names \ + --stacktrace \ + --example-directory="${ROOT_DIR}/data-model/resources/examples/valid" \ + "${ROOT_DIR}/data-model/margo-data-model.linkml.yaml" + +mkdir -p system-design/data-model +mv generated/markdown/* "${MERGED_DIR}/data-model/" + +"${THIS_DIR}/generate-class-diagram.bash" + +cp "${ROOT_DIR}/generated/diagrams/DataModel-ClassDiagram.svg" "${MERGED_DIR}/figures/" +cp "${ROOT_DIR}/generated/diagrams/DataModel-ClassDiagram.png" "${MERGED_DIR}/figures/" + +"${THIS_DIR}/generate-openapi.bash" +# generate-openapi.bash writes to both generated/openapi/ and system-design/. +# Since system-design/ was already copied into merged/ above, copy the spec +# into merged/ now so mkdocs build can find it. +cp "${ROOT_DIR}/generated/openapi/workload-management-api-1.0.0.openapi.yaml" \ + "${MERGED_DIR}/specification/margo-management-interface/workload-management-api-1.0.0.yaml" + +# JSON Schemas: copy for download +JSON_SCHEMA_DIR="${ROOT_DIR}/generated/json-schemas" +MERGED_JSON_SCHEMA_DIR="${MERGED_DIR}/json-schemas" + +mkdir -p "${MERGED_JSON_SCHEMA_DIR}" + +for schema_file in "${JSON_SCHEMA_DIR}"/*.schema.json; do + if [ -f "${schema_file}" ]; then + cp "${schema_file}" "${MERGED_JSON_SCHEMA_DIR}/" + fi +done diff --git a/data-model/tools/generate-json-schemas.bash b/data-model/tools/generate-json-schemas.bash new file mode 100755 index 00000000..c2e93c54 --- /dev/null +++ b/data-model/tools/generate-json-schemas.bash @@ -0,0 +1,26 @@ +#!/usr/bin/env bash + +set -eu + +THIS_SCRIPT="$(readlink -f "${0}")" +THIS_DIR="$(dirname "${THIS_SCRIPT}")" + +ROOT_DIR="$(dirname "$(dirname "${THIS_DIR}")")" + +TGT_DIR="${ROOT_DIR}/generated/json-schemas" + +if command -v poetry &>/dev/null; then + RUN="poetry run" +else + if ! command -v linkml &>/dev/null; then + echo "The command 'linkml' is missing" + exit 1 + fi + RUN="" +fi + +mkdir -p "${TGT_DIR}" + +for schema in "application-deployment" "application-description" "deployment-status" "desired-state-manifest" "device-capabilities"; do + ${RUN} linkml generate json-schema "${ROOT_DIR}/data-model/${schema}.linkml.yaml" >"${TGT_DIR}/${schema}.schema.json" +done diff --git a/data-model/tools/generate-openapi.bash b/data-model/tools/generate-openapi.bash new file mode 100755 index 00000000..44e3006c --- /dev/null +++ b/data-model/tools/generate-openapi.bash @@ -0,0 +1,35 @@ +#!/usr/bin/env bash + +set -eu + +THIS_SCRIPT="$(readlink -f "${0}")" +THIS_DIR="$(dirname "${THIS_SCRIPT}")" + +ROOT_DIR="$(dirname "$(dirname "${THIS_DIR}")")" + +TGT_DIR="${ROOT_DIR}/generated/openapi" +TGT_FILE="${TGT_DIR}/workload-management-api-1.0.0.openapi.yaml" + +# Tracked location in system-design/ — this is what mkdocs and the repo use +SYSTEM_DESIGN_FILE="${ROOT_DIR}/system-design/specification/margo-management-interface/workload-management-api-1.0.0.yaml" + +if command -v poetry &>/dev/null; then + RUN="poetry run" +else + if ! command -v linkml &>/dev/null; then + echo "The command 'linkml' is missing" + exit 1 + fi + RUN="" +fi + +mkdir -p "${TGT_DIR}" + +${RUN} linkml generate openapi \ + --template "${THIS_DIR}/workload-management-api-1.0.0.openapi.yaml" \ + "${ROOT_DIR}/data-model/margo-data-model.linkml.yaml" \ + >"${TGT_FILE}" + +# Copy to the tracked location in system-design/ so it is picked up by +# generate-docs.bash (which copies system-design/ → merged/) and by git +cp "${TGT_FILE}" "${SYSTEM_DESIGN_FILE}" diff --git a/system-design/specification/margo-management-interface/workload-management-api-1.0.0.yaml b/data-model/tools/workload-management-api-1.0.0.openapi.yaml similarity index 51% rename from system-design/specification/margo-management-interface/workload-management-api-1.0.0.yaml rename to data-model/tools/workload-management-api-1.0.0.openapi.yaml index e88632e6..769395a7 100644 --- a/system-design/specification/margo-management-interface/workload-management-api-1.0.0.yaml +++ b/data-model/tools/workload-management-api-1.0.0.openapi.yaml @@ -235,7 +235,7 @@ paths: content: application/vnd.margo.manifest.v1+json: schema: - $ref: '#/components/schemas/UnsignedAppStateManifest' + $ref: '#/components/schemas/DesiredStateManifest' '304': description: Not Modified - Manifest has not changed '406': @@ -358,341 +358,4 @@ components: Base64-encoded payload signature using SHA-256 and device certificate. Format: public_key;digital_signature schemas: - ManifestVersion: - type: number - description: > - Monotonically increasing unsigned 64-bit integer in the inclusive range [1, 2^64-1]. - Prevents rollback attacks. The first manifest MUST use 1. - DeploymentBundleRef: - type: object - nullable: true - description: > - Describes a single archive containing all ApplicationDeployment documents. If there are zero deployments (deployments array is empty) the property MUST be present with the value null (it MUST NOT be omitted). - properties: - mediaType: - type: string - description: > - MUST be application/vnd.margo.bundle.v1+tar+gzip; a gzip-compressed tar whose root contains one or more ApplicationDeployment YAML files. If there are zero deployments then bundle MUST be null (an empty archive MUST NOT be served). The archive MUST contain exactly the set of YAML files referenced by deployments. - digest: - type: string - description: > - The digest of the bundle archive. MUST equal the digest computed over the exact sequence of bytes (per Exact Bytes Rule) in the bundle endpoint's HTTP 200 OK response body. - sizeBytes: - type: number - description: > - Unsigned 64-bit advisory estimate of the decoded payload length in bytes for the bundle archive. Provided for bandwidth estimation and update planning. MUST NOT be used for integrity; digest verification remains mandatory. - url: - type: string - description: > - Content-addressable retrieval endpoint of the form /api/v1/clients/{clientId}/bundles/{digest} where {digest} equals bundle.digest. - DeploymentManifestRef: - type: object - description: > - Reference to a deployment manifest with content addressing and integrity verification. - required: - - deploymentId - - digest - - url - properties: - deploymentId: - type: string - description: > - The unique UUID from the ApplicationDeployment's metadata.annotations.id. - digest: - type: string - description: > - The digest of the individual ApplicationDeployment YAML file. MUST equal the digest computed over the exact sequence of bytes (per Exact Bytes Rule) in that deployment endpoint's HTTP 200 OK response body. - sizeBytes: - type: number - description: > - Unsigned 64-bit advisory estimate of the decoded payload length in bytes for the deployment YAML. Provided for planning or progress display. MUST NOT be used for integrity; digest verification remains mandatory. - url: - type: string - description: > - Content-addressable endpoint of the form /api/v1/clients/{clientId}/deployments/{deploymentId}/{digest}. The {digest} MUST equal deployments[].digest; the referenced resource is immutable - UnsignedAppStateManifest: - type: object - required: - - manifestVersion - - bundle - - bundle.mediaType - - bundle.digest - - bundle.url - - deployments - properties: - manifestVersion: - $ref: '#/components/schemas/ManifestVersion' - bundle: - $ref: '#/components/schemas/DeploymentBundleRef' - deployments: - type: array - description: A list of deployment object references for the device. The reference contains some meta info and reference to the url where the deployment is available. - items: - $ref: '#/components/schemas/DeploymentManifestRef' - DeviceCapabilitiesManifest: - type: object - required: [apiVersion, kind, properties] - properties: - apiVersion: - type: string - kind: - type: string - enum: [DeviceCapabilitiesManifest] - properties: - type: object - required: [id, vendor, modelNumber, serialNumber, roles, resources] - properties: - id: - type: string - vendor: - type: string - modelNumber: - type: string - serialNumber: - type: string - roles: - type: array - items: - type: string - enum: [Standalone Cluster, Cluster Leader, Standalone Device] - resources: - type: object - required: [cpu, memory, storage, peripherals, interfaces] - properties: - cpu: - type: object - required: [cores] - properties: - cores: - type: number - architecture: - type: string - enum: [amd64, arm64, arm] - memory: - type: string - storage: - type: string - peripherals: - type: array - items: - $ref: '#/components/schemas/DevicePeripheral' - interfaces: - type: array - items: - $ref: '#/components/schemas/DeviceCommunicationInterface' - - ComponentStatus: - type: object - required: [name, state] - properties: - name: - type: string - state: - type: string - enum: [pending, installing, installed, failed, removing, removed] - error: - type: object - properties: - code: - type: string - message: - type: string - - DeploymentStatusManifest: - type: object - required: [apiVersion, kind, deploymentId, status, components] - properties: - apiVersion: - type: string - kind: - type: string - enum: [DeploymentStatusManifest] - deploymentId: - type: string - status: - type: object - required: [state] - properties: - state: - type: string - enum: [pending, installing, installed, failed, removing, removed] - error: - type: object - properties: - code: - type: string - message: - type: string - components: - type: array - items: - $ref: '#/components/schemas/ComponentStatus' - DevicePeripheral: - type: object - required: [type] - properties: - type: - type: string - enum: [gpu, display, camera, microphone, speaker] - manufacturer: - type: string - model: - type: string - - DeviceCommunicationInterface: - type: object - required: [type] - properties: - type: - type: string - enum: [ethernet, wifi, cellular, bluetooth, usb, canbus, rs232] - - # app deployment struct added here for ease of programming, the code generators will generate the structs - # for the actual app deployment yaml and parsing would be easy to do - appDeploymentManifest: - type: object - description: Application Deployment manifest - required: [apiVersion, kind, metadata, spec] - properties: - apiVersion: - type: string - default: margo.org - description: API version - kind: - type: string - default: ApplicationDeployment - description: Resource kind - metadata: - $ref: '#/components/schemas/appDeploymentMetadata' - spec: - $ref: '#/components/schemas/appDeploymentSpec' - appDeploymentMetadata: - type: object - required: [name] - properties: - id: - type: string - description: Unique identifier for the application deployment - name: - type: string - description: Name of the resource - namespace: - type: string - description: Namespace of the resource - labels: - type: object - additionalProperties: { type: string } - description: Labels for the resource - annotations: - type: object - additionalProperties: { type: string } - description: Annotations for the resource - helmApplicationDeploymentProfileComponent: - type: object - description: Helm Application Deployment Profile Component - required: [name, properties] - properties: - name: - type: string - description: Name of the component - properties: - type: object - required: [repository] - properties: - repository: - type: string - description: Repository of the component - revision: - type: string - description: Revision of the component - timeout: - type: string - description: Timeout for the component - wait: - type: boolean - description: Wait for the component to be ready - composeApplicationDeploymentProfileComponent: - type: object - description: Compose Application Deployment Profile Component - required: [name, properties] - properties: - name: - type: string - description: Name of the component - properties: - type: object - required: [packageLocation] - properties: - packageLocation: - type: string - description: The URL indicating the Compose package's location. It should be a direct path to the compose.yaml or compose.yaml file archived in tar.gz - keyLocation: - type: string - description: Key location of the component - timeout: - type: string - description: Timeout for the component - wait: - type: boolean - description: Wait for the component to be ready - appDeploymentProfile: - type: object - description: Application Deployment Profile - required: [type, components] - properties: - type: - type: string - enum: ["helm", "compose"] - description: Type of deployment profile - components: - type: array - items: - oneOf: - - $ref: '#/components/schemas/helmApplicationDeploymentProfileComponent' - - $ref: '#/components/schemas/composeApplicationDeploymentProfileComponent' - description: Components of the deployment profile - appParameterTarget: - type: object - description: Application Parameter Target - required: [pointer, components] - properties: - pointer: - type: string - description: Pointer to the parameter - components: - type: array - items: - type: string - description: Components of the parameter - appParameterValue: - type: object - description: Application Parameter Value - required: [value, targets] - properties: - value: - # type: object - description: Value of the parameter - additionalProperties: true - x-go-type: interface{} - targets: - type: array - items: - $ref: '#/components/schemas/appParameterTarget' - description: Targets of the parameter - appDeploymentParams: - type: object - description: Application Parameters - additionalProperties: - $ref: '#/components/schemas/appParameterValue' - appDeploymentSpec: - type: object - description: Application Deployment specification - required: [appPackageRef, deploymentProfile] - properties: - deploymentProfile: - $ref: '#/components/schemas/appDeploymentProfile' - description: Deployment profile - parameters: - $ref: '#/components/schemas/appDeploymentParams' - description: Parameters for the deployment \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index 8a2af410..acfe196a 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1,5 +1,5 @@ site_name: Specification (pre-draft) -docs_dir: system-design +docs_dir: merged nav: - What is Margo?: index.md @@ -8,9 +8,11 @@ nav: - specification/margo-management-interface/api-requirements-and-security.md - specification/margo-management-interface/certificate-api.md - specification/margo-management-interface/device-client-onboarding.md - - specification/margo-management-interface/device-capabilities.md + - specification/margo-management-interface/device-capabilities-api.md + - Device Capabilities: specification/margo-management-interface/device-capabilities.md - specification/margo-management-interface/desired-state.md - - specification/margo-management-interface/deployment-status.md + - specification/margo-management-interface/deployment-status-api.md + - Deployment Status: specification/margo-management-interface/deployment-status.md - specification/margo-management-interface/management-interface-swagger.md - Applications: - specification/applications/application-description.md @@ -21,6 +23,15 @@ nav: - specification/observability/publishing-workload-observability-data.md - specification/observability/collecting-workload-observability-data.md - specification/observability/consuming-workload-observability-data.md + - Data Model: + - Overview: data-model/index.md + - ApplicationDescription: data-model/ApplicationDescription.md + - ApplicationDeployment: data-model/ApplicationDeployment.md + - DeviceCapabilitiesManifest: data-model/DeviceCapabilitiesManifest.md + - DesiredStateManifest: data-model/DesiredStateManifest.md + - DeploymentStatusManifest: data-model/DeploymentStatusManifest.md + - Make a Contribution: + - make-a-contribution/contributing.md theme: name: material @@ -49,6 +60,8 @@ theme: markdown_extensions: - toc: toc_depth: 2 + - admonition + - pymdownx.details - pymdownx.superfences: custom_fences: - name: mermaid @@ -62,6 +75,8 @@ extra_css: extra_javascript: - assets/swagger-ui-bundle.js - assets/swagger-ui-standalone-preset.js + - assets/svg-pan-zoom.min.js + - assets/svg-pan-zoom-init.js extra: diff --git a/poetry.lock b/poetry.lock index 661c5c89..bde4e717 100644 --- a/poetry.lock +++ b/poetry.lock @@ -1,4 +1,16 @@ -# This file is automatically @generated by Poetry 2.0.1 and should not be changed by hand. +# This file is automatically @generated by Poetry 2.4.1 and should not be changed by hand. + +[[package]] +name = "alabaster" +version = "1.0.0" +description = "A light, configurable Sphinx theme" +optional = false +python-versions = ">=3.10" +groups = ["main"] +files = [ + {file = "alabaster-1.0.0-py3-none-any.whl", hash = "sha256:fc6786402dc3fcb2de3cabd5fe455a2db534b371124f1f21de8731783dec828b"}, + {file = "alabaster-1.0.0.tar.gz", hash = "sha256:c00dca57bca26fa62a6d7d0a9fcce65f3e026e9bfe33e9c538fd3fbb2144fd9e"}, +] [[package]] name = "annotated-types" @@ -56,12 +68,12 @@ files = [ ] [package.extras] -benchmark = ["cloudpickle", "hypothesis", "mypy (>=1.11.1)", "pympler", "pytest (>=4.3.0)", "pytest-codspeed", "pytest-mypy-plugins", "pytest-xdist[psutil]"] -cov = ["cloudpickle", "coverage[toml] (>=5.3)", "hypothesis", "mypy (>=1.11.1)", "pympler", "pytest (>=4.3.0)", "pytest-mypy-plugins", "pytest-xdist[psutil]"] -dev = ["cloudpickle", "hypothesis", "mypy (>=1.11.1)", "pre-commit", "pympler", "pytest (>=4.3.0)", "pytest-mypy-plugins", "pytest-xdist[psutil]"] +benchmark = ["cloudpickle ; platform_python_implementation == \"CPython\"", "hypothesis", "mypy (>=1.11.1) ; platform_python_implementation == \"CPython\" and python_version >= \"3.9\"", "pympler", "pytest (>=4.3.0)", "pytest-codspeed", "pytest-mypy-plugins ; platform_python_implementation == \"CPython\" and python_version >= \"3.9\" and python_version < \"3.13\"", "pytest-xdist[psutil]"] +cov = ["cloudpickle ; platform_python_implementation == \"CPython\"", "coverage[toml] (>=5.3)", "hypothesis", "mypy (>=1.11.1) ; platform_python_implementation == \"CPython\" and python_version >= \"3.9\"", "pympler", "pytest (>=4.3.0)", "pytest-mypy-plugins ; platform_python_implementation == \"CPython\" and python_version >= \"3.9\" and python_version < \"3.13\"", "pytest-xdist[psutil]"] +dev = ["cloudpickle ; platform_python_implementation == \"CPython\"", "hypothesis", "mypy (>=1.11.1) ; platform_python_implementation == \"CPython\" and python_version >= \"3.9\"", "pre-commit", "pympler", "pytest (>=4.3.0)", "pytest-mypy-plugins ; platform_python_implementation == \"CPython\" and python_version >= \"3.9\" and python_version < \"3.13\"", "pytest-xdist[psutil]"] docs = ["cogapp", "furo", "myst-parser", "sphinx", "sphinx-notfound-page", "sphinxcontrib-towncrier", "towncrier (<24.7)"] -tests = ["cloudpickle", "hypothesis", "mypy (>=1.11.1)", "pympler", "pytest (>=4.3.0)", "pytest-mypy-plugins", "pytest-xdist[psutil]"] -tests-mypy = ["mypy (>=1.11.1)", "pytest-mypy-plugins"] +tests = ["cloudpickle ; platform_python_implementation == \"CPython\"", "hypothesis", "mypy (>=1.11.1) ; platform_python_implementation == \"CPython\" and python_version >= \"3.9\"", "pympler", "pytest (>=4.3.0)", "pytest-mypy-plugins ; platform_python_implementation == \"CPython\" and python_version >= \"3.9\" and python_version < \"3.13\"", "pytest-xdist[psutil]"] +tests-mypy = ["mypy (>=1.11.1) ; platform_python_implementation == \"CPython\" and python_version >= \"3.9\"", "pytest-mypy-plugins ; platform_python_implementation == \"CPython\" and python_version >= \"3.9\" and python_version < \"3.13\""] [[package]] name = "babel" @@ -245,14 +257,14 @@ files = [ [[package]] name = "click" -version = "8.1.7" +version = "8.3.3" description = "Composable command line interface toolkit" optional = false -python-versions = ">=3.7" +python-versions = ">=3.10" groups = ["main"] files = [ - {file = "click-8.1.7-py3-none-any.whl", hash = "sha256:ae74fb96c20a0277a1d615f1e4d73c8414f5a98db8b799a7931d1582f3390c28"}, - {file = "click-8.1.7.tar.gz", hash = "sha256:ca9853ad459e787e2192211578cc907e7594e294c7ccc834310722b41b9ca6de"}, + {file = "click-8.3.3-py3-none-any.whl", hash = "sha256:a2bf429bb3033c89fa4936ffb35d5cb471e3719e1f3c8a7c3fff0b8314305613"}, + {file = "click-8.3.3.tar.gz", hash = "sha256:398329ad4837b2ff7cbe1dd166a4c0f8900c3ca3a218de04466f38f6497f18a2"}, ] [package.dependencies] @@ -310,7 +322,7 @@ files = [ wrapt = ">=1.10,<2" [package.extras] -dev = ["PyTest", "PyTest-Cov", "bump2version (<1)", "jinja2 (>=3.0.3,<3.1.0)", "setuptools", "sphinx (<2)", "tox"] +dev = ["PyTest", "PyTest-Cov", "bump2version (<1)", "jinja2 (>=3.0.3,<3.1.0)", "setuptools ; python_version >= \"3.12\"", "sphinx (<2)", "tox"] [[package]] name = "distlib" @@ -324,6 +336,18 @@ files = [ {file = "distlib-0.3.9.tar.gz", hash = "sha256:a60f20dea646b8a33f3e7772f74dc0b2d0772d2837ee1342a00645c81edf9403"}, ] +[[package]] +name = "docutils" +version = "0.22.4" +description = "Docutils -- Python Documentation Utilities" +optional = false +python-versions = ">=3.9" +groups = ["main"] +files = [ + {file = "docutils-0.22.4-py3-none-any.whl", hash = "sha256:d0013f540772d1420576855455d050a2180186c91c15779301ac2ccb3eeb68de"}, + {file = "docutils-0.22.4.tar.gz", hash = "sha256:4db53b1fde9abecbb74d91230d32ab626d94f6badfc575d6db9194a49df29968"}, +] + [[package]] name = "et-xmlfile" version = "2.0.0" @@ -351,7 +375,7 @@ files = [ [package.extras] docs = ["furo (>=2024.8.6)", "sphinx (>=8.0.2)", "sphinx-autodoc-typehints (>=2.4.1)"] testing = ["covdefaults (>=2.3)", "coverage (>=7.6.1)", "diff-cover (>=9.2)", "pytest (>=8.3.3)", "pytest-asyncio (>=0.24)", "pytest-cov (>=5)", "pytest-mock (>=3.14)", "pytest-timeout (>=2.3.1)", "virtualenv (>=20.26.4)"] -typing = ["typing-extensions (>=4.12.2)"] +typing = ["typing-extensions (>=4.12.2) ; python_version < \"3.11\""] [[package]] name = "fqdn" @@ -407,7 +431,7 @@ description = "Lightweight in-process concurrent programming" optional = false python-versions = ">=3.7" groups = ["main"] -markers = "python_version < \"3.13\" and (platform_machine == \"aarch64\" or platform_machine == \"ppc64le\" or platform_machine == \"x86_64\" or platform_machine == \"amd64\" or platform_machine == \"AMD64\" or platform_machine == \"win32\" or platform_machine == \"WIN32\")" +markers = "python_version == \"3.12\" and (platform_machine == \"aarch64\" or platform_machine == \"ppc64le\" or platform_machine == \"x86_64\" or platform_machine == \"amd64\" or platform_machine == \"AMD64\" or platform_machine == \"win32\" or platform_machine == \"WIN32\")" files = [ {file = "greenlet-3.1.1-cp310-cp310-macosx_11_0_universal2.whl", hash = "sha256:0bbae94a29c9e5c7e4a2b7f0aae5c17e8e90acbfd3bf6270eeba60c39fce3563"}, {file = "greenlet-3.1.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0fde093fb93f35ca72a556cf72c92ea3ebfda3d79fc35bb19fbe685853869a83"}, @@ -515,6 +539,18 @@ files = [ [package.extras] all = ["flake8 (>=7.1.1)", "mypy (>=1.11.2)", "pytest (>=8.3.2)", "ruff (>=0.6.2)"] +[[package]] +name = "imagesize" +version = "1.5.0" +description = "Getting image size from png/jpeg/jpeg2000/gif file" +optional = false +python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,>=2.7" +groups = ["main"] +files = [ + {file = "imagesize-1.5.0-py2.py3-none-any.whl", hash = "sha256:32677681b3f434c2cb496f00e89c5a291247b35b1f527589909e008057da5899"}, + {file = "imagesize-1.5.0.tar.gz", hash = "sha256:8bfc5363a7f2133a89f0098451e0bcb1cd71aba4dc02bbcecb39d99d40e1b94f"}, +] + [[package]] name = "iniconfig" version = "2.0.0" @@ -615,37 +651,6 @@ files = [ [package.dependencies] hbreader = "*" -[[package]] -name = "jsonpatch" -version = "1.33" -description = "Apply JSON-Patches (RFC 6902)" -optional = false -python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*, !=3.6.*" -groups = ["main"] -files = [ - {file = "jsonpatch-1.33-py2.py3-none-any.whl", hash = "sha256:0ae28c0cd062bbd8b8ecc26d7d164fbbea9652a1a3693f3b956c1eae5145dade"}, - {file = "jsonpatch-1.33.tar.gz", hash = "sha256:9fcd4009c41e6d12348b4a0ff2563ba56a2923a7dfee731d004e212e1ee5030c"}, -] - -[package.dependencies] -jsonpointer = ">=1.9" - -[[package]] -name = "jsonpath-ng" -version = "1.7.0" -description = "A final implementation of JSONPath for Python that aims to be standard compliant, including arithmetic and binary comparison operators and providing clear AST for metaprogramming." -optional = false -python-versions = "*" -groups = ["main"] -files = [ - {file = "jsonpath-ng-1.7.0.tar.gz", hash = "sha256:f6f5f7fd4e5ff79c785f1573b394043b39849fb2bb47bcead935d12b00beab3c"}, - {file = "jsonpath_ng-1.7.0-py2-none-any.whl", hash = "sha256:898c93fc173f0c336784a3fa63d7434297544b7198124a68f9a3ef9597b0ae6e"}, - {file = "jsonpath_ng-1.7.0-py3-none-any.whl", hash = "sha256:f3d7f9e848cba1b6da28c55b1c26ff915dc9e0b1ba7e752a53d6da8d5cbd00b6"}, -] - -[package.dependencies] -ply = "*" - [[package]] name = "jsonpointer" version = "3.0.0" @@ -660,14 +665,14 @@ files = [ [[package]] name = "jsonschema" -version = "4.23.0" +version = "4.24.1" description = "An implementation of JSON Schema validation for Python" optional = false -python-versions = ">=3.8" +python-versions = ">=3.9" groups = ["main"] files = [ - {file = "jsonschema-4.23.0-py3-none-any.whl", hash = "sha256:fbadb6f8b144a8f8cf9f0b89ba94501d143e50411a1278633f56a7acf7fd5566"}, - {file = "jsonschema-4.23.0.tar.gz", hash = "sha256:d71497fef26351a33265337fa77ffeb82423f3ea21283cd9467bb03999266bc4"}, + {file = "jsonschema-4.24.1-py3-none-any.whl", hash = "sha256:6b916866aa0b61437785f1277aa2cbd63512e8d4b47151072ef13292049b4627"}, + {file = "jsonschema-4.24.1.tar.gz", hash = "sha256:fe45a130cc7f67cd0d67640b4e7e3e2e666919462ae355eda238296eafeb4b5d"}, ] [package.dependencies] @@ -676,7 +681,7 @@ fqdn = {version = "*", optional = true, markers = "extra == \"format\""} idna = {version = "*", optional = true, markers = "extra == \"format\""} isoduration = {version = "*", optional = true, markers = "extra == \"format\""} jsonpointer = {version = ">1.13", optional = true, markers = "extra == \"format\""} -jsonschema-specifications = ">=2023.03.6" +jsonschema-specifications = ">=2023.3.6" referencing = ">=0.28.4" rfc3339-validator = {version = "*", optional = true, markers = "extra == \"format\""} rfc3987 = {version = "*", optional = true, markers = "extra == \"format\""} @@ -705,32 +710,31 @@ referencing = ">=0.31.0" [[package]] name = "linkml" -version = "1.8.5" +version = "1.11.0" description = "Linked Open Data Modeling Language" optional = false -python-versions = "<4.0.0,>=3.8.1" +python-versions = ">=3.10" groups = ["main"] files = [ - {file = "linkml-1.8.5-py3-none-any.whl", hash = "sha256:5a45577a4bb380f3a128f45764545cb2da92dd3310a110de7dc5355796d5ac43"}, - {file = "linkml-1.8.5.tar.gz", hash = "sha256:8f31834560ade4b7f1aebc973d22b31951d7061643d32bdcba258a650db9b140"}, + {file = "linkml-1.11.0-py3-none-any.whl", hash = "sha256:8145e5ae2d90d304d135e26aac5ddff4d752d89a80e476f65a14588dfb43e7b2"}, + {file = "linkml-1.11.0.tar.gz", hash = "sha256:bbe7f31dbfaac10047feb951b56520e9467a763d7ae08cf5d38783b70965ad53"}, ] [package.dependencies] antlr4-python3-runtime = ">=4.9.0,<4.10" -click = ">=7.0" +click = ">=8.2" graphviz = ">=0.10.1" hbreader = "*" isodate = ">=0.6.0" jinja2 = ">=3.1.0" jsonasobj2 = ">=1.0.3,<2.0.0" jsonschema = {version = ">=4.0.0", extras = ["format"]} -linkml-dataops = "*" -linkml-runtime = ">=1.8.1,<2.0.0" +linkml-runtime = ">=1.10.0,<2.0.0" openpyxl = "*" parse = "*" prefixcommons = ">=0.1.7" prefixmaps = ">=0.2.2" -pydantic = ">=1.0.0,<3.0.0" +pydantic = ">=2.0.0,<3.0.0" pyjsg = ">=0.11.6" pyshex = ">=0.7.20" pyshexc = ">=0.8.3" @@ -738,44 +742,20 @@ python-dateutil = "*" pyyaml = "*" rdflib = ">=6.0.0" requests = ">=2.22" +sphinx-click = ">=6.0.0" sqlalchemy = ">=1.4.31" watchdog = ">=0.9.0" -[package.extras] -black = ["black (>=24.0.0)"] -numpydantic = ["numpydantic (>=1.6.1)"] -shacl = ["pyshacl (>=0.25.0,<0.26.0)"] -tests = ["black (>=24.0.0)", "numpydantic (>=1.6.1)", "pyshacl (>=0.25.0,<0.26.0)"] - -[[package]] -name = "linkml-dataops" -version = "0.1.0" -description = "LinkML Data Operations API" -optional = false -python-versions = ">=3.7" -groups = ["main"] -files = [ - {file = "linkml_dataops-0.1.0-py3-none-any.whl", hash = "sha256:193cf7f659e5f07946d2c2761896910d5f7151d91282543b1363801f68307f4c"}, - {file = "linkml_dataops-0.1.0.tar.gz", hash = "sha256:4550eab65e78b70dc3b9c651724a94ac2b1d1edb2fbe576465f1d6951a54ed04"}, -] - -[package.dependencies] -jinja2 = "*" -jsonpatch = "*" -jsonpath-ng = "*" -linkml-runtime = ">=1.1.6" -"ruamel.yaml" = "*" - [[package]] name = "linkml-runtime" -version = "1.8.3" +version = "1.10.0" description = "Runtime environment for LinkML, the Linked open data modeling language" optional = false -python-versions = "<4.0,>=3.8" +python-versions = ">=3.10" groups = ["main"] files = [ - {file = "linkml_runtime-1.8.3-py3-none-any.whl", hash = "sha256:0750920f1348fffa903d99e7b5834ce425a2a538285aff9068dbd96d05caabd1"}, - {file = "linkml_runtime-1.8.3.tar.gz", hash = "sha256:5b7f682eef54aaf0a59c50eeacdb11463b43b124a044caf496cde59936ac05c8"}, + {file = "linkml_runtime-1.10.0-py3-none-any.whl", hash = "sha256:b7caf806e1b49bf62005d8f398b070c282742c5f6626469fdc1660add0c9da58"}, + {file = "linkml_runtime-1.10.0.tar.gz", hash = "sha256:899889d584ce8056c5c44512b2d247bdc84a8484c3aa228aeb2db283e3a9d2ec"}, ] [package.dependencies] @@ -793,6 +773,9 @@ pyyaml = "*" rdflib = ">=6.0.0" requests = "*" +[package.extras] +dev = ["coverage", "requests-cache"] + [[package]] name = "markdown" version = "3.7" @@ -921,7 +904,7 @@ watchdog = ">=2.0" [package.extras] i18n = ["babel (>=2.9.0)"] -min-versions = ["babel (==2.9.0)", "click (==7.0)", "colorama (==0.4)", "ghp-import (==1.0)", "importlib-metadata (==4.4)", "jinja2 (==2.11.1)", "markdown (==3.3.6)", "markupsafe (==2.0.1)", "mergedeep (==1.3.4)", "mkdocs-get-deps (==0.2.0)", "packaging (==20.5)", "pathspec (==0.11.1)", "pyyaml (==5.1)", "pyyaml-env-tag (==0.1)", "watchdog (==2.0)"] +min-versions = ["babel (==2.9.0)", "click (==7.0)", "colorama (==0.4) ; platform_system == \"Windows\"", "ghp-import (==1.0)", "importlib-metadata (==4.4) ; python_version < \"3.10\"", "jinja2 (==2.11.1)", "markdown (==3.3.6)", "markupsafe (==2.0.1)", "mergedeep (==1.3.4)", "mkdocs-get-deps (==0.2.0)", "packaging (==20.5)", "pathspec (==0.11.1)", "pyyaml (==5.1)", "pyyaml-env-tag (==0.1)", "watchdog (==2.0)"] [[package]] name = "mkdocs-get-deps" @@ -1098,18 +1081,6 @@ files = [ dev = ["pre-commit", "tox"] testing = ["pytest", "pytest-benchmark"] -[[package]] -name = "ply" -version = "3.11" -description = "Python Lex & Yacc" -optional = false -python-versions = "*" -groups = ["main"] -files = [ - {file = "ply-3.11-py2.py3-none-any.whl", hash = "sha256:096f9b8350b65ebd2fd1346b12452efe5b9607f7482813ffca50c22722a807ce"}, - {file = "ply-3.11.tar.gz", hash = "sha256:00c7c1aaa88358b9c765b6d3000c6eec0ba42abca5351b095321aef446081da3"}, -] - [[package]] name = "prefixcommons" version = "0.1.12" @@ -1163,7 +1134,7 @@ typing-extensions = ">=4.12.2" [package.extras] email = ["email-validator (>=2.0.0)"] -timezone = ["tzdata"] +timezone = ["tzdata ; python_version >= \"3.9\" and platform_system == \"Windows\""] [[package]] name = "pydantic-core" @@ -1603,19 +1574,20 @@ rdflib-jsonld = "0.6.1" [[package]] name = "referencing" -version = "0.35.1" +version = "0.37.0" description = "JSON Referencing + Python" optional = false -python-versions = ">=3.8" +python-versions = ">=3.10" groups = ["main"] files = [ - {file = "referencing-0.35.1-py3-none-any.whl", hash = "sha256:eda6d3234d62814d1c64e305c1331c9a3a6132da475ab6382eaa997b21ee75de"}, - {file = "referencing-0.35.1.tar.gz", hash = "sha256:25b42124a6c8b632a425174f24087783efb348a6f1e0008e63cd4466fedf703c"}, + {file = "referencing-0.37.0-py3-none-any.whl", hash = "sha256:381329a9f99628c9069361716891d34ad94af76e461dcb0335825aecc7692231"}, + {file = "referencing-0.37.0.tar.gz", hash = "sha256:44aefc3142c5b842538163acb373e24cce6632bd54bdb01b21ad5863489f50d8"}, ] [package.dependencies] attrs = ">=22.2.0" rpds-py = ">=0.7.0" +typing-extensions = {version = ">=4.4.0", markers = "python_version < \"3.13\""} [[package]] name = "regex" @@ -1770,6 +1742,18 @@ files = [ {file = "rfc3987-1.3.8.tar.gz", hash = "sha256:d3c4d257a560d544e9826b38bc81db676890c79ab9d7ac92b39c7a253d5ca733"}, ] +[[package]] +name = "roman-numerals" +version = "4.1.0" +description = "Manipulate well-formed Roman numerals" +optional = false +python-versions = ">=3.10" +groups = ["main"] +files = [ + {file = "roman_numerals-4.1.0-py3-none-any.whl", hash = "sha256:647ba99caddc2cc1e55a51e4360689115551bf4476d90e8162cf8c345fe233c7"}, + {file = "roman_numerals-4.1.0.tar.gz", hash = "sha256:1af8b147eb1405d5839e78aeb93131690495fe9da5c91856cb33ad55a7f1e5b2"}, +] + [[package]] name = "rpds-py" version = "0.21.0" @@ -1870,82 +1854,6 @@ files = [ {file = "rpds_py-0.21.0.tar.gz", hash = "sha256:ed6378c9d66d0de903763e7706383d60c33829581f0adff47b6535f1802fa6db"}, ] -[[package]] -name = "ruamel-yaml" -version = "0.18.6" -description = "ruamel.yaml is a YAML parser/emitter that supports roundtrip preservation of comments, seq/map flow style, and map key order" -optional = false -python-versions = ">=3.7" -groups = ["main"] -files = [ - {file = "ruamel.yaml-0.18.6-py3-none-any.whl", hash = "sha256:57b53ba33def16c4f3d807c0ccbc00f8a6081827e81ba2491691b76882d0c636"}, - {file = "ruamel.yaml-0.18.6.tar.gz", hash = "sha256:8b27e6a217e786c6fbe5634d8f3f11bc63e0f80f6a5890f28863d9c45aac311b"}, -] - -[package.dependencies] -"ruamel.yaml.clib" = {version = ">=0.2.7", markers = "platform_python_implementation == \"CPython\" and python_version < \"3.13\""} - -[package.extras] -docs = ["mercurial (>5.7)", "ryd"] -jinja2 = ["ruamel.yaml.jinja2 (>=0.2)"] - -[[package]] -name = "ruamel-yaml-clib" -version = "0.2.12" -description = "C version of reader, parser and emitter for ruamel.yaml derived from libyaml" -optional = false -python-versions = ">=3.9" -groups = ["main"] -markers = "platform_python_implementation == \"CPython\" and python_version < \"3.13\"" -files = [ - {file = "ruamel.yaml.clib-0.2.12-cp310-cp310-macosx_13_0_arm64.whl", hash = "sha256:11f891336688faf5156a36293a9c362bdc7c88f03a8a027c2c1d8e0bcde998e5"}, - {file = "ruamel.yaml.clib-0.2.12-cp310-cp310-manylinux2014_aarch64.whl", hash = "sha256:a606ef75a60ecf3d924613892cc603b154178ee25abb3055db5062da811fd969"}, - {file = "ruamel.yaml.clib-0.2.12-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:fd5415dded15c3822597455bc02bcd66e81ef8b7a48cb71a33628fc9fdde39df"}, - {file = "ruamel.yaml.clib-0.2.12-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f66efbc1caa63c088dead1c4170d148eabc9b80d95fb75b6c92ac0aad2437d76"}, - {file = "ruamel.yaml.clib-0.2.12-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:22353049ba4181685023b25b5b51a574bce33e7f51c759371a7422dcae5402a6"}, - {file = "ruamel.yaml.clib-0.2.12-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:932205970b9f9991b34f55136be327501903f7c66830e9760a8ffb15b07f05cd"}, - {file = "ruamel.yaml.clib-0.2.12-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:a52d48f4e7bf9005e8f0a89209bf9a73f7190ddf0489eee5eb51377385f59f2a"}, - {file = "ruamel.yaml.clib-0.2.12-cp310-cp310-win32.whl", hash = "sha256:3eac5a91891ceb88138c113f9db04f3cebdae277f5d44eaa3651a4f573e6a5da"}, - {file = "ruamel.yaml.clib-0.2.12-cp310-cp310-win_amd64.whl", hash = "sha256:ab007f2f5a87bd08ab1499bdf96f3d5c6ad4dcfa364884cb4549aa0154b13a28"}, - {file = "ruamel.yaml.clib-0.2.12-cp311-cp311-macosx_13_0_arm64.whl", hash = "sha256:4a6679521a58256a90b0d89e03992c15144c5f3858f40d7c18886023d7943db6"}, - {file = "ruamel.yaml.clib-0.2.12-cp311-cp311-manylinux2014_aarch64.whl", hash = "sha256:d84318609196d6bd6da0edfa25cedfbabd8dbde5140a0a23af29ad4b8f91fb1e"}, - {file = "ruamel.yaml.clib-0.2.12-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bb43a269eb827806502c7c8efb7ae7e9e9d0573257a46e8e952f4d4caba4f31e"}, - {file = "ruamel.yaml.clib-0.2.12-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:811ea1594b8a0fb466172c384267a4e5e367298af6b228931f273b111f17ef52"}, - {file = "ruamel.yaml.clib-0.2.12-cp311-cp311-musllinux_1_1_i686.whl", hash = "sha256:cf12567a7b565cbf65d438dec6cfbe2917d3c1bdddfce84a9930b7d35ea59642"}, - {file = "ruamel.yaml.clib-0.2.12-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:7dd5adc8b930b12c8fc5b99e2d535a09889941aa0d0bd06f4749e9a9397c71d2"}, - {file = "ruamel.yaml.clib-0.2.12-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:1492a6051dab8d912fc2adeef0e8c72216b24d57bd896ea607cb90bb0c4981d3"}, - {file = "ruamel.yaml.clib-0.2.12-cp311-cp311-win32.whl", hash = "sha256:bd0a08f0bab19093c54e18a14a10b4322e1eacc5217056f3c063bd2f59853ce4"}, - {file = "ruamel.yaml.clib-0.2.12-cp311-cp311-win_amd64.whl", hash = "sha256:a274fb2cb086c7a3dea4322ec27f4cb5cc4b6298adb583ab0e211a4682f241eb"}, - {file = "ruamel.yaml.clib-0.2.12-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:20b0f8dc160ba83b6dcc0e256846e1a02d044e13f7ea74a3d1d56ede4e48c632"}, - {file = "ruamel.yaml.clib-0.2.12-cp312-cp312-manylinux2014_aarch64.whl", hash = "sha256:943f32bc9dedb3abff9879edc134901df92cfce2c3d5c9348f172f62eb2d771d"}, - {file = "ruamel.yaml.clib-0.2.12-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:95c3829bb364fdb8e0332c9931ecf57d9be3519241323c5274bd82f709cebc0c"}, - {file = "ruamel.yaml.clib-0.2.12-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:749c16fcc4a2b09f28843cda5a193e0283e47454b63ec4b81eaa2242f50e4ccd"}, - {file = "ruamel.yaml.clib-0.2.12-cp312-cp312-musllinux_1_1_i686.whl", hash = "sha256:bf165fef1f223beae7333275156ab2022cffe255dcc51c27f066b4370da81e31"}, - {file = "ruamel.yaml.clib-0.2.12-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:32621c177bbf782ca5a18ba4d7af0f1082a3f6e517ac2a18b3974d4edf349680"}, - {file = "ruamel.yaml.clib-0.2.12-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:b82a7c94a498853aa0b272fd5bc67f29008da798d4f93a2f9f289feb8426a58d"}, - {file = "ruamel.yaml.clib-0.2.12-cp312-cp312-win32.whl", hash = "sha256:e8c4ebfcfd57177b572e2040777b8abc537cdef58a2120e830124946aa9b42c5"}, - {file = "ruamel.yaml.clib-0.2.12-cp312-cp312-win_amd64.whl", hash = "sha256:0467c5965282c62203273b838ae77c0d29d7638c8a4e3a1c8bdd3602c10904e4"}, - {file = "ruamel.yaml.clib-0.2.12-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:4c8c5d82f50bb53986a5e02d1b3092b03622c02c2eb78e29bec33fd9593bae1a"}, - {file = "ruamel.yaml.clib-0.2.12-cp313-cp313-manylinux2014_aarch64.whl", hash = "sha256:e7e3736715fbf53e9be2a79eb4db68e4ed857017344d697e8b9749444ae57475"}, - {file = "ruamel.yaml.clib-0.2.12-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0b7e75b4965e1d4690e93021adfcecccbca7d61c7bddd8e22406ef2ff20d74ef"}, - {file = "ruamel.yaml.clib-0.2.12-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:96777d473c05ee3e5e3c3e999f5d23c6f4ec5b0c38c098b3a5229085f74236c6"}, - {file = "ruamel.yaml.clib-0.2.12-cp313-cp313-musllinux_1_1_i686.whl", hash = "sha256:3bc2a80e6420ca8b7d3590791e2dfc709c88ab9152c00eeb511c9875ce5778bf"}, - {file = "ruamel.yaml.clib-0.2.12-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:e188d2699864c11c36cdfdada94d781fd5d6b0071cd9c427bceb08ad3d7c70e1"}, - {file = "ruamel.yaml.clib-0.2.12-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:4f6f3eac23941b32afccc23081e1f50612bdbe4e982012ef4f5797986828cd01"}, - {file = "ruamel.yaml.clib-0.2.12-cp313-cp313-win32.whl", hash = "sha256:6442cb36270b3afb1b4951f060eccca1ce49f3d087ca1ca4563a6eb479cb3de6"}, - {file = "ruamel.yaml.clib-0.2.12-cp313-cp313-win_amd64.whl", hash = "sha256:e5b8daf27af0b90da7bb903a876477a9e6d7270be6146906b276605997c7e9a3"}, - {file = "ruamel.yaml.clib-0.2.12-cp39-cp39-macosx_12_0_arm64.whl", hash = "sha256:fc4b630cd3fa2cf7fce38afa91d7cfe844a9f75d7f0f36393fa98815e911d987"}, - {file = "ruamel.yaml.clib-0.2.12-cp39-cp39-manylinux2014_aarch64.whl", hash = "sha256:bc5f1e1c28e966d61d2519f2a3d451ba989f9ea0f2307de7bc45baa526de9e45"}, - {file = "ruamel.yaml.clib-0.2.12-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5a0e060aace4c24dcaf71023bbd7d42674e3b230f7e7b97317baf1e953e5b519"}, - {file = "ruamel.yaml.clib-0.2.12-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:e2f1c3765db32be59d18ab3953f43ab62a761327aafc1594a2a1fbe038b8b8a7"}, - {file = "ruamel.yaml.clib-0.2.12-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:d85252669dc32f98ebcd5d36768f5d4faeaeaa2d655ac0473be490ecdae3c285"}, - {file = "ruamel.yaml.clib-0.2.12-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:e143ada795c341b56de9418c58d028989093ee611aa27ffb9b7f609c00d813ed"}, - {file = "ruamel.yaml.clib-0.2.12-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:2c59aa6170b990d8d2719323e628aaf36f3bfbc1c26279c0eeeb24d05d2d11c7"}, - {file = "ruamel.yaml.clib-0.2.12-cp39-cp39-win32.whl", hash = "sha256:beffaed67936fbbeffd10966a4eb53c402fafd3d6833770516bf7314bc6ffa12"}, - {file = "ruamel.yaml.clib-0.2.12-cp39-cp39-win_amd64.whl", hash = "sha256:040ae85536960525ea62868b642bdb0c2cc6021c9f9d507810c0c604e66f5a7b"}, - {file = "ruamel.yaml.clib-0.2.12.tar.gz", hash = "sha256:6c8fbb13ec503f99a91901ab46e0b07ae7941cd527393187039aec586fdfd36f"}, -] - [[package]] name = "shexjsg" version = "0.8.2" @@ -1973,6 +1881,18 @@ files = [ {file = "six-1.16.0.tar.gz", hash = "sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926"}, ] +[[package]] +name = "snowballstemmer" +version = "3.0.1" +description = "This package provides 32 stemmers for 30 languages generated from Snowball algorithms." +optional = false +python-versions = "!=3.0.*,!=3.1.*,!=3.2.*" +groups = ["main"] +files = [ + {file = "snowballstemmer-3.0.1-py3-none-any.whl", hash = "sha256:6cd7b3897da8d6c9ffb968a6781fa6532dce9c3618a4b127d920dab764a19064"}, + {file = "snowballstemmer-3.0.1.tar.gz", hash = "sha256:6d5eeeec8e9f84d4d56b847692bacf79bc2c8e90c7f80ca4444ff8b6f2e52895"}, +] + [[package]] name = "sortedcontainers" version = "2.4.0" @@ -2023,6 +1943,158 @@ docs = ["sphinx (<5)", "sphinx-rtd-theme"] keepalive = ["keepalive (>=0.5)"] pandas = ["pandas (>=1.3.5)"] +[[package]] +name = "sphinx" +version = "9.1.0" +description = "Python documentation generator" +optional = false +python-versions = ">=3.12" +groups = ["main"] +files = [ + {file = "sphinx-9.1.0-py3-none-any.whl", hash = "sha256:c84fdd4e782504495fe4f2c0b3413d6c2bf388589bb352d439b2a3bb99991978"}, + {file = "sphinx-9.1.0.tar.gz", hash = "sha256:7741722357dd75f8190766926071fed3bdc211c74dd2d7d4df5404da95930ddb"}, +] + +[package.dependencies] +alabaster = ">=0.7.14" +babel = ">=2.13" +colorama = {version = ">=0.4.6", markers = "sys_platform == \"win32\""} +docutils = ">=0.21,<0.23" +imagesize = ">=1.3" +Jinja2 = ">=3.1" +packaging = ">=23.0" +Pygments = ">=2.17" +requests = ">=2.30.0" +roman-numerals = ">=1.0.0" +snowballstemmer = ">=2.2" +sphinxcontrib-applehelp = ">=1.0.7" +sphinxcontrib-devhelp = ">=1.0.6" +sphinxcontrib-htmlhelp = ">=2.0.6" +sphinxcontrib-jsmath = ">=1.0.1" +sphinxcontrib-qthelp = ">=1.0.6" +sphinxcontrib-serializinghtml = ">=1.1.9" + +[[package]] +name = "sphinx-click" +version = "6.2.0" +description = "Sphinx extension that automatically documents click applications" +optional = false +python-versions = ">=3.10" +groups = ["main"] +files = [ + {file = "sphinx_click-6.2.0-py3-none-any.whl", hash = "sha256:1fb1851cb4f2c286d43cbcd57f55db6ef5a8d208bfc3370f19adde232e5803d7"}, + {file = "sphinx_click-6.2.0.tar.gz", hash = "sha256:fc78b4154a4e5159462e36de55b8643747da6cda86b3b52a8bb62289e603776c"}, +] + +[package.dependencies] +click = ">=8.0" +docutils = "*" +sphinx = ">=4.0" + +[package.extras] +docs = ["reno"] +test = ["pytest", "pytest-cov"] + +[[package]] +name = "sphinxcontrib-applehelp" +version = "2.0.0" +description = "sphinxcontrib-applehelp is a Sphinx extension which outputs Apple help books" +optional = false +python-versions = ">=3.9" +groups = ["main"] +files = [ + {file = "sphinxcontrib_applehelp-2.0.0-py3-none-any.whl", hash = "sha256:4cd3f0ec4ac5dd9c17ec65e9ab272c9b867ea77425228e68ecf08d6b28ddbdb5"}, + {file = "sphinxcontrib_applehelp-2.0.0.tar.gz", hash = "sha256:2f29ef331735ce958efa4734873f084941970894c6090408b079c61b2e1c06d1"}, +] + +[package.extras] +lint = ["mypy", "ruff (==0.5.5)", "types-docutils"] +standalone = ["Sphinx (>=5)"] +test = ["pytest"] + +[[package]] +name = "sphinxcontrib-devhelp" +version = "2.0.0" +description = "sphinxcontrib-devhelp is a sphinx extension which outputs Devhelp documents" +optional = false +python-versions = ">=3.9" +groups = ["main"] +files = [ + {file = "sphinxcontrib_devhelp-2.0.0-py3-none-any.whl", hash = "sha256:aefb8b83854e4b0998877524d1029fd3e6879210422ee3780459e28a1f03a8a2"}, + {file = "sphinxcontrib_devhelp-2.0.0.tar.gz", hash = "sha256:411f5d96d445d1d73bb5d52133377b4248ec79db5c793ce7dbe59e074b4dd1ad"}, +] + +[package.extras] +lint = ["mypy", "ruff (==0.5.5)", "types-docutils"] +standalone = ["Sphinx (>=5)"] +test = ["pytest"] + +[[package]] +name = "sphinxcontrib-htmlhelp" +version = "2.1.0" +description = "sphinxcontrib-htmlhelp is a sphinx extension which renders HTML help files" +optional = false +python-versions = ">=3.9" +groups = ["main"] +files = [ + {file = "sphinxcontrib_htmlhelp-2.1.0-py3-none-any.whl", hash = "sha256:166759820b47002d22914d64a075ce08f4c46818e17cfc9470a9786b759b19f8"}, + {file = "sphinxcontrib_htmlhelp-2.1.0.tar.gz", hash = "sha256:c9e2916ace8aad64cc13a0d233ee22317f2b9025b9cf3295249fa985cc7082e9"}, +] + +[package.extras] +lint = ["mypy", "ruff (==0.5.5)", "types-docutils"] +standalone = ["Sphinx (>=5)"] +test = ["html5lib", "pytest"] + +[[package]] +name = "sphinxcontrib-jsmath" +version = "1.0.1" +description = "A sphinx extension which renders display math in HTML via JavaScript" +optional = false +python-versions = ">=3.5" +groups = ["main"] +files = [ + {file = "sphinxcontrib-jsmath-1.0.1.tar.gz", hash = "sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8"}, + {file = "sphinxcontrib_jsmath-1.0.1-py2.py3-none-any.whl", hash = "sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178"}, +] + +[package.extras] +test = ["flake8", "mypy", "pytest"] + +[[package]] +name = "sphinxcontrib-qthelp" +version = "2.0.0" +description = "sphinxcontrib-qthelp is a sphinx extension which outputs QtHelp documents" +optional = false +python-versions = ">=3.9" +groups = ["main"] +files = [ + {file = "sphinxcontrib_qthelp-2.0.0-py3-none-any.whl", hash = "sha256:b18a828cdba941ccd6ee8445dbe72ffa3ef8cbe7505d8cd1fa0d42d3f2d5f3eb"}, + {file = "sphinxcontrib_qthelp-2.0.0.tar.gz", hash = "sha256:4fe7d0ac8fc171045be623aba3e2a8f613f8682731f9153bb2e40ece16b9bbab"}, +] + +[package.extras] +lint = ["mypy", "ruff (==0.5.5)", "types-docutils"] +standalone = ["Sphinx (>=5)"] +test = ["defusedxml (>=0.7.1)", "pytest"] + +[[package]] +name = "sphinxcontrib-serializinghtml" +version = "2.0.0" +description = "sphinxcontrib-serializinghtml is a sphinx extension which outputs \"serialized\" HTML files (json and pickle)" +optional = false +python-versions = ">=3.9" +groups = ["main"] +files = [ + {file = "sphinxcontrib_serializinghtml-2.0.0-py3-none-any.whl", hash = "sha256:6e2cb0eef194e10c27ec0023bfeb25badbbb5868244cf5bc5bdc04e4464bf331"}, + {file = "sphinxcontrib_serializinghtml-2.0.0.tar.gz", hash = "sha256:e9d912827f872c029017a53f0ef2180b327c3f7fd23c87229f7a8e8b70031d4d"}, +] + +[package.extras] +lint = ["mypy", "ruff (==0.5.5)", "types-docutils"] +standalone = ["Sphinx (>=5)"] +test = ["pytest"] + [[package]] name = "sqlalchemy" version = "2.0.36" @@ -2197,7 +2269,7 @@ files = [ ] [package.extras] -brotli = ["brotli (>=1.0.9)", "brotlicffi (>=0.8.0)"] +brotli = ["brotli (>=1.0.9) ; platform_python_implementation == \"CPython\"", "brotlicffi (>=0.8.0) ; platform_python_implementation != \"CPython\""] h2 = ["h2 (>=4,<5)"] socks = ["pysocks (>=1.5.6,!=1.5.7,<2.0)"] zstd = ["zstandard (>=0.18.0)"] @@ -2221,7 +2293,7 @@ platformdirs = ">=3.9.1,<5" [package.extras] docs = ["furo (>=2023.7.26)", "proselint (>=0.13)", "sphinx (>=7.1.2,!=7.3)", "sphinx-argparse (>=0.4)", "sphinxcontrib-towncrier (>=0.2.1a0)", "towncrier (>=23.6)"] -test = ["covdefaults (>=2.3)", "coverage (>=7.2.7)", "coverage-enable-subprocess (>=1)", "flaky (>=3.7)", "packaging (>=23.1)", "pytest (>=7.4)", "pytest-env (>=0.8.2)", "pytest-freezer (>=0.4.8)", "pytest-mock (>=3.11.1)", "pytest-randomly (>=3.12)", "pytest-timeout (>=2.1)", "setuptools (>=68)", "time-machine (>=2.10)"] +test = ["covdefaults (>=2.3)", "coverage (>=7.2.7)", "coverage-enable-subprocess (>=1)", "flaky (>=3.7)", "packaging (>=23.1)", "pytest (>=7.4)", "pytest-env (>=0.8.2)", "pytest-freezer (>=0.4.8) ; platform_python_implementation == \"PyPy\" or platform_python_implementation == \"CPython\" and sys_platform == \"win32\" and python_version >= \"3.13\"", "pytest-mock (>=3.11.1)", "pytest-randomly (>=3.12)", "pytest-timeout (>=2.1)", "setuptools (>=68)", "time-machine (>=2.10) ; platform_python_implementation == \"CPython\""] [[package]] name = "watchdog" @@ -2355,5 +2427,5 @@ files = [ [metadata] lock-version = "2.1" -python-versions = ">=3.12,<4.0" -content-hash = "af27ef8edba6d8bb3c5b19ab7675d604bf134266a66f4ceaa25a9ed8b1391d53" +python-versions = ">=3.12,<3.14" +content-hash = "a2c662c0741520867938e0257bc4ed61da44a1434f1fbece800a35514741e9b1" diff --git a/pyproject.toml b/pyproject.toml index abb64ebb..8a49fb4a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -10,12 +10,12 @@ description = "Margo Specification" authors = [] license = "Open Web Foundation - CLA Copyright Grant 0.9" readme = "README.md" -requires-python = ">=3.12,<4.0" +requires-python = ">=3.12,<3.14" dependencies = [ 'mkdocs>=1.6.1', 'mkdocs-markdownextradata-plugin>=0.2.6', 'mkdocs-material>=9.5.44', - 'linkml>=1.8.5', + 'linkml>=1.11.0', ] [tool.poetry] diff --git a/system-design/assets/svg-pan-zoom-init.js b/system-design/assets/svg-pan-zoom-init.js new file mode 100644 index 00000000..86e5737b --- /dev/null +++ b/system-design/assets/svg-pan-zoom-init.js @@ -0,0 +1,56 @@ +/** + * Initialize svg-pan-zoom on containers marked with data-svg-pan-zoom. + * Each container should have a data-svg-src attribute pointing to the SVG file. + * The SVG is fetched, inlined into the DOM, and then svg-pan-zoom is attached. + */ +document.addEventListener("DOMContentLoaded", function () { + var containers = document.querySelectorAll("[data-svg-pan-zoom]"); + containers.forEach(function (container) { + var svgSrc = container.getAttribute("data-svg-src"); + if (!svgSrc) return; + + fetch(svgSrc) + .then(function (response) { return response.text(); }) + .then(function (svgText) { + // Parse the SVG and insert it into the container + var parser = new DOMParser(); + var svgDoc = parser.parseFromString(svgText, "image/svg+xml"); + var svgEl = svgDoc.documentElement; + + // Make the SVG fill the container + svgEl.setAttribute("width", "100%"); + svgEl.setAttribute("height", "100%"); + svgEl.style.width = "100%"; + svgEl.style.height = "100%"; + + container.appendChild(svgEl); + + // Initialize svg-pan-zoom + var panZoomInstance = svgPanZoom(svgEl, { + zoomEnabled: true, + controlIconsEnabled: true, + fit: true, + center: true, + minZoom: 0.25, + maxZoom: 20, + zoomScaleSensitivity: 0.3 + }); + + // Handle resize + window.addEventListener("resize", function () { + panZoomInstance.resize(); + panZoomInstance.fit(); + panZoomInstance.center(); + }); + }) + .catch(function (err) { + console.error("Failed to load SVG for pan-zoom:", err); + // Fallback: show as a regular image + var img = document.createElement("img"); + img.src = svgSrc; + img.alt = container.getAttribute("data-svg-alt") || "SVG diagram"; + img.style.width = "100%"; + container.appendChild(img); + }); + }); +}); diff --git a/system-design/assets/svg-pan-zoom.min.js b/system-design/assets/svg-pan-zoom.min.js new file mode 100644 index 00000000..4904d12d --- /dev/null +++ b/system-design/assets/svg-pan-zoom.min.js @@ -0,0 +1,3 @@ +// svg-pan-zoom v3.6.1 +// https://github.com/ariutta/svg-pan-zoom +!function s(r,a,l){function u(e,t){if(!a[e]){if(!r[e]){var o="function"==typeof require&&require;if(!t&&o)return o(e,!0);if(h)return h(e,!0);var n=new Error("Cannot find module '"+e+"'");throw n.code="MODULE_NOT_FOUND",n}var i=a[e]={exports:{}};r[e][0].call(i.exports,function(t){return u(r[e][1][t]||t)},i,i.exports,s,r,a,l)}return a[e].exports}for(var h="function"==typeof require&&require,t=0;tthis.options.maxZoom*n.zoom&&(t=this.options.maxZoom*n.zoom/this.getZoom());var i=this.viewport.getCTM(),s=e.matrixTransform(i.inverse()),r=this.svg.createSVGMatrix().translate(s.x,s.y).scale(t).translate(-s.x,-s.y),a=i.multiply(r);a.a!==i.a&&this.viewport.setCTM(a)},i.prototype.zoom=function(t,e){this.zoomAtPoint(t,a.getSvgCenterPoint(this.svg,this.width,this.height),e)},i.prototype.publicZoom=function(t,e){e&&(t=this.computeFromRelativeZoom(t)),this.zoom(t,e)},i.prototype.publicZoomAtPoint=function(t,e,o){if(o&&(t=this.computeFromRelativeZoom(t)),"SVGPoint"!==r.getType(e)){if(!("x"in e&&"y"in e))throw new Error("Given point is invalid");e=a.createSVGPoint(this.svg,e.x,e.y)}this.zoomAtPoint(t,e,o)},i.prototype.getZoom=function(){return this.viewport.getZoom()},i.prototype.getRelativeZoom=function(){return this.viewport.getRelativeZoom()},i.prototype.computeFromRelativeZoom=function(t){return t*this.viewport.getOriginalState().zoom},i.prototype.resetZoom=function(){var t=this.viewport.getOriginalState();this.zoom(t.zoom,!0)},i.prototype.resetPan=function(){this.pan(this.viewport.getOriginalState())},i.prototype.reset=function(){this.resetZoom(),this.resetPan()},i.prototype.handleDblClick=function(t){var e;if((this.options.preventMouseEventsDefault&&(t.preventDefault?t.preventDefault():t.returnValue=!1),this.options.controlIconsEnabled)&&-1<(t.target.getAttribute("class")||"").indexOf("svg-pan-zoom-control"))return!1;e=t.shiftKey?1/(2*(1+this.options.zoomScaleSensitivity)):2*(1+this.options.zoomScaleSensitivity);var o=a.getEventPoint(t,this.svg).matrixTransform(this.svg.getScreenCTM().inverse());this.zoomAtPoint(e,o)},i.prototype.handleMouseDown=function(t,e){this.options.preventMouseEventsDefault&&(t.preventDefault?t.preventDefault():t.returnValue=!1),r.mouseAndTouchNormalize(t,this.svg),this.options.dblClickZoomEnabled&&r.isDblClick(t,e)?this.handleDblClick(t):(this.state="pan",this.firstEventCTM=this.viewport.getCTM(),this.stateOrigin=a.getEventPoint(t,this.svg).matrixTransform(this.firstEventCTM.inverse()))},i.prototype.handleMouseMove=function(t){if(this.options.preventMouseEventsDefault&&(t.preventDefault?t.preventDefault():t.returnValue=!1),"pan"===this.state&&this.options.panEnabled){var e=a.getEventPoint(t,this.svg).matrixTransform(this.firstEventCTM.inverse()),o=this.firstEventCTM.translate(e.x-this.stateOrigin.x,e.y-this.stateOrigin.y);this.viewport.setCTM(o)}},i.prototype.handleMouseUp=function(t){this.options.preventMouseEventsDefault&&(t.preventDefault?t.preventDefault():t.returnValue=!1),"pan"===this.state&&(this.state="none")},i.prototype.fit=function(){var t=this.viewport.getViewBox(),e=Math.min(this.width/t.width,this.height/t.height);this.zoom(e,!0)},i.prototype.contain=function(){var t=this.viewport.getViewBox(),e=Math.max(this.width/t.width,this.height/t.height);this.zoom(e,!0)},i.prototype.center=function(){var t=this.viewport.getViewBox(),e=.5*(this.width-(t.width+2*t.x)*this.getZoom()),o=.5*(this.height-(t.height+2*t.y)*this.getZoom());this.getPublicInstance().pan({x:e,y:o})},i.prototype.updateBBox=function(){this.viewport.simpleViewBoxCache()},i.prototype.pan=function(t){var e=this.viewport.getCTM();e.e=t.x,e.f=t.y,this.viewport.setCTM(e)},i.prototype.panBy=function(t){var e=this.viewport.getCTM();e.e+=t.x,e.f+=t.y,this.viewport.setCTM(e)},i.prototype.getPan=function(){var t=this.viewport.getState();return{x:t.x,y:t.y}},i.prototype.resize=function(){var t=a.getBoundingClientRectNormalized(this.svg);this.width=t.width,this.height=t.height;var e=this.viewport;e.options.width=this.width,e.options.height=this.height,e.processCTM(),this.options.controlIconsEnabled&&(this.getPublicInstance().disableControlIcons(),this.getPublicInstance().enableControlIcons())},i.prototype.destroy=function(){var e=this;for(var t in this.beforeZoom=null,this.onZoom=null,this.beforePan=null,this.onPan=null,(this.onUpdatedCTM=null)!=this.options.customEventsHandler&&this.options.customEventsHandler.destroy({svgElement:this.svg,eventsListenerElement:this.options.eventsListenerElement,instance:this.getPublicInstance()}),this.eventListeners)(this.options.eventsListenerElement||this.svg).removeEventListener(t,this.eventListeners[t],!this.options.preventMouseEventsDefault&&h);this.disableMouseWheelZoom(),this.getPublicInstance().disableControlIcons(),this.reset(),c=c.filter(function(t){return t.svg!==e.svg}),delete this.options,delete this.viewport,delete this.publicInstance,delete this.pi,this.getPublicInstance=function(){return null}},i.prototype.getPublicInstance=function(){var o=this;return this.publicInstance||(this.publicInstance=this.pi={enablePan:function(){return o.options.panEnabled=!0,o.pi},disablePan:function(){return o.options.panEnabled=!1,o.pi},isPanEnabled:function(){return!!o.options.panEnabled},pan:function(t){return o.pan(t),o.pi},panBy:function(t){return o.panBy(t),o.pi},getPan:function(){return o.getPan()},setBeforePan:function(t){return o.options.beforePan=null===t?null:r.proxy(t,o.publicInstance),o.pi},setOnPan:function(t){return o.options.onPan=null===t?null:r.proxy(t,o.publicInstance),o.pi},enableZoom:function(){return o.options.zoomEnabled=!0,o.pi},disableZoom:function(){return o.options.zoomEnabled=!1,o.pi},isZoomEnabled:function(){return!!o.options.zoomEnabled},enableControlIcons:function(){return o.options.controlIconsEnabled||(o.options.controlIconsEnabled=!0,s.enable(o)),o.pi},disableControlIcons:function(){return o.options.controlIconsEnabled&&(o.options.controlIconsEnabled=!1,s.disable(o)),o.pi},isControlIconsEnabled:function(){return!!o.options.controlIconsEnabled},enableDblClickZoom:function(){return o.options.dblClickZoomEnabled=!0,o.pi},disableDblClickZoom:function(){return o.options.dblClickZoomEnabled=!1,o.pi},isDblClickZoomEnabled:function(){return!!o.options.dblClickZoomEnabled},enableMouseWheelZoom:function(){return o.enableMouseWheelZoom(),o.pi},disableMouseWheelZoom:function(){return o.disableMouseWheelZoom(),o.pi},isMouseWheelZoomEnabled:function(){return!!o.options.mouseWheelZoomEnabled},setZoomScaleSensitivity:function(t){return o.options.zoomScaleSensitivity=t,o.pi},setMinZoom:function(t){return o.options.minZoom=t,o.pi},setMaxZoom:function(t){return o.options.maxZoom=t,o.pi},setBeforeZoom:function(t){return o.options.beforeZoom=null===t?null:r.proxy(t,o.publicInstance),o.pi},setOnZoom:function(t){return o.options.onZoom=null===t?null:r.proxy(t,o.publicInstance),o.pi},zoom:function(t){return o.publicZoom(t,!0),o.pi},zoomBy:function(t){return o.publicZoom(t,!1),o.pi},zoomAtPoint:function(t,e){return o.publicZoomAtPoint(t,e,!0),o.pi},zoomAtPointBy:function(t,e){return o.publicZoomAtPoint(t,e,!1),o.pi},zoomIn:function(){return this.zoomBy(1+o.options.zoomScaleSensitivity),o.pi},zoomOut:function(){return this.zoomBy(1/(1+o.options.zoomScaleSensitivity)),o.pi},getZoom:function(){return o.getRelativeZoom()},setOnUpdatedCTM:function(t){return o.options.onUpdatedCTM=null===t?null:r.proxy(t,o.publicInstance),o.pi},resetZoom:function(){return o.resetZoom(),o.pi},resetPan:function(){return o.resetPan(),o.pi},reset:function(){return o.reset(),o.pi},fit:function(){return o.fit(),o.pi},contain:function(){return o.contain(),o.pi},center:function(){return o.center(),o.pi},updateBBox:function(){return o.updateBBox(),o.pi},resize:function(){return o.resize(),o.pi},getSizes:function(){return{width:o.width,height:o.height,realZoom:o.getZoom(),viewBox:o.viewport.getViewBox()}},destroy:function(){return o.destroy(),o.pi}}),this.publicInstance};var c=[];e.exports=function(t,e){var o=r.getSvg(t);if(null===o)return null;for(var n=c.length-1;0<=n;n--)if(c[n].svg===o)return c[n].instance.getPublicInstance();return c.push({svg:o,instance:new i(o,e)}),c[c.length-1].instance.getPublicInstance()}},{"./control-icons":1,"./shadow-viewport":2,"./svg-utilities":5,"./uniwheel":6,"./utilities":7}],5:[function(t,e,o){var l=t("./utilities"),s="unknown";document.documentMode&&(s="ie"),e.exports={svgNS:"http://www.w3.org/2000/svg",xmlNS:"http://www.w3.org/XML/1998/namespace",xmlnsNS:"http://www.w3.org/2000/xmlns/",xlinkNS:"http://www.w3.org/1999/xlink",evNS:"http://www.w3.org/2001/xml-events",getBoundingClientRectNormalized:function(t){if(t.clientWidth&&t.clientHeight)return{width:t.clientWidth,height:t.clientHeight};if(t.getBoundingClientRect())return t.getBoundingClientRect();throw new Error("Cannot get BoundingClientRect for SVG.")},getOrCreateViewport:function(t,e){var o=null;if(!(o=l.isElement(e)?e:t.querySelector(e))){var n=Array.prototype.slice.call(t.childNodes||t.children).filter(function(t){return"defs"!==t.nodeName&&"#text"!==t.nodeName});1===n.length&&"g"===n[0].nodeName&&null===n[0].getAttribute("transform")&&(o=n[0])}if(!o){var i="viewport-"+(new Date).toISOString().replace(/\D/g,"");(o=document.createElementNS(this.svgNS,"g")).setAttribute("id",i);var s=t.childNodes||t.children;if(s&&0 Note: This assumes consistent connection to the WFM, we will address intermittent or extended disconnection scenarios in the future. + +## Route and HTTP Methods + +```https +POST /api/v1/clients/{clientId}/deployments/{deploymentId}/status +``` + +### Route Parameters + +|Parameter | Type | Required? | Description| +|----------|------|-----------|------------| +| {clientId} | string | Y | The unique identifier of the (device) client registered with the WFM during onboarding. | +| {deploymentId} | string | Y | The UUID of the `ApplicationDeployment` YAML being reported. + +### Response Codes + +| Code | Description | +|------|-------------| +| 200 OK | The deployment status was added, or updated, successfully. | +| 400 Bad Request | Missing or invalid content-digest header. Ensure the SHA256 hash of the base64-encoded payload is included. | +| 401 Unauthorized | Signature verification failed. Ensure you are signing with the correct X.509 private key. | +| 403 Forbidden | Client certificate is not trusted or has been revoked. | +| 422 Unprocessable Content | Request body includes a semantic error. | + + +## Request Body + +[DeploymentStatus](./deployment status.md) document describing the status of an application deployment. + diff --git a/system-design/specification/margo-management-interface/desired-state.md b/system-design/specification/margo-management-interface/desired-state.md new file mode 100644 index 00000000..5e3781ef --- /dev/null +++ b/system-design/specification/margo-management-interface/desired-state.md @@ -0,0 +1,534 @@ +# Desired State + +In order for the Workload Fleet Manager (WFM) to manage workloads on an Edge Compute Device, the device's Workload Fleet Management Client must periodically retrieve its desired workload configuration - referred to as the Desired State - from the WFM. + +The Desired State defines *what* workloads (applications) should run on the device and *how* they should be configured. +It is distributed using a lightweight, pull-based HTTP API that allows devices to stay synchronized with the WFM. + +At the center of this process is the State Manifest, a JSON document that lists all workloads assigned to the device. Each workload is represented by an [`ApplicationDeployment`](#applicationdeployment-yaml-definition) YAML - a self-contained object defining configuration, components, and parameters for that workload. + +The manifest includes two complementary ways for the client to obtain the same `ApplicationDeployment` YAMLs: + +- Individual YAMLs - each `ApplicationDeployment` YAML fetched separately using its own URL. +- A bundle archive - a single compressed archive containing multiple `ApplicationDeployment` YAMLs. + +Both references describe the same content. The bundle is simply a packaging optimization. +This design allows the Workload Fleet Management Client to choose the optimal retrieval strategy depending on network conditions or update size. + +| Retrieval Method | Typical Use | Advantages | +| ---------------- | ----------- | ---------- | +| Bundle | Initial onboarding, large updates, high-latency or high-round-trip networks | Single request with minimal overhead | +| Individual YAMLs | Incremental updates, bandwidth-limited or metered links | Only changed workloads are downloaded | + +The Workload Fleet Management Client compares the manifest with its current state and reconciles any differences by deploying, updating, or removing workloads. +For every change in deployment state - including installation, updates, removals, and failures - the client MUST report the corresponding status to the WFM using the [Deployment Status API](../../specification/margo-management-interface/deployment-status.md). + +## Endpoints - State Manifest + +This section defines the API endpoint used by a client to retrieve the State Manifest from the Workload Fleet Manager, representing the complete desired workload configuration assigned to the device. + +### Route and HTTP Methods + +```https +GET /api/v1/clients/{clientId}/deployments +``` + +### Route Parameters + +| Parameter | Type | Required? | Description | +| --------- | ---- | --------- | ----------- | +| `{clientId}` | string | Y | The unique identifier of the (device) client registered with the WFM during onboarding. | + +### Request Headers + +| Header | Description | +| ------ | ------------| +| `If-None-Match` *(optional)* | The `ETag` value from the last successfully retrieved manifest. | +| `Accept` *(optional)* | The client SHOULD request the manifest in the `application/vnd.margo.manifest.v1+json` format. If the `Accept` header lists only unsupported types, the server MUST return `406 Not Acceptable`. If omitted, the server MUST return this format by default. | + +### Response Codes + +| Code | Description | +| ----- | ---------- | +| 200 OK | The response body contains the manifest. The server MUST include a valid `ETag` and `Content-Type: application/vnd.margo.manifest.v1+json`. | +| 304 Not Modified | The response body is empty. Returned if the `If-None-Match` `ETag` matches, i.e. the cached response body has not changed since the last retrieved version. | +| 406 Not Acceptable | The server cannot return a representation matching the `Accept` header. | + +### Example State Manifest Response + +```json +{ + "manifestVersion": 101, + "bundle": { + "mediaType": "application/vnd.margo.bundle.v1+tar+gzip", + "digest": "sha256:b5c6d7e8f9...", + "url": "/api/v1/clients/1234/bundles/sha256:b5c6d7e8f9..." + }, + "deployments": [ + { + "deploymentId": "a3e2f5dc-912e-494f-8395-52cf3769bc06", + "digest": "sha256:a4e01b2c3d...", + "url": "/api/v1/clients/1234/deployments/a3e2f5dc-912e-494f-8395-52cf3769bc06/sha256:a4e01b2c3d..." + } + ] +} +``` + +### Response Body Attributes + +| Field | Type | Required? | Description | +| ----- | ---- | --------- | ----------- | +| `manifestVersion` | number | Y | Monotonically increasing unsigned 64-bit integer in the inclusive range `[1, 2^64-1]`. Each new manifest for the same (device) client MUST have a strictly greater value than the previous. The first manifest for a given client MUST use the value 1. | +| `bundle` | object | Y | Describes an archive containing all referenced `ApplicationDeployment` YAMLs. If there are zero deployments (i.e., the `deployments` array is empty), this field MUST be present with the value `null`. An empty archive MUST NOT be served. | +| `bundle.mediaType` | string | Y | MUST be `application/vnd.margo.bundle.v1+tar+gzip`, which denotes a gzip-compressed tar archive (commonly delivered as a .tar.gz) whose root contains one or more `ApplicationDeployment` YAML files. Servers MUST set the HTTP `Content-Type` to this media type. The archive MUST contain exactly the set of YAML files referenced by `deployments`. | +| `bundle.digest` | string | Y | Digest of the bundle archive. MUST equal the digest computed over the exact sequence of bytes in the [bundle endpoint's](#endpoints-deployment-bundle) HTTP `200 OK` response body. See [Protocol - Digest](#protocol-digest) for further details. | +| `bundle.sizeBytes` | number | N | Optional unsigned 64-bit advisory estimate of the decoded payload length in bytes for the bundle archive. Provided for bandwidth estimation and update planning. MUST NOT be used for integrity verification. | +| `bundle.url` | string | Y | Content-addressable retrieval endpoint for the bundle of the form `/api/v1/clients/{clientId}/bundles/{digest}` where `{digest}` equals `bundle.digest`. | +| `deployments` | array | Y | List of deployment objects describing each workload. | +| `deployments[].deploymentId` | string | Y | The UUID of the deployment. MUST equal [`metadata.annotations.id`](#annotations-attributes) in the `ApplicationDeployment`. | +| `deployments[].digest` | string | Y | Digest of the corresponding `ApplicationDeployment` YAML file. MUST equal the digest computed over the exact sequence of bytes in the [individual deployment endpoint's](#endpoints-individual-deployment-yaml) HTTP `200 OK` response body. See [Protocol - Digest](#protocol-digest) for further details. | +| `deployments[].sizeBytes` | number | N | Optional unsigned 64-bit advisory estimate of the decoded payload length in bytes for the `ApplicationDeployment` YAML. Provided for bandwidth estimation and update planning. MUST NOT be used for integrity verification. | +| `deployments[].url` | string | Y | Content-addressable retrieval endpoint for the `ApplicationDeployment` YAML of the form `/api/v1/clients/{clientId}/deployments/{deploymentId}/{digest}` where `{digest}` equals `deployments[].digest`. | + +> **Note:** The `ETag` returned from this endpoint is a digest of the entire JSON response body (after serialization). It is independent of `bundle.digest` and individual deployment digests (`deployments[].digest`). See [ETag and Caching](#protocol-etag-and-caching) for details. + +### Client Validation Rules + +- The client MUST verify the digest of every fetched artifact before use. +- If any digest validation fails, the client MUST abort the update and retain the previous state. +- The client MUST persist both the last accepted `manifestVersion` and `ETag` to prevent rollback across restarts. + +## Endpoints - Individual Deployment YAML + +This section defines the API endpoint used by a client to retrieve a single `ApplicationDeployment` YAML for incremental synchronization and targeted updates. + +### Route and HTTP Methods + +```https +GET /api/v1/clients/{clientId}/deployments/{deploymentId}/{digest} +``` + +### Route Parameters + +| Parameter | Type | Required? | Description | +| --------- | ---- | --------- | ----------- | +| `{clientId}` | string | Y | The unique identifier of the (device) client registered with the WFM during onboarding. | +| `{deploymentId}` | string | Y | The UUID of the served `ApplicationDeployment` YAML. This MUST equal to `metadata.annotations.id`. | +| `{digest}` | string | Y | Content-addressable digest of the served `ApplicationDeployment` YAML. See [Protocol - Digest](#protocol-digest) for further details. | + +### Response Codes + +| Code | Description | +| ---- | ----------- | +| 200 OK | The response body contains the raw `ApplicationDeployment` YAML (`Content-Type: application/yaml`). Server MUST set `ETag` to the quoted digest and SHOULD return `Cache-Control: public, max-age=31536000, immutable`. | +| 404 Not Found | The referenced digest does not exist on the server. `404 Not Found` indicates only that this specific digest is unavailable. It MUST NOT be interpreted as a deletion signal by a client; deletion of workloads is determined solely by absence from the state manifest. | + +> **Note:** Servers MAY apply HTTP `Content-Encoding` (e.g., gzip, br). The client advertises support via `Accept-Encoding`. Digests and ETags always refer to the decoded representation (i.e., the exact bytes of the response body after decompressing any HTTP `Content-Encoding` such as gzip). Servers SHOULD include `Vary: Accept-Encoding` if compression is used. + +## Endpoints - Deployment Bundle + +This section defines the API endpoint used by a client to retrieve a compressed bundle containing all `ApplicationDeployment` YAMLs for efficient bulk synchronization. + +### Route and HTTP Methods + +```https +GET /api/v1/clients/{clientId}/bundles/{digest} +``` + +### Route Parameters + +| Parameter | Type | Required? | Description | +| --------- | ---- | --------- | ----------- | +| `{clientId}` | string | Y | The unique identifier of the (device) client registered with the WFM during onboarding. | +| `{digest}` | string | Y | Content-addressable digest of the served bundle archive. See [Protocol - Digest](#protocol-digest) for further details. | + +### Response Codes + +| Code | Description | +| ---- | ----------- | +| 200 OK | The bundle was successfully retrieved. The server MUST set `Content-Type` to the manifest-declared `bundle.mediaType`, `ETag` to the quoted digest, and SHOULD return `Cache-Control: public, max-age=31536000, immutable`. | +| 404 Not Found | The referenced digest does not exist on the server. `404 Not Found` indicates only that this specific digest is unavailable. It MUST NOT be interpreted as a deletion signal by a client; deletion of workloads is determined solely by absence from the state manifest. | + +> **Note:** Servers MAY apply `Content-Encoding` (e.g., gzip, br) and SHOULD include `Vary: Accept-Encoding` if they do. + +## Protocol - Digest + +All Desired State artifacts - including the manifest, bundle archives, and individual `ApplicationDeployment` YAMLs - use a canonical digest to ensure content integrity and consistency across client and server implementations. + +A digest has the form `algorithm:encoded`, where both parts are lowercase. Clients and servers MUST NOT add prefixes, suffixes, or whitespace. The required algorithm is `sha256`, and the encoded portion is a 64-character lowercase hexadecimal string. + +- The digest MUST be computed over the exact bytes of the decoded HTTP response body - that is, after decompressing any HTTP `Content-Encoding` (for example, gzip or br). +- No reformatting, re-serialization, or newline normalization is permitted during digest computation. + +The resulting digest value is used consistently across all API representations: + +- In JSON responses: `"digest": "sha256:a4e01b2c3d..."` +- In URLs: `/api/v1/clients/{clientId}/deployments/{deploymentId}/sha256:a4e01b2c3d...` +- In HTTP headers: `ETag: "sha256:a4e01b2c3d..."` + +Clients MUST verify that the digest they compute for every retrieved artifact matches the value provided in the manifest. Any mismatch MUST cause the client to abort the update and preserve the previous state. +If a manifest references a digest using an unsupported algorithm, the client MUST treat the manifest as invalid and abort processing. + +**Example:** + +```text +sha256:a1b2c3d4e5f67890123456789abcdef0123456789abcdef0123456789abcdef +``` + +> **Note:** +> This API defines the digest format `sha256:` (lowercase) for JSON fields, URLs, and ETags. +> This digest represents content identity, ensuring that each artifact (manifest, bundle, or `ApplicationDeployment` YAML) can be uniquely verified and referenced within the Desired State API. +> +> By contrast, the mechanism described in the [API Requirements and Security Details](../../specification/margo-management-interface/api-requirements-and-security.md) document - which follows [RFC 9421](https://datatracker.ietf.org/doc/html/rfc9421) - provides HTTP message-level integrity through signed payloads. +> While both rely on the SHA-256 algorithm, they operate at different layers of the protocol: +> +> - The digest in this API defines immutable content identity for artifacts. +> - The RFC 9421 mechanism ensures end-to-end message integrity and authenticity during HTTP transport. + +## Protocol - ETag and Caching + +All Desired State endpoints implement standard HTTP caching semantics to optimize synchronization between the Workload Fleet Manager (WFM) and the Workload Fleet Management Client. ETags are used to detect content changes and avoid redundant data transfers. Two caching models are defined: one for the mutable State Manifest, and one for immutable, content-addressable resources such as individual deployments and bundles. + +### State Manifest Endpoint + +For the [State Manifest](#endpoints-state-manifest) endpoint, servers use strong ETags as defined in [RFC 9110 § 8.8.3](https://datatracker.ietf.org/doc/html/rfc9110#section-8.8.3). + +- The `ETag` MUST be a strong validator computed as a digest of the exact serialized JSON response body. + The format MUST follow the digest grammar defined in [Protocol – Digest](#protocol-digest): + `":"`, for example: + `"sha256:a4e01b2c3d..."`. +- Servers SHOULD serialize JSON deterministically (for example, per [RFC 8785](https://datatracker.ietf.org/doc/html/rfc8785)) so that logically identical manifests yield identical bytes and therefore identical ETags. +- Manifest responses MUST NOT be marked immutable (e.g., by applying `Cache-Control: immutable` or excessively long `max-age` values). Freshness is controlled through periodic polling using `If-None-Match` revalidation requests. +- When a client presents an `ETag` that matches the current manifest, the server MUST respond with `304 Not Modified`, omitting the response body. + +### Content-Addressable Endpoints + +For [Individual Deployment YAML](#endpoints-individual-deployment-yaml) and [Deployment Bundle](#endpoints-deployment-bundle) endpoints, the resources are immutable and uniquely identified by their digest. + +- The `ETag` MUST equal the quoted digest embedded in the resource’s URL (e.g., `ETag: "sha256:a4e01b2c3d..."`). + This constitutes a strong validator per [RFC 9110 § 8.8.3](https://datatracker.ietf.org/doc/html/rfc9110#section-8.8.3). +- Servers SHOULD include `Cache-Control: public, max-age=31536000, immutable` to enable long-term caching of immutable artifacts. +- If compression is applied, servers SHOULD include `Vary: Accept-Encoding` to ensure cache correctness across encodings. +- Clients MAY send `If-None-Match` when revalidating cached resources; servers MAY return `304 Not Modified` if the artifact has not changed. + +## Deployment Workflow + +This section defines the end-to-end workflow followed by a client to retrieve, validate, and reconcile its Desired State with the Workload Fleet Manager. + +- The client polls the WFM for the latest manifest using the last known `ETag` (if any): + + ```https + GET /api/v1/clients/{clientId}/deployments + ``` + + - If the manifest is unchanged, the WFM responds with `304 Not Modified`. + - If the manifest has changed (`200 OK`), the client: + - Verifies the `manifestVersion` is strictly greater than the stored version. If not, the update MUST be rejected and SHOULD be logged as a security event. The specific requirements for logging security events are not currently defined and will be addressed in a future version of the specification. + - Parses the manifest and decides whether to fetch the bundle or individual deployments. + - Downloads and verifies digests for all referenced `ApplicationDeployment` YAMLs. + - The client reconciles its local workloads: + - Adds or updates workloads that appear in the new manifest. + - Removes workloads no longer present in the new manifest. + - For each change in workload state, the client reports progress and results to the WFM using the [Deployment Status API](../../specification/margo-management-interface/deployment-status.md). + - Once reconciliation succeeds, the client MUST durably persist the new `manifestVersion` and associated `ETag` for use in the next poll cycle. + +### Sequence Diagram + +```mermaid +sequenceDiagram + autonumber + participant Client as Workload Fleet
Management Client + participant WFM as Workload Fleet
Manager + + loop Poll for updates + Client->>+WFM: GET /api/v1/clients/{clientId}/deployments
Header: If-None-Match: "sha256:abc..." + alt State unchanged + WFM-->>-Client: 304 Not Modified + else State updated + WFM-->>Client: 200 OK
Header: ETag: "sha256:xyz..."
Body: Manifest JSON + Client->>Client: Compare manifestVersion and current state + + alt Initial sync or major update + Client->>WFM: GET (bundle URL) + WFM-->>Client: 200 OK
Body: Bundle archive + else Incremental update + Client->>WFM: GET (deployment URLs) + WFM-->>Client: 200 OK
Body: YAMLs + end + + Client->>Client: Verify digests and reconcile workloads + Client->>WFM: POST /api/v1/clients/{clientId}/deployment/{deploymentId}/status
Report progress + end + end +``` + +## ApplicationDeployment YAML Definition + +This section defines the structure and YAML schema of an `ApplicationDeployment`, providing a normative reference for Desired State configuration objects. + +Each workload is represented as an `ApplicationDeployment` YAML file that specifies its components, configuration, and parameters. This resource is delivered via the Desired State API and referenced by `id` in the Deployment Status API. + +```yaml +apiVersion: application.margo.org/v1alpha1 +kind: ApplicationDeployment +metadata: + annotations: + id: + applicationId: + name: + namespace: +spec: + deploymentProfile: + type: + components: + - name: + properties: + parameters: + param: + value: + targets: + - pointer: + components:[] +``` + +| Attribute | Type | Required? | Description | +| --- | --- | --- | --- | +| apiVersion | string | Y | Identifier of the version of the API the object definition follows.| +| kind | string | Y | Must be `ApplicationDeployment`.| +| metadata | Metadata | Y | Metadata element specifying characteristics about the application deployment. See the [Metadata Attributes](#metadata-attributes) section below.| +| spec | Spec | Y | Spec element that defines deployment profile and parameters associated with the application deployment. See the [Spec Attributes](#spec-attributes) section below.| + + +#### Metadata Attributes + +| Attribute | Type | Required? | Description | +| --- | --- | --- | --- | +| annotations | Annotations | Y | Defines the application ID and unique identifier associated to the deployment specification. Needs to be assigned by the Workload Orchestration Software. See the [Annotation Attributes](#annotations-attributes) section below.| +| name | string | Y | When deploying to Kubernetes, the manifests name. The name is chosen by the workload orchestration vendor and is not displayed anywhere.| +| namespace | string | Y | When deploying to Kubernetes, the namespace the manifest is added under. The namespace is chosen by the workload orchestration solution vendor.| + + +#### Annotations Attributes + +| Attribute | Type | Required? | Description | +| --- | --- | --- | --- | +| applicationId | string | Y | An identifier for the application. The id is used to help create unique identifiers where required, such as namespaces. The id must be lower case letters and numbers and MAY contain dashes. Uppercase letters, underscores and periods MUST NOT be used. The id MUST NOT be more than 200 characters. The applicationId MUST match the associated application package Metadata "id" attribute.| +| id | string | Y | The unique identifier UUID of the deployment specification. Needs to be assigned by the Workload Orchestration Software.| + + +#### Spec Attributes + +| Attribute | Type | Required? | Description | +| --- | --- | --- | --- | +| deploymentProfile | DeploymentProfile | Y | Section that defines deployment details including type and components.| +| parameters | map[string][Parameter] | Y | Describes the configured parameters applied via the end-user.| + + +#### DeploymentProfile Attributes + +| Attribute | Type | Required? | Description | +| --- | --- | --- | --- | +| type | string | Y | The type of deployment profile (e.g., helm.v3, compose).| +| components | Component | Y | Components of the application| + + +#### ComposeDeploymentProfile Attributes + +| Attribute | Type | Required? | Description | +| --- | --- | --- | --- | + + +#### Component Attributes + +| Attribute | Type | Required? | Description | +| --- | --- | --- | --- | +| name | string | Y | The name of the component.| +| properties | map[string][string] | Y | Properties associated with the component.| + + +#### ComposeComponent Attributes + +| Attribute | Type | Required? | Description | +| --- | --- | --- | --- | + + +#### Parameter Attributes + +| Attribute | Type | Required? | Description | +| --- | --- | --- | --- | +| name | string | Y | None| +| value | string | Y | The value of the parameter.| +| targets | Target | Y | The targets associated with the parameter.| + + +#### Target Attributes + +| Attribute | Type | Required? | Description | +| --- | --- | --- | --- | +| pointer | string | Y | The pointer indicating the location of the target.| +| components | string | Y | The components associated with the target.| + + +### Example: Cluster Enabled Application Deployment Specification + +```yaml +apiVersion: application.margo.org/v1alpha1 +kind: ApplicationDeployment +metadata: + annotations: + applicationId: com-northstartida-digitron-orchestrator + id: a3e2f5dc-912e-494f-8395-52cf3769bc06 + name: com-northstartida-digitron-orchestrator-deployment + namespace: margo-poc +spec: + deploymentProfile: + type: helm.v3 + components: + - name: database-services + properties: + repository: oci://quay.io/charts/realtime-database-services + revision: 2.3.7 + timeout: 8m30s + wait: "true" + - name: digitron-orchestrator + properties: + repository: oci://northstarida.azurecr.io/charts/northstarida-digitron-orchestrator + revision: 1.0.9 + wait: "true" + parameters: + adminName: + value: Some One + targets: + - pointer: administrator.name + components: + - digitron-orchestrator + adminPrincipalName: + value: someone@somewhere.com + targets: + - pointer: administrator.userPrincipalName + components: + - digitron-orchestrator + cpuLimit: + value: "4" + targets: + - pointer: settings.limits.cpu + components: + - digitron-orchestrator + idpClientId: + value: 123-ABC + targets: + - pointer: idp.clientId + components: + - digitron-orchestrator + idpName: + value: Azure AD + targets: + - pointer: idp.name + components: + - digitron-orchestrator + idpProvider: + value: aad + targets: + - pointer: idp.provider + components: + - digitron-orchestrator + idpUrl: + value: https://123-abc.com + targets: + - pointer: idp.providerUrl + components: + - digitron-orchestrator + - pointer: idp.providerMetadata + components: + - digitron-orchestrator + memoryLimit: + value: "16384" + targets: + - pointer: settings.limits.memory + components: + - digitron-orchestrator + pollFrequency: + value: "120" + targets: + - pointer: settings.pollFrequency + components: + - digitron-orchestrator + - database-services + siteId: + value: SID-123-ABC + targets: + - pointer: settings.siteId + components: + - digitron-orchestrator + - database-services +``` + +### Example: Standalone Device Application Deployment Specification + +```yaml +apiVersion: application.margo.org/v1alpha1 +kind: ApplicationDeployment +metadata: + annotations: + applicationId: com-northstartida-digitron-orchestrator + id: ad9b614e-8912-45f4-a523-372358765def + name: com-northstartida-digitron-orchestrator-deployment + namespace: margo-poc +spec: + deploymentProfile: + type: compose + components: + - name: digitron-orchestrator-docker + properties: + keyLocation: https://northsitarida.com/digitron/docker/public-key.asc + packageLocation: https://northsitarida.com/digitron/docker/digitron-orchestrator.tar.gz + parameters: + adminName: + value: Some One + targets: + - pointer: ENV.ADMIN_NAME + components: + - digitron-orchestrator-docker + adminPrincipalName: + value: someone@somewhere.com + targets: + - pointer: ENV.ADMIN_PRINCIPALNAME + components: + - digitron-orchestrator-docker + idpClientId: + value: 123-ABC + targets: + - pointer: ENV.IDP_CLIENT_ID + components: + - digitron-orchestrator-docker + idpName: + value: Azure AD + targets: + - pointer: ENV.IDP_NAME + components: + - digitron-orchestrator-docker + idpProvider: + value: aad + targets: + - pointer: ENV.IDP_PROVIDER + components: + - digitron-orchestrator-docker + idpUrl: + value: https://123-abc.com + targets: + - pointer: ENV.IDP_URL + components: + - digitron-orchestrator-docker + pollFrequency: + value: "120" + targets: + - pointer: ENV.POLL_FREQUENCY + components: + - digitron-orchestrator-docker + siteId: + value: SID-123-ABC + targets: + - pointer: ENV.SITE_ID + components: + - digitron-orchestrator-docker +``` \ No newline at end of file diff --git a/system-design/specification/margo-management-interface/device-capabilities-api.md b/system-design/specification/margo-management-interface/device-capabilities-api.md new file mode 100644 index 00000000..ff4f52e0 --- /dev/null +++ b/system-design/specification/margo-management-interface/device-capabilities-api.md @@ -0,0 +1,35 @@ +# Device Capabilities API + +Devices MUST provide the Workload Fleet Management service with its capabilities and characteristics. This is done by calling the Device API's `device capabilities` endpoint. Reporting the device capabilities is the final step in the onboarding of the device's client. + +To ensure the WFM is kept up to date, the device's client MUST send updated capabilities information if any changes occur to the information originally provided (i.e., additional memory is added to the device). + +- Requests to this endpoint MUST be authenticated using the HTTP Message Signature method as defined in the [Payload Security](../margo-management-interface/api-requirements-and-security.md#payload-security-method) section. + +## Route and HTTP Methods + +```https +POST /api/v1/clients/{clientId}/capabilities +PUT /api/v1/clients/{clientId}/capabilities +``` + +### Route Parameters + +|Parameter | Type | Required? | Description| +|----------|------|-----------|------------| +| {clientId} | string | Y | The unique identifier of the (device) client registered with the WFM during onboarding. | + +### Response Codes + +| Code | Description | +|------|-------------| +| 201 OK | The device capabilities document was added, or updated, successfully | +| 400 Bad Request | Missing or invalid content-digest header. Ensure the SHA256 hash of the base64-encoded payload is included. | +| 401 Unauthorized | Signature verification failed. Ensure you are signing with the correct X.509 private key. | +| 403 Forbidden | Client certificate is not trusted or has been revoked. | +| 422 Unprocessable Content | Request body includes a semantic error. | + +## Request Body + +[DeviceCapabilitiesManifest](./device-capabilities.md) document describing a device's capabilities and characteristics. +