Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/check-version.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Check for changes in the upstream template. If changes are found, an issue is created
name: Template check.
on:
workflow_dispatch:
workflow_dispatch:
schedule:
- cron: '0 0 1 * *' # Runs at 00:00 UTC on the 1st day of every month

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pr-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ jobs:
uses: actions/upload-artifact@v4
with:
name: integration-test-logs-${{ matrix.os }}
path: tests/integration/resources/geo_boundaries/logs
path: tests/integration/resources/module/logs
if-no-files-found: ignore
retention-days: 30
- name: Fail if integration or linting failed
Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ repos:
# Repo quality
- id: check-added-large-files
args: [--enforce-all]
exclude: ^pixi\.lock$
exclude: ^(?:pixi\.lock|figures/architecture\.drawio\.png)$
- id: forbid-submodules
- id: check-case-conflict
- id: check-illegal-windows-names
Expand Down
22 changes: 13 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,15 +22,15 @@ and the `snakemake` [documentation](https://snakemake.readthedocs.io/en/stable/s
Data processing steps:

<p align="center">
<img src="./figures/rulegraph.png" width="60%">
<img src="./figures/architecture.drawio.png" width="90%">
</p>


1. The configuration file is read to identify the datasets to use as well as the specific countries and regional aggregation (`subtype` in the configuration) to process.
- Country area data: [GADM](https://gadm.org/), [Overture Maps](https://overturemaps.org/) and [NUTS](https://ec.europa.eu/eurostat/web/gisco/geodata/statistical-units/territorial-units-statistics) divisions are supported.
- Exclusive Economic Zone (EEZ) data: [Marine regions](https://www.marineregions.org/).
- Country landmass data: [eurostat NUTS](https://ec.europa.eu/eurostat/web/gisco/geodata/statistical-units/territorial-units-statistics), [GADM](https://gadm.org/), [geoBoundaries](https://www.geoboundaries.org/), and [Overture Maps](https://overturemaps.org/) are supported.
- Exclusive Economic Zone (EEZ) data: [MarineRegions.org](https://www.marineregions.org/).
2. Individual country files are downloaded and harmonised to fit a standardised schema.
- Contested regions are removed at this stage.
- If identified, contested regions are removed at this stage.
- Land is clipped using maritime Exclusive Economic Zones (EEZ).
- Optionally, a Voronoi algorithm is run to separate EEZ areas to fit subnational regions.
3. Each country file is combined and then clipped using its neighbours to minimise overlapping polygons.
Expand All @@ -40,7 +40,7 @@ Data processing steps:
> Use the references at the bottom of this page for more details.

> [!CAUTION]
> To increase the replicability of your workflow, we recommend using NUTS and GADM as sources whenever possible as they are more stable than Overture Maps.
> To increase the replicability of your workflow, we recommend using NUTS and geoBoundaries as sources whenever possible as they have more stable hosting methods than Overture Maps and GADM.

## Configuration
<!-- Please describe how to configure this module below -->
Expand Down Expand Up @@ -86,11 +86,15 @@ snakemake --use-conda --cores 2 # run the workflow!
This module is based on the following research and datasets.
We encourage users to cite both the original source and our workflow.

- eurostat NUTS (various years). Nomenclature of territorial units for statistics (NUTS).
- License: reuse is authorised provided the source is acknowledged. <https://ec.europa.eu/eurostat/statistics-explained/index.php?title=Copyright/licence_policy>
- GADM 4.1. (2018). Global Administrative Areas (GADM).
- License: GADM data is freely available for academic and non-commercial use. <https://gadm.org/license.html>.
- NUTS (various years). Nomenclature of territorial units for statistics (NUTS).
- License: reuse is authorised provided the source is acknowledged. <https://ec.europa.eu/eurostat/statistics-explained/index.php?title=Copyright/licence_policy>
- Overture Maps Divisions database (most recent version). Overture Maps Foundation.
- License: ODbL. See <https://docs.overturemaps.org/attribution/> and <https://opendatacommons.org/licenses/odbl/summary/> for details.
- geoBoundaries (most recent version). William & Mary geoLab.
- License: varies per dataset type (from CC-BY 4.0 compliant to non-commercial use only).
Consult their documentation for details.
<https://www.geoboundaries.org/>.
- Marine Regions World EEZ v12 (2023). Flanders Marine Institute (MarineRegions.org).
- License: CC-By. See <https://www.marineregions.org/disclaimer.php>.
- Overture Maps Divisions database (most recent version). Overture Maps Foundation.
- License: ODbL. See <https://docs.overturemaps.org/attribution/> and <https://opendatacommons.org/licenses/odbl/summary/> for details.
1 change: 0 additions & 1 deletion config/china_example.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
# Example of regional disaggregation of a large country
# China has contested borders, which will be removed by the module
overture_release: "latest"
crs:
projected: "epsg:3857"
geographic: "epsg:4326"
Expand Down
11 changes: 6 additions & 5 deletions config/config.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
# A minimal example of how to configure this module
overture_release: "latest"
voronoi_eez:
enabled: True
sample_spacing: 10000 # sample every 10 km
Expand All @@ -16,11 +15,13 @@ countries:
subtype: "country"
source: "overture"
NLD:
subtype: "0"
source: "gadm"
subtype: 0
source: "geoboundaries"
release_type: "gbOpen"
BEL:
subtype: "country"
source: "overture"
subtype: 1
source: "geoboundaries"
release_type: "gbOpen"
CHE:
subtype: "country"
source: "overture"
Expand Down
156 changes: 99 additions & 57 deletions config/europe_example.yaml
Original file line number Diff line number Diff line change
@@ -1,115 +1,157 @@
# A minimal example of how to configure this module
overture_release: "latest"
voronoi_eez:
enabled: True
sample_spacing: 10000
crs:
projected: "epsg:3035"
geographic: "epsg:4326"
countries:
"ALB":
subtype: "1"
source: "geoboundaries"
release_type: "gbOpen"
"AUT":
subtype: "1"
source: "gadm"
source: "geoboundaries"
release_type: "gbOpen"
"BEL":
subtype: "1"
source: "gadm"
source: "geoboundaries"
release_type: "gbOpen"
"BGR":
subtype: "1"
source: "gadm"
"HRV":
source: "geoboundaries"
release_type: "gbOpen"
"BIH":
subtype: "1"
source: "gadm"
source: "geoboundaries"
release_type: "gbOpen"
"CHE":
subtype: "1"
source: "geoboundaries"
release_type: "gbOpen"
"CYP":
subtype: "1"
source: "gadm"
source: "geoboundaries"
release_type: "gbOpen"
"CZE":
subtype: "1"
source: "gadm"
source: "geoboundaries"
release_type: "gbOpen"
"DEU":
subtype: "1"
source: "geoboundaries"
release_type: "gbOpen"
"DNK":
subtype: "2"
subtype: "1"
source: "nuts"
resolution: 03M
resolution: 01M
year: 2024
"ESP":
subtype: "1"
source: "geoboundaries"
release_type: "gbOpen"
extra_eez: 8364
"EST":
subtype: "1"
source: "gadm"
source: "geoboundaries"
release_type: "gbOpen"
"FIN":
subtype: "1"
source: "gadm"
source: "geoboundaries"
release_type: "gbOpen"
"FRA":
subtype: "1"
source: "gadm"
"DEU":
subtype: "1"
source: "gadm"
source: "geoboundaries"
release_type: "gbOpen"
"GBR":
subtype: "region"
source: "overture"
"GGY":
subtype: "0"
source: "geoboundaries"
release_type: "gbOpen"
"GRC":
subtype: "1"
source: "gadm"
source: "geoboundaries"
release_type: "gbOpen"
"HRV":
subtype: "1"
source: "geoboundaries"
release_type: "gbOpen"
"HUN":
subtype: "1"
source: "gadm"
source: "geoboundaries"
release_type: "gbOpen"
"IMN":
subtype: "0"
source: "geoboundaries"
release_type: "gbOpen"
"IRL":
subtype: "region"
source: "overture"
"ITA":
subtype: "1"
source: "gadm"
"LVA":
subtype: "1"
source: "gadm"
source: "geoboundaries"
release_type: "gbOpen"
"LTU":
subtype: "1"
source: "gadm"
source: "geoboundaries"
release_type: "gbOpen"
"LUX":
subtype: "1"
source: "gadm"
source: "geoboundaries"
release_type: "gbOpen"
"LVA":
subtype: "1"
source: "geoboundaries"
release_type: "gbOpen"
"MKD":
subtype: "1"
source: "geoboundaries"
release_type: "gbOpen"
"MNE":
subtype: "1"
source: "geoboundaries"
release_type: "gbOpen"
"NLD":
subtype: "region"
source: "overture"
"NOR":
subtype: "1"
source: "geoboundaries"
release_type: "gbOpen"
"POL":
subtype: "1"
source: "gadm"
source: "geoboundaries"
release_type: "gbOpen"
"PRT":
subtype: "2"
subtype: "1"
source: "nuts"
resolution: 03M
resolution: 01M
year: 2024
extra_eez: [8361, 8363]
"ROU":
subtype: "1"
source: "gadm"
source: "geoboundaries"
release_type: "gbOpen"
"SRB":
subtype: "1"
source: "geoboundaries"
release_type: "gbOpen"
"SVK":
subtype: "1"
source: "gadm"
source: "geoboundaries"
release_type: "gbOpen"
"SVN":
subtype: "1"
source: "gadm"
"ESP":
subtype: "1"
source: "gadm"
source: "geoboundaries"
release_type: "gbOpen"
"SWE":
subtype: "1"
source: "gadm"
"GBR":
subtype: "region"
source: "overture"
"ALB":
subtype: "1"
source: "gadm"
"BIH":
subtype: "1"
source: "gadm"
"MKD":
subtype: "1"
source: "gadm"
"MNE":
subtype: "1"
source: "gadm"
"NOR":
subtype: "1"
source: "gadm"
"SRB":
subtype: "1"
source: "gadm"
"CHE":
source: "geoboundaries"
release_type: "gbOpen"
"XKX":
subtype: "1"
source: "gadm"
source: "geoboundaries"
release_type: "gbOpen"
1 change: 0 additions & 1 deletion config/usa_example.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
# Example of regional disaggregation of a large country
# USA has multiple marine zones, which can be appended as extras
overture_release: "latest"
voronoi_eez:
enabled: True
sample_spacing: 10000 # sample every 10 km
Expand Down
Binary file added figures/architecture.drawio.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified figures/rulegraph.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified figures/shapes.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading