implementation of absolute path for inputs and outputs in Docker + re…

sen2vm-notebook/README_Notebooks.md

@@ -16,6 +16,15 @@
 # Sen2VM Notebook Processing Workflow
 
 This repository provides a complete workflow to run **Sen2VM** inside Docker and generate orthorectified and mosaicked outputs from Sentinel-2 L1B data.  
+
+Two notebooks are provided:
+
+* **inputs-download-notebook.ipynb**  
+  Downloads all required input data (GEOID and GIPP) from the Git repository.
+
+* **notebook.ipynb**  
+  Runs Sen2VM using the downloaded inputs, along with the IERS file and L1B product, to produce orthorectified and mosaicked images.
+
 The project is designed and tested on **Linux**. It may not work reliably on **Windows**.
 Actually, this notebook can generate an inverse grid but can not use it to apply the orthorectification. 
 
@@ -32,25 +41,47 @@ Actually, this notebook can generate an inverse grid but can not use it to apply
 * Required data:
 
   * **Sentinel-2 L1B product** : /DATASTRIP and /GRANULE.  
-  * **DEM files** placed in the appropriate directory
+  * **DEM files**
 
 ## Mandatory Directory Structure
 
+Only a `WORKDIR` folder is required. All outputs, as well as intermediate files generated during execution, will be stored there.
+
+## Inputs
+
+Several inputs are needed :
+* L1B S2 product under EUP.SAFE format :
+```bash
+ <L1B_product>/
+    ├── DATASTRIP  # Required
+    ├── GRANULE   # Required
+    ├── S2*OPER_MTD_SAFL1B_PDMC*.xml #Required
+    └──  ...
+```
+* Digital Elevation Model (DEM)
+
+* IERS (prediction of Earth orientation, IERS can be donwloaded using **notebook.ipynb** (cell number 2))
+
+* GIPP (GIPP can be downloaded from the GIT repositery using **inputs-download-notebook.ipynb**)
+
+* GEOID (GEOID can also be downloaded from the GIT repositery using **inputs-download-notebook.ipynb** )
+
+Thanks to a recent update (#issue 56), now the differents inputs path are absolute path.
+
+## Directory Structure after execution
 ```bash
 WORKDIR
 │
-├── DATA/
-│ ├── DEM/ # Put your DEM files here
-| ├── GEOID/ # Put your GEOID files here (Optional)
-│ └── <L1B_product>/ # Place the full L1B product here
-|   ├── DATASTRIP  # Required
-|   ├── GRANULE   # Required
-|   └──  ...
-└── ...
+├── bulletin*.txt # IERS prediction of earth exploration (downloaded by the cell n°3)  
+│
+├── output/ # output after orthorectification and mosaic (.tif)
+│ ├── GDAL_OUTPUT_ORTHO
+│ └── GDAL_OUTPUT_MOSAIC
+│
+├── src/ # contains a .sh used to run in the gdal docker generated by the notebook
+└── UserConf # contains .txt config files generated by the notebook
 ```
 
-If the GEOID folder is empty, the notebopok will automaticaly use the geoid provided by sen2vm-core
-
 ## Python Environment Setup
 
 In the notebook directory create your venv :
@@ -63,37 +94,58 @@ pip install -r requirements.txt
 
 Select the virtual environment kernel in your Jupyter session.
 
-## Notebook Configuration
+## Notebooks Configuration
+
+In **inputs-download-notebook.ipynb** :
+In the first cell of the notebook :
+
+1. Set the paths to:
 
-In the first cell of the notebook:
+   * The directory where you want the GIPP files to be downloaded  
+   * The directory where you want the GEOID files to be downloaded  
 
-1. Set the path to:
+In **notebook.ipynb** :
+In the first cell of the notebook :
+
+1. Set the absolute paths to:
 
    * The working directory 
    * The L1B product
+   * The GIPP directory
+   * The GEOID directory
+   * The DEM directory
    * The output directory
-
+   * The IERS file (if you do not have the IERS file, put `""`, and one will be automatically  ed to the working directory)
+
 2. Adjust configuration parameters for:
 
    * Sen2VM
    * Orthorectification settings
+
+3. Specify whether the docker images should be removed 
 
 ## Processing Steps
 
-Execute the notebook cell by cell in the following order:
+If you do not have your own GIPP and GEOID files, or if they are not already downloaded,
+execute the **inputs-download-notebook.ipynb** cell by cell in the following order :
 
-1. Variable definitions  
+1. Path definitions
 2. Clone `sen2vm-gipp`, and manage GIPP assets  
-3. Automatic download of the IERS bulletin 
-4. Generation of `config.json` in: `/WORKDIR/UserConf`
-5. Generation of `params.json` in: `/WORKDIR/UserConf`
-6. Execution of sen2vm inside Docker  
-7. Generation of a `.sh` script, then execution inside a second Docker container running the latest GDAL:
+3. Copy `Geoid` folder from `"*/sen2vm-core/src/test/resources/DEM_GEOID"`
+
+Then, execute the **notebook.ipynb** cell by cell :
+
+1. Variable and path definitions 
+2. Automatic download of the IERS bulletin if none is provided by the user
+3. Generation of `config.json` in: `/WORKDIR/UserConf`
+4. Generation of `params.json` in: `/WORKDIR/UserConf`
+5. Execution of sen2vm inside Docker  
+6. Generation of a `.sh` script, then execution inside a second Docker container running the latest GDAL:
 
     * Orthorectification by band  
     * Mosaicking  
 
-Docker images are automatically cleaned up after each execution.
+Docker images can be removed or kept depending on the value of `REMOVE_DOCKER_IMAGE` in step 1. 
 
 ## Execution
 

sen2vm-notebook/src/inputs-download-notebook.ipynb

@@ -0,0 +1,152 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "e71d30a3-b1dc-48f9-b7a0-1ee82f8cc775",
+   "metadata": {},
+   "source": [
+    "# === USER CONFIGURATION ==="
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "0d377684-c4b1-4faa-a16f-02d7c8a58105",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#In this cell, you may indicate where you want the inputs to be donwloaded\n",
+    "\n",
+    "# Path where to put the GIPP directory\n",
+    "PATH_TO_GIPP = \"PATH/WHERE/TO/PUT/GIPP\"\n",
+    "\n",
+    "# Path where to put the GEOID directory\n",
+    "PATH_TO_GEOID = \"PATH/WHERE/TO/PUT/GEOID\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "93bc15d2-9c05-4cc2-82e5-3533e1bbcfd0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# GIPP database download"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "52f77292-b1d3-42cd-817f-24b9bea7e520",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# === GIPPs ===\n",
+    "# This step is optional if the Database was already downloaded or if you want to use your own GIPP,\n",
+    "# It will donwload from GITHUB the directory sen2vm-gipp-database at PATH_GIPP\n",
+    "\n",
+    "import os\n",
+    "import shutil\n",
+    "import tarfile\n",
+    "import re\n",
+    "from datetime import datetime\n",
+    "import sys\n",
+    "\n",
+    "if not os.path.exists(PATH_TO_GIPP):\n",
+    "    print(\"The folder where you want to download the GIPP is missing\")\n",
+    "    sys.exit()          \n",
+    "\n",
+    "# =====================================================================\n",
+    "# 1) CLONE sen2vm-gipp-database\n",
+    "# =====================================================================\n",
+    "\n",
+    "gipp_repo_name = \"sen2vm-gipp-database\"\n",
+    "PATH_GIPP = os.path.join(PATH_TO_GIPP, gipp_repo_name)\n",
+    "\n",
+    "# Delete repository if exists\n",
+    "if os.path.exists(PATH_GIPP):\n",
+    "    print(f\"Removing existing repository: {PATH_GIPP}\")\n",
+    "    shutil.rmtree(PATH_GIPP)\n",
+    "\n",
+    "# Clone repository fresh\n",
+    "print(\"Cloning sen2vm-gipp-database...\")\n",
+    "!git clone https://github.com/sen2vm/sen2vm-gipp-database.git {PATH_GIPP}\n",
+    "print(\"Clone complete.\\n\")\n",
+    "\n",
+    "print(\"GIPP processing finished successfully.\")\n",
+    "print(\"GIPP donwloaded in :\", PATH_GIPP)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0e8b2d6d-5e67-4145-aa04-42d99671d36b",
+   "metadata": {},
+   "source": [
+    "# GEOID folder copy from github"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c781bbdc-3aa7-48d1-9284-5f5c507ed265",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# === GEOID ===\n",
+    "# This step is optional if the GEOID are already copied, or if you want to you use your own GEOID,\n",
+    "# It will copy the folder GEOID from \"*/sen2vm-core/src/test/resources/DEM_GEOID\" to PATH_TO_GEOID\n",
+    "\n",
+    "import os\n",
+    "import sys\n",
+    "import shutil\n",
+    "\n",
+    "if not os.path.exists(PATH_TO_GEOID):\n",
+    "    print(\"The folder where you want to download the GEOID is missing\")\n",
+    "    sys.exit()   \n",
+    "\n",
+    "GEOID_DIR = os.path.join(PATH_TO_GEOID, \"GEOID\")\n",
+    "\n",
+    "if os.path.exists(GEOID_DIR):\n",
+    "    shutil.rmtree(GEOID_DIR)\n",
+    "\n",
+    "os.makedirs(GEOID_DIR)\n",
+    "\n",
+    "# Notebook location (NOT relative to CWD)\n",
+    "notebook_dir = os.getcwd()\n",
+    "\n",
+    "# Relative path to DEM_GEOID from notebook\n",
+    "internal_geoid_dir = os.path.abspath(os.path.join(\n",
+    "    notebook_dir,\n",
+    "    \"..\", \"..\", \"src\", \"test\", \"resources\", \"DEM_GEOID\"\n",
+    "))\n",
+    "\n",
+    "for f in os.listdir(internal_geoid_dir):\n",
+    "    src = os.path.join(internal_geoid_dir, f)\n",
+    "    dst = os.path.join(GEOID_DIR, f)\n",
+    "    shutil.copy(src, dst)\n",
+    "print(\"Done copying\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.13.11"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}