Refactor REST API client and update imports to new helpers #100
Open
jlegrand62 wants to merge 36 commits into
Open
Refactor REST API client and update imports to new helpers #100jlegrand62 wants to merge 36 commits into
jlegrand62 wants to merge 36 commits into
Conversation
…REST API registration - Switch imports from `plantdb.client` to `plantdb.commons` in client modules (`rest_api.py`, `plantdb_client.py`) and tests - Relocate `api_endpoints` to `plantdb.commons`, add missing `home` endpoint and update all usage examples - Rewrite server CLI (`fsdb_rest_api.py`) to import endpoints directly and replace verbose `api.add_resource` calls with a concise mapping helper - Reformat function signatures, logging calls, and type hints for consistency - Update docstrings and examples to reflect the new import path and endpoint structure
…pers - Introduce comprehensive path constants (`HOME`, `HEALTH`, `REFRESH`, `REGISTER`, `LOGIN`, `LOGOUT`, `TOKEN_REFRESH`, `TOKEN_VALIDATION`, `CREATE_API_TOKEN`, `SCANS`, `SCANS_INFO`, `SCAN`, `SCAN_MD`, `SCAN_FILESETS`, `FILESET`, `FILESET_MD`, `FILESET_FILES`, `FILE`, `FILE_MD`, `IMAGE`, `POINTCLOUD`, `MESH`, `SKELETON`, `ARCHIVE`, `FILE_PATH`) in **`src/commons/plantdb/commons/api_endpoints.py`**. - Replace all hard‑coded endpoint strings with the new constants and use `.format(...)` for dynamic segments. - Update endpoint functions: - **`pointcloud`** with size, coords, and type validation. - **`mesh`** with size and coords handling. - **`skeleton`** returning the skeleton path. - **`file_path`** endpoint to use the `FILE_PATH` constant. - Adjust docstrings and examples to reflect the new constant‑based paths and updated signatures.
- Replace all JSON error payloads from `{'message': ...}` to `{'error': ...}` across asset, image, pointcloud, mesh, and zip handling endpoints.
- Add `resource_file` helper to retrieve a `File` object with unified error handling and consistent `error` key.
- Refactor `PointCloud`, `Mesh`, `CurveSkeleton`, and `AnglesAndInternodes` resources to use `resource_file`, removing redundant `fileset_id`/`file_id` parameters and related sanitizers.
- Extend PointCloud endpoint to support a `type` query parameter (`default` or `gt`) and simplify request signature.
- Remove `PointCloudGroundTruth` resource as it is now accessible from `PointCloud`.
- Update logging and success responses to align with the new error key convention.
…ource registration - Replace bulk import of `plantdb.commons.api_endpoints` with explicit constant imports (e.g., `ARCHIVE`, `FILE`, `SCAN_MD`, `SCAN_FILESETS`, etc.) - Reorder typing imports for clarity (`Optional` and `Union` on separate lines) - Simplify `_register_resources` by removing the custom `_add` helper, using a flat `RESOURCE_MAP` with endpoint constants formatted via ``.format`` and registering each with `api.add_resource(..., resource_class_args=(db, logger))` - Update function signatures in `_configure_api`, `_setup_test_database`, and `rest_api` for consistent indentation and type hint style - Adjust import ordering and formatting throughout the file for readability
- Define constructor `def __init__(self, db, logger=None)` with detailed docstring - Store `self.db: FSDB = db` and `self.logger: logging.Logger` (fallback to `get_logger`) - Enables dependency injection of database instance and logger for the home endpoint.
- Replace all JSON error payloads from `{'message': ...}` to `{'error': ...}` across authentication endpoints (register, login, logout, token validation, refresh, and API token creation).
- Consolidate response construction using a single `response` variable with proper status codes and return it at the end of each method.
- Add success response for user registration (`User <username> successfully created`).
- Refactor logout to always return a response variable and use the `error` key for failure cases.
- Simplify token validation flow: early return on missing token and unified error handling.
- Update login to build the successful response in the `else` block and use `error` for failure cases.
- Adjust refresh token and API token creation endpoints to follow the new error-key convention and to return responses consistently.
- Remove unused imports (`requests`, `jsonify`, `make_response`).
- Change health‑check error response key from `message` to `error` and move exception handling before the success `else` block.
- Update `Refresh` resource:
- Return `{'error': ...}` on failure instead of `{'message': ...}`.
- Add explicit `else` block for successful reload response.
- Update full‑database reload endpoint:
- Use `{'error': ...}` for exception case.
- Add `else` block for successful reload message.
- Apply consistent try/except/else pattern across the modified endpoints.
- Move `resource_file`, `is_within_directory`, and `is_directory_in_archive` to new **`plantdb/server/api/utils.py`** - Clean up unused imports in `assets.py` (e.g., `os`, `pathlib`, `BytesIO`, `ZipFile`, `numpy`, `requests`) - Import `resource_file` from utils in `assets.py` - Delete the original helper implementations from `assets.py` to avoid duplication - Keep existing functionality unchanged while improving module organization.
- Replace all `{'message': ...}` responses with `{'error': ...}` in **scan**, **fileset**, **file**, and **assets** endpoints
- Update related docstrings and success response structures to reflect the new error key
- Adjust metadata, metadata‑update, and file‑upload error handling accordingly
- Minor refactor: improve logger initialization formatting and simplify `wants_base64` flag parsing in assets API.
…umentation
- Rename `FILE_PATH` constant to use `/files/{scan_id}/{file_path}`
- Minor comment formatting tweak for lower‑case JSON‑style boolean handling
- Import `ARCHIVE`, `FILE`, `HEALTH`, `IMAGE`, `LOGIN`, `SCAN`, and `SCANS` from `plantdb.commons.api_endpoints` - Replace hard‑coded URL strings with the corresponding formatted constants in all request calls - Adjust request constructions for health check, login, scans retrieval, scan metadata, file serving, image thumbnails, and archive download - Add a temporary print of the retrieved file name (debug aid)
- Updated endpoint hierarchy comment block to reflect new `/auth` and `/assets` routes - Prefixed authentication paths (`/register`, `/login`, `/logout`, token routes) with `/auth` and adjusted corresponding constants (`REGISTER`, `LOGIN`, `LOGOUT`, `TOKEN_REFRESH`, `TOKEN_VALIDATION`, `CREATE_API_TOKEN`) - Moved all static asset routes under `/assets` and updated related constants (`IMAGE`, `POINTCLOUD`, `MESH`, `SEQUENCE`, `SKELETON`, `ARCHIVE`, `FILE_PATH`) - Revised URL building functions to use the new constants, including updated example docstrings for `register`, `login`, `logout`, `token_refresh`, `token_validation`, `create_api_token`, `scans_info`, `scan`, `scan_metadata`, `scan_filesets_list`, `fileset`, `fileset_metadata`, `fileset_files_list`, `file_metadata`, `image`, `sequence`, `pointcloud`, `mesh`, `skeleton`, `archive` - Modified `file_path` helper signature to accept only `file_path` (removed `scan_id` argument) and formatted URL with the new `FILE_PATH` pattern - Adjusted all related import paths and comments to match the new namespace structure.
- Import `api_endpoints` in `scan.py` and replace hard‑coded paths with `api_endpoints.archive`, `api_endpoints.file_path`, and `api_endpoints.image` - Update thumbnail URI construction to use `api_endpoints.image(..., size="thumb")` - Refactor task file URI creation in both `get_scan_info` and `get_scan_data` to use `api_endpoints.file_path` instead of the removed `get_file_uri` - Remove obsolete `get_file_uri` and `get_image_uri` functions from `utils.py` and update docstrings to reference `api_endpoints.image` - Adjust imports in `utils.py` to include `api_endpoints` and drop unused `urllib.parse` usage - Clean up related comment sections and ensure all new calls use the central endpoint helpers.
…endpoint - Add `no_auth=True` to `test_database` calls in `webcache.py`, `services/scan.py`, and `core/utils.py` examples - Change example API URLs from `/scan/...` to `/scans/...` in `api/scan.py` docstrings - Adjust related example paths for image, pointcloud, mesh, and metadata requests accordingly
- Import `api_endpoints` in all server test modules and replace hard‑coded URL strings with the appropriate helper functions (e.g. `api_endpoints.health()`, `api_endpoints.login()`, `api_endpoints.scans()`, `api_endpoints.file(...)`, etc.). - Update request constructions for health checks, authentication, scans, filesets, files, images, pointclouds, meshes, skeletons, sequences, archives, and token operations to use the new helpers. - Remove now‑unused imports such as `tempfile`, `os`, and redundant `time` imports. - Adjust docstrings and comments to reflect the use of `api_endpoints`. - Clean up import ordering and eliminate dead code in the test suite.
- Extend `scan_metadata`, `fileset_metadata`, and `file_metadata` signatures to accept `key: str | None = None`. - Document the new `key` argument in each function’s docstring. - Build a query string with `parse.urlencode` when `key` is provided and append it to the returned URL. - Update example calls in docstrings for `fileset`, `fileset_metadata`, `file`, `file_metadata`, and `image` to use consistent spacing. - Minor cleanup of example strings and formatting in `api_endpoints.py`.
…/api_endpoints.py` - Adjust spacing and alignment in the endpoint hierarchy comment block for `health`, `refresh`, scan retrieval, creation, and fileset listings to improve readability.
- Parse `filterQuery` from JSON request body (`request.get_json`) instead of URL args in `src/server/plantdb/server/api/scan.py`
- Retrieve `fuzzy` flag early and use it consistently across endpoints
- Rename local result variable to `scans_list` for clarity
- Wrap database queries in `try/except` blocks, returning `{'error': ...}` with HTTP 500 on failure
- Apply identical query parsing and error handling to `ScansTable` and the fileset‑listing endpoint, passing `query` and `fuzzy` as named arguments to `list_filesets`
…y bodies
- Replace hard‑coded URL strings with `api_endpoints` functions (`scans()`, `scan()`, `scan_metadata()`, `scan_filesets_list()`, etc.) across all docstring examples in `src/server/plantdb/server/api/scan.py`
- Show how to start and stop a `TestRestApiServer` in examples, providing a complete test flow
- Switch filter queries from URL‑encoded strings to JSON payloads (`json={"filterQuery": query}`) for fuzzy searches
- Use `api_endpoints` to build URLs for metadata, login, logout, and file operations
- Add `self.db.reload(scan_id)` after creating a scan to load the newly created entry
- Provide default empty dict when calling `scan.get_metadata(key, default={})` to avoid `None` returns
- Update example code to demonstrate creating scans with and without metadata, retrieving metadata, and cleaning up the test server
- Minor cleanup of imports and example comments for consistency.
…and test server - Replace hard‑coded URL strings with `api_endpoints` functions (`login()`, `fileset()`, `fileset_metadata()`, `file_metadata()`, `fileset_files_list()`) throughout the examples in `src/server/plantdb/server/api/fileset.py` - Demonstrate starting and stopping a `TestRestApiServer` (`server = TestRestApiServer(test=True); server.start(); …; server.stop()`) for a complete test flow - Show login via the endpoint and populate the `Authorization` header with the obtained `token` - Use `response.ok` and concise JSON access (e.g. `response.json()['metadata']['description']`) instead of verbose status checks - Add examples for retrieving specific metadata keys, updating file metadata, and fetching the list of files in a fileset - Include logout calls (`api_endpoints.logout()`) and cleanup steps in each example block
… test server - Import and demonstrate `TestRestApiServer` for starting/stopping a test REST API server in all examples - Replace hard‑coded URL strings with `api_endpoints` functions (`file()`, `file_metadata()`, `login()`, `logout()`, etc.) for building request URLs - Show login flow using `api_endpoints.login()` and populate `Authorization` header with the retrieved token - Use `response.ok` for success checks instead of raw status code comparisons - Add logout calls (`api_endpoints.logout()`) and server shutdown (`server.stop()`) to each example block - Demonstrate fetching specific metadata keys via `api_endpoints.file_metadata(..., key="...")` - Update metadata update example to include token handling and response inspection - Clean up imports and example comments for consistency throughout `src/server/plantdb/server/api/file.py`
… deprecate `DatasetFile` resource - Replace hard‑coded URL strings in examples across **assets**, **image**, **pointcloud**, **mesh**, **skeleton**, **sequence**, and **archive** endpoints with calls to `api_endpoints` (e.g. `api_endpoints.file_path`, `api_endpoints.image`, `api_endpoints.pointcloud`, etc.). - Demonstrate starting and stopping a `TestRestApiServer` in every example (`server = TestRestApiServer(test=True); server.start(); …; server.stop()`). - Add `server.stop()` calls at the end of example blocks for clean shutdown. - Update request constructions to use the generated URIs (`"http://127.0.0.1:5000" + file_uri`). - Refactor flag parsing in the point‑cloud resource: - Simplify `coords` flag handling with a set of valid true values. - Clarify `type` flag logic and default task name selection. - Remove the `DatasetFile` class, emitting a deprecation warning and noting its future replacement by the Archive resource.
…piServer` - Import and demonstrate `TestRestApiServer` (`server = TestRestApiServer(test=True); server.start(); …; server.stop()`) in every docstring example. - Replace hard‑coded URLs with calls to `api_endpoints` (e.g. `api_endpoints.login()`, `api_endpoints.token_validation()`, `api_endpoints.logout()`). - Add `server.stop()` calls at the end of each example block for clean shutdown. - Use `response.ok` for success checks instead of raw status codes in the login example. - Remove the explicit “no token supplied” check in the token‑validation endpoint and rely on service‑layer handling. - Update imports to include `api_endpoints` where required.
- Import `api_endpoints` and all endpoint constants in `src/server/plantdb/server/api/base.py` - Replace hard‑coded URL strings in docstrings with calls to `api_endpoints` (e.g. `api_endpoints.health()`, `api_endpoints.refresh()`) - Update the generated API information dictionary to use f‑strings with the imported endpoint constants - Extend examples to demonstrate starting and stopping a `TestRestApiServer`, including health‑check and refresh calls - Add inline examples for rate‑limit handling and server shutdown using the new helpers - Minor formatting cleanup and removal of redundant blank lines.
- Change `self.assertEqual(r.status_code, 400)` to `self.assertEqual(r.status_code, 401)` in `src/server/tests/test_auth_api.py`
- Remove unused `splitport` import and replace with manual host parsing in `login_url` port handling - Update authentication endpoints to include `/auth` prefix (`login_url`, `logout_url`, `register_url`, `token_validation_url`, `token_refresh_url`, `api_token_url`) - Change scan-related URLs to use `/scans` and asset paths (`scan_url`, `scan_preview_image_url`, `scan_image_url`, `archive_url`) - Consolidate image and asset URLs under `/assets` prefix - Simplify `scan_file_url` signature to drop `scan_id` and accept only `file_path`; adjust all callers (`scan_config_url`, `scan_reconstruction_url`, `get_toml_file`, etc.) to build combined paths - Pass `**kwargs` to `api_endpoints` calls where needed (`token_validation_url`, `token_refresh_url`, `api_token_url`) - Update `list_task_images_uri` comprehension formatting - Minor formatting adjustments in `request_new_token` and related request construction - Adjust related helper functions (`get_toml_file`, `get_scan_config`, `get_reconstruction_config`, `get_angles_and_internodes_data`) to use new URL building logic and simplified signatures.
- Import `os`, `MagicMock`, and `patch` to enable environment and request mocking in `test_rest_api.py` - Add `setUp` helper with `host`, `port`, and `prefix` for reuse across tests - New unit tests for URL builders: - `origin_url` basic and SSL‑override behavior - `plantdb_url` with and without SSL/prefix - Authentication endpoints (`login_url`, `logout_url`, `register_url`, `token_validation_url`, `token_refresh_url`, `api_token_url`) - Scan‑related URLs using `/scans` and asset paths (`scan_url`, `scan_image_url`, `archive_url`) - Asset helpers (`scan_file_url`, `scan_config_url`, `scan_reconstruction_url`) - Added tests for request helper functions: - `request_login`, `request_check_username`, `request_logout`, `request_token_validation`, `request_token_refresh`, `request_api_token`, `request_new_user` - Scan list/info helpers (`request_scan_names_list`, `request_scans_info`, `request_scan_data`) - Verified `make_api_request` handling for `GET`, `POST`, and invalid methods - Test that centralized environment variables (`PLANTDB_HOST`, `PLANTDB_PORT`, `PLANTDB_PREFIX`) default correctly - Removed outdated `sanitize_name` tests as they are no longer part of the client API.
- Add `src/client/plantdb/client/rest_api/urls.py` - Add `src/client/plantdb/client/rest_api/requests.py` - Add `src/client/plantdb/client/rest_api/parsers.py`
- Updated all interactive examples in `plantdb_client.py` to import `plantdb_url` from `plantdb.client.rest_api.urls` instead of the now‑removed `plantdb.client.rest_api`. - Removed the unused `functools.wraps` import. - Deleted an extraneous blank line after logger initialization. - Ensured consistency across all example blocks (login, scan creation, metadata retrieval, etc.) with the new import path.
- Changed example imports in `src/client/plantdb/client/sync.py` to `from plantdb.client.rest_api.requests import request_scan_names_list` and `request_refresh` - Updated runtime imports in `src/client/plantdb/client/sync.py` to pull `request_archive_download` and `request_archive_upload` from `plantdb.client.rest_api.requests` instead of the former module path - Adjusted corresponding docstring example blocks to reflect the new import location.
- Changed imports in `src/client/plantdb/client/cli/fsdb_rest_api_sync.py` to use `plantdb.client.rest_api.requests`
- Replace direct `client` imports with `parsers`, `requests`, and `urls` in `test_client_against_server.py` - Update all API calls in the test suite to reference the new helper modules (`requests.*`, `urls.*`, `parsers.*`) - Add shebang and module header to `test_rest_api.py` and switch imports to the new `requests` and `urls` sub‑modules - Adjust `patch` targets in `test_rest_api.py` to mock `plantdb.client.rest_api.requests.make_api_request` instead of the previous location - Ensure all referenced functions (`request_login`, `request_scan_names_list`, `scan_url`, etc.) point to their new locations across the test files
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implemented a comprehensive restructuring of the REST API client:
urls.py,requests.py, andparsers.pymodules to centralize endpoint generation, request handling, and response parsing.plantdb.client.rest_api.requestsandplantdb.client.rest_api.urls.api_endpoints.errorkey and aligned error handling with the new helper structure.Resource Hierarchy