Skip to content

Feature/add preprocessing smoothing - #45

Draft
alessandrofelder wants to merge 6 commits into
mainfrom
feature/add-preprocessing-smoothing
Draft

Feature/add preprocessing smoothing#45
alessandrofelder wants to merge 6 commits into
mainfrom
feature/add-preprocessing-smoothing

Conversation

@alessandrofelder

Copy link
Copy Markdown
Collaborator

This PR is an attempt at incorporating preprocessing ("workflow steps B+C of the GEBCO cookbook") into remove-restore.py.
See #32 and #43 for details.

This PR mainly adds

  • a preprocessing function doing blockmedian, surface, grdfilter and grdsample on combined base and update grid data.
  • and incorporates this functionality into the main() remove-restore workflow

We need to think more about

  • how much this really helps us
  • if it does help, is there a better way to make the preprocessing part of the workflow?
    • do we want to control this via input arguments?
    • do we want a separate entry point for preprocessing?
    • how should it work in the case when the user provides several input grids?

remove unused import.
ensure diff_threshold arg is parsed to float.
will be resampled as part of preprocessing step instead.
this is needed because low-pass filter in preprocessing may
not have values everywhere.
@alessandrofelder

Copy link
Copy Markdown
Collaborator Author

close-up-comparison
This is a representative close-up comparison of using vs. not using the preprocessing steps B+C (in their current implementation).
I ran:

remove-restore --base gebco_2020_n54.0_s43.0_w-130.0_e-117.0.nc --output=vancouver-island.nc --plot --spacing 0.00028 NOAA_South_VancouverIsland_30m.tif --region_of_interest -123.4 -123 47.8 48.0 --diff_threshold 0.1

It doesn't seem to make any real difference on the smoothness of the boundary? 🤔
For orientation, this is here:
image

@ghost

ghost commented May 21, 2021

Copy link
Copy Markdown

Just to confirm:
Input base grid is GEBCO ~300m
Input update grid is for Vacouver ~30m
Output spacing ~ 30m

Now: (Step B +C p. 148 in GEBCO Cookbook)

  • All input data combined and blockmedian filtered at 30m*4 = 120m grid
  • Blockmedianed data gridded at 120m using surface
  • Gridded data smoothed at 120m*3 = 3600m using grdflter
  • Filtered data resampled at 30m using grdsample (i.e. Step C of Cookbook)
    Output at this stage is value at nodes of 30m grid.

Next: (remove-restore)

  • All input data combined and blockmedian filtered at 30m grid (Output here is cell-centred!?)
  • Find difference between blockmedianed data and resampled data (i.e. output of Step B+C)
  • If difference above a threshold, replace node data in 30m grid with blockmedian data.

@alessandrofelder

Copy link
Copy Markdown
Collaborator Author

@Devaraj-G essentially, yes, that is what is happening.

The main difference between your description and what is happening in the code is that only the update grid gets put back into the output of Step B+C.

(small corrections, not important: 3*120=360, not 3600, and Step C is the grdfilter not the grdsample step)

@alessandrofelder alessandrofelder self-assigned this Jun 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant