Add Python demo for variational image inpainting with smoothed TV regularization on irregular masks

[jb2178-star]

Hi all,

I would like to propose a new Python demo for DOLFINx showing variational image inpainting on an irregular mask using a smoothed total variation regularization. This demo would set up the inpainting problem as a nonlinear variational problem. The fidelity term on the known part of the image together with a smoothed TV regularization term on the full domain. It would derive the week form and implement the residual and Jacobian in UFL. For geometry, the plan is to use the Gmsh Python interface to build a conforming unstructured mesh of the full image domain, with the irregular mask boundary represented inside the domain. That would make it possible to tag the known and unknown regions and apply the fidelity term only where image data is available. The solve would use dolfinx.fem.petsc.NonlinearProblem together with PETSc SNES / Newton iteration. I would aim to keep the example self-contained, probably using a synthetic image and mask generated directly in code. I think this could be a useful demo because it combines several pieces of DOLFINx functionality in one example.

If it fits for this project, my assumption is that it would go in the Python demo collection under python/demo/ as a new demo file. A likely scope would be:

• formulate the smoothed TV inpainting problem
• derive the weak form
• generate a conforming mesh with an irregular interior mask/interface
• tag the relevant regions
• implement the nonlinear problem in UFL
• solve it with NonlinearProblem and SNES
• show basic reconstruction results and diagnostics
• have documentation

I have tried to use the Slack link mentioned in the project resources, but it said the link was no longer active. I'm opening this issue first to check whether this would be a welcome contribution and whether you would prefer it as a core demo or as an external tutorial.

Sincerely,

Jack Brockett

[jb2178-star]

Hi again, I was wondering if this was an ok demo for the dolfinx library?

[jorgensd]

First of all thanks for showing an interest in the project.
Before you dive into implementing this, could you list what external libraries you would require for this demo if it is made through synthetic data

I would aim to keep the example self-contained, probably using a synthetic image and mask generated directly in code.

Furthermore, what kind of derivation of the weak for are we talking about

derive the weak form
is there a specific paper you will base your derivation on?

Finally, do you think this demo will show off specific DOLFINx library features not exposed in other demos? If so, could you list them or briefly explain them.

[jb2178-star]

Hi,

Thank you for taking the time to review my issue.

For dependencies, if the demo is kept fully synthetic, I would only need NumPy in addition to the DOLFINx library. The synthetic image and mask can be generated directly in code, and the known and unknown regions can be represented on a background mesh using subdomain tags.

By “derive the weak form,” I mean the standard first-variation / Euler–Lagrange derivation of a smoothed TV inpainting functional. The residual would be: find (u) such that for all test functions (v),

$$ F(u;v)=\alpha \int_{\Omega}{\nabla u\cdot\nabla v\over \sqrt{|\nabla u|^2+\varepsilon^2}},\mathrm{d}x+\lambda \int_{\Omega_{\mathrm{known}}}(u-f)v,\mathrm{d}x=0. $$

This is directly expressible in UFL, and the Jacobian can be obtained with derivative(). I would base the formulation on the standard TV/inpainting literature, for example Chan–Shen (2001).

In terms of what it adds, the main value I had in mind is the combination in one example: a nonquadratic nonlinear variational problem, subdomain-restricted integration over a tagged interior region, and solution with dolfinx.fem.petsc.NonlinearProblem and PETSc SNES / Newton methods. I realize that some ingredients already appear separately in the current demos, so I do not mean to claim that each piece is new on its own; my thought was that the combination might still be useful in one compact self-contained example.