Fix tessellation artifacts

[Paramveersingh-S]

Fix tessellation artifacts on empty/hollow faces

Following up on the conversation from the previous PR regarding the diagonal line artifacts appearing on the detector's empty/hollow faces.

As you correctly diagnosed, this is a side effect of the tessellation process. When the GLU tessellator breaks >3-sided polygons into triangles, OpenGL faithfully draws all those internal triangle edges because the polygon mode is set to GL_LINE (for hollow faces).

While your idea of treating the filling and borders separately with two rendering passes would definitely work, there is actually a built-in OpenGL solution that is much cleaner, more performant, and avoids having to rewrite the rendering pipeline for multi-pass drawing.

The Fix:
I have implemented a fix by utilizing the GLU_TESS_EDGE_FLAG callback in our GLU tessellator (wsgl_tess.c).

When this callback is provided, the GLU tessellator automatically emits glEdgeFlag(GL_TRUE) for the true outer boundaries of the original polygon, and glEdgeFlag(GL_FALSE) for the internal diagonal lines it generates. OpenGL's line rendering respects this state natively and automatically hides the internal diagonals while drawing the true borders, resulting in the exact clean output you showed in the expected image.

The implementation details:

1. Created tessEdgeFlagCB that passes the boolean edge flag directly to OpenGL via glEdgeFlag().
2. Registered the callback in wsgl_draw_tess_polygon().
3. Added a call to glEdgeFlag(GL_TRUE) at the end of the tessellation function to ensure the global state is reset safely for subsequent rendering commands.

Please test this and tell if there is error or not

[Paramveersingh-S]

test_f3

[schwicke]

Looks much better with this patch, thanks a lot!

[Paramveersingh-S]

Awesome @schwicke , Just a heads up, the final commits in this PR also resolved Issue #34 (transparency) and Issue #35 (missing Fortran back-face properties) so you should be able to color the back-faces in test_f3 now!"

[schwicke]

#34 is not fixed yet. This can be addressed later.