Feature request: point lights, opacity, and vertex color textured triangles in the 3D renderer

[XanCraft21]

I would have posted this in discussions, but it is not available to me here.

Hello, i really like this library, and i am planning to use it for a project. There are a few features i would like added if possible.

The first feature i would like to have added is point lights. I can possibly simulate this using directional lights and some math for now, but if possible i think point lights will be great for making a scene with a lamp on a table in a room.

The second feature is opacity. This feature already exists for the 2D image api, but not for the 3D renderer. I would try to combine the 3D and 2D APIs to make transparent models, but then i would lose lighting and z buffer functionality. I think it would be cool if both textured and vertex color triangles could be transparent to make windows and glass models.

The last feature is textured vertex color triangles. We already have textured and vertex color triangles separately, but not both together. I would love it if we could make a rainbow wooden crate without doing any photoshop work.

I hope it won’t be too hard to add these new features. Thank you for taking the time to read this, looking forward to your reply.

[vindar]

Hi,

I would have posted this in discussions, but it is not available to me here.

Hello, i really like this library, and i am planning to use it for a project. There are a few features i would like added if possible.

The first feature i would like to have added is point lights. I can possibly simulate this using directional lights and some math for now, but if possible i think point lights will be great for making a scene with a lamp on a table in a room.

Yes, adding point/spot lights is indeed one of my top priorities! In fact, if you check out the latest version of the library, I have already added a MAX_SPOT_LIGHTS template parameter to the Renderer3D class for this purpose, although the code is not implemented yet.

The way I currently see it, the user will be able to add spotlights: that is, lights restricted to a given cone, which may also degenerate into full omnidirectional lighting, like a light bulb. Most importantly, the light intensity will decrease with the distance from the source, probably with a cutoff at a given distance.

There are a few performance concerns and compromises required to make this efficient enough for MCUs, but it should be doable. I expect to add this very soon. :)

The second feature is opacity. This feature already exists for the 2D image api, but not for the 3D renderer. I would try to combine the 3D and 2D APIs to make transparent models, but then i would lose lighting and z buffer functionality. I think it would be cool if both textured and vertex color triangles could be transparent to make windows and glass models.

Yes, this is another feature I would like to add, but it is more complicated because the current shaders make many speed-oriented compromises that are not really compatible with blending/opacity. What should be doable in the near future is to add a shader with a “mask” color that will be considered fully transparent. This should already be useful for sprites, windows, etc. Also, binary transparency, unlike alpha blending, does not require Z-sorting the triangles before rendering.

Once this is done, I will think about how to introduce proper transparency handling.

The last feature is textured vertex color triangles. We already have textured and vertex color triangles separately, but not both together. I would love it if we could make a rainbow wooden crate without doing any photoshop work.

It is indeed possible to write such a shader, but it would require additional per-pixel computation, which is quite expensive. So, whenever possible, it is probably better to bake this effect into the texture rather than compute it at runtime.

I hope it won’t be too hard to add these new features. Thank you for taking the time to read this, looking forward to your reply.

I have some free time coming up, so I expect to add some of these features within the next few weeks! And please don't hesitate to share suggestions about what would be most useful in practice, and how you think these features should be handled or implemented. Insights are always welcome. :)

Best,

[XanCraft21]

Thank you for your reply. I also have 2 new ideas i came up with earlier today, so that is about 5 feature suggestions.

The 4th idea is optional perspective corrected vertex coloring, which can be turned off to save performance. I thought of this idea since i noticed the colors moving around when the camera is rotated. I think it can be implemented the same way texture perspective correction is done, except with the color values instead of texture coords.

The 5th idea is maybe some more texture and vertex color manipulation math when using the solid material color, of which can also be disabled for performance. In my recent testing the solid material color only seems to affect flat shaded untextured triangles, but not vertex color or textured triangles.

I am curious to see how increasing the spotlight cutoff angle could mostly simulate a point light, i am looking forward to when that feature is added.

If proper transparency of the triangles will take too much performance then i think a simple no draw color could be good enough. Is it too much to also possibly add polygon stipple from the C-Chads TinyGL library so a simple dithering algorithm can be used to make decent fake full transparency? I know TinyGL is not Arduino compatible but i have gotten it to work with very little modifications.

I am assuming adding fog rendering with variable intensity and color might be too performance heavy, because at this point i’m just telling you my dream for the absolute best Arduino compatible 3D renderer library on earth, which i feel like with the right techniques this library might just become that in a few years from now.

I must apologize for asking for so much when we need to keep performance high for lower end devices. I have plans to make an Arduino compatible game console using a teensy 4.1, and i feel like this library might be the right choice for me. I have been trying to write my own 3D renderer from scratch and i can’t make it perform as good as this library.

I am also waiting for a stronger cpu architecture to become available, like the ARM Cortex M85. As far as i know only one company is using such cpu, but there is no Arduino support for it and probably won’t be for a long time.

I know probably not all of these ideas will be added because of performance, but i can’t just assume that without waiting patiently first. I have a lot of faith in this project, and i can picture it in my head what this library could do in the future. Sorry about the nonsensical rambling, maybe i’m just too excited about this library.

[vindar]

Hi, these are all good ideas, thanks :-)

I'll see what I can add. There is already quite a lot to explore but your suggestions look interesting.

I just released version 1.1.4, which adds in particular:

• affine texturing (faster than perspective-correct texturing). This is useful for finely tessellated models or when speed matters more than accuracy.
• point/spot light support. The renderer can now use multiple directional, point and spot lights in the same scene. I think local lights are really useful for creating an 'atmosphere'.
• New examples showing point/spot lights on ESP32/M5Stack/Pico/Teensy boards in the examples/ directory.

Please let me know if you try the new examples and what you think about the new lighting modes.

Next, I'm going to look at the best way to add transparency. I will probably start with something simple and cheap, like a mask/no-draw color, and then see what can be added without hurting performance too much.

[XanCraft21]

I’m glad you like my suggestions, thank you and you’re welcome!

You’ve finished working on this update pretty fast, and so far things are pretty good. I tested the directional light stuff a little and did some experiments with point lights, and i have yet to test the spot lights cones. So far i think i’m happy with the current results of my testing.

There are a couple oddities that come with the latest update, it runs slightly slower than before when not using the new features, but it’s not that much slower, just a few fps. And texture wrapping is not as accurate when pushed to the extreme as before. I was creating scrolling textures by offsetting the texture coords when drawing individual triangles, increasing in speed faster and faster over time with the texture wrap pow 2 shader enabled, and it could go a few minutes before textures get deformed and warped. Now it seems to take only a few seconds before the textures get deformed.

I probably shouldn’t be pushing texture wrapping so hard, it’s just a silly experiment. But i also like texture scrolling since it makes certain animations easier to create, and is similar to what is done in some older video games.

I look forward to seeing the no draw color transparency, and possibly a small adjustment to fix the unstable texture wrapping. I still plan to create extreme test cases lol. Thank you for the latest update.

[vindar]

I’m glad you like my suggestions, thank you and you’re welcome!

You’ve finished working on this update pretty fast, and so far things are pretty good. I tested the directional light stuff a little and did some experiments with point lights, and i have yet to test the spot lights cones. So far i think i’m happy with the current results of my testing.

There are a couple oddities that come with the latest update, it runs slightly slower than before when not using the new features, but it’s not that much slower, just a few fps.

Yes, that is unfortunately to be expected. I try to add new features as unobtrusively as possible by using templates and conditional compilation, but the compiler may still change the code layout in performance critical sections, which can slow down (or sometimes speed up!) the frame rate by a few percent. I find this especially noticeable on slower MCUs such as the Pico family (probably because of their limited instruction cache from flash). However, if the same code runs more than 10% slower with the new version, then it is definitely something I should investigate.

And texture wrapping is not as accurate when pushed to the extreme as before. I was creating scrolling textures by offsetting the texture coords when drawing individual triangles, increasing in speed faster and faster over time with the texture wrap pow 2 shader enabled, and it could go a few minutes before textures get deformed and warped. Now it seems to take only a few seconds before the textures get deformed.

That is very interesting. I cannot immediately see which of my changes could have caused this new behavior. Could you please provide a complete example that I can run to reproduce the problem and try to fix it?

I probably shouldn’t be pushing texture wrapping so hard, it’s just a silly experiment. But i also like texture scrolling since it makes certain animations easier to create, and is similar to what is done in some older video games.

I look forward to seeing the no draw color transparency, and possibly a small adjustment to fix the unstable texture wrapping. I still plan to create extreme test cases lol. Thank you for the latest update.

Yes please keep creating extreme test cases ! They are very useful for identifying unexpected issues. 😃

[XanCraft21]

Sorry for the late reply, i’ve been a bit busy and haven’t had the time to reply.

I discovered that the small slowdown mentioned before seems to affect bilinear texture filtering the most as far as i can tell. I can probably just turn it off and use higher resolution textures to get more speed, i just like how bilinear filtering makes pixilated textures look better.

For the code example on the texture scrolling it will take me a bit to get it posted, i am probably gonna make a github gist to make it look more professional to some extent. But for now i can give you a basic overview of what I did.

I stored my vertices, normals, and texture coords in an array, i have a single floating point scroll value i increment constantly over time, then i created s function and passed the arrays and scroll value into that function and increased the texture u coordinate by the scroll value before drawing individual triangles. To create faster scrolling increase the scroll value by a larger amount of make it go faster and faster over time with an increasing increment value over time.

Here’s another thing that I’ve noticed, but it could just be me misremembering this. I think specular lighting is supposed to make a little blob on the model that overwrites the texture or solid color and replace it with the solid specular color during rendering, but i could be totally wrong about that. I just wanted to say that i thought your library gave the models that extra bright shine before the latest update, now it just looks like the normal texture color where the specular blob is, so it no longer has that shine. Like i said it could be just me misremembering.

I will work on posting that code example when i can. I will say before hand i am using a different display driver library Arduino_GFX (also goes by GFX_Library_For_Arduino), and i am cheating when i create the texture images using a void star pointer cast since i couldn’t get it to work normally.