A 3D perspective transformation and rendering API featuring both pytorch and numpy
Using machine learning to inference on the position and fov of cameras within the scene. See tutorial.
Here is a demo video on the training process.
learning.mp4
The pink points indicate the cameras whose positions are to be learned. The green points indicates the true camera positions. The lines are the ray tracing lines for inferencing the positions and fovs.
After calibrating the cameras, the two cameras are now able to detect the 3D position of any items in the scene.
Here is the object placed in the scene:
Then, both cameras captures images of the scene:
Two tracing rays are generated from the above two images. Here is the position detection result:
The red point is the true object center (1/2, 1/2, 1/3), and the cyan point is the detected object position (0.4969, 0.5038, 0.3421). The pink rays indicate the tracing rays from the captured.