About the Transformation from velo to image

[ShijieXue749]

Hi,
I really appreciate your work.
I'm reading your code about calculating the Transformation from Velodyne to the camera image plane at code,
but I get confused.
For convenience, I capture part of the code below:

        P_cam_nwu = np.asarray([[0, -1, 0, 0], 
                                [0, 0, -1, 0], 
                                [1, 0, 0, 0], 
                                [0, 0, 0, 1]], dtype=pc_np.dtype)
        P_nwu_cam = np.linalg.inv(P_cam_nwu)

        # now the point cloud is in CAMERA coordinate
        Pr_Pcamnwu = np.dot(Pr, P_cam_nwu)
        pc_np = transform_pc_np(Pr_Pcamnwu, pc_np)
        sn_np = transform_pc_np(Pr_Pcamnwu, sn_np)

        # assemble P. P * pc will get the point cloud in the camera image coordinate
        PcPnwucamPrinv = np.dot(Pc, np.dot(P_nwu_cam, Pr_inv))
        P = np.dot(Pji, PcPnwucamPrinv)  # 4x4

• according to my understanding:
  • P_cam_nwu is actually close to Tr in calib.txt, representing the coordinate transformation from velo to cam.
  • Pr is the random rotation matrix
  • Pc is P2 @ Tr or P3 @ Tr, representing the transformation from velo coordinate to the camera image plane (2 or 3)
  • Pji is the relative pose, j'th frame wrt. i'th frame
• Question:
  • P = Pji @ Pc @ P_nwu_cam @ Pr_inv is equivalent to P = Pji @ (P2 @ Tr) @ (Tr_inv) @ Pr_inv = Pji @ P2 @ Pr_inv
    • assemble P can transform i'th frame in Velodyne coordinate into j'th frame in camera image plane (include random rotation).
    • but if there lacks the coordinate transfoamation from velo to cam (Tr)?
    • I mean, the ground truth may be P = Pji @ P2 @ Tr @ Pr_inv?
  • pc_np = transform_pc_np(Pr_Pcamnwu, pc_np) is equivalent to pc_np = Pr @ P_cam_nwu @ pc_np
    • so if the Pr is the random transform on points under the camera coordinate rather than the NMU coordinate? (annotation at Line 352)
    • this actually doesn't affect the results, just for rigor.

May I ask if there is something wrong in my understanding, if so, Could you please give me a brief explanation?
Thank you in advance and look forward to your reply.