The issue of calibration

[yulan0215]

I did the calibration but the problem is:

and when I used more images and point cloud to do calibration it showed:

I used ubuntu 16.04 and opencv4.2, thx!

[heethesh]

Pick around 20-30+ corresponding points to get good results. You can reset the points collected so far by deleting the img_corners.npy and pcl_corners.npy files.

[yulan0215]

Pick around 20-30+ corresponding points to get good results. You can reset the points collected so far by deleting the img_corners.npy and pcl_corners.npy files.

Actually I used about 20 correspondences but when I calibrated the second corresponding image and point cloud, it showed the problem like I showed on the top.

[heethesh]

Can you verify the shapes of points2D and points3D before the LM refinement steps? Seems like the initial estimation working but fails during LM refinement

[yulan0215]

You mean, check the points and pixels in the .npy file? The file pcl_corner and img_corner?

[heethesh]

Yes

[yulan0215]

I checked in the image and pcd, I selected 12 pixels and 2 points in one image and one frame point cloud.... You said use 20 to 30 correspondences by using just 1 frame point cloud and 1 image, is it?

[heethesh]

You should select the same corresponding points. Not sure what you mean by 12 pixels and 2 points. Corresponding points means you pick 12 pixels on the image and the same corresponding 12 points from the point cloud, both from the same timestamps. The total number of correspondences don't have to be picked from the same frame/timestamp. It's okay to pick 3 correspondences (in both image and point cloud each) at time=1 and pick 5 more correspondences at time=2.

[yulan0215]

You should select the same corresponding points. Not sure what you mean by 12 pixels and 2 points. Corresponding points means you pick 12 pixels on the image and the same corresponding 12 points from the point cloud, both from the same timestamps. The total number of correspondences don't have to be picked from the same frame/timestamp. It's okay to pick 3 correspondences (in both image and point cloud each) at time=1 and pick 5 more correspondences at time=2.

Thanks for your reply, I have a question that the time stamp of my camera and LIDAR are not synchronised and they have about 0.5 second difference, do u think it will influence the result or not? Thx

[heethesh]

Don't set your slop too high in the time synchronizer. 0.5 secs should be fine as long as the scene is static for a while and the calibration targets aren't moving too much.

[yulan0215]

Don't set your slop too high in the time synchronizer. 0.5 secs should be fine as long as the scene is static for a while and the calibration targets aren't moving too much.

Thx,I recorded one pose of calibration target for 5s, and I recorded several bags and play them one by one, is it ok?

[heethesh]

Yes, just delete the npy files once and then for every successive correspondences picked, they will continually be updated.

[yulan0215]

Yes, just delete the npy files once and then for every successive correspondences picked, they will continually be updated.

Thank you for your reply and I encountered another issue that I calibrated the camera and LiDAR sensor but I found that when I reprojected the point to image via the data which I used to do calibration, the result was acceptable:

But when I appied this extrinsic parameter into another dataset, the result was strange:

Do you know the problem? I used the same intrinsic parameter and extrinsic parameter to do data fusion. Thx！

[heethesh]

Try to pick more correspondences by placing the checkerboard in various orientations, distances, and different perspectives of the checkerboard plane and cover as much of the LiDARs field of view as possible, your estimates will improve. See the example video in the README.