Imperfect alignment registration & the cube's pose changes following the head movement

[nocardio]

Hi all ,
I did calibration of my HL2 by taking 47 pictures from various angle and distance. I preprocess the captured images (resize & jpg-to-png conversion) and run calibrate.py to calculate the camera parameter. I then applied the calculated camera parameter in the HoloLensARToolkit sample to track the cube marker.

When I run the sample in the HL2, it can detect the marker and display the virtual cube. However, the cube is positioned almost on top of the cube and there are some offset. It is not perfectly aligned and registered.
Furthermore, the cube position is also slightly changed following my head movement (left, right, or near/far distance). There is a little judder too when I move the head position.

Fyi, I used the same cube marker provided in the sample.

Thank you all for the helps

[zxzkf1992]

Hello, Can I ask you a question about capturing images? I use HoloLensARToolKit to obtain RGB images and compress its size, but currently v0.3 can only obtain grayscale images. If you have time, can you tell me in detail how you obtained the RGB images from the HL2 camera and resize it?

Thank you for your help !!!

[nocardio]

Hello @zxzkf1992 , I consecutively captured the images in HL2 by manually selected Camera Icon in the START menu and used an air tap gestures. I modified calibrate.py in the Github to resize the image into 1504x845 dimension and save the file with the *.png format.
I then used the preprocessed image as the input for the calibration.

[nocardio]

Hi @qian256 When I am wearing the HL2, I can see that the virtual cube can partly align to the marker. Even though there is some offset and also judder when I move my head or the marker.
The interesting one is the difference between my real-time HL2 view with the video-captured of my view. In the captured video, the virtual cube was really not aligned as it's displayed in the HL2 view.

To resume, I have two problems: 1) the misalignment between virtual and real cube 2) how to record the HL2 view with the exact result.

Do you know the reason and solution for these problems?
@qian256 and anyone can help me to solve these problems?

Thank you in advance!!