It seems some scenes in BLEFF have invalid gt poses

[cmh1027]

For instance, gt_metas.json of bed scene has corresponding c2w data.

            [
                0.14657878875732422,
                -0.0010548068676143885,
                -0.2047007977962494,
                -1.303330659866333
            ],
            [
                -0.20465244352817535,
                -0.00637618824839592,
                -0.14651136100292206,
                -0.7851966619491577
            ],
            [
                -0.004570294171571732,
                0.25168848037719727,
                -0.004569551907479763,
                1.4914559125900269
            ],
            [
                0.0,
                0.0,
                0.0,
                1.0
            ]

If you check whether it is a valid SE(3) matrix (check if rotation part is orthonormal and determinant is 1), you'll notice that it is not a valid SE(3) matrix. Is there any possibility that c2w data is corrupted? Or, I just notice that if I multiply some value so that the determinant becomes 1, they suddenly become reasonable.

[ziruiw-dev]

Hi @cmh1027,

I did some quick checks and the rotation part is indeed not normalised. I am sorry for making this mistake in the dataset. Thanks for pointing this out!

I need to dig a little bit more to figure out the exact reason when I have time. I had a quick check with my blender-related code, and the issue might be caused by:

1. the format of blender's rotation matrix;
2. the way I add SE3 noise.

I also need to check whether all scenes and all noise levels have this issue when I have time.

Luckily I think the rotation part of c2ws is still orthogonal, just not normalised. For now, a quick fix is to normalise the rotation part with svd() after loading gt_metas.json, for example:

def normalise_rotmat(R_in):
    """
    R_in:  (N, 3, 3)
    R_out: (N, 3, 3)
    """
    u, s, vh = np.linalg.svd(R_in)
    R_out = u @ vh
    return R_out

R_normed = normalise_rotmat(c2ws[:, :3, :3])
c2ws[:, :3, :3] = R_normed

Let me know if this works for you!

Best,
Zirui

[cmh1027]

Thanks for reply!