Evaluation of BLEFF rotation and translation error

[HannahHaensen]

Question about the calcualtion of the pose error for the BLEFF dataset.
As the colmap ground truth and the gt_metas differ i was wondering how the evaluation of COLMAP vs real ground truth is done.

can you provide some insights on that?

[HannahHaensen]

so do you also extract a GT file directly from blender as poses_bounds.npy or how is the metric caluclated?

[ziruiw-dev]

Hi @HannahHaensen,

Yes in BLEFF dataset package there should be a json file for each scene that contains poses exported from blender. Evaluating COLMAP vs GT is as same as evaluating our trajectory vs COLMAP, i.e. align two trajectory with a Sim(3) transformation and compute ATE (using ATE toolbox).

[HannahHaensen]

@ziruiw-dev thank you! i was wondering how in this case the near and far bounds where defined for training?

[ziruiw-dev]

Hi @HannahHaensen, NeRFmm only works in forward facing scenes and we use NDC coordinate (check original NeRF) so the near and far are always 0.0 and 1.0 for all scenes.

[HannahHaensen]

@ziruiw-dev Thank you! one last question regarding the GT of BLEFF, the orientation of c2ws in the blender meta info in which format is it?
because for colmap we normally face that issue
# correct c2ws: original c2ws has rotation in form "down right back", change to "right up back".

[ziruiw-dev]

Hi @HannahHaensen,

Thanks for the question and sorry for the delay. This work has been a while and I forgot these details. This reminds me I didn't specify that in the data release and haven't updated the dataloader code for BLEFF. Sorry about that!

Anyway, after checking, the c2w loaded from gt_meta.json is already in OpenGL coordinate, i.e. x: right, y: up and z back. Let me know if this works for you and feel free to re-open the issue!

Best,
Zirui

[HannahHaensen]

thank you very much, i assumed so too :D then but wanted to double check for a correct eval. also visualized it and looks good so far.

thank you very much for being so responsive! @ziruiw-dev