ETH3D dataset

[narsisn]

Hi,
Thanks for your interesting work.
Could you please explain how you downloaded, prepared, and evaluated your model on the ETH3D dataset?
And also for Long3D, how did you calculate the mertics? Did you calculate for all catagories (e.g, Classroom, dormitory, ...)?? I can not reproduce your reported results for this dataset.
Thanks for your time.

[00tim00]

Hi,
Thank you for your interest in our work! Here are the details regarding your questions:

1. ETH3D Dataset Preparation and Evaluation:

To download the ETH3D dataset, you can use the script provided in the Pi3 repository:
https://github.com/yyfz/Pi3/blob/evaluation/datasets/preprocess/prepare_eth3d.sh
Please note that you only need to use this script for downloading; you do not need to perform preprocessing steps for our method.

Regarding the implementation, our codebase already includes the necessary setup for ETH3D. You don't need to write a new dataloader from scratch. You can simply import ETH3D and add it to your dataset configuration.
Here is an example of how to configure it in the code:

from eval.mv_recon.data import SevenScenes, NRGBD, ETH3D

# Add this to your dataset configuration dictionary
"ETH3D": ETH3D(
    split="test",
    ROOT="/path/to/your/eth3d", # Please replace this with your actual local path
    resolution=resolution,
    num_seq=1,
    full_video=True,
    kf_every=1,
),

2. Long3D Metrics and Reproducibility:

Yes, for Long3D, we calculated the metrics across all categories (Classroom, Dormitory, etc.) and then took the overall mean. Regarding the reproducibility issue you mentioned: since there is random sampling involved in the evaluation process, we fixed the random seed to 42 to ensure consistent results.

[narsisn]

Thanks for your reply. I will try with seed 42 and run it again. Are you using this code for Long3d dataset, right ?
https://github.com/AutoLab-SAI-SJTU/InfiniteVGGT/tree/main/src/eval/mv_recon_long

[00tim00]

Thanks for following up!

Regarding the Long3D dataset evaluation, we did not use the specific code you linked. That script appears to be a recently released evaluation code, and at the time of our work, those evaluation details were not yet available.

Instead, we used our own post-processing approach, which we detailed in the supplementary material of our paper. The process is somewhat complex because aligning ultra-long sequences is quite challenging due to accumulated tracking errors. However, since the mv_recon_long script is now available as a reference, you can certainly try using it you may be able to obtain similar results.

Additionally, due to unavoidable accumulated errors, if you want better reconstruction quality for long sequences, chunk-based methods like VGGT-Long might be a better choice (even though they are not direct streaming approaches).

[narsisn]

Thanks for your reply!

Could you please share the code you used for the Long3D dataset evaluation? Even after applying your post-processing approach, I was not able to reproduce your results. I really appreciate it.