The FGD differ from your paper on BEAT dataset.

[lovemino]

Thank you for your contributions. However, I have a significant question regarding the discrepancy in the FGD metric results. When following your test dataset requirements and test code, the FGD metric results differ greatly from those reported in your paper. Specifically, when using the test_RAG_beat.py script from scripts_beat to test the original BEAT dataset, the final datasets obtained through your various processing scripts (my6d_bvh_rot_2_4_6_8_cache) yield FGD test metrics that significantly deviate from those in your paper. I would greatly appreciate any clarification you could provide on this matter.

[zyhbili]

Thanks for your attention! Do you test on the env with same torch and cuda version? You may check issue7 for more details. It may due to some torch implementaion problems in latter torch and cuda version.

[lovemino]

Thanks for your attention! Do you test on the env with same torch and cuda version? You may check issue7 for more details. It may due to some torch implementaion problems in latter torch and cuda version.

Thank you very much for your response. Here is my environment: the torch version is correct, but I am getting an FGD of 738.47, while your paper reports 7.561. I followed your data processing steps to obtain the data.mdb file and did not modify the inference script. The test_RAG_beat.py script runs successfully, but the metric differs significantly from your paper. Could you please provide the data.mdb file processed from your BEAT dataset? Thank you very much!
absl-py 1.4.0
audioread 3.0.0
backcall 0.2.0
blessed 1.20.0
blobfile 2.0.2
cachetools 5.3.0
certifi 2023.5.7
cffi 1.15.1
charset-normalizer 3.1.0
clip 1.0
ConfigArgParse 1.5.3
cycler 0.11.0
decorator 5.1.1
einops 0.4.1
fasttext 0.9.2
ffmpeg 1.4
ffmpeg-python 0.2.0
filelock 3.12.0
fonttools 4.38.0
fsspec 2023.1.0
ftfy 6.1.1
future 0.18.3
google-auth 2.18.0
google-auth-oauthlib 0.4.6
gpustat 1.1
grpcio 1.54.0
huggingface-hub 0.14.1
idna 3.4
importlib-metadata 6.6.0
ipython 7.34.0
jedi 0.18.2
Jinja2 3.1.4
joblib 1.2.0
kiwisolver 1.4.4
librosa 0.9.2
llvmlite 0.39.1
lmdb 1.4.1
loguru 0.7.0
lxml 4.9.2
Markdown 3.4.3
MarkupSafe 2.1.2
matplotlib 3.5.3
matplotlib-inline 0.1.6
mpmath 1.3.0
networkx 3.1
numba 0.56.4
numpy 1.19.5
nvidia-cublas-cu12 12.1.3.1
nvidia-cuda-cupti-cu12 12.1.105
nvidia-cuda-nvrtc-cu12 12.1.105
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu12 8.9.2.26
nvidia-cufft-cu12 11.0.2.54
nvidia-curand-cu12 10.3.2.106
nvidia-cusolver-cu12 11.4.5.107
nvidia-cusparse-cu12 12.1.0.106
nvidia-ml-py 11.525.112
nvidia-nccl-cu12 2.18.1
nvidia-nvjitlink-cu12 12.5.82
nvidia-nvtx-cu12 12.1.105
oauthlib 3.2.2
opencv-python 4.6.0.66
packaging 23.1
pandas 1.3.5
parso 0.8.3
pexpect 4.8.0
pickleshare 0.7.5
Pillow 9.5.0
pip 24.0
platformdirs 3.5.1
pooch 1.7.0
prompt-toolkit 3.0.38
protobuf 3.20.3
psutil 5.9.5
ptyprocess 0.7.0
pyarrow 4.0.0
pyasn1 0.5.0
pyasn1-modules 0.3.0
pybind11 2.10.4
pycparser 2.21
pycryptodomex 3.17
Pygments 2.15.1
pyparsing 3.0.9
python-dateutil 2.8.2
pytz 2023.3
PyYAML 6.0
regex 2023.5.5
requests 2.30.0
requests-oauthlib 1.3.1
resampy 0.4.2
rsa 4.9
scikit-learn 1.0.2
scipy 1.7.3
setuptools 69.5.1
six 1.16.0
SoundFile 0.10.3.post1
sympy 1.13.0
tensorboard 2.11.2
tensorboard-data-server 0.6.1
tensorboard-plugin-wit 1.8.1
termcolor 2.3.0
TextGrid 1.5
threadpoolctl 3.1.0
tokenizers 0.13.3
torch 1.7.1+cu110
torchvision 0.8.2+cu110
tqdm 4.64.0
traitlets 5.9.0
transformers 4.29.1
triton 2.1.0
typing_extensions 4.6.2
umap 0.1.1
urllib3 1.26.15
wcwidth 0.2.6
Werkzeug 2.2.3
wheel 0.43.0
zipp 3.15.0

[zyhbili]

Here is my testset. What's your python version and gpu device, and do you encounter same issue on TED?

[lovemino]

Here is my testset. What's your python version and gpu device, and do you encounter same issue on TED?
I used the RAG.pt model provided by you and best_rec_200.bin as the motion autoencoder. I created the data.mdb file using the 2, 4, 6, 8 dataset and ensured that the Python version is 3.7, matching your setup. However, there is still a significant difference in the FGD results. If you could provide the processed Beat dataset to help me replicate your results, I would greatly appreciate it. I didn't test the ted datasets as I don't need to use it.

[lovemino]

Reference in new issue
Thank you for your support. If you could send the data.mdb, I would greatly appreciate it.

[lovemino]

Here is my testset. What's your python version and gpu device, and do you encounter same issue on TED?

I greatly appreciate your response. I used the test set you provided, but I couldn't get the Beat code to run successfully. This dataset seems not to have been cut into 34-frame segments. When the dataloader's batch size is set to 256, it throws an error and only one batch can be obtained. Changing the batch size to 1 also results in an error when running the sample_fn function in infer_from_testloader:
sample = sample_fn( mdm_model, (B, 47, 6, frames_perclip), clip_denoised=False, model_kwargs=cond, skip_timesteps=skipsteps, # 0 is the default value - i.e. don't skip any step init_image=decoded_motions, progress=False, dump_steps=None, noise=None, const_noise=False, )
With frames_perclip=34, the corresponding shape in init_image=decoded_motions would be its actual frame count, such as 1150.

[zyhbili]

Sorry, i lost the backup for the testset with 34-frames segments. It's the raw testset. You need to process them with scripts, and you could contact my email for faster debugging. [email protected]