FID score

[mlyarthur]

Do you calculate fid score by comparing the training set and the generated images ? I cannot reproduce the same fid in the paper. And which fid git repo you choose to evaluate the results.

[fnzhan]

The provided weight has been checked before uploading, the FID score of UNITE paper should be reproduced with the weights.
You should compute the FID between testing set and generated images.
For the FID calculation, I use https://github.com/mseitzer/pytorch-fid.
The testing set images and generated images should be saved in .jpg format.

[mlyarthur]

Thanks for your quick reply. In fact, i can only reproduce fid = 15.0 in deepfashion dataset. That's weird. Can you supply one ground-truth test image and the corresponding generated image on deepfashion for me to check where the problem is ? I think it may be caused by the data (i use .jpg format) inconsistentce error.

[mlyarthur]

My setting on deepfashion
1..jpg format with 4993 generated images and test images
2.img size :256x256
3.fid calculation script is the same with yours https://github.com/mseitzer/pytorch-fid.

[mlyarthur]

the deepfashion fid score i say is the cocosnet result. And the ade score i calculated is not the same as well. It bothers me a lot.

[fnzhan]

not sure about the problem. Here I provide a sample of generated image & gt:

[mlyarthur]

Thanks for your reply and your kind help. I will check my results.

[mlyarthur]

Sorry to bother you, Can you provide the image id or name of these two images?

[fnzhan]

It is the first sample in https://github.com/fnzhan/UNITE/blob/main/data/deepfashion_ref_test.txt

[mlyarthur]

https://drive.google.com/file/d/1o1GteDHPbYldOge3fAjRx1zF9wBtNF6d/view?usp=sharing .Here are the results of cocosnet using the pretrained model, I can only produce 13.05 using the https://github.com/shepnerd/pytorch-fid and 15.00 using the https://github.com/mseitzer/pytorch-fid which are inconsistent with the reported score 14.4 in the paper. can you help to check the data. Thank you in advance.

[mlyarthur]

It is the first sample in https://github.com/fnzhan/UNITE/blob/main/data/deepfashion_ref_test.txt

[mlyarthur]

not sure about the problem. Here I provide a sample of generated image & gt:

I find that your gt image is not the same as mine by subtracting the pixel values.

Did you generate the gt image via test.py or use the original file in the dataset. I generate the gt via test.py with

[fido20160817]

H, where to download the pretrained model on deepfashion, coco and celeb? I just find ade20 on the provided link