About data inputs

[UdonDa]

Hi @fnzhan !

Thank you for providing your nice implementation.

I have a question about inputs for networks, especially for a celeba edge case.

Correspondence predictor is given RGB images and seg_map (https://github.com/fnzhan/UNITE/blob/main/models/networks/correspondence.py#L200).

Celeb segmaps (15 channel) are created via a get_label_tensor function(https://github.com/fnzhan/UNITE/blob/main/data/celebahqedge_dataset.py#L77).
It seems that celeba segmaps include not only an edge but also distanceTransformed images.

Why did you use additional information such as semantic maps?
Do your work not work well for a dataset having no additional labels e.g. AFHQ -- animal face dataset?

Thanks.

[fnzhan]

Hi @UdonDa , thanks for your interest. The code inherits from the implementation of CoCosNet which includes other information for building correspondence. It is indeed somewhat different from the description in CoCosNet paper, we just follow the setting for fair comparison.

The original edge map is sparse which tends to be suboptimal for feature extraction, so a distanceTransformed image is included for more dense representation.

I don't check the performance on CelebA without other information, but I believe the performance will be degraded significantly under this setting.

[UdonDa]

Thank you for your quickly replying and telling me.

I know that the distanceTransformed image is significantly important.
I'll try to learn a network without this information.

Thank you!