Question Regarding the Difference Between Zero Tensors and Black Image Embeds in CLIP Vision Model

[jiagongfu]

Thank you for your excellent work on this project! I’ve come across some discussions about unconditional embeddings, but I have a specific question regarding their implementation.

What is the difference between setting CLIP image embeddings to zero tensors directly and passing a black image(or a noised one) to the CLIP vision model? More specifically, what semantic meaning (if any) is encoded in the zeroed image embeddings?

[jiagongfu]

Additionally, what are the respective use cases for these two approaches when constructing negative image embeddings? For instance, in the context of the basic version of the IP-Adapter model (as described in the paper), which approach is more suitable? Would the first method (zero tensors) be preferable, or does it depend on other factors?