Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add error-checking branch to catch lists of non-tensors being passed as ip_adapter_image_embeds #7755

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

beyarkay
Copy link

@beyarkay beyarkay commented Apr 23, 2024

What does this PR do?

This PR just adds an extra branch to the parameter checking (in self.check_inputs) which makes sure that the value being passed in as ip_adapter_image_embeds is indeed a list of items which have a ndim attribute. I mistakenly passed in a list of strings and got hit with:

  File "/Users/brk/projects/quick-ideas/.venv/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl.py", line 681, in check_inputs
    elif ip_adapter_image_embeds[0].ndim not in [3, 4]:
AttributeError: 'str' object has no attribute 'ndim'

Which took looking into the source code to figure out. After this commit, the error is:

  File "/Users/brk/projects/quick-ideas/.venv/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl.py", line 682, in check_inputs
    raise ValueError(
ValueError: `ip_adapter_image_embeds` must be a list of tensors, but is a list of <class 'str'>

Before submitting

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

(tagging because pipelines: @sayakpaul @yiyixuxu)

@sayakpaul sayakpaul requested a review from yiyixuxu June 1, 2024 07:18
@sayakpaul
Copy link
Member

@fabiorigano WDYT about this?

Copy link
Contributor

@fabiorigano fabiorigano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi everyone, thanks, @sayakpaul, for adding me here
I suggested a change to the current solution. All pipelines should be updated, right?

@@ -678,6 +678,10 @@ def check_inputs(
raise ValueError(
f"`ip_adapter_image_embeds` has to be of type `list` but is {type(ip_adapter_image_embeds)}"
)
elif not hasattr(ip_adapter_image_embeds[0], "ndim"):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
elif not hasattr(ip_adapter_image_embeds[0], "ndim"):
elif not isinstance(ip_adapter_image_embeds[0], torch.Tensor):

Both torch.Tensor and numpy.ndarray have the ndim attribute. If an user passes a list of numpy.ndarray objects as ip_adapter_image_embeds, he will get an error in the prepare_ip_adapter_image_embeds method, because this object does not have a chunk method.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants