Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable Volume Generation from Concrete Labelmaps in MAISI ControlNet #8314

Open
nvahmadi opened this issue Jan 27, 2025 · 0 comments
Open

Enable Volume Generation from Concrete Labelmaps in MAISI ControlNet #8314

nvahmadi opened this issue Jan 27, 2025 · 0 comments

Comments

@nvahmadi
Copy link
Contributor

Is your feature request related to a problem? Please describe.
Image generation with MAISI

Describe the solution you'd like
Afaiu, currently, MAISI can "only" generate volumes conditionally on controlnet_cond. These are relative size changes of certain anatomical labels. But it cannot generate a volume from a concrete labelmap that is given.

Describe alternatives you've considered
In ControlNet's forward() method, I did not see an alternative that would allow embedding a concrete labelmap. The Diffusion Model requires embeddings from ControlNet, and cannot accept concrete labelmaps either.

Additional context
Figure 2 in the MAISI paper also seems to indicate that ControlNet can only accept "task-specific conditions", and "Diffusion Model" depends on ControlNet embeddings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant