Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

something wrong with 'blend_shape = torch.einsum('bl,mkl->bmk', [betas, shape_disps])' #66

Open
wjrzm opened this issue Mar 18, 2023 · 3 comments
Open

something wrong with 'blend_shape = torch.einsum('bl,mkl->bmk', [betas, shape_disps])' #66

wjrzm opened this issue Mar 18, 2023 · 3 comments

Comments

@wjrzm
Copy link

wjrzm commented Mar 18, 2023

I have configured the environment according to the installation process in README, but when I run ik_example_joints.py, I have the following error.

(py37)  ✘ wjrzm@DESKTOP-QJIK4OK  ~/human_body_prior/tutorials   master ±  /home/wjrzm/anaconda3/envs/pytorch1.7
.1/bin/python /home/wjrzm/human_body_prior/tutorials/ik_example_joints.py
No module named 'body_visualizer'
psbody.mesh based visualization could not be started. skipping ...
sample_amass keys:  ['poses', 'dmpls', 'labels', 'markers', 'mocap_framerate', 'betas', 'gender', 'vids', 'trans']
smplx (10475, 3, 486)
2023-03-17 15:25:20.281 | INFO     | human_body_prior.tools.model_loader:load_model:97 - Loaded model in eval mode with trained weights: ../support_data/dowloads/vposer_v2_05/snapshots/V02_05_epoch=13_val_loss=0.03.ckpt
[133 498 170 438]
smplx (10475, 3, 486)
Traceback (most recent call last):
  File "/home/wjrzm/human_body_prior/tutorials/ik_example_joints.py", line 105, in <module>
    ik_res = ik_engine(source_pts, target_pts)
  File "/home/wjrzm/anaconda3/envs/pytorch1.7.1/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/wjrzm/human_body_prior/src/human_body_prior/models/ik_engine.py", line 292, in forward
    optimizer.step(lambda: closure(wts, free_vars))
  File "/home/wjrzm/anaconda3/envs/pytorch1.7.1/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 26, in decorate_context
    return func(*args, **kwargs)
  File "/home/wjrzm/anaconda3/envs/pytorch1.7.1/lib/python3.7/site-packages/torch/optim/lbfgs.py", line 311, in step
    orig_loss = closure()
  File "/home/wjrzm/anaconda3/envs/pytorch1.7.1/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 26, in decorate_context
    return func(*args, **kwargs)
  File "/home/wjrzm/human_body_prior/src/human_body_prior/models/ik_engine.py", line 292, in <lambda>
    optimizer.step(lambda: closure(wts, free_vars))
  File "/home/wjrzm/human_body_prior/src/human_body_prior/models/ik_engine.py", line 142, in fit
    res = source_kpts_model(free_vars)
  File "/home/wjrzm/anaconda3/envs/pytorch1.7.1/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/wjrzm/human_body_prior/tutorials/ik_example_joints.py", line 50, in forward
    new_body = self.bm(**body_parms)
  File "/home/wjrzm/anaconda3/envs/pytorch1.7.1/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/wjrzm/human_body_prior/src/human_body_prior/body_model/body_model.py", line 261, in forward
    dtype=self.dtype)
  File "/home/wjrzm/human_body_prior/src/human_body_prior/body_model/lbs.py", line 209, in lbs
    v_shaped = v_template + blend_shapes(betas, shapedirs)
  File "/home/wjrzm/human_body_prior/src/human_body_prior/body_model/lbs.py", line 299, in blend_shapes
    blend_shape = torch.einsum('bl,mkl->bmk', [betas, shape_disps])
  File "/home/wjrzm/anaconda3/envs/pytorch1.7.1/lib/python3.7/site-packages/torch/functional.py", line 342, in einsum
    return einsum(equation, *_operands)
  File "/home/wjrzm/anaconda3/envs/pytorch1.7.1/lib/python3.7/site-packages/torch/functional.py", line 344, in einsum
    return _VF.einsum(equation, operands)  # type: ignore
RuntimeError: size of dimension does not match previous size, operand 1, dim 2

I searched for a lot of corresponding errors on the Internet. They tried to remind me of the BUG of the corresponding function of Pytorch, so I upgraded the version of Pytorch.
The following is the error report using Pytorch1.12.1 and 1.13.1.

(py37)  wjrzm@DESKTOP-QJIK4OK  ~/human_body_prior/tutorials   master ±  /home/wjrzm/anaconda3/envs/py37/bin/pyt
hon /home/wjrzm/human_body_prior/tutorials/ik_example_joints.py
Platform does not define a GLUT font retrieval function
psbody.mesh based visualization could not be started. skipping ...
sample_amass keys:  ['poses', 'dmpls', 'labels', 'markers', 'mocap_framerate', 'betas', 'gender', 'vids', 'trans']
smplx (10475, 3, 486)
2023-03-17 15:31:24.031 | INFO     | human_body_prior.tools.model_loader:load_model:97 - Loaded model in eval mode with trained weights: ../support_data/dowloads/vposer_v2_05/snapshots/V02_05_epoch=13_val_loss=0.03.ckpt
[123 110  26 423]
smplx (10475, 3, 486)
Traceback (most recent call last):
  File "/home/wjrzm/human_body_prior/tutorials/ik_example_joints.py", line 105, in <module>
    ik_res = ik_engine(source_pts, target_pts)
  File "/home/wjrzm/anaconda3/envs/py37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wjrzm/human_body_prior/src/human_body_prior/models/ik_engine.py", line 292, in forward
    optimizer.step(lambda: closure(wts, free_vars))
  File "/home/wjrzm/anaconda3/envs/py37/lib/python3.7/site-packages/torch/optim/optimizer.py", line 113, in wrapper
    return func(*args, **kwargs)
  File "/home/wjrzm/anaconda3/envs/py37/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/home/wjrzm/anaconda3/envs/py37/lib/python3.7/site-packages/torch/optim/lbfgs.py", line 311, in step
    orig_loss = closure()
  File "/home/wjrzm/anaconda3/envs/py37/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/home/wjrzm/human_body_prior/src/human_body_prior/models/ik_engine.py", line 292, in <lambda>
    optimizer.step(lambda: closure(wts, free_vars))
  File "/home/wjrzm/human_body_prior/src/human_body_prior/models/ik_engine.py", line 142, in fit
    res = source_kpts_model(free_vars)
  File "/home/wjrzm/anaconda3/envs/py37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wjrzm/human_body_prior/tutorials/ik_example_joints.py", line 50, in forward
    new_body = self.bm(**body_parms)
  File "/home/wjrzm/anaconda3/envs/py37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wjrzm/human_body_prior/src/human_body_prior/body_model/body_model.py", line 261, in forward
    dtype=self.dtype)
  File "/home/wjrzm/human_body_prior/src/human_body_prior/body_model/lbs.py", line 209, in lbs
    v_shaped = v_template + blend_shapes(betas, shapedirs)
  File "/home/wjrzm/human_body_prior/src/human_body_prior/body_model/lbs.py", line 299, in blend_shapes
    blend_shape = torch.einsum('bl,mkl->bmk', [betas, shape_disps])
  File "/home/wjrzm/anaconda3/envs/py37/lib/python3.7/site-packages/torch/functional.py", line 358, in einsum
    return einsum(equation, *_operands)
  File "/home/wjrzm/anaconda3/envs/py37/lib/python3.7/site-packages/torch/functional.py", line 360, in einsum
    return _VF.einsum(equation, operands)  # type: ignore[attr-defined]
RuntimeError: einsum(): operands do not broadcast with remapped shapes [original->remapped]: [4, 16]->[4, 1, 1, 16] [10475, 3, 10]->[1, 10475, 3, 10]

(pytorch)  wjrzm@DESKTOP-QJIK4OK  ~/human_body_prior/tutorials   master ±  /home/wjrzm/anaconda3/envs/pytorch/b
in/python /home/wjrzm/human_body_prior/tutorials/ik_example_joints.py
Platform does not define a GLUT font retrieval function
psbody.mesh based visualization could not be started. skipping ...
sample_amass keys:  ['poses', 'dmpls', 'labels', 'markers', 'mocap_framerate', 'betas', 'gender', 'vids', 'trans']
smplx (10475, 3, 486)
2023-03-17 15:56:27.134 | INFO     | human_body_prior.tools.model_loader:load_model:97 - Loaded model in eval mode with trained weights: ../support_data/dowloads/vposer_v2_05/snapshots/V02_05_epoch=13_val_loss=0.03.ckpt
[460 154 324 100]
smplx (10475, 3, 486)
Traceback (most recent call last):
  File "/home/wjrzm/human_body_prior/tutorials/ik_example_joints.py", line 105, in <module>
    ik_res = ik_engine(source_pts, target_pts)
  File "/home/wjrzm/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wjrzm/human_body_prior/src/human_body_prior/models/ik_engine.py", line 292, in forward
    optimizer.step(lambda: closure(wts, free_vars))
  File "/home/wjrzm/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/optim/optimizer.py", line 140, in wrapper
    out = func(*args, **kwargs)
  File "/home/wjrzm/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/home/wjrzm/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/optim/lbfgs.py", line 312, in step
    orig_loss = closure()
  File "/home/wjrzm/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/home/wjrzm/human_body_prior/src/human_body_prior/models/ik_engine.py", line 292, in <lambda>
    optimizer.step(lambda: closure(wts, free_vars))
  File "/home/wjrzm/human_body_prior/src/human_body_prior/models/ik_engine.py", line 142, in fit
    res = source_kpts_model(free_vars)
  File "/home/wjrzm/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wjrzm/human_body_prior/tutorials/ik_example_joints.py", line 50, in forward
    new_body = self.bm(**body_parms)
  File "/home/wjrzm/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/wjrzm/human_body_prior/src/human_body_prior/body_model/body_model.py", line 261, in forward
    dtype=self.dtype)
  File "/home/wjrzm/human_body_prior/src/human_body_prior/body_model/lbs.py", line 209, in lbs
    v_shaped = v_template + blend_shapes(betas, shapedirs)
  File "/home/wjrzm/human_body_prior/src/human_body_prior/body_model/lbs.py", line 299, in blend_shapes
    blend_shape = torch.einsum('bl,mkl->bmk', [betas, shape_disps])
  File "/home/wjrzm/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/functional.py", line 373, in einsum
    return einsum(equation, *_operands)
  File "/home/wjrzm/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/functional.py", line 378, in einsum
    return _VF.einsum(equation, operands)  # type: ignore[attr-defined]
RuntimeError: einsum(): subscript l has size 10 for operand 1 which does not broadcast with previously seen size 16

I am using
image
image

Could you help me find a solution?
@oneThousand1000
Copy link

Hi, in ik_example_joints.py, you could try to set num_betas = 10 in IK_Engine like:

ik_engine = IK_Engine(vposer_expr_dir=vposer_expr_dir,
                      verbosity=2,
                      display_rc= (2, 2),
                      data_loss=data_loss,
                      stepwise_weights=stepwise_weights,
                      optimizer_args=optimizer_args,
                      num_betas = 10 # ADD
                      ).to(comp_device)

Hope this can help you.

@wjrzm
Copy link
Author

wjrzm commented Mar 21, 2023

Hi, in ik_example_joints.py, you could try to set num_betas = 10 in IK_Engine like:

ik_engine = IK_Engine(vposer_expr_dir=vposer_expr_dir,
                      verbosity=2,
                      display_rc= (2, 2),
                      data_loss=data_loss,
                      stepwise_weights=stepwise_weights,
                      optimizer_args=optimizer_args,
                      num_betas = 10 # ADD
                      ).to(comp_device)

Hope this can help you.

Thanks, this helped me a lot! !

@bring-nirachornkul
Copy link

Hi, in ik_example_joints.py, you could try to set num_betas = 10 in IK_Engine like:

ik_engine = IK_Engine(vposer_expr_dir=vposer_expr_dir,
                      verbosity=2,
                      display_rc= (2, 2),
                      data_loss=data_loss,
                      stepwise_weights=stepwise_weights,
                      optimizer_args=optimizer_args,
                      num_betas = 10 # ADD
                      ).to(comp_device)

Hope this can help you.

It's working. Thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@oneThousand1000 @wjrzm @bring-nirachornkul