Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

UV map to SMPL model #41

Open
Amran2k16 opened this issue Aug 7, 2019 · 23 comments
Open

UV map to SMPL model #41

Amran2k16 opened this issue Aug 7, 2019 · 23 comments

Comments

@Amran2k16
Copy link

Hi,

Once i have the UV maps for the images after using create_uv_maps.py , how do i construct a SMPL model out of it?
@liwenssss
Copy link

hi, @Amran2k16 , have you done it, I want to do it too

@Amran2k16
Copy link
Author

Yeah, you need to use the resample function that can be found in the uv_map_generator python file. This file can be found in the data_utils folder and takes a uv map as input argument

@liwenssss
Copy link

thanks for your response ,I will try later~

@liwenssss
Copy link

hi, @Amran2k16 , I use create_UV_maps.py to generate some uv_map, and I use the resample function like this:

    file_prefix = 'template'
    generator = UV_Map_Generator(
        UV_height=256,
        UV_pickle=file_prefix+'.pickle'
    )
    uv_map = cv2.imread("/home/liwensh/data/human36m_UV_map_/h36m_0001/frame0000.jpg")
    resample_mesh = generator.resample(uv_map)

    device = torch.device('cuda')

    model = SMPLModel(
        device=device,
        model_path='./model_lsp.pkl',
        data_type=torch.float32
    )

    model.write_obj(resample_mesh, './test.obj')

the uv_map is like this:
frame0000
but it seem I got the wrong res, the generate obj file show like this:
image
any advice can you give?

@Amran2k16
Copy link
Author

Can you try and use smpl_fbx_temate instead of the radvani ones and show me what results you get?

@Lotayou
Copy link
Owner

Lotayou commented Aug 23, 2019

Hi @liwenssss I think you made a mistake here:

uv_map = cv2.imread("/home/liwensh/data/human36m_UV_map_/h36m_0001/frame0000.jpg")
# uv_map = uv_map.astype(np.float) / 255.0
resample_mesh = generator.resample(uv_map)

Maybe you should convert the uv_map to float between [0,1] first. Also, I would not suggest storing uv maps as jpg, since the lossy compression would change the coordinates. It's better to save them as lossless image format like png or bmp.

@liwenssss
Copy link

@Amran2k16 are you it convenient to send me your generate smpl_fbx_temate file. Actually， I fail to generate it so that I use the radvani's. Maybe it is the reasaon.
@Lotayou , thanks for your advice , but it seems not the reason. I do the following change:

   uv_map = imread("/home/liwensh/data/human36m_UV_map_/h36m_0001/frame0001.png")
    uv_map = uv_map.astype(np.float) / 255.0
    resample_mesh = generator.resample(uv_map)

    device = torch.device('cuda')

    model = SMPLModel(
        device=device,
        model_path='./model_lsp.pkl',
        data_type=torch.float32
    )

    model.write_obj(resample_mesh, './test.obj')

and get the res like this:
image

@Amran2k16
Copy link
Author

@liwenssss Hi, what changes when you stop dividing your UV_map by 255? So
uv_map = uv_map.astype(np.float) this instead of uv_map = uv_map.astype(np.float) / 255.0.

@liwenssss
Copy link

@Amran2k16 Hi, I tried. nothing changed.

uv_map = imread("/home/liwensh/data/human36m_UV_map_/h36m_0001/frame0001.png")
    uv_map = uv_map.astype(np.float)
    print(uv_map.min(), uv_map.max())
    resample_mesh = generator.resample(uv_map)

    device = torch.device('cuda')

    model = SMPLModel(
        device=device,
        model_path='./model_lsp.pkl',
        data_type=torch.float32
    )

    model.write_obj(resample_mesh, './test.obj')

and got print info:
Find cached pickle file...
0.0 254.0
(7670, 2)
7670
Tensor J_regressor shape: torch.Size([24, 6890])
Tensor joint_regressor shape: torch.Size([6890, 14])
Tensor weights shape: torch.Size([6890, 24])
Tensor posedirs shape: torch.Size([6890, 3, 207])
Tensor v_template shape: torch.Size([6890, 3])
Tensor shapedirs shape: torch.Size([6890, 3, 10])

by the way , imread is from skimage.io

@Amran2k16
Copy link
Author

@liwenssss Sorry, not entirely sure what the issue is in that case. Maybe it might be the template issue, but i doubt it. Unfortunately I don't think i can send you the generate smpl_fbx_temate file due to the SMPL license.

As for generating it, these are the steps i took

  1. Import SMPL FBX file into blender (You will need to get this file from SMPL website)
  2. Export it as an obj file, using the triangulation options that can be selected (this way you will not need to use triangulation.py method, I myself did not use this but discovered it much later)

@Lotayou
Copy link
Owner

Lotayou commented Aug 23, 2019

@liwenssss This is quite odd. Seems your resampled mesh still looks like a person, so it's minor mistake... How about sending me an working example and I'll try it out on my machine?

@liwenssss
Copy link

liwenssss commented Aug 23, 2019 via email

@liwenssss
Copy link

liwenssss commented Aug 23, 2019 via email

@Lotayou
Copy link
Owner

Lotayou commented Aug 23, 2019

okay, good luck

@liwenssss
Copy link

hi, @Amran2k16 ,have you met the following phenomenon when you generate SMPL model:
image
as we can see some bulge around the left hand and the right feet.
if you met it and solve it ,bother to tell me how? or any advice can you give me ?

@Amran2k16
Copy link
Author

Hi, I believe this is an issue of UV MAP size. I am guessing this was reshaped from a 256 * 256 image. Instead try to do it with a 512 * 512 image size. Also did you find that triangulation.py was not a necessary step if it was done in blender already?

@tszhang97
Copy link

image
this is my resample result.Some tips:

  1. when normalize to [0,1], the result will be different from the groud truth
  2. may not use dilate, this will cause those bugle
  3. using bilinear interpolation rather than nearest integer. And find the nearest nonzero point if 4 nearing points all equal 0

@Amran2k16
Copy link
Author

@willie1997 How did you manage to create UV maps without normalizing to [0,1]? And which part of the process did you use bilinear interpolation in exactly?

@tszhang97
Copy link

normalization is a problem which I haven't figured out. If you want to remain the correct propotion, my solution is saving vmin and vmax.
vt*256 will calculate a result like [150.22, 223.15], you can use the value of its nearest four points to interpolation. @Amran2k16

@yongyct yongyct mentioned this issue Oct 5, 2019
Closed
@tszhang97
Copy link

Good news! I finally fix the bug and resample a very smooth smpl model. The tip is remove the vts equal zero in v_to_vt list and only use the good verts .
image

@gaizixuan0128
Copy link

@willie1997 Hello！Could you share your resample code？I have followed your tips，but still couldn‘t solve the bulge problem.

@electronicliujiang
Copy link

@liwenssss
hello!I don't have smpl_fbx_template file.so,I use the radvani's.this is my result,do you know how cai i do or where to find smpl_fbx_template file.I found this file is gone on the official website.
image

@PerhapsThat
Copy link

@liwenssss
hello!I don't have smpl_fbx_template file.so,I use the radvani's.this is my result,do you know how cai i do or where to find smpl_fbx_template file.I found this file is gone on the official website.
image

hi,have u solved the problem? I have encontered the same bug as yours
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@liwenssss @Amran2k16 @gaizixuan0128 @PerhapsThat @Lotayou @electronicliujiang @tszhang97