-
Notifications
You must be signed in to change notification settings - Fork 376
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
KeyError: "BEVFusion: 'encoders.camera.backbone.stages.0.blocks.0.attn.w_msa.relative_position_bias_table'" #597
Comments
Pls did you solve it, I also had the same problem |
I faced the same problem, when trying to use the new saved checkpoints from training a cam-only centerhead detector. Let me be specific.
torchpack dist-run -np 1 python tools/train.py
torchpack dist-run -np 1 python tools/train.py \ configs/nuscenes/det/transfusion/secfpn/camera+lidar/swint_v0p075/convfuser.yaml And the errors are as follow: Traceback (most recent call last): |
The model you trained in (1) means you obtain a single-modality model which just uses the camera images to do the object detection. That means you cannot use it as the camera backbone to train the fusion model. |
Hi fdy61, I notice the same "pretained/swint-nuimages-pretrained.pth" was used as a checkpoint for training the cam-only detector, and also as a checkpoint to train the cam+lidar detector. This is why I was of the impression that the saved cam-only checkpoints "runs/run-531bf67d-d3138be2/epoch_20.pth" will be usable to train the cam+lidar detector. May I have your kind advise, how then can I train an appropriate camera backbone to train the cam+lidar fusion model? Thanks |
pretained/swint-nuimages-pretrained.pth is just a image backbone model, if I remember correctly,which has the same structure compared with SwinTransformer. But runs/run-531bf67d-d3138be2/epoch_20.pth is an entire model, and its' weights parameter have been changed completely. you can print out the model weights name to see. |
Hi fdy61, Thanks for your tips. Indeed when I "ls -l" the 2 files, they are very different:
I am not sure how else can I print examine their "model weight name". Can you kindly advise me? So the question then becomes how do I re-train the cam+lidar fusion model, using my new image data? I believe you may have the same goal, when you mentioned in your first post "I use my own lidar-only and camera-only pth to train the fusion model....". You also have the camera-only pth checkpoint? Did you manage to solve it? If so, can you advise me please? Thanks |
Can I do the following instead: torchpack dist-run -np 8 That is, have 2 "--load_from", loading both the cam-only checkpoint and the lidar-only checkpoint. Thanks |
torchpack dist-run -np 8 |
After some debugging, it seems that the issue is that the state_dict keys for
differ from that of
by the prefix "encoders.camera.backbone". I am hoping that if I change the key to skip this prefix, then this error will go away! |
You just use the pretrained/swint-nuimages-pretrained.pth and it's done. |
Hi fdy61, Hence, my thinking is to use this "epoch_20.pth" checkpoint to train the cam+lidar fusion model. OK, I will try to check/confirm if the state_dict keys of "epoch_20.pth" are totally different from the "camera-only-det.pth", or if they only differ by the prefix "encoders.camera.backbone". I will post the update here. Thanks |
I use my own lidar-only and camera-only pth to train the fusion model,and encountered this problem,How can I solve it?
The text was updated successfully, but these errors were encountered: