Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clicking on a training camera should (ideally) snap to the exact position of this camera #3094

Open
andynewman10 opened this issue Apr 19, 2024 · 0 comments

Comments

@andynewman10
Copy link

Is your feature request related to a problem? Please describe.

I need to verify the validity of the rendering performed from a given pose. More precisely, I need to compare the result of the neural rendering with the image of a training camera so as to verify that the neural rendering performed from a training pose indeed matches the corresponding training image. I of course don't expect a pixel-perfect match, my goal is to check that there is no "pose mismatch", in other words, that the rendering generated at that pose matches the pose of the training image.

Steps to describe the procedure I want to follow

  • install the latest version of Nerfstudio
  • use one of the popular datasets, eg. ns-download-data nerfstudio --capture-name=sculpture
  • train the model and let the train process make some progress, just a minute will be sufficient. I use splatfacto for rendering.
  • open your browser, and click one of the known camera poses. The image matching the camera pose in the dataset is displayed "full screen", but the exact camera position does not seem to match the position of the displayed view frustrum on screen. You can see that by clicking the Hide train cams button and back Show train cams again : the renderings should be identical, shouldn't they? When you click this button, I expect the rendering to actually match what I was seeing before clicking the button. Doing so would somehow prove that the neural rendering actually matches this training image from this training pose.

Describe the solution you'd like

When you click a camera, it would be ideal to reach exactly the position of the training camera pose.

Describe alternatives you've considered

Currently I am:

  • clicking on a camera
  • use mouse wheel and roll the wheel button towards me once to go slightly backwards. Unfortunately, the produced rendering from this new position does not seem to completely match the training image. It's a close match, but not perfect.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant