Skip to content

Images from an RGB-D camera are used to detect/classify objects in 2D, then detections are projected on the 3D point cloud.

License

Notifications You must be signed in to change notification settings

fvilmos/object_classification_2d_to_3d

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Object classification from 2D images to 3D point cloud

This approach uses the available methods for image object classification in 2D then projects the results based on RGB-D depth information to the 3D space. Object classification (in this case) is made with Cascade classifier, but easily can be changed with a Deep Neuronal Network.

Processig steps:

  1. RGB and Depth information is loaded

600

2. RGB image is applied to a Cascade classifier. Results are filtered with Non-Maximum-Suppression

500

3. Object coordinated are projected in 3D space using Kinect claibration parameters. Detection are plotted on the 3D Point Cloud

500

Note: for visualization purposes, the pyvista and itk viewer is used. Please follow the installation instructions: https://github.com/InsightSoftwareConsortium/itkwidgets

More details on the implementation, algorithms can be found here:

TODO

  • Optimize Classifier - change with i.e. Yolo or Mobilenet
  • with a good classifier optimize away the Non-Maximum-Suppression step

Any contribution is welcomed!

Resources

  1. Cascade_tools
  2. Non-Maximum-Suppression - cascade

/Enjoy.

About

Images from an RGB-D camera are used to detect/classify objects in 2D, then detections are projected on the 3D point cloud.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published