Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Tools - Captioning, Image Processing, Generic OD & Seg #157

Merged
merged 10 commits into from
Jul 1, 2024

Conversation

shankar-vision-eng
Copy link
Collaborator

@shankar-vision-eng shankar-vision-eng commented Jun 27, 2024

This PR includes the following features

  • Adds capability to extract_frames to extract frames from youtube videos
  • Adds Florancev2 image captioning in two modes - Brief and Detailed.
  • Adds Generic OD and Segmentation tool to detect common objects with text prompts
  • Adds Florancev2 based image question answering with context
  • Adds following image generation/processing tools
    • Image to Depth - Use Depth Anything V2
    • Image to Pose - Generates Open pose stick image which can interpreted by Open pose libs
    • Image to Soft Edge Image - Generates HED or Soft edge image of object boundaries
    • Image to Normal - Generates Normal image where RGB values represent XYZ surface normals
  • Adds template matching tool to find cropped ROI in original image

Copy link
Member

@dillonalaird dillonalaird left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR looks good, but can you use the model names in to tool function names? For example image_question_answering_with_context and the generic tools

@shankar-vision-eng
Copy link
Collaborator Author

PR looks good, but can you use the model names in to tool function names? For example image_question_answering_with_context and the generic tools

Renamed tools with actual model names

Copy link
Member

@dillonalaird dillonalaird left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@shankar-vision-eng shankar-vision-eng merged commit 1305146 into main Jul 1, 2024
8 checks passed
@shankar-vision-eng shankar-vision-eng deleted the tools_upgrade_6/26 branch July 1, 2024 18:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants