Support Apple's "Vision" framework for on-device OCR text recognition

[jasongitmail]

Is your feature request related to a problem? Please describe.

I'd like to be able to use Apple's excellent, on-device OCR text recognition "vision" framework, available since iOS 13. (For clarity, this is not for VisionOS head set.)

It offers high accuracy and speed for OCR, due to being on-device, using machine learning for accuracy, and optimized by Apple in recent years.

Describe the solution you'd like

An API in TypeScript to provide an image and get a result of both text and positions of the text within the image, using Apple's Vision OCR APIs.

Describe alternatives you've considered

• https://market.nativescript.org/plugins/nativescript-ocr/ - 3rd party, quite old, OCR tech has improved greatly in recent years.
• React vision camera has this available via transformers, but it's React Native and the transformers are React hooks.

Anything else?

• Apple Vision Framework: https://developer.apple.com/documentation/vision/recognizing_text_in_images
• Succinct blog post explainer: https://nemecek.be/blog/38/how-to-implement-ocr-with-vision-framework-in-ios-13

Please accept these terms

• I have searched the existing issues as well as StackOverflow and this has not been posted before
• I agree to follow this project's Code of Conduct

[NathanWalker]

Completely agree. We have used the vision kit with apps before and it’s quite nice. RN vision camera from Margelo is likely possible to be used with NativeScript as Marc mentioned its libraries were recently decoupled from react; we want to try sometime early this year.

[farfromrefug]

this is already possible in N using native iOS APIs through JS? I am not sure I understand what more needs to be added to N

[NathanWalker]

I believe @jasongitmail is just mentioning a condensed API to make it a bit more palatable. We have some examples of vision kit usage in NativeScript apps I’ll have to share sometime.

[farfromrefug]

@NathanWalker oh ok. I will switch my app to using it too (using tesseract right now) so will have another real time example at that point.
Should we move this ticket to discussions?