Does rapid ocr supporting batch inference?

[krystalzeng]

Hello, trying to get a faster inference time. Been looking at the code and can't seem to see any support for batching. Am I missing something or is this not supported at the moment? Thanks!

[SWHL]

RapidOCR is mainly divided into three parts, one is text detection and the other is text recognition. Among them, the text detection part does not support batch inference, and the text recognition part used batch inference by defatult.

If you want faster inference, try using rapidocr_openvino.

[krystalzeng]

Thanks for the fast response! Appreciate it!

[krystalzeng]

@SWHL Sorry, another question. I have been trying rapidocr_openvino on CPU, it's somehow a lot slower than rapidocr_onnxruntime. I have a very high volume of images of millions, any pointers as to how I can improve the latency please? Have access to GPU too, so the solution is not limited to CPU

[SWHL]

I can think of two solutions: one is to open multiple processes, if resources are sufficient; the other is to use PaddleOCR, which will be faster using GPU inference.

[SWHL]

We recently encapsulated a library that uses GPU reasoning, rapidocr_paddle, based on the paddle inference engine. The inference speed has been improved by orders of magnitude. You can try it out.
For specific documentation, see: https://rapidai.github.io/RapidOCRDocs/docs/install_usage/rapidocr_paddle/