Vertical text detection and extraction?

[mllife]

I am working on extracting text, but the framework is unable to handle vertical text. For example -

Can someone help, if I am missing something?

[mllife]

@fg-mindee , please help.

[felixdittrich92]

Hi @mllife 👋,

You need to pass assume_straight_pages=False to the ocr_predictor instance :)
I suggest to use the main branch instead of v0.8.1 and fast_tiny or fast_base as detection model (fast_base will be published today so at the moment of this answer only fast_tiny is pretrained available) - only with pytorch as backend available TF checkpoints will follow in the next weeks

[felixdittrich92]

CC @odulcy-mindee the output is from fast_base if we can reproduce the quality with TF we could change the default det arch to fast which works in general more robust (also with rotations), needs less memory and is a bit faster compared to db_resnet50 (the current default)

[mllife]

Thanks @felixdittrich92 for your immediate response.I am able to get the "python-doctr-0.9.0a0" from the main branch, and "fast_base" works perfectly with the detection part, but the recognition is still bad on rotated text. I am using reco_arch = "parseq".
See from the image for reconstructed text. Are there any immediate plans to update recognition models as well for indentifying the rotated text? I am also very much impressed DocumentStructure class representation for the final result, but it is also lacking orientation attribute. Is that something that is doable in the upcoming release.

[mllife]

Will the orientation attribute be available with version 0.9 in the DocumentStructure output?

[felixdittrich92]

It is already you need only to add detect_orientation=True to the predictor :)
But i have faiced that the fast models are the most robust for rotations atm compared to db and linknet and the orientation angle detection depends on the detection model returned bin map

[mllife]

Yes, but "detect_orientation=True" only gives page orientation. I am talking about the individual word bboxes and their orientation, which makes more sense.
For example:
'words': [{'value': 'Total', 'confidence': 0.9999986886978149, 'geometry': ((0.18405330181121826, 0.0888671875), (0.18405330181121826, 0.0693359375), (0.19416359066963196, 0.0693359375), (0.19416359066963196, 0.0888671875)), 'orientation': -90}}

[mllife]

Can you add information about the 4 tuples now in geometry, it was just 2 in DocumentStructure ((x_min, y_min), (x_max, y_max))?

[felixdittrich92]

Yeah that could be indeed a great addition :)

Yep with assume-straight_pages=False you get polygons ((x1, y1), (x2, y2), (x3, y3), (x4, y4)) back

for page in result.pages:
        h, w = page.dimensions

        for block in page.blocks:
            for line in block.lines:
                for word in line.words:
                    coords = word.geometry
                    if len(coords) == 4:
                        x_coords = [int(point[0] * w) for point in coords]
                        y_coords = [int(point[1] * h) for point in coords]

                        x1, x2, x3, x4, y1, y2, y3, y4 = x_coords + y_coords

[mllife]

Thanks, @felixdittrich92. I hope the text/word orientation can be added in the upcoming version as it will help a lot of people.

[felixT2K]

Hi @mllife short question about the "word orientation".
What we could include is the general orientation of the text box ([0, 90, 180, 270] from the crop orientation classifier) is this what you mean or do you mean the orientation of the word in the crop ?

[mllife]

yes, as mentioned in the image example image
the text total is rotated by 90 with respect to the page, if it can be added into the document structure with each word it will be really helpful.
@felixT2K , thanks for looking into it. waiting for the next update to include this feature.

[Muaz08]

@felixdittrich92 I hope you are doing well. I wonder is there any way we could get the response of the doctr in the csv, excel and html that retains the same table structure as it was in the image. Thanks.