Exclude / filter text segments

[Pique7]

Is it possible to exclude / filter text segments?

I managed to exclude single characters using page.filter() as described here: #89 .
For example, I have following text:

UPPER CASE: REMOVE! 
lower case: keep! 

With page.filter() excluding all upper-case letters I get (of course):

 : ! 
lower case: keep! 

But I want this result:

lower case: keep! 

Is this doable?

[jsvine]

The .filter(...) method operates only on individual objects (chars, rects, etc.). To achieve the goal described here, regular expressions might be a better fit. E.g.,:

import re
text = "UPPER CASE: REMOVE!\nlower case: keep!"
print(re.sub(r"\b[A-Z]+[!:]?", "", text))

... returns:

  
lower case: keep!

[Pique7]

Thanks for your advice to use regular expressions. I think this is suitable (although I presented a simplified example).
Actually I would have preferred all unwanted text to be totally ignored by the entire table recognition process. That's why I initially asked this question. Meanwhile I think that there is no easy way to achieve this.

[EdmundsEcho]

@jsvine Does the table analysis work on char-level or word-level input? I ask because the keep_blank_chars option in the extract_words method on page maintains a good amount of layout information that I imagine is useful for identifying table layouts (in my experience it maintains table "cell" information). If the table method uses extract_words perhaps there is value in having a filter at the word level... that then feeds the table analysis?

[jsvine]

When using the "horizontal/vertical_strategy": "text" setting, the table analysis uses word-level information:

pdfplumber/pdfplumber/table.py

Lines 618 to 619 in c562774

	if v_strat == "text" or h_strat == "text":
	words = self.page.extract_words(**(settings.text_settings or {}))