[HLP] Weird table extraction

[Kochise]

Hi, trying to extract data from :

https://www.se.com/uk/en/product/RM17JC00MW/modular-current-control-relay-8-a-1-co-overcurrent-function-250-v-ac-dc/
-> https://www.se.com/uk/en/product/download-pdf/RM17JC00MW?filename=Schneider+Electric_Harmony-Control-Relays_RM17JC00MW.pdf

⚠️ Tables don't have vertical lines.

Settings

import sys
sys.path.insert(0, "..")

import pdfplumber

G_TABLE_SETTINGS	= {
	"vertical_strategy": "text",
	"horizontal_strategy": "lines",
#	"explicit_vertical_lines": [],
#	"explicit_horizontal_lines": [],
#	"snap_tolerance": 3,
#	"snap_x_tolerance": 3,
#	"snap_y_tolerance": 3,
#	"join_tolerance": 3,
#	"join_x_tolerance": 3,
#	"join_y_tolerance": 3,
#	"edge_min_length": 3,
#	"min_words_vertical": 3,		# Change behavior at 4
#	"min_words_horizontal": 1,
#	"intersection_tolerance": 3,
#	"intersection_x_tolerance": 3,
#	"intersection_y_tolerance": 3,
#	"text_tolerance": 3,
#	"text_x_tolerance": 3,
#	"text_y_tolerance": 3,
#	"text_*": …, # See below
}

str_file = "Schneider Electric_Harmony-Control-Relays_RM17JC00MW.pdf"

with pdfplumber.open(str_file) as obj_pdf:
	obj_page = obj_pdf.pages[0]

	str_text = obj_page.extract_text()
	lst_text = str_text.split('\n')
	lst_table = obj_page.extract_table(table_settings = G_TABLE_SETTINGS)

	img_page = obj_page.to_image(resolution = 150, antialias = True)
	img_page.reset().debug_tablefinder(G_TABLE_SETTINGS)
	img_page.show()

	print(f"lst_text({obj_page.page_number})={lst_text}")
	print(f"")
	print(f"lst_table({obj_page.page_number})={lst_table}")

First page

⚠️ Notice the vertical line on the right border.

Console

lst_text(1)=['Product datasheet', 'Specifications', 'Modular current control relay, 8 A,', '1 CO, overcurrent function, 250 V', 'AC/DC', 'Local distributor code:', '389834764 RM17JC00MW', 'EAN Code: 3389119405034', 'Main', 'Range Of Product Harmony Control Relays', 'Relay Type Current control relay', 'Product Or Component Type Current control relay', 'Relay Name RM17JC', 'Relay Monitored Parameters Overcurrent detection', 'Time Delay Without', 'Switching Capacity In Va 1250 VA', 'Minimum Switching Current 10 mA at 5 V DC', 'Maximum Power Consumption In 3 VA', 'Va', 'Measurement Range 2...20 A AC', 'Utilisation Category AC-12 conforming to IEC 60947-5-1', 'AC-13 conforming to IEC 60947-5-1', 'AC-14 conforming to IEC 60947-5-1', 'AC-15 conforming to IEC 60947-5-1', 'DC-12 conforming to IEC 60947-5-1', 'DC-13 conforming to IEC 60947-5-1', 'DC-14 conforming to IEC 60947-5-1', 'Contacts Type And Composition 1 C/O', 'Complementary', 'Maximum Switching Voltage 250 V AC/DC', '[Us] Rated Supply Voltage AC/DC', 'Supply Voltage Limits 20.4…264 V AC/DC', 'Operating Voltage Tolerance - 15 % + 10 % Un', 'Maximum Power Consumption In 1 W', 'W', 'Control Circuit Frequency 40...70 Hz sinusoidal', 'Output Contacts 1 C/O', 'Nominal Output Current 5 A', 'Maximum Measuring Cycle 30 ms measurement cycle as true rms value', 'Hysteresis 15 % fixed of threshold setting', 'Delay At Power Up 0.5 s', 'Measurement Accuracy +/- 10 % of the full scale value', 'Repeat Accuracy +/- 0.5 % for input and measurement circuit', '16 Apr 2024 1', 'snoitacilppa', 'resu', 'cificeps', 'rof', 'stcudorp', 'eseht', 'fo', 'ytilibailer', 'ro', 'ytilibatius', 'gninimreted', 'rof', 'desu', 'eb', 'ot', 'ton', 'si', 'dna', 'rof', 'etutitsbus', 'a', 'sa', 'dednetni', 'ton', 'si', 'noitatnemucod', 'sihT', ':remialcsiD']

⚠️ Some weird output at the end, maybe the vertical line on the right border.

lst_table(1)=[[None, None, 'Local distributor code:\n389834764 RM17JC00MW'], [None, None, 'EAN Code: 3389119405034'], ['Range Of Product', 'Harmony Control', 'Relays'], ['Relay Type', 'Current control rel', 'ay'], ['Product Or Component Type', 'Current control rel', 'ay'], ['Relay Name', 'RM17JC', ''], ['Relay Monitored Parameters', 'Overcurrent detect', 'ion'], ['Time Delay', 'Without', ''], ['Switching Capacity In Va', '1250 VA', ''], ['Minimum Switching Current', '10 mA at 5 V DC', ''], ['Maximum Power Consumption In\nVa', '3 VA', ''], ['Measurement Range', '2...20 A AC', ''], ['Utilisation Category', 'AC-12 conforming\nAC-13 conforming\nAC-14 conforming\nAC-15 conforming\nDC-12 conforming\nDC-13 conforming\nDC-14 conforming', 'to IEC 60947-5-1\nto IEC 60947-5-1\nto IEC 60947-5-1\nto IEC 60947-5-1\nto IEC 60947-5-1\nto IEC 60947-5-1\nto IEC 60947-5-1'], ['Contacts Type And Composition', '1 C/O', ''], ['Complementary', '', ''], ['Maximum Switching Voltage', '250 V AC/DC', ''], ['[Us] Rated Supply Voltage', 'AC/DC', ''], ['Supply Voltage Limits', '20.4…264 V AC/D', 'C'], ['Operating Voltage Tolerance', '- 15 % + 10 % Un', ''], ['Maximum Power Consumption In\nW', '1 W', ''], ['Control Circuit Frequency', '40...70 Hz sinusoi', 'dal'], ['Output Contacts', '1 C/O', ''], ['Nominal Output Current', '5 A', ''], ['Maximum Measuring Cycle', '30 ms measureme', 'nt cycle as true rms value'], ['Hysteresis', '15 % fixed of thres', 'hold setting'], ['Delay At Power Up', '0.5 s', ''], ['Measurement Accuracy', '+/- 10 % of the full', 'scale value'], ['Repeat Accuracy', '+/- 0.5 % for input', 'and measurement circuit']]

Expected

If possible, get 2 tables with 2 columns :

Main and Complementary

Perhaps also the header with :

['Modular current control relay, 8 A, 1 CO, overcurrent function, 250 V AC/DC', 'Local distributor code:', '389834764 RM17JC00MW', 'EAN Code: 3389119405034'],

Regards.

[Kochise]

Maybe have a look at these projects to be able to detect and create sub groups :

https://github.com/MulongXie/UIED
https://github.com/MulongXie/GUI-Perceptual-Grouping

Regards.

[jsvine]

Thanks for sharing those resources with the community. Although perceptual grouping is outside of pdfplumber's intended scope, these links may serve as inspiration to others.

[jsvine]

Thanks for showing this example. Given the particular structure of that product sheet, some custom logic — taking advantage of the particular structural elements on the page — might work better than .extract_table(...), given that the layout is somewhat different from a traditional table.