Reconstruct parsing tree based on different grammars

[marwern]

Hi, today I tried to understand the 'Python 3 to Python 2 converter' and TemplateConf, TemplateTranslator.

I want to reconstruct the source code based on a given parsing tree. This works so far. The question that is bothering me is how to translate the tree of one grammar to the tree of another grammar.

Is there any way to cast a string into a 'tree datatype'?
I basically just want to replace a few nodes + tokens to match the other grammar.

Thank you!

[MegaIng]

What do you mean with 'tree datatype'? The Tree.data attributes are just normal strings.

[marwern]

For example, let's imagine that lark returns me a tree1: 'Tree(Token('RULE', 'start'), [Tree(Token('RULE', 'rule_grammar1'), ... )'

I would like to translate it into a 'tree2': 'Tree(Token('RULE', 'start'), [Tree(Token('RULE', 'rule_grammar2'), ... )' to be able to reconstruct the source code based on grammar2.

[erezsh]

how to translate the tree of one grammar to the tree of another grammar.

That's what Transformers are for. But it's your responsibility to make sure the resulting tree has the exact right structure.

[marwern]

Thank you for your detailed answer!

I think my problem is to understand how I have to define/configure 'translations_3to2' in detail or to be more precisely, how to use '$'.

[erezsh]

The templates can help make it easier, but they can't describe every transformation you might need. So keep in mind you may have to change the tree yourself, either with a transformer, or even manually by assigning to data/children.

[marwern]

"or even manually by assigning to data/children". I would like to try this and have a simple scenario in mind:

Source Code:

Hello,World

grammar1:

	start: a "," b
	a: WORD
	b: WORD
	%import common.WORD
	%ignore " "

tree1:

	Tree(Token('RULE', 'start'), [Tree(Token('RULE', 'a'), [Token('WORD', 'Hello')]), Tree(Token('RULE', 'b'), [Token('WORD', 'World')])])

tree2:

	Tree(Token('RULE', 'start'), [Tree(Token('RULE', 'b'), [Token('WORD', 'Hello')]), Tree(Token('RULE', 'a'), [Token('WORD', 'World')])])	

grammar2:

	start: b "," a
	a: WORD
	b: WORD
	%import common.WORD
	%ignore " "

Target Code:

	Hello,World

Assuming I want to translate 'tree1' to 'tree2' and not use a transformer (although it should be easy in this case).

How can I generally construct 'tree2' using a manual assignment of data/children?

Tree(Token('RULE', 'b'), [Token('WORD', 'World')]) ---> Tree(Token('RULE', 'a'), [Token('WORD', 'World')])
Tree(Token('RULE', 'a'), [Token('WORD', 'Hello')]) ---> Tree(Token('RULE', 'b'), [Token('WORD', 'Hello')])

I can find the 'append' function in some files and documentation, but no example that recreates a tree from scratch (tree2 in this case).

Which commands can I use for that?

[erezsh]

It's fairly basic Python stuff. Just for example:

from lark import Tree

t = Tree('a', ['world'])
print(t)
t.data = 'b'
t.children[:0] = ['hello']
print(t)

[marwern]

Hi @erezsh , sry for my late reply. Thank you very much! This simple example is helpful for me.