How to construct the AST programatically?

[tad-lispy]

Hi! Thanks for working on Comrak and sharing it with us.

I'm new to Rust and I'm having trouble figuring out how to manipulate the Comrak AST. Here's a snippet of code that doesn't compile:

fn item_to_markdown<'a>(item: &'a AstNode<'a>) -> String {
    // construct the new document from item's children
    let root  = AstNode::from(NodeValue::Document);
    for child in item.children() {
        root.append(child);
    }

    // format the newly created document as string
    let mut markdown = vec![];
    comrak::format_commonmark(&root, &ComrakOptions::default(), &mut markdown)
        .expect("Formatting of the item as markdown failed");
    let description =
        String::from_utf8(markdown).expect("Converting the markdown buffer to a String failed");
    description
}

The intention of this function is to get a single list item node, extract all it's content and return a string with this content as top-level blocks. For example from:

- First **paragraph**

  Second paragraph.

to

First **paragraph**

Second paragraph.

The error I'm having is the following.

$ cargo run
   Compiling markdown-to-icalendar-export v0.1.0 (/home/tad/Projects/markdown-to-ical-export)
error[E0597]: `root` does not live long enough
   --> src/main.rs:148:9
    |
145 | fn item_to_markdown<'a>(item: &'a AstNode<'a>) -> String {
    |                     -- lifetime `'a` defined here
...
148 |         root.append(&child);
    |         ^^^^^^^^^^^^^^^^^^^
    |         |
    |         borrowed value does not live long enough
    |         argument requires that `root` is borrowed for `'a`
...
156 | }
    | - `root` dropped here while still borrowed

error[E0597]: `root` does not live long enough
   --> src/main.rs:151:31
    |
145 | fn item_to_markdown<'a>(item: &'a AstNode<'a>) -> String {
    |                     -- lifetime `'a` defined here
...
151 |     comrak::format_commonmark(&root, &ComrakOptions::default(), &mut markdown)
    |     --------------------------^^^^^-------------------------------------------
    |     |                         |
    |     |                         borrowed value does not live long enough
    |     argument requires that `root` is borrowed for `'a`
...
156 | }
    | - `root` dropped here while still borrowed

For more information about this error, try `rustc --explain E0597`.
error: could not compile `markdown-to-icalendar-export` due to 2 previous errors

It seems like I am using the language wrong - something about lifecycles, ownership and borrowing. I'm still wrapping my head around those concepts. But I have a suspicion that I'm also doing something wrong with comrak. I mean: the code should be structured differently to do what I need.

I think it's somewhat related to #186 but I'm not asking for any API changes. Just an advice how to use the existing API.

So my question is "How to construct the AST programatically?"

[emschwartz]

Before you try to append one node to another, I think you need to allocate space for them in the arena. Something like:

fn item_to_markdown<'a>(arena: &'a Arena<AstNode<'a>>, item: &'a AstNode<'a>) -> String {
    // construct the new document from item's children
    let root = arena.alloc(AstNode::from(NodeValue::Document));
    for child in item.children() {
        root.append(child);
    }

    // format the newly created document as string
    let mut markdown = vec![];
    comrak::format_commonmark(&root, &ComrakOptions::default(), &mut markdown)
        .expect("Formatting of the item as markdown failed");
    let description =
        String::from_utf8(markdown).expect("Converting the markdown buffer to a String failed");
    description
}

[tad-lispy]

Got it! Thank you @emschwartz.

For future reference, to get it to work I needed to remove the ampersand before the root argument to comrak::format_commonmark, like this:

     // format the newly created document as string
     let mut markdown = vec![];
-    comrak::format_commonmark(&root, &ComrakOptions::default(), &mut markdown)
+    comrak::format_commonmark(root, &ComrakOptions::default(), &mut markdown)
         .expect("Formatting of the item as markdown failed");
     let description =
         String::from_utf8(markdown).expect("Converting the markdown buffer to a String failed");

[tad-lispy]

A follow up question if I may. Is there a way to avoid passing the reference to arena from the main function where it's created all the way down to the item_to_markdown? There are several functions being called in between and now I'm passing an extra argument to each of them. It's not a huge deal, but would be nice to avoid.

I thought to create a new Arena in the scope of item_to_markdown, like this:

fn item_to_markdown<'a>(item: &'a AstNode<'a>) -> String {
    let arena = Arena::new();
    let root = arena.alloc(AstNode::from(NodeValue::Document));
    for child in item.children() {
        root.append(child);
    };
    // ...
}

But then I'm getting the following error:

`arena` does not live long enough

borrowed value does not live long enough
argument requires that `arena` is borrowed for `'a`

I badly struggle with memory related stuff in Rust. I guess I'm a spoiled child of garbage collectors 🤣 Please educate me.

[emschwartz]

One option would be to create a struct that wraps the arena and has the same lifetime as the arena and then define all of your functions as methods on that struct.

pub struct Converter<'a> {
    arena: &'a Arena<AstNode<'a>>,
    root: &'a AstNode<'a>,
}

impl<'a> Converter<'a> {
  fn item_to_markdown<'a>(&self, item: &'a AstNode<'a>) -> String {
    ...
  }
}