unified-engine: Respect directory hierarchy when outputting to a different directory

[NickSto]

I'm trying to wrap my head around unified-engine, and whether it can be used for what I'd like to do.

I'd like to take a directory full of many Markdown files in many subdirectories and write the output to a separate directory, maintaining the same directory tree, directory names, and filenames. Basically I'd like a copy of the original directory, but with certain modifications to the Markdown content.

I found that I can direct the output to a different directory using options.output, but when I do that the directory structure gets flattened and all files end up in the same directory. Is there a way to get it to maintain the structure?

Also, I was curious about whether there's a way to specify the output file extension? If I want to convert .md to .html that'd be useful.

[wooorm]

Q1: keep directory structure intact

Nope, not supported.
I think Gulp does some interesting things here (which was briefly discussed before here), but how should it work?

Some examples, with the following input files:

input/a/readme.md
input/b/readme.md
readme.md

$ ./node_modules/.bin/remark -u html -o output/ input/
$ ./node_modules/.bin/remark -u html -o output/ input/ readme.md
$ ./node_modules/.bin/remark -u html -o output/ input/{a,b}
$ ./node_modules/.bin/remark -u html -o output/ **/*.md

What should the structure of output/ be?

Q2: Extensions

Plugins can change files. In the above examples, remark-html is used, which sets extnames of .html, and hence the written files will have .html extensions

[wooorm]

Nah, I'm assuming you're posting pseudo code, I'm doing that too, but I guess my Q is: how can the code know 'input/' is the base directory to skip?

[NickSto]

Oh okay, I think I see what you're asking. Yeah, I'm not sure how to allow giving multiple paths to the files option when there's only one path given to output. I've only been thinking of the situation where you want to copy the contents of a single directory into another, single directory.

Maybe when it's operating in this mode (i.e. flatten: false), you only allow one path in the files array. Or you give the single path to a new option.

Either way, unified-engine gets one input path, and assumes that's the root directory whose contents you want to duplicate into the output directory.

[wooorm]

I’d rather not add a new option, that is so specific as to only allow a single input file. I’d have a hard time documenting and explaining it to users in a sensible way.

Assuming a path to a directory is given, the path to that directory could be stripped, and the rest remain. Same for a glob, like Gulp, the non-magic part starting it off could be stripped as well.
Still, it seems complex and needs some fiddling to test it out and see if it works

[janaka]

Hi,
I think what I'm needing to do is the same as @NickSto. I have a folder with a bunch of sub folders which contain MD files. I need to process each file and output as copy elsewhere maintaining the relative folder structure. So if the source was in relative for /a then the output is created in folder/a relative to output, folder created if doesn't exist. To add a bit more colour, my MD files are docs with code blocks in language A. I'm creating version of the same docs for language B and C.

original source docs

/orig/a/doc1.md
/orig/b/doc2.md
/orig/index.md

converted version for java

/java/a/doc1.md
/java/b/doc2.md
/java/index.md

files: ['/orig/'].
output: '/java/'

As mentioned above, it doesn't do that, which I was a little surprised given folder are integrated on the input side.

Given the state of things my only option is to not use unifed-engine and instead write the input/output code myself right?

[wooorm]

You can also call the cli several times?

This is still rather vague about how it would work, there are many edge cases to think through before this can be worked on