add import and extractArchive methods

[ETLaurent]

• rename export route to export-batch route
• remove "archive" notion
• extract and read from archive

Importing docs and attachments is still to be done.

[linear]

PRO-4271 As an editor, I can import Apostrophe documents in my site

As an editor, I am able to import documents, related ones and related attachments to another site. These are contained in a ZIP file generated from a documents export in another site (or the current one).

The ZIP file itself must be imported as it is to be sure everything is correctly imported and its content unchanged.

When the import button from the import modal is triggered, the zip file must be sent to the server through a single POST request for this iteration (so will be limited in size).

Read from ZIP file

The backend will unzip the file using the same lib used to generate it, We would prefer using the node native lib.

You might look a way to stream the content of the ZIP file to avoid having to load everything in memory before to read it. It should be ok now since we don't handle big loads of data, but in the future we will.

Attachments

It must import attachments and documents from aposAttachments first. There is a problem here, we want to keep the IDs of the aposAttachments documents to avoid having to rewrite apoDocs relationships.

The insert method of apostrophecms/attachments writes the file and create the associated aposAttachments document,

you might need to make some modifications here to be able to write a file and to insert an existing attachment, you need at least, to keep the ID.

If the attachment already exist (by ID) it must not be created, and the file not uploaded. We'll consider here that it has been imported in the past.

Documents

Then we need to import aposDocs documents, they must be imported following the order provided during the export. We will need to keep track of imported documents IDs. this way, if two documents have a related one in common (or a related one is imported too) it won't be imported twice or, at least, treated like a duplicated one.

We will also need to keep track of duplicated documents IDs, to ask later to the user if we must override them.

You must use the proper manager for each piece depending its type, and to use the insert method of the concerned module. If the document already exists we must stack it somewhere as explained above, it might work since data load won't be huge for now. An other solution would be to store the duplicated documents into the job directly but would involve more work and doesn't seem necessary right now.

the import process must be wrapped by the run method of the job module, it'll allow to manage the notification progress bar.

See this ticket for more information about the notification progress bar implementation.

Singletons

If the piece is from a module that has the option singleton: true (there is only one piece allowed eg. apostrophecms/global, apostrophecms-pro/palette) it overrides by default the existing one, without asking the user.

Pages

For this iteration we won't manage the download of all the exported page children. We still want to check the location of the page in the tree, if this one exists then we want to import the page at the same location (at least for the parent).

If the path doesn't exists, simply put the page as a Home page child.

Permissions

Since we will use the official insert/update Apostrophe methods, if they fail due to permissions the import should stop with an appropriate error.

Acceptance Criteria

• When clicking on the import button of the modal the process starts and I can see the notification progress bar.
• Attachments and documents are successfully imported with the same IDs and relations keep working.
• Duplicated documents must not be imported but kept in memory for the last step (asking the user for override).
• Pieces that are singletons automatically override the existing one (if there is one) during import.
• For pages, if the tree location exists we want to keep it, otherwise to locate the page as a home page child.

[ETLaurent]

Suggested change

	exportPath = await format.input(file.path, file.name);
	exportPath = await format.input(file.path);

[ETLaurent]

Suggested change

	const { EJSON } = require("bson");
	const { EJSON } = require('bson');