Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

handle different locale #48

Merged
merged 19 commits into from
Nov 28, 2023
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,13 @@

## UNRELEASED

### Adds

* Import now detects if the locale found in the exported docs is different from the current site one.
If the site has only one locale configured, then the docs are automatically re-written with the site locale.
If the site has multiple locales configured, the user is given the possibility to abort the import or re-write the docs with the site locale.
Please note that this change is dependent on core changes found in 3.60.0.

### Changes

* Hide "Duplicates Detected." notification.
Expand Down
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,3 +139,11 @@ module.exports = {
label: 'Article',
pluralLabel: 'Articles'
```

## Importing documents from another locale

Exported documents maintain their locale settings. If the locale during import differs from the export locale, and only one locale is configured in the @apostrophecms/i18n module, the documents will be automatically rewritten to align with the new import locale.

If multiple locales are set up, the user will be prompted to choose between canceling the import or proceeding with it.

![Screenshot highlighting the confirm modal letting the user choose between aborting on continuing the import when the docs locale is different from the site one.](https://static.apostrophecms.com/apostrophecms/import-export/images/different-locale-modal.png)
2 changes: 2 additions & 0 deletions i18n/en.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@
"exporting": "Exporting {{ type }}...",
"import": "Import {{ type }}",
"importCleanFailed": "The cleaning of the imported file on server failed: {{ exportPath }}",
"importWithCurrentLocaleHeading": "Different locale detected",
"importWithCurrentLocaleDescription": "This file was exported from the \"{{ docsLocale }}\" locale. Are you sure you want to import it into the \"{{ currentLocale }}\" locale?",
"importDuplicateContinue": "Continue Import",
"importDuplicateDetected": "Duplicates Detected.",
"importDuplicateMessage": "Check the items you'd like to overwrite during import.",
Expand Down
Binary file added images/different-locale-modal.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
128 changes: 109 additions & 19 deletions lib/methods/import.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,65 @@ const { EJSON } = require('bson');
module.exports = self => {
return {
async import(req, moduleName) {
const {
docs, attachmentsInfo, exportPath, filePath
} = await self.readExportFile(req);
if (!req.user) {
throw self.apos.error('forbidden');
}

const overrideLocale = self.apos.launder.boolean(req.body.overrideLocale);
let exportPath = self.apos.launder.string(req.body.exportPath);

let docs;
let attachmentsInfo;

if (exportPath) {
const filesData = await self.getFilesData(exportPath);

({ docs, attachmentsInfo } = filesData);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what means the fact that this method receives an exportPath? Does it mean that the gzip file has already been cleaned?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The gzip file is now cleaned as soon as it's extracted (cf. comment below)

This method is provided exportPath when the user chooses Yes (continue the import with the current locale), because is archive has already been extracted during the import method first call.
It just early returned to display the confirmation modal.
So at the 2nd call, we simply give the method back the exportPath so it knows where the files were extracted during the 1st call

} else {
const exportFile = await self.readExportFile(req);

({ docs, attachmentsInfo } = exportFile);
exportPath = exportFile.exportPath;

await self.cleanFile(exportFile.filePath);
Copy link
Contributor Author

@ETLaurent ETLaurent Nov 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we now clean the archive right after having extracted it so we don't have to think about removing it at several other places

}

const differentDocsLocale = self.getFirstDifferentLocale(req, docs);
const siteHasMultipleLocales = Object.keys(self.apos.i18n.locales).length > 1;

if (differentDocsLocale) {
if (siteHasMultipleLocales && !overrideLocale) {
// Display confirmation modal to ask the user if he wants to
// continue the import with the current locale overriding the docs one:
await self.apos.notify(req, ' ', {
type: 'warning',
event: {
name: 'import-locale-differs',
data: {
moduleName,
exportPath,
content: {
heading: req.t('aposImportExport:importWithCurrentLocaleHeading'),
description: req.t('aposImportExport:importWithCurrentLocaleDescription', {
docsLocale: differentDocsLocale,
currentLocale: req.locale
}),
negativeLabel: 'apostrophe:no',
affirmativeLabel: 'apostrophe:yes'
}
}
},
classes: [ 'apos-notification--hidden' ]
});

return;
}

// Re-write if user decided to continue the import (`overrideLocale` param sent)
// or if the site has only one locale configured
self.rewriteDocsWithCurrentLocale(req, docs);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose this method is called twice, another time with the confirmation to change locale?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method is called only during the 2nd call (when the user chooses Yes), see the early return above

}

const total = docs.length + attachmentsInfo.length;
const {
reporting, jobId, notificationId
Expand All @@ -21,6 +73,7 @@ module.exports = self => {
const {
duplicatedDocs, duplicatedIds, failedIds
} = await self.insertDocs(req, docs, reporting);

const importedAttachments = await self.insertAttachments(req, {
attachmentsInfo,
reporting,
Expand Down Expand Up @@ -52,7 +105,6 @@ module.exports = self => {
});

await self.cleanFile(exportPath);
await self.cleanFile(filePath);
return;
}

Expand All @@ -68,6 +120,7 @@ module.exports = self => {
}

const results = {
overrideLocale,
duplicatedDocs,
importedAttachments,
type: moduleName,
Expand All @@ -87,8 +140,6 @@ module.exports = self => {
classes: [ 'apos-notification--hidden' ]
});

self.cleanFile(filePath);

return results;
},

Expand Down Expand Up @@ -180,6 +231,34 @@ module.exports = self => {
};
},

getFirstDifferentLocale(req, docs) {
const doc = docs
.find(doc => self.isLocaleDifferent(req, doc));

return doc && self.extractLocale(doc.aposLocale);
},

rewriteDocsWithCurrentLocale(req, docs) {
for (const doc of docs) {
if (self.isLocaleDifferent(req, doc)) {
const [ id ] = doc._id.split(':');

doc._id = [ id, req.locale, doc.aposMode ].join(':');
doc.aposLocale = [ req.locale, doc.aposMode ].join(':');
}
}
},

isLocaleDifferent(req, doc) {
return doc.aposLocale && self.extractLocale(doc.aposLocale) !== req.locale;
},

extractLocale(aposLocale) {
const [ locale ] = aposLocale.split(':');

return locale;
},

async insertDocs(req, docs, reporting) {
const duplicatedDocs = [];
const duplicatedIds = [];
Expand Down Expand Up @@ -342,25 +421,26 @@ module.exports = self => {
}
},

async cleanFile(exportPath) {
async cleanFile(path) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor change: cleanFile also cleans the archive path

try {
const stat = await fsp.lstat(exportPath);
const stat = await fsp.lstat(path);
if (stat.isDirectory()) {
await fsp.rm(exportPath, {
await fsp.rm(path, {
recursive: true,
force: true
});
} else {
await fsp.unlink(exportPath);
await fsp.unlink(path);
}
} catch (err) {
self.apos.util.error(
`Error while trying to remove the file or folder: ${exportPath}. You might want to remove it yourself.`
`Error while trying to remove the file or folder: ${path}. You might want to remove it yourself.`
);
}
},

async overrideDuplicates(req) {
const overrideLocale = self.apos.launder.boolean(req.body.overrideLocale);
const exportPath = self.apos.launder.string(req.body.exportPath);
const docIds = self.apos.launder.strings(req.body.docIds);
const jobId = self.apos.launder.string(req.body.jobId);
Expand All @@ -372,6 +452,16 @@ module.exports = self => {

const { docs, attachmentsInfo } = await self.getFilesData(exportPath, docIds);

const differentDocsLocale = self.getFirstDifferentLocale(req, docs);
const siteHasMultipleLocales = Object.keys(self.apos.i18n.locales).length > 1;

// Re-write locale if `overrideLocale` param is passed-on from the import process
// (i.e if the user chose "Yes")
// or re-write locale automatically on a single-locale site
if (differentDocsLocale && (!siteHasMultipleLocales || overrideLocale)) {
self.rewriteDocsWithCurrentLocale(req, docs);
}

Comment on lines +455 to +464
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we also need to override the locale here!

for (const doc of docs) {
try {
const attachmentsToOverride = self.getRelatedDocsFromSchema(req, {
Expand Down Expand Up @@ -428,15 +518,15 @@ module.exports = self => {
const jobId = self.apos.launder.string(req.body.jobId);
const notificationId = self.apos.launder.string(req.body.notificationId);

const jobManager = self.apos.modules['@apostrophecms/job'];
const job = await jobManager.db.findOne({ _id: jobId });

jobManager.end(job, true);
if (jobId) {
const jobManager = self.apos.modules['@apostrophecms/job'];
const job = await jobManager.db.findOne({ _id: jobId });
jobManager.end(job, true);
}
if (notificationId) {
self.apos.notification.dismiss(req, notificationId, 2000).catch(self.apos.util.error);
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clean job and notification only if existing

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see /api/v1/@apostrophecms/import-export/clean-export call in app.js below, no job nor notification is instantiated at this step, but we still want to remove the export path


self.apos.notification.dismiss(req, notificationId, 2000)
.catch((error) => {
self.apos.util.error(error);
});
await self.cleanFile(exportPath);
}
};
Expand Down
Loading