Use batch processing to migrate attachments

[fklein-lu]

The WP CLI migrate-attachments command uses a WP_Query with posts_per_page set to -1.

With a larger data set, this will lead to issues. We should investigate on how we could use batch processing for the migration.

[danielbachhuber]

You can use WP_CLI\Iterators\Query: https://github.com/wp-cli/wp-cli/blob/master/php/WP_CLI/Iterators/Query.php

[fklein-lu]

Awesome! 👍

[joehoyle]

@fklein-lu I've actually been considering removing migrate-attachments, really it's a lot more robust to use upload-directory and then run a find-replace via wp-cli. Perhaps we could repurpose migrate-attachments to be migrate-uploads which would just copy all the uploads to the S3 bucket location, then run a search-replace. Doing the attachments one-by-one in this command is problematic as it can miss files that might be stored in uploads but not directly referenced by attachment metadata.

[dev169]

@joehoyle That would make sense. As things are currently, is there any reason to prefer migrate-attachments over upload-directory?

[joehoyle]

@dev169 not really, I added the migrate-attachments command before the upload-directory one which is why it exists.