[FEATURE]Training in 3D - How to process 2D trainings data?

[klingann]

Dear Cellpose Community,

We have drawn 3D ROIs and now want to use them to train our own model in Cellpose.
Following the issue (#745) we want to extract XY, ZX, YZ slices from the 3D labelled data and want to use them for 2D training.
However, the question now arises as to how to annotate the specific slices as xy, xz, and yz, while considering the different aspect ratios (anisotropy of the 3D data).
Can Cellpose be instructed to identify xy, xz, and yz slices separately?
Is it necessary to resample (interpolate) xz and yz slices to have the same voxel size in all dimensions?
Alternatively, is it preferable to train using only xy slices?

Looking forward to your suggestions. Thanks you!

[AndreZeug]

I have the same problem. How to use 3D ROI labelled objects for training when Cellpose only allows 2D data for training and my xyz stacks have a particular anisotropy.
Providing xy, xz, yz slices for training require anisotropy information for the training as well, not only for predicting.
Is there some HowTo available?

[carsen-stringer]

You can train a separate model on XY and then on XZ/YZ, if you have different anisotropy, and training on all together is not working well. If you want to train one model on XY, XZ, and YZ, then yes you need to interpolate so the pixels/um are all the same.

[AndreZeug]

I understand that training XY, XZ, and YZ is not working well when having anisotropy. Interpolating XZ/YZ would be an workaround, but shapes look substantially different in XZ/YZ, compared to XY, thus the best option seems to train a separate model on XY and a second model on XZ/YZ. But how do I combine both models for prediction? The first predicting in XY and the second for the orthogonal views?
Or did I understand something wrong?

[pawlowska]

@AndreZeug I stumbled upon the exact same questions some months ago when attempting 3D training. Have you found a good solution?