Use CozoDB with PyTorch

[chengzegang]

Hi, I am DL researcher and recently working on SLAM. We are going to use cozodb with PyTorch to store landmarks and frames, but it is quite annoying to persist tensors. To make use of the NN algorithm, we have to first turn tensors into Python lists and then store it into cozo. This has two problems: first, cast tensor into native list will blow up the memory since Python has no type, and, second, it lost gradient. So we stored a binary blob (for gradient) and a list (for vector indexing) to "workaround" the situation. Is there a more elegant way to achieve it?

[zh217]

Unfortunately there is currently no better way to do it. Ideally we could just pass in pytorch or numpy tensors into the parameters and PyCozo would recognize it as a vector, unfortunately this involves two language barriers (C/C++ -> Python -> Rust) and we don't want to depend on the pytorch or numpy C/C++ library. Maybe we could have a function that converts blobs to vectors in Cozo, at least to deal with the memory problem.

MAYBE there's some black magic that can be done to determine the object passed in is a numpy/pytorch tensor AND get its underlying bytes. If that's possible AND the procedure does not need any knowlegde about numpy/pytorch at compile time, then everything is solved. If anyone knows how to do it with PyO3 (which CozoDB uses to interop with Python) I'm glad to hear.

[chengzegang]

ok. The document said Cozo would be interoperable with PyG and Pytorch, and if that is achieved, I can tell there must be a way for us to encode everything into pytorch modules.

[infogulch]

zh217 have you heard of Apache Arrow memory format? Its designed to be a language agnostic schema to pass blobs of memory between languages with no copying or [de]serialization required. Might be useful for cozo considering the breadth of supported platforms.