fix: widen ravel_multi_index to int64 on overflow#16
Open
andrinr wants to merge 1 commit into
Open
Conversation
NumpyBackend.ravel_multi_index unconditionally cast the linearised
result to the caller's multi_index.dtype. When that dtype was int32
and the shape product exceeded 2**31, the cast silently wrapped to
negative and downstream callers (CompressedSparseMatrix.compress in
phiml/math/_sparse.py) surfaced it as
ValueError: index <negative> is out of bounds for array with size <product>
from np.unravel_index.
Preserve the caller's dtype when it fits, but widen to int64 when the
linear index space exceeds np.iinfo(dtype).max. Mirrors the JAX
backend's behaviour, which never downcasts.
Closes tum-pbs#15.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
NumpyBackend.ravel_multi_indexunconditionally cast the linearised result to the caller'smulti_index.dtype. When that dtype wasint32and the shape product exceeded2**31, the cast silently wrapped to negative. Downstream callers — primarilyCompressedSparseMatrix.compressinphiml/math/_sparse.py— then surface it as:from
np.unravel_indexon the next line.Fix: preserve the caller's dtype when it fits, widen to
int64whennp.prod(shape) > np.iinfo(dtype).max. The JAX backend already doesn't downcast, so the patch brings the numpy backend in line.Reproducer
In real use this surfaces in
phi.physics.fluid.make_incompressibleon a 3-D Navier-Stokes grid as soon asc_dims.volume * u_dims.volume > 2**31— for anN³velocity-on-faces problem, that's aroundN ≥ 48.What changed
phiml/backend/_numpy_backend.py:362-378— drop the unconditional.astype(multi_index.dtype), pickint64when overflow would occur.tests/commit/backend/test__backend.py— newtest_ravel_multi_index_int32_overflowregression case (large-shape round-trip + small-shape dtype preservation).Testing done
tests/commit/backend/test__backend.py(43 cases) andtests/commit/math/test__sparse.pypass with the patch. One pre-existing failure ontest_convert(JAX/DLPack GPU issue, unrelated to this PR) was confirmed to also fail onmain.Closes #15.