iris concatenate_cube() issue

[wu841227]

Hello,

I am using iris to analyse a series of data files for the same parameter under the same spatial grids but for different years.

However, I encounter following error when I call concatenate_cube().
The script and data are also included to reproduce the error.

I believe this is due to inconsistency in the data files. But I don't know how to spot this.
Attached include a minimum demo.

Best, Lianghai

Archive.zip

---

Traceback (most recent call last):
File "/Users/lwu/iris_test.py", line 39, in
cubes_conc()
File "/Users/lwu/iris_test.py", line 34, in cubes_conc
print(clts.concatenate_cube())
File "/Users/lwu/mambaforge/lib/python3.10/site-packages/iris/cube.py", line 614, in concatenate_cube
raise iris.exceptions.ConcatenateError(msgs)
iris.exceptions.ConcatenateError: failed to concatenate into a single cube.
An unexpected problem prevented concatenation.
Expected only a single cube, found 2.

[acchamber]

Hi Lianghai

You're correct, this is a result of a inconsistency in the data files. Iris merging/concatenate is very picky about the input data files which can be an issue with malformed CMIP5 data. As a general rule, when iris is giving a very non-specific concatenation error like this, the cause is a difference in the coordinates you are trying to concatenate over (latitude & and longitude, in this case). So I had a look at the example data you provided.

cubel = iris.load(files)

cubel[0].coord("longitude")  == cubel[1].coord("longitude") == cubel[2].coord("longitude")
True

cubel[0].coord("latitude")  == cubel[1].coord("latitude") == cubel[2].coord("latitude")
False

So there's something wrong with the latitude coordinates. In particular

cubel[0].coord("latitude")  == cubel[1].coord("latitude")
True

cubel[1].coord("latitude") == cubel[2].coord("latitude")
False

cubel[0].coord("latitude") == cubel[2].coord("latitude")
False

There's a issue with the last cube in the list. If we try to concatenate the first two cubes, that works, confirming our suspicions. But given we probably need the data from the third cube and it should be identical, what is the problem?

print( cubel[1].coord("latitude").points - cubel[2].coord("latitude").points)
[ 0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00 -9.9475983e-14
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
 -9.9475983e-14  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00 -1.0658141e-14  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  1.0658141e-14  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  9.9475983e-14
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  9.9475983e-14  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00
  0.0000000e+00  0.0000000e+00  0.0000000e+00  0.0000000e+00]

So we can see here that for a a few points, there is a tiny (-9.9475983e-14, it' mostly likely just a floating point rounding error) difference in latitude points in the data. If you examine the bounds, you will find the same tiny differences. So how can we fix this?

Given that this is all model data from the same model on the same spatial grid, we can just workaround this by assigning the latitude points and bounds for one of the cubes to all of them. Here is a fix for your example.

for c in cubel:
     points = cubel[0].coord("latitude").points
     bounds = cubel[0].coord("latitude").bounds
     c.coord("latitude").points = points
     c.coord("latitude").bounds = bounds


cubel.concatenate_cube()
<iris 'Cube' of cloud_area_fraction / (%) (time: 36; latitude: 256; longitude: 512)>

[wu841227]

Hello Alex,

Thanks a lot for your reply! Now I understand where the issues is and how to fix it.
Thanks again!
Lianghai