Large JP2 Read Error

[guymac]

I've converted some large TIFFs to lossless JP2 (using gdal_translate), the TIFFs could be read with ImageReader, but the converted JP2s (same pixel dimension, 66680x66680) cannot. I'm using opi.formats-gpl 7.3.1.

java.lang.NegativeArraySizeException
at ome.codecs.JPEG2000Codec.decompress(JPEG2000Codec.java:261)
at loci.formats.codec.WrappedCodec.decompress(WrappedCodec.java:132)
at loci.formats.codec.JPEG2000Codec.decompress(JPEG2000Codec.java:63)
at loci.formats.in.JPEG2000Reader.openBytes(JPEG2000Reader.java:182)
at loci.formats.FormatReader.openBytes(FormatReader.java:922)
at loci.formats.ImageReader.openBytes(ImageReader.java:450)
at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:336)
at loci.formats.gui.BufferedImageReader.openImage(BufferedImageReader.java:86)

[scuniff]

I’m fairly new to Bioformats and was interested in this issue. I did some research and found the following that may be related.

I found this Bioformats link regarding NegativeArraySize error: Reporting a bug — Bio-Formats 7.2.0 documentation

Here is a screenshot:

Mentions 2Gb of data being some sort of maximum…..

A lot of info, I need to keep researching/learning.😎

Also, you mentioned you used ImageReader. What exactly is that? BioFormats?, ImageJ?,Fiji?, etc?

[guymac]

Here's some example code:

`
loci.formats.ImageReader imr = new loci.formats.ImageReader();
imr.setId(image);

    BufferedImageReader reader = BufferedImageReader.makeBufferedImageReader(imr);

`

I just tried this again (on a 47420 x 47420 jp2), and got a slightly different error:

java.lang.IllegalArgumentException: Dimensions (width=47420 height=47420) are too large
at java.awt.image.SampleModel.(SampleModel.java:130)
at java.awt.image.ComponentSampleModel.(ComponentSampleModel.java:146)
at java.awt.image.PixelInterleavedSampleModel.(PixelInterleavedSampleModel.java:87)
at java.awt.image.PixelInterleavedSampleModel.createCompatibleSampleModel(PixelInterleavedSampleModel.java:144)
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KReadState.readAsRaster(J2KReadState.java:429)
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KImageReader.readRaster(J2KImageReader.java:556)
at ome.codecs.services.JAIIIOServiceImpl.readRaster(JAIIIOServiceImpl.java:177)
at ome.codecs.JPEG2000Codec.decompress(JPEG2000Codec.java:296)
at ome.codecs.JPEG2000Codec.decompress(JPEG2000Codec.java:267)
at loci.formats.codec.WrappedCodec.decompress(WrappedCodec.java:132)
at loci.formats.codec.JPEG2000Codec.decompress(JPEG2000Codec.java:63)
at loci.formats.in.JPEG2000Reader.openBytes(JPEG2000Reader.java:182)
at loci.formats.FormatReader.openBytes(FormatReader.java:922)
at loci.formats.ImageReader.openBytes(ImageReader.java:450)
at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:336)
at loci.formats.gui.BufferedImageReader.openImage(BufferedImageReader.java:86)

[guymac]

Same error with the recommended IFormatReader openBytes method. Somehow the TIFF reader is able to internally handle these large images (I don't think they are tiled, but don't really know how to check), but the JPEG2000 reader can't.

[scuniff]

Sounds good.

I just completed getting bio-formats built in Eclipse.

I think I need to read this link some to come up more to speed on using the API. 😎

https://bio-formats.readthedocs.io/en/v7.2.0/developers/java-library.html

I had run bftools/showinf (https://docs.openmicroscopy.org/bio-formats/5.7.1/users/comlinetools/index.html) on a 66k x 66k grayscale jp2 image and got this at the end of the output:

c:\steve\Bioformats\bftools> showinf c:\steve\temp\temp20\IMG_0067-66x66-grayscale.jp2
....
....
....

Reading series #0 pixel data (0-0)
Exception in thread "main" loci.formats.FormatException: Image plane too large. Only 2GB of data can be extracted at one time. You can work around the problem by opening the plane in tiles; for further details, see: https://docs.openmicroscopy.org/bio-formats/7.2.0/about/bug-reporting.html#common-issues-to-check
at loci.formats.FormatReader.openBytes(FormatReader.java:916)
at loci.formats.ImageReader.openBytes(ImageReader.java:451)
at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:336)
at loci.formats.gui.BufferedImageReader.openImage(BufferedImageReader.java:86)
at loci.formats.tools.ImageInfo.readPixels(ImageInfo.java:829)
at loci.formats.tools.ImageInfo.testRead(ImageInfo.java:1063)
at loci.formats.tools.ImageInfo.main(ImageInfo.java:1129)
Caused by: java.lang.IllegalArgumentException: Array size too large: 66000 x 66000 x 1 x 1
at loci.common.DataTools.safeMultiply32(DataTools.java:1286)
at loci.common.DataTools.allocate(DataTools.java:1259)
at loci.formats.FormatReader.openBytes(FormatReader.java:913)
... 6 more

Another reference to 2G

I hope to run some tests in Eclipse later today.....

[scuniff]

I’ve been trying the api mentioned in this link Reporting a bug — Bio-Formats 8.0.0 documentation.

It mentions trying to open just part of an image with openBytes(int, int, int, int, int).

Scrennshot from link above

I tried reading a 1000x1000 portion of a 66Kx66K image starting in the upper left corner(x=0, y=0).

byte[] buf = reader.openBytes(0, 0, 0, 1000, 1000);

Complete code:

import java.io.IOException;
import loci.formats.gui.BufferedImageReader;

public class testJp2 {

	public static void main(String[] args) {

		String image = "C:\\steve\\temp\\IMG-66Kx66K-grayscale.jp2";
		loci.formats.ImageReader imr = new loci.formats.ImageReader();

		try {
			imr.setId(image);

			BufferedImageReader reader = BufferedImageReader.makeBufferedImageReader(imr);

			System.out.println("dimensions before openBytes:  " + reader.getSizeX() + "  " + reader.getSizeY());

			byte[] buf = reader.openBytes(0, 0, 0, 1000, 1000);

			System.out.println("dimensions:  " + reader.getSizeX() + "  " + reader.getSizeY());
			System.out.println("buf len is "+ buf.length );
			
		} catch (Exception e) {
			e.printStackTrace();
		}
	}
}

I get this error when running above code: java.lang.IllegalArgumentException: Dimensions (width=66000 height=66000) are too large

Output from code:

dimensions before openBytes: 66000 66000

java.lang.IllegalArgumentException: Dimensions (width=66000 height=66000) are too large
at java.awt.image.SampleModel.(SampleModel.java:130)
at java.awt.image.ComponentSampleModel.(ComponentSampleModel.java:146)
at java.awt.image.PixelInterleavedSampleModel.(PixelInterleavedSampleModel.java:87)
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KReadState.getSampleModel(J2KReadState.java:1048)
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KReadState.readAsRaster(J2KReadState.java:427)
at com.sun.media.imageioimpl.plugins.jpeg2000.J2KImageReader.readRaster(J2KImageReader.java:556)
at ome.codecs.services.JAIIIOServiceImpl.readRaster(JAIIIOServiceImpl.java:177)
at ome.codecs.JPEG2000Codec.decompress(JPEG2000Codec.java:296)
at ome.codecs.JPEG2000Codec.decompress(JPEG2000Codec.java:267)
at loci.formats.codec.WrappedCodec.decompress(WrappedCodec.java:132)
at loci.formats.codec.JPEG2000Codec.decompress(JPEG2000Codec.java:63)
at loci.formats.in.JPEG2000Reader.openBytes(JPEG2000Reader.java:182)
at loci.formats.FormatReader.openBytes(FormatReader.java:922)
at loci.formats.ImageReader.openBytes(ImageReader.java:450)
at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:336)
at testJp2.main(testJp2.java:18)