-
Notifications
You must be signed in to change notification settings - Fork 74
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
JPEG-LS LSE ID=02h | LSE ID = 03h Mapping Table support #90
Comments
Hi, The main reason that support for mapping tables have never been implemented is the fact that there has never been a request for it. The ISO/IEC 14495-1 documents in section C.2.4.1.2 only a palette use case:: This Recommendation | International Standard does not define an interpretation for the Wt bytes in an entry. Images that use a palette are rare however (MCGA video mode is very old). Medical standard like DICOM support also palette images, but DICOM requires that the palette table is stored in DICOM attributes and not in the encoded pixel data. It is of course possible for certain types of images to create a mapping table that would allow the JPEG-LS algorithm to achieve a better compression ratio and even with the added mapping table data would result in a smaller image. I could imagine a 16-bit image that only is using a very small subset of certain pixel values and not the full 16-bit range would benefit from it. Note: image data that needs to be encoded in 16 bits, but only uses a limit range could also use the option of custom JPEG-LS preset coding parameters to achieve better encoding (which is supported by CharLS). The functional implementation of mapping tables would not be that big, but the effort to create all the unit tests and error handling code for all the exceptional cases would be significant. It is possible however to put this feature on the backlog if you expect that you have a real-world usage case for it. |
Generally I agree, except that using preset to limit the RANGE would yield near-lossless images (loss in sample precision), while using Mapping Table can create lossless images (without any loss of sample precision), and still compress further the original data. I've been testing this with NEAR = 0, and NEAR = 1 on 16-bit images. Yes my primary target are medical images, and devices storing them. suffixes on files:
And if you notice on post-processed images compression goes additional ~22% when using Mapping Tables. True, I need more testing on bulk of non-post-processes and post-processed images to get final conclusion what is the real gain here. |
My proposal to limit the range would be limited to the actual values that are used to ensure the compression remains lossless. It is of course also possible to scan the pixels and determine the actual MAXVAL, which may be helpful if you have images that are sometimes "dark" but not always. Would it be possible to share some RAW images that generate a larger file when setting the limit to 14-bit. It is indeed strange that these files are larger, but I have never experimented with it. |
Tried just now (on g16i03.raw), both MAXVAL = 11261, and MAXVAL = (1 << 14) - 1, gain is almost negligible:
But I am not concern by those images as they are RAW images from the Flat Panel Detector, and are almost never stored to PACS devices. |
I did some analysis on the development costs. Adding table support to the encoder would require 2 functions:
to cover all possible scenario's. Adding support to the decoder would be much more complex however. Tables can be present or not and can be defined at several locations in the JPEG-LS stream (as long as they are defined before the scan that uses them). Did you looked into the compressions gains that tables could bring for your use case? |
sorry, was on something else in mean time. I have done the following: I've uploaded initial tests with post-processed X-RAY images without/with mapping tables at the following location: JPEG-LS-TestGround > MappingTables Explanation on suffixes on files (but it can be viewed in JLS headers also):
Also pasting here the JLS headers info and compression levels (less is better) from above files, and you'll notice huge difference between without/with mapping tables compression levels.
|
I will have a look at the data\images. Viewing JPEG-LS images It cannot handle tables at the moment, but as it just a wrapper around CharLS, you can point it to your CharLS submodule branch. Note that this WIC wrapper is still in development and not all things work. Another solution would be to create a plug-in for Gimp, which could be a cross-platform solution. Never found the time to create such a plug-in however. |
Was reading wiki docs and noted you have stated the following:
"How complete is CharLS?
CharLS is a conforming ISO/IEC 14495-1:1999 implementation, with some exceptions that have little or no impact in real-world usage. All exceptions are listed in the README.md"
And README.md has quite lengthy list:
"The following JPEG-LS options are not supported by the CharLS implementation. Most of these options are rarely used in practice.
No support for JPEG restart markers (RST). Restart markers make it possible to recover from corrupted JPEG files, but are seldom used for data recovery scenarios.
No support for sub-sampled scans. Sub-sampling is a lossly encoding mechanism and not used in lossless scenarios.
No support for multi component frames with mixed component counts in a single scan. While technical possible all known JPEG-LS codecs put multi-component (color) images in a single scan or in multiple scans, but not use a mix of these in one file.
No support for oversize image dimension. Maximum supported image dimensions are [1, 65535] by [1, 65535].
No support for JPEG-LS mapping tables.
No support for point transform. Point transform is a lossly encoding mechanism and not used in lossless scenarios."
JPEG-LS mapping tables are added to give better compression to both the lossless and near-lossless.
It it's just mapping pixel value to mapping table index, and vice verso when decoding.
It affects only function for retrieving Rx (Ra, Rb, Rc, Rd) values.
So why not implementing it?
Note: I've already tested with lowering the MAXVAL to Mapping Table entries count - 1, resulting image is less compressed.
This was tested on 16-bit image, 8-bit may have different results.
But with 16-bit image I got 6% more compression,
Also there was no speed loss on doing histogram and building mapping table and getting map index for samples.
Gain in compression (write bits) compensates for time needed to build mapping table. Guess the same would apply for decoding process also (need to write mapping table decoder to test the assumption).
`
Test[A]
in : Y:\VISARIS_C\VISARIS_STORING\RAW\1\02.RAW
size : 16 588 800
out : Y:\VISARIS_C\VISARIS_STORING\RAW\1\02-b16-N0.jls
size : 7 984 976 / encoded: 7 984 934
comp : 48.13 % / 48.13 %
Test[B]
in : Y:\VISARIS_C\VISARIS_STORING\RAW\1\02.RAW
size : 16 588 800
out : Y:\VISARIS_C\VISARIS_STORING\RAW\1\02-b16-N0mt.jls
size : 6 989 911 / encoded: 6 961 398
comp : 42.14 % / 41.96 %
`
I agree mapping table has more compression only with larger images using it on small images would actually produce larger files. But since JPEG-LS compression is for medical, astronomy and similar use, I doubt there will be small images. Also this 21st century (Phase One camera has 100 MP), everything these days has higher and higher resolution.
The text was updated successfully, but these errors were encountered: