Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sporadic java.lang.ArrayIndexOutOfBoundsException/Can't deserialize lambda if RuntimeException is deserialized #1558

Open
1 of 2 tasks
johannschenkl opened this issue Apr 23, 2024 · 9 comments
Open
1 of 2 tasks

Sporadic java.lang.ArrayIndexOutOfBoundsException/Can't deserialize lambda if RuntimeException is deserialized #1558

johannschenkl opened this issue Apr 23, 2024 · 9 comments
Labels
bug Something isn't working

Comments

@johannschenkl
Copy link

Search before asking

  • I had searched in the issues and found no similar issues.

Version

0.4.1

Component(s)

Java

Minimal reproduce step

We are using fury as a replacement for jdk serialization in a java rpc application. If on the server side an exception is thrown, this exception in serialized and sent to the client.
In some cases we seen and exception when the serialization is done.
Thanks for your help and for that great project!

Fury setup:

var fury = Fury.builder() .withLanguage(Language.JAVA) .withRefTracking(true) .requireClassRegistration(false) .withAsyncCompilation(true).build();

Relevant parts of logs:

io.fury.serializer.LambdaSerializer.read(LambdaSerializer.java:88)
io.fury.Fury.readRef(Fury.java:828)
io.fury.resolver.FieldResolver.skipDataBy4(FieldResolver.java:433)
com.mycompany.SomeExceptionFuryRefCompatibleCodec_1_815033865_986095905.readAndSetFields(SomeExceptionFuryRefCompatibleCodec_1_815033865_986095905.java:258)
com.mycompany.SomeExceptionFuryRefCompatibleCodec_1_815033865_986095905.read(SomeExceptionFuryRefCompatibleCodec_1_815033865_986095905.java:324)
com.mycompany.RemoteInvocationResultFuryRefCodec_1_815033865_718325701.readFields$(RemoteInvocationResultFuryRefCodec_1_815033865_718325701.java:111)
com.mycompany.RemoteInvocationResultFuryRefCodec_1_815033865_718325701.read(RemoteInvocationResultFuryRefCodec_1_815033865_718325701.java:127)
io.fury.Fury.readDataInternal(Fury.java:899)
io.fury.Fury.readRef(Fury.java:801)
io.fury.Fury.deserializeJavaObjectAndClass(Fury.java:1135)
io.fury.Fury.deserializeJavaObjectAndClass(Fury.java:1121)
com.mycompany.Serializer.deserialize(Serializer.java:35)

...

Caused by:
Index -1 out of bounds for length 512
java.lang.ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 512
io.fury.collection.IntArray.pop(IntArray.java:51)
io.fury.resolver.MapRefResolver.reference(MapRefResolver.java:194)
io.fury.builder.SerializedLambdaFuryRefCodec_1_1361449601.read(SerializedLambdaFuryRefCodec_1_1361449601.java:193)
io.fury.serializer.LambdaSerializer.read(LambdaSerializer.java:85)
io.fury.Fury.readRef(Fury.java:828)
io.fury.resolver.FieldResolver.skipDataBy4(FieldResolver.java:433)
com.mycompany.SomeExceptionFuryRefCompatibleCodec_1_815033865_986095905.readAndSetFields(SomeExceptionFuryRefCompatibleCodec_1_815033865_986095905.java:258)
com.mycompany.SomeExceptionFuryRefCompatibleCodec_1_815033865_986095905.read(SomeExceptionFuryRefCompatibleCodec_1_815033865_986095905.java:324)
com.mycompany.RemoteInvocationResultFuryRefCodec_1_815033865_718325701.readFields$(RemoteInvocationResultFuryRefCodec_1_815033865_718325701.java:111)
com.mycompany.RemoteInvocationResultFuryRefCodec_1_815033865_718325701.read(RemoteInvocationResultFuryRefCodec_1_815033865_718325701.java:127)
io.fury.Fury.readDataInternal(Fury.java:899)
io.fury.Fury.readRef(Fury.java:801)
io.fury.Fury.deserializeJavaObjectAndClass(Fury.java:1135)
io.fury.Fury.deserializeJavaObjectAndClass(Fury.java:1121)
com.mycompany.Serializer.deserialize(Serializer.java:35)

What did you expect to see?

The deserialized payload.

What did you see instead?

An ArrayIndexOutOfBoundsException.

Anything Else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!
@johannschenkl johannschenkl added the bug Something isn't working label Apr 23, 2024
@chaokunyang
Copy link
Collaborator

Hi @johannschenkl ， thanks for reporting this bug. It seems that our current implementation has some inconsistency between interpreter mode and codegen mode. This has been reported in #1455, #1325 and #1176.

The fix won't be difficult. But none of this can provide reproduction code, so I can't debug to find out what's going wrong.

@chaokunyang
Copy link
Collaborator

Will it still raise execpetion if you disabled asyncCompilation?

@chaokunyang
Copy link
Collaborator

It would be really great if you can provide reproduction code. I try to fix it for a long time

@johannschenkl
Copy link
Author

Hi @chaokunyang, thanks for the quick response. Currently, the issue is only reproducible on a qa machine and only for some users. I was not able to reproduce it by myself. As of yet, it only happend in conjunction with a serialized RuntimeException.
I'll check if disabling asyncCompilation helps and if I can somehow build a test case.

@chaokunyang
Copy link
Collaborator

Hi @johannschenkl . Is this issue still persisits?

@johannschenkl
Copy link
Author

Hi @chaokunyang, unfortunately yes. I'm still trying to get a reproducer working.

@johannschenkl
Copy link
Author

johannschenkl commented Apr 29, 2024 •
edited

We're self-managing Fury in a ThreadLocal, is that the issue, maybe?

private static final ThreadLocal<Fury> THE_FURY = ThreadLocal.withInitial(() -> {
        var fury = Fury.builder().withLanguage(Language.JAVA)
                .withRefTracking(true)
                .requireClassRegistration(false)
                .withAsyncCompilation(true)
                .suppressClassRegistrationWarnings(true) // as checker is enabled, see below
                .build();
        fury.getClassResolver().setClassChecker(...);
        return fury;
    });

I know there's also the ...buildThreadSafeFury/buildThreadLocalFury implementations. But we did not see a way to register a classchecker on them.

@chaokunyang
Copy link
Collaborator

This is right, or you can use org.apache.fury.ThreadLocalFury#ThreadLocalFury, which let you to set anything in the factory

@chaokunyang chaokunyang mentioned this issue May 6, 2024
Merged
2 tasks
chaokunyang added a commit that referenced this issue May 6, 2024
@chaokunyang
fix(java): clear serializer for collection/map (#1606)
7d912ca 
## What does this PR do?
Some collectionSerializer may overwrite write/read method, then clear
element serializer may not got invoked.

This PR clears serializer for collection/map to avoid container use
wrong serializer for nested elements.

## Related issues

#1558
#1455,
#1325 and
#1176.

## Does this PR introduce any user-facing change?

<!--
If any user-facing interface changes, please [open an
issue](https://github.com/apache/incubator-fury/issues/new/choose)
describing the need to do so and update the document if necessary.
-->

- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?


## Benchmark

<!--
When the PR has an impact on performance (if you don't know whether the
PR will have an impact on performance, you can submit the PR first, and
if it will have impact on performance, the code reviewer will explain
it), be sure to attach a benchmark data here.
-->
@johannschenkl
Copy link
Author

Hi @chaokunyang, sorry for the late response. After using the provided factory & disabling async compilation, the issue did not show up again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@chaokunyang @johannschenkl