Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixes stack overflow possibility with merge operators #2616

Merged
merged 5 commits into from
Feb 15, 2025

Conversation

geoffmacd
Copy link
Contributor

@geoffmacd geoffmacd commented Jul 26, 2024

Fixes #2615 See issue for details

when completing Merge operator iterations (concat(), concatMap() and merge(maxConcurrent:), fixes subscribing immediately to the next in the queue, which can produce values immediately which can re-enter and cause stack overflows. ultimately uses CurrentThreadScheduler and the isScheduleRequired prop to know if it needs to schedule or not. This is similar to Producer.

All tests pass

fix subscribing immediately in merge operators can produce values immediately which can re-enter and cause stack overflows
@geoffmacd geoffmacd changed the title fixes stackoverflows in MergeLimitedSink fixes stack overflow possibility with merge operators Jul 26, 2024
@danielt1263
Copy link
Collaborator

This looks like a good candidate for a dot release @freak4pc. What do you think?

@@ -170,7 +170,10 @@ private final class MergeLimitedSinkIter<SourceElement, SourceSequence: Observab
case .completed:
self.parent.group.remove(for: self.disposeKey)
if let next = self.parent.queue.dequeue() {
self.parent.subscribe(next, group: self.parent.group)
_ = CurrentThreadScheduler.instance.schedule(()) { _ in
self.parent.subscribe(next, group: self.parent.group)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how important this is, but scheduling the subscribe call will take the corresponding modification of self.parent.group out of the lock that is held while synchronized_on is executing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea, I think you are right that there is an issue and I need to ensure the NSRecursiveLock is re-locking inside this block. I think for the current locking cycle from synchronized_on, there is no more mutation after this line to worry about. The lock will recursively lock on the next inner subscribe.

There is a separate issue here which is that the the disposable from _ = CurrentThreadScheduler.instance.schedule(()) needs to be added to the group otherwise this will not dispose if the CompositeDisposable is disposed. This is minor though since the parent.subscribe would have no impact.

@freak4pc
Copy link
Member

freak4pc commented Oct 3, 2024

Hey @geoffmacd, sorry for the long delay on this.

  1. Did you use your own fix in prod by now? Is it stable?
  2. Is the code ready from your perspective ?
  3. Let's rebase / merge with main and have CI run all the tests to make sure we're good.

Thanks!

fix subscribing immediately in merge operators can produce values immediately which can re-enter and cause stack overflows
@freak4pc freak4pc force-pushed the fix-merge-stack-overflow branch from 33dfd45 to 168f887 Compare October 4, 2024 18:09
@freak4pc freak4pc closed this Oct 4, 2024
@freak4pc freak4pc reopened this Oct 4, 2024
@geoffmacd
Copy link
Contributor Author

Hey @freak4pc

Sorry for delay.

  1. Yes this has been integrated in dropbox for 6 months which heavily uses Rx and is stable.
  2. yes.
  3. Will do.

@freak4pc
Copy link
Member

Much appreciated, thank you :)

@geoffmacd
Copy link
Contributor Author

ok, updated from main. ready to merge

@freak4pc freak4pc merged commit baf816f into ReactiveX:main Feb 15, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Stack overflows caused by MergeLimitedSink operators
4 participants