Skip to content

Fix crash for web socket in some race conditions - #22439

Closed
zhongwuzw wants to merge 3 commits into
react:masterfrom
zhongwuzw:fix_webSocket_crash
Closed

Fix crash for web socket in some race conditions#22439
zhongwuzw wants to merge 3 commits into
react:masterfrom
zhongwuzw:fix_webSocket_crash

Conversation

@zhongwuzw

@zhongwuzwzhongwuzw commented Nov 28, 2018

Copy link
Copy Markdown
Contributor

Fixes#21086.
Fixes#6117.

This PR fixes a crash caused by a race condition when webSocket deallocated and NSStream delegate callback, because NSStream's delegate callback be called on RCTSR_networkRunLoop.

This PR mainly changes:

  • Remove unnecessary nil operation in dealloc method.
  • Add a new method _scheduleCleanUp to schedule webSocket cleanup also on RCTSR_networkRunLoop.
  • In stream:(NSStream *)aStream handleEvent:(NSStreamEvent)eventCode delegate method, add a wself to make safe further.

Test Plan:

Reproduce steps:
Run any react-native project, and keep it running, it would crash at some point.

Changelog:

[iOS] [BUGFIX] [RCTSRWebSocket] - Fix crash for web socket in some race conditions.

@facebook-github-botfacebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 28, 2018
@pull-bot

Copy link
Copy Markdown
Warnings
⚠️

📋 Changelog - This PR may have incorrectly formatted Changelog.

Generated by 🚫 dangerJS

@kelset

Copy link
Copy Markdown
Contributor

Thanks for opening this new PR :)

@cpojer

Copy link
Copy Markdown
Contributor

This PR unfortunately doesn't come with a reproducible test plan of the before/after behavior and the change itself is quite large without any form of automated testing. In order to land this, would you mind working both on the test plan and possibly adding some form of tests?

@zhongwuzw

Copy link
Copy Markdown
ContributorAuthor

@cpojer Emm, I did try to find a way to test it, but I have no idea currently, because it's a race condition, RCTSRWebSocket is deallocating and another runloop calls its method.

@zhongwuzw

Copy link
Copy Markdown
ContributorAuthor

Runloop callback the delegate method, but delegate object already released:

image

@qingluanchou

qingluanchou commented Dec 16, 2018

Copy link
Copy Markdown

[In environment Xcode10.1 react-native:0.55.4 react:16.3.1 ,I have modified RCTSRWebSocket.m and still crashes,What is the cause? @zhongwuzw
image

@janicduplessis

Copy link
Copy Markdown
Contributor

I can confirm that I'm also seeing this crash in my app. No idea how to repro though.

OS Version: iOS 12.1 (16B92)
Report Version: 104
Exception Type: EXC_BAD_ACCESS (SIGBUS)
Exception Codes: BUS_NOOP at 0x0000000000000010
Crashed Thread: 13
Application Specific Information:
class > respondsToSelector: > stream:handleEvent: > XTUM >
Attempted to dereference garbage pointer 0x10.
Thread 13 Crashed:
0 libobjc.A.dylib 0x3abb07d70 objc_msgSend
1 libobjc.A.dylib 0x3abaebe7c -[NSObject respondsToSelector:]
2 CoreFoundation 0x3ad5aeed4 _inputStreamCallbackFunc
3 CoreFoundation 0x3ad5db030 _signalEventSync
4 CoreFoundation 0x3ad5d8134 _cfstream_solo_signalEventSync
5 CoreFoundation 0x3ad5d7f14 _CFStreamSignalEvent
6 CFNetwork 0x3ae3ade98 SocketStream::dispatchSignalFromSocketCallbackUnlocked(SocketStreamSignalHolder*)
7 CFNetwork 0x3ae3b3768 SocketStream::socketCallback(__CFSocket*, unsigned long, __CFData const*, void const*)
8 CFNetwork 0x3ae3af4a0 SocketStream::_SocketCallBack_stream(__CFSocket*, unsigned long, __CFData const*, void const*, void*)
9 CoreFoundation 0x3ad5cf5a8 __CFSocketPerformV0
10 CoreFoundation 0x3ad5c81cc __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__
11 CoreFoundation 0x3ad5c814c __CFRunLoopDoSource0
12 CoreFoundation 0x3ad5c7a30 __CFRunLoopDoSources0
13 CoreFoundation 0x3ad5c28fc __CFRunLoopRun
14 CoreFoundation 0x3ad5c21cc CFRunLoopRunSpecific
15 Foundation 0x3aea4a404 -[NSRunLoop(NSRunLoop) runMode:beforeDate:]
16 th3rdwave 0x202fe523c _ZN5folly6detail8function14FunctionTraitsIKFbRKNS_7dynamicES5_EE10uninitCallERNS1_4DataES5_S5_
17 Foundation 0x3aeb7d1ac __NSThread__start__
18 libsystem_pthread.dylib 0x3acf7f2ac _pthread_body
19 libsystem_pthread.dylib 0x3acf7f20c _pthread_start

@janicduplessisjanicduplessis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great overall, thanks for working on this. This code is pretty tricky so we need to be careful.

@cpojer The changes are actually less scary than I though, mostly just whitespace.

Were you able to test this in production to make sure the crash does not happen and that there are no regressions?


// Cleanup NSStream's delegate in the same RunLoop used by the streams themselves:
// This way we'll prevent race conditions between handleEvent and SRWebsocket's dealloc
NSTimer *timer = [NSTimer timerWithTimeInterval:(0.0f) target:self selector:@selector(_cleanupSelfReference:) userInfo:nil repeats:NO];

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason why you can't use dispatch_async to schedule cleanup here?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@janicduplessis Emm, we need to schedule cleanup on RCTSR_networkRunLoop, seems we have no dispatch_queue here. Maybe we can use performSelector, but we can keep use timer, because they are the same actually.

Comment threadLibraries/WebSocket/RCTSRWebSocket.m Outdated
assert(_workQueue != NULL);

// _workQueue cannot be NULL
if (!_workQueue) { return; }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If work queue cannot be null can we just remove this?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@janicduplessis In theory, we don't need to check work queue nullable, the reason I add this is to add the further safe check, because if work queue is null, it would crash immediately.

- (void)_cleanupSelfReference:(NSTimer *)timer
{
// Remove the streams, right now, from the networkRunLoop
[_inputStream close];

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we check the stream status here before trying to close it like we do here https://github.com/facebook/react-native/blob/master/Libraries/WebSocket/RCTSRWebSocket.m#L1034

Might be worth extracting to a shared method to safely close streams.

@zhongwuzwzhongwuzwDec 20, 2018

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@janicduplessis 👍 We can extract it to a separate method.

@kelset

Copy link
Copy Markdown
Contributor

@janicduplessis we have been using this patch in our prod app for months now (more or less) and it's working super well for us.

And yes I don't have a way to repro either, it's a weird prod edge case.

@zhongwuzw

Copy link
Copy Markdown
ContributorAuthor

[In environment Xcode10.1 react-native:0.55.4 react:16.3.1 ,I have modified RCTSRWebSocket.m and still crashes,What is the cause? @zhongwuzw
image

Emm, seems it's the apple's internal bug? maybe it's not related to this issue?

@facebook-github-botfacebook-github-bot added the Import Started This pull request has been imported. This does not imply the PR has been approved. label Jan 2, 2019

@facebook-github-botfacebook-github-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cpojer is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@react-native-bot

Copy link
Copy Markdown
Collaborator

@zhongwuzw merged commit dd209bb into facebook:master.

@reactreact locked as resolved and limited conversation to collaborators Jan 2, 2019
@react-native-botreact-native-bot added the Merged This PR has been merged. label Jan 2, 2019
@zhongwuzw
zhongwuzw deleted the fix_webSocket_crash branch January 2, 2019 02:01
@hramoshramos removed Import Started This pull request has been imported. This does not imply the PR has been approved. labels Feb 6, 2019
bruchim pushed a commit to wix-playground/react-native that referenced this pull request Feb 12, 2019
Summary:
Fixesreact#21086.
Fixesreact#6117.
This PR fixes a crash caused by a race condition when `webSocket` deallocated and `NSStream` delegate callback, because `NSStream`'s delegate callback be called on `RCTSR_networkRunLoop`.
This PR mainly changes:
* Remove unnecessary `nil` operation in `dealloc` method.
* Add a new method `_scheduleCleanUp` to schedule `webSocket` cleanup also on `RCTSR_networkRunLoop`.
* In `stream:(NSStream *)aStream handleEvent:(NSStreamEvent)eventCode` delegate method, add a `wself` to make safe further.
Pull Request resolved: react#22439
Differential Revision: D13564247
Pulled By: cpojer
fbshipit-source-id: 675c1b2805aa45c54d7708d796f5843ef7ea34e2
marcinolek pushed a commit to marcinolek/react-native that referenced this pull request Feb 13, 2019
Summary:
Fixesreact#21086.
Fixesreact#6117.
This PR fixes a crash caused by a race condition when `webSocket` deallocated and `NSStream` delegate callback, because `NSStream`'s delegate callback be called on `RCTSR_networkRunLoop`.
This PR mainly changes:
* Remove unnecessary `nil` operation in `dealloc` method.
* Add a new method `_scheduleCleanUp` to schedule `webSocket` cleanup also on `RCTSR_networkRunLoop`.
* In `stream:(NSStream *)aStream handleEvent:(NSStreamEvent)eventCode` delegate method, add a `wself` to make safe further.
Pull Request resolved: react#22439
Differential Revision: D13564247
Pulled By: cpojer
fbshipit-source-id: 675c1b2805aa45c54d7708d796f5843ef7ea34e2
grabbou pushed a commit that referenced this pull request Feb 18, 2019
Summary:
Fixes#21086.
Fixes#6117.
This PR fixes a crash caused by a race condition when `webSocket` deallocated and `NSStream` delegate callback, because `NSStream`'s delegate callback be called on `RCTSR_networkRunLoop`.
This PR mainly changes:
* Remove unnecessary `nil` operation in `dealloc` method.
* Add a new method `_scheduleCleanUp` to schedule `webSocket` cleanup also on `RCTSR_networkRunLoop`.
* In `stream:(NSStream *)aStream handleEvent:(NSStreamEvent)eventCode` delegate method, add a `wself` to make safe further.
Pull Request resolved: #22439
Differential Revision: D13564247
Pulled By: cpojer
fbshipit-source-id: 675c1b2805aa45c54d7708d796f5843ef7ea34e2
aleclarson pushed a commit to aleclarson/react-native-macos that referenced this pull request Jun 16, 2019
Summary:
Fixes #21086.
Fixes #6117.
This PR fixes a crash caused by a race condition when `webSocket` deallocated and `NSStream` delegate callback, because `NSStream`'s delegate callback be called on `RCTSR_networkRunLoop`.
This PR mainly changes:
* Remove unnecessary `nil` operation in `dealloc` method.
* Add a new method `_scheduleCleanUp` to schedule `webSocket` cleanup also on `RCTSR_networkRunLoop`.
* In `stream:(NSStream *)aStream handleEvent:(NSStreamEvent)eventCode` delegate method, add a `wself` to make safe further.
Pull Request resolved: react/react-native#22439
Differential Revision: D13564247
Pulled By: cpojer
fbshipit-source-id: 675c1b2805aa45c54d7708d796f5843ef7ea34e2
t-nanava pushed a commit to microsoft/react-native-macos that referenced this pull request Jun 17, 2019
Summary:
Fixesreact#21086.
Fixesreact#6117.
This PR fixes a crash caused by a race condition when `webSocket` deallocated and `NSStream` delegate callback, because `NSStream`'s delegate callback be called on `RCTSR_networkRunLoop`.
This PR mainly changes:
* Remove unnecessary `nil` operation in `dealloc` method.
* Add a new method `_scheduleCleanUp` to schedule `webSocket` cleanup also on `RCTSR_networkRunLoop`.
* In `stream:(NSStream *)aStream handleEvent:(NSStreamEvent)eventCode` delegate method, add a `wself` to make safe further.
Pull Request resolved: react#22439
Differential Revision: D13564247
Pulled By: cpojer
fbshipit-source-id: 675c1b2805aa45c54d7708d796f5843ef7ea34e2
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

BugCLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.MergedThis PR has been merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

com.squareup.SocketRocket.NetworkThread(18): EXC_BAD_ACCESS com.squareup.SocketRocket.NetworkThread(18): EXC_BAD_ACCESS

9 participants

@zhongwuzw@pull-bot@kelset@cpojer@qingluanchou@janicduplessis@react-native-bot@facebook-github-bot@hramos