Enforce 64KB event payload size limit on EventPipe by sywhang · Pull Request #50600 · dotnet/runtime

sywhang · 2021-04-01T18:46:21Z

ETW has a maximum of 64KB event payload size limit that is not enforced on EventPipe. This causes issues like #50515 to occur where an event that's greater than 100KB can cause the flushing thread's event block to be too small to fit a single event payload, which crashes the application.

josalem · 2021-04-01T19:05:21Z

 	ep_return_false_if_nok (ep_event_is_enabled (ep_event));

+	// Check that the payload size is less than 64 KB (max size for ETW events)
+	ep_return_false_if_nok (ep_event_payload_get_size (payload) <= 64 * 1024);


ETW might limit its payloads to 64KB, but does EventPipe need to? EventSource doesn't explicitly prevent you from having payloads this large, e.g., your test.

It doesn't. But ETW will drop the event if it's greater than 64KB and I'd like to enforce the same behavior here since:
We need some kind of limit regardless. Currently >100KB payloads crash the app. We could enforce a 100KB limit instead of 64KB.
But if we do enforce 100KB limit, that leads to a discrepancy between ETW/EventPipe behavior which we then need to go and figure out how customers can find out whether big events are getting dropped on ETW but not on EventPipe and document why such behavior differences exist.
Another approach is to enforce 1MB payload (which we currently do - just not explicitly but because we just fail to allocate buffers greater than that) and increase the max flushing block size to 1MB but that's a pretty inefficient use of space for sessions listening on much smaller-sized payloads (i.e. counters).

Fair point. Let me mull this over, but I think 64KB is fine. Should we remove the DebugBreak in the EP_UNLIKELY macro expansion?

I'd actually like to keep that. It lets us catch exceptions like this rather than silently failing and us never noticing the failures.

noahfalk

A few comments inline, looks good otherwise : )

sywhang · 2021-04-05T23:27:36Z

I'm merging this since CI failure is due to #50753.

…shim_mono # By Aaron Robinson (10) and others # Via GitHub * upstream/main: (108 commits) [mbr] Add Apple sample (dotnet#50740) make EstablishProxyTunnelAsync throw on failure status code from proxy (dotnet#50763) Improve RGB Min Max evaluation performance by using 2 or 3 comparison… (dotnet#50622) [mono] More domain cleanups (dotnet#50479) Fix Crossgen2 of PlatformDefaultMemberFunction methods and calls. (dotnet#50754) Disable EventSource generator in design-time builds (dotnet#50741) Fix X509 test failures on Android (dotnet#50301) Do not confuse fgDispBasicBlocks in fgMorphBlocks (dotnet#50703) Enforce 64KB event payload size limit on EventPipe (dotnet#50600) Reorganize CoreCLR native build to reduce CMake reconfigures when the build system is untouched (dotnet#49906) [mbr] Turn on hot reload for iOS, tvOS and MacCatalyst (dotnet#50458) improve connection scavenge logic by doing zero-byte read (dotnet#50545) Resolve call mdtokens when making tier 1 inline observations (dotnet#50675) Annotate APIs in System.Private.Xml (dotnet#49682) Support compiling against OpenSSL 3 headers Change Configuration.Json to use a regular Dictionary. (dotnet#50611) Remove unused BigNumFromBinary P/Invoke (dotnet#50670) Make Ninja the default CMake generator on Windows for the repo (dotnet#49715) [AppleAppBuilder] Entitlements to run tests on catalyst using the JIT (dotnet#50637) [mono] Fix delegate invokes to dynamic methods in mixed mode. (dotnet#50547) ... # Conflicts: # src/mono/dlls/mscordbi/CMakeLists.txt

sywhang · 2021-04-01T18:46:21Z

ETW has a maximum of 64KB event payload size limit that is not enforced on EventPipe. This causes issues like #50515 to occur where an event that's greater than 100KB can cause the flushing thread's event block to be too small to fit a single event payload, which crashes the application.

josalem · 2021-04-01T19:05:21Z

 	ep_return_false_if_nok (ep_event_is_enabled (ep_event));

+	// Check that the payload size is less than 64 KB (max size for ETW events)
+	ep_return_false_if_nok (ep_event_payload_get_size (payload) <= 64 * 1024);


ETW might limit its payloads to 64KB, but does EventPipe need to? EventSource doesn't explicitly prevent you from having payloads this large, e.g., your test.

It doesn't. But ETW will drop the event if it's greater than 64KB and I'd like to enforce the same behavior here since:
We need some kind of limit regardless. Currently >100KB payloads crash the app. We could enforce a 100KB limit instead of 64KB.
But if we do enforce 100KB limit, that leads to a discrepancy between ETW/EventPipe behavior which we then need to go and figure out how customers can find out whether big events are getting dropped on ETW but not on EventPipe and document why such behavior differences exist.
Another approach is to enforce 1MB payload (which we currently do - just not explicitly but because we just fail to allocate buffers greater than that) and increase the max flushing block size to 1MB but that's a pretty inefficient use of space for sessions listening on much smaller-sized payloads (i.e. counters).

Fair point. Let me mull this over, but I think 64KB is fine. Should we remove the DebugBreak in the EP_UNLIKELY macro expansion?

I'd actually like to keep that. It lets us catch exceptions like this rather than silently failing and us never noticing the failures.

noahfalk

A few comments inline, looks good otherwise : )

sywhang · 2021-04-05T23:27:36Z

I'm merging this since CI failure is due to #50753.

…shim_mono # By Aaron Robinson (10) and others # Via GitHub * upstream/main: (108 commits) [mbr] Add Apple sample (dotnet#50740) make EstablishProxyTunnelAsync throw on failure status code from proxy (dotnet#50763) Improve RGB Min Max evaluation performance by using 2 or 3 comparison… (dotnet#50622) [mono] More domain cleanups (dotnet#50479) Fix Crossgen2 of PlatformDefaultMemberFunction methods and calls. (dotnet#50754) Disable EventSource generator in design-time builds (dotnet#50741) Fix X509 test failures on Android (dotnet#50301) Do not confuse fgDispBasicBlocks in fgMorphBlocks (dotnet#50703) Enforce 64KB event payload size limit on EventPipe (dotnet#50600) Reorganize CoreCLR native build to reduce CMake reconfigures when the build system is untouched (dotnet#49906) [mbr] Turn on hot reload for iOS, tvOS and MacCatalyst (dotnet#50458) improve connection scavenge logic by doing zero-byte read (dotnet#50545) Resolve call mdtokens when making tier 1 inline observations (dotnet#50675) Annotate APIs in System.Private.Xml (dotnet#49682) Support compiling against OpenSSL 3 headers Change Configuration.Json to use a regular Dictionary. (dotnet#50611) Remove unused BigNumFromBinary P/Invoke (dotnet#50670) Make Ninja the default CMake generator on Windows for the repo (dotnet#49715) [AppleAppBuilder] Entitlements to run tests on catalyst using the JIT (dotnet#50637) [mono] Fix delegate invokes to dynamic methods in mixed mode. (dotnet#50547) ... # Conflicts: # src/mono/dlls/mscordbi/CMakeLists.txt

sywhang · 2021-04-01T18:46:21Z

ETW has a maximum of 64KB event payload size limit that is not enforced on EventPipe. This causes issues like #50515 to occur where an event that's greater than 100KB can cause the flushing thread's event block to be too small to fit a single event payload, which crashes the application.

josalem · 2021-04-01T19:05:21Z

 	ep_return_false_if_nok (ep_event_is_enabled (ep_event));

+	// Check that the payload size is less than 64 KB (max size for ETW events)
+	ep_return_false_if_nok (ep_event_payload_get_size (payload) <= 64 * 1024);


ETW might limit its payloads to 64KB, but does EventPipe need to? EventSource doesn't explicitly prevent you from having payloads this large, e.g., your test.

It doesn't. But ETW will drop the event if it's greater than 64KB and I'd like to enforce the same behavior here since:
We need some kind of limit regardless. Currently >100KB payloads crash the app. We could enforce a 100KB limit instead of 64KB.
But if we do enforce 100KB limit, that leads to a discrepancy between ETW/EventPipe behavior which we then need to go and figure out how customers can find out whether big events are getting dropped on ETW but not on EventPipe and document why such behavior differences exist.
Another approach is to enforce 1MB payload (which we currently do - just not explicitly but because we just fail to allocate buffers greater than that) and increase the max flushing block size to 1MB but that's a pretty inefficient use of space for sessions listening on much smaller-sized payloads (i.e. counters).

Fair point. Let me mull this over, but I think 64KB is fine. Should we remove the DebugBreak in the EP_UNLIKELY macro expansion?

I'd actually like to keep that. It lets us catch exceptions like this rather than silently failing and us never noticing the failures.

noahfalk

A few comments inline, looks good otherwise : )

sywhang · 2021-04-05T23:27:36Z

I'm merging this since CI failure is due to #50753.

…shim_mono # By Aaron Robinson (10) and others # Via GitHub * upstream/main: (108 commits) [mbr] Add Apple sample (dotnet#50740) make EstablishProxyTunnelAsync throw on failure status code from proxy (dotnet#50763) Improve RGB Min Max evaluation performance by using 2 or 3 comparison… (dotnet#50622) [mono] More domain cleanups (dotnet#50479) Fix Crossgen2 of PlatformDefaultMemberFunction methods and calls. (dotnet#50754) Disable EventSource generator in design-time builds (dotnet#50741) Fix X509 test failures on Android (dotnet#50301) Do not confuse fgDispBasicBlocks in fgMorphBlocks (dotnet#50703) Enforce 64KB event payload size limit on EventPipe (dotnet#50600) Reorganize CoreCLR native build to reduce CMake reconfigures when the build system is untouched (dotnet#49906) [mbr] Turn on hot reload for iOS, tvOS and MacCatalyst (dotnet#50458) improve connection scavenge logic by doing zero-byte read (dotnet#50545) Resolve call mdtokens when making tier 1 inline observations (dotnet#50675) Annotate APIs in System.Private.Xml (dotnet#49682) Support compiling against OpenSSL 3 headers Change Configuration.Json to use a regular Dictionary. (dotnet#50611) Remove unused BigNumFromBinary P/Invoke (dotnet#50670) Make Ninja the default CMake generator on Windows for the repo (dotnet#49715) [AppleAppBuilder] Entitlements to run tests on catalyst using the JIT (dotnet#50637) [mono] Fix delegate invokes to dynamic methods in mixed mode. (dotnet#50547) ... # Conflicts: # src/mono/dlls/mscordbi/CMakeLists.txt

sywhang · 2021-04-01T18:46:21Z

ETW has a maximum of 64KB event payload size limit that is not enforced on EventPipe. This causes issues like #50515 to occur where an event that's greater than 100KB can cause the flushing thread's event block to be too small to fit a single event payload, which crashes the application.

josalem · 2021-04-01T19:05:21Z

 	ep_return_false_if_nok (ep_event_is_enabled (ep_event));

+	// Check that the payload size is less than 64 KB (max size for ETW events)
+	ep_return_false_if_nok (ep_event_payload_get_size (payload) <= 64 * 1024);


ETW might limit its payloads to 64KB, but does EventPipe need to? EventSource doesn't explicitly prevent you from having payloads this large, e.g., your test.

It doesn't. But ETW will drop the event if it's greater than 64KB and I'd like to enforce the same behavior here since:
We need some kind of limit regardless. Currently >100KB payloads crash the app. We could enforce a 100KB limit instead of 64KB.
But if we do enforce 100KB limit, that leads to a discrepancy between ETW/EventPipe behavior which we then need to go and figure out how customers can find out whether big events are getting dropped on ETW but not on EventPipe and document why such behavior differences exist.
Another approach is to enforce 1MB payload (which we currently do - just not explicitly but because we just fail to allocate buffers greater than that) and increase the max flushing block size to 1MB but that's a pretty inefficient use of space for sessions listening on much smaller-sized payloads (i.e. counters).

Fair point. Let me mull this over, but I think 64KB is fine. Should we remove the DebugBreak in the EP_UNLIKELY macro expansion?

I'd actually like to keep that. It lets us catch exceptions like this rather than silently failing and us never noticing the failures.

noahfalk

A few comments inline, looks good otherwise : )

sywhang · 2021-04-05T23:27:36Z

I'm merging this since CI failure is due to #50753.

…shim_mono # By Aaron Robinson (10) and others # Via GitHub * upstream/main: (108 commits) [mbr] Add Apple sample (dotnet#50740) make EstablishProxyTunnelAsync throw on failure status code from proxy (dotnet#50763) Improve RGB Min Max evaluation performance by using 2 or 3 comparison… (dotnet#50622) [mono] More domain cleanups (dotnet#50479) Fix Crossgen2 of PlatformDefaultMemberFunction methods and calls. (dotnet#50754) Disable EventSource generator in design-time builds (dotnet#50741) Fix X509 test failures on Android (dotnet#50301) Do not confuse fgDispBasicBlocks in fgMorphBlocks (dotnet#50703) Enforce 64KB event payload size limit on EventPipe (dotnet#50600) Reorganize CoreCLR native build to reduce CMake reconfigures when the build system is untouched (dotnet#49906) [mbr] Turn on hot reload for iOS, tvOS and MacCatalyst (dotnet#50458) improve connection scavenge logic by doing zero-byte read (dotnet#50545) Resolve call mdtokens when making tier 1 inline observations (dotnet#50675) Annotate APIs in System.Private.Xml (dotnet#49682) Support compiling against OpenSSL 3 headers Change Configuration.Json to use a regular Dictionary. (dotnet#50611) Remove unused BigNumFromBinary P/Invoke (dotnet#50670) Make Ninja the default CMake generator on Windows for the repo (dotnet#49715) [AppleAppBuilder] Entitlements to run tests on catalyst using the JIT (dotnet#50637) [mono] Fix delegate invokes to dynamic methods in mixed mode. (dotnet#50547) ... # Conflicts: # src/mono/dlls/mscordbi/CMakeLists.txt

sywhang · 2021-04-01T18:46:21Z

ETW has a maximum of 64KB event payload size limit that is not enforced on EventPipe. This causes issues like #50515 to occur where an event that's greater than 100KB can cause the flushing thread's event block to be too small to fit a single event payload, which crashes the application.

josalem · 2021-04-01T19:05:21Z

 	ep_return_false_if_nok (ep_event_is_enabled (ep_event));

+	// Check that the payload size is less than 64 KB (max size for ETW events)
+	ep_return_false_if_nok (ep_event_payload_get_size (payload) <= 64 * 1024);


ETW might limit its payloads to 64KB, but does EventPipe need to? EventSource doesn't explicitly prevent you from having payloads this large, e.g., your test.

It doesn't. But ETW will drop the event if it's greater than 64KB and I'd like to enforce the same behavior here since:
We need some kind of limit regardless. Currently >100KB payloads crash the app. We could enforce a 100KB limit instead of 64KB.
But if we do enforce 100KB limit, that leads to a discrepancy between ETW/EventPipe behavior which we then need to go and figure out how customers can find out whether big events are getting dropped on ETW but not on EventPipe and document why such behavior differences exist.
Another approach is to enforce 1MB payload (which we currently do - just not explicitly but because we just fail to allocate buffers greater than that) and increase the max flushing block size to 1MB but that's a pretty inefficient use of space for sessions listening on much smaller-sized payloads (i.e. counters).

Fair point. Let me mull this over, but I think 64KB is fine. Should we remove the DebugBreak in the EP_UNLIKELY macro expansion?

I'd actually like to keep that. It lets us catch exceptions like this rather than silently failing and us never noticing the failures.

noahfalk

A few comments inline, looks good otherwise : )

sywhang · 2021-04-05T23:27:36Z

I'm merging this since CI failure is due to #50753.

…shim_mono # By Aaron Robinson (10) and others # Via GitHub * upstream/main: (108 commits) [mbr] Add Apple sample (dotnet#50740) make EstablishProxyTunnelAsync throw on failure status code from proxy (dotnet#50763) Improve RGB Min Max evaluation performance by using 2 or 3 comparison… (dotnet#50622) [mono] More domain cleanups (dotnet#50479) Fix Crossgen2 of PlatformDefaultMemberFunction methods and calls. (dotnet#50754) Disable EventSource generator in design-time builds (dotnet#50741) Fix X509 test failures on Android (dotnet#50301) Do not confuse fgDispBasicBlocks in fgMorphBlocks (dotnet#50703) Enforce 64KB event payload size limit on EventPipe (dotnet#50600) Reorganize CoreCLR native build to reduce CMake reconfigures when the build system is untouched (dotnet#49906) [mbr] Turn on hot reload for iOS, tvOS and MacCatalyst (dotnet#50458) improve connection scavenge logic by doing zero-byte read (dotnet#50545) Resolve call mdtokens when making tier 1 inline observations (dotnet#50675) Annotate APIs in System.Private.Xml (dotnet#49682) Support compiling against OpenSSL 3 headers Change Configuration.Json to use a regular Dictionary. (dotnet#50611) Remove unused BigNumFromBinary P/Invoke (dotnet#50670) Make Ninja the default CMake generator on Windows for the repo (dotnet#49715) [AppleAppBuilder] Entitlements to run tests on catalyst using the JIT (dotnet#50637) [mono] Fix delegate invokes to dynamic methods in mixed mode. (dotnet#50547) ... # Conflicts: # src/mono/dlls/mscordbi/CMakeLists.txt

sywhang · 2021-04-01T18:46:21Z

ETW has a maximum of 64KB event payload size limit that is not enforced on EventPipe. This causes issues like #50515 to occur where an event that's greater than 100KB can cause the flushing thread's event block to be too small to fit a single event payload, which crashes the application.

josalem · 2021-04-01T19:05:21Z

 	ep_return_false_if_nok (ep_event_is_enabled (ep_event));

+	// Check that the payload size is less than 64 KB (max size for ETW events)
+	ep_return_false_if_nok (ep_event_payload_get_size (payload) <= 64 * 1024);


ETW might limit its payloads to 64KB, but does EventPipe need to? EventSource doesn't explicitly prevent you from having payloads this large, e.g., your test.

It doesn't. But ETW will drop the event if it's greater than 64KB and I'd like to enforce the same behavior here since:
We need some kind of limit regardless. Currently >100KB payloads crash the app. We could enforce a 100KB limit instead of 64KB.
But if we do enforce 100KB limit, that leads to a discrepancy between ETW/EventPipe behavior which we then need to go and figure out how customers can find out whether big events are getting dropped on ETW but not on EventPipe and document why such behavior differences exist.
Another approach is to enforce 1MB payload (which we currently do - just not explicitly but because we just fail to allocate buffers greater than that) and increase the max flushing block size to 1MB but that's a pretty inefficient use of space for sessions listening on much smaller-sized payloads (i.e. counters).

Fair point. Let me mull this over, but I think 64KB is fine. Should we remove the DebugBreak in the EP_UNLIKELY macro expansion?

I'd actually like to keep that. It lets us catch exceptions like this rather than silently failing and us never noticing the failures.

noahfalk

A few comments inline, looks good otherwise : )

sywhang · 2021-04-05T23:27:36Z

I'm merging this since CI failure is due to #50753.

…shim_mono # By Aaron Robinson (10) and others # Via GitHub * upstream/main: (108 commits) [mbr] Add Apple sample (dotnet#50740) make EstablishProxyTunnelAsync throw on failure status code from proxy (dotnet#50763) Improve RGB Min Max evaluation performance by using 2 or 3 comparison… (dotnet#50622) [mono] More domain cleanups (dotnet#50479) Fix Crossgen2 of PlatformDefaultMemberFunction methods and calls. (dotnet#50754) Disable EventSource generator in design-time builds (dotnet#50741) Fix X509 test failures on Android (dotnet#50301) Do not confuse fgDispBasicBlocks in fgMorphBlocks (dotnet#50703) Enforce 64KB event payload size limit on EventPipe (dotnet#50600) Reorganize CoreCLR native build to reduce CMake reconfigures when the build system is untouched (dotnet#49906) [mbr] Turn on hot reload for iOS, tvOS and MacCatalyst (dotnet#50458) improve connection scavenge logic by doing zero-byte read (dotnet#50545) Resolve call mdtokens when making tier 1 inline observations (dotnet#50675) Annotate APIs in System.Private.Xml (dotnet#49682) Support compiling against OpenSSL 3 headers Change Configuration.Json to use a regular Dictionary. (dotnet#50611) Remove unused BigNumFromBinary P/Invoke (dotnet#50670) Make Ninja the default CMake generator on Windows for the repo (dotnet#49715) [AppleAppBuilder] Entitlements to run tests on catalyst using the JIT (dotnet#50637) [mono] Fix delegate invokes to dynamic methods in mixed mode. (dotnet#50547) ... # Conflicts: # src/mono/dlls/mscordbi/CMakeLists.txt

sywhang · 2021-04-01T18:46:21Z

ETW has a maximum of 64KB event payload size limit that is not enforced on EventPipe. This causes issues like #50515 to occur where an event that's greater than 100KB can cause the flushing thread's event block to be too small to fit a single event payload, which crashes the application.

josalem · 2021-04-01T19:05:21Z

 	ep_return_false_if_nok (ep_event_is_enabled (ep_event));

+	// Check that the payload size is less than 64 KB (max size for ETW events)
+	ep_return_false_if_nok (ep_event_payload_get_size (payload) <= 64 * 1024);


ETW might limit its payloads to 64KB, but does EventPipe need to? EventSource doesn't explicitly prevent you from having payloads this large, e.g., your test.

It doesn't. But ETW will drop the event if it's greater than 64KB and I'd like to enforce the same behavior here since:
We need some kind of limit regardless. Currently >100KB payloads crash the app. We could enforce a 100KB limit instead of 64KB.
But if we do enforce 100KB limit, that leads to a discrepancy between ETW/EventPipe behavior which we then need to go and figure out how customers can find out whether big events are getting dropped on ETW but not on EventPipe and document why such behavior differences exist.
Another approach is to enforce 1MB payload (which we currently do - just not explicitly but because we just fail to allocate buffers greater than that) and increase the max flushing block size to 1MB but that's a pretty inefficient use of space for sessions listening on much smaller-sized payloads (i.e. counters).

Fair point. Let me mull this over, but I think 64KB is fine. Should we remove the DebugBreak in the EP_UNLIKELY macro expansion?

I'd actually like to keep that. It lets us catch exceptions like this rather than silently failing and us never noticing the failures.

noahfalk

A few comments inline, looks good otherwise : )

sywhang · 2021-04-05T23:27:36Z

I'm merging this since CI failure is due to #50753.

…shim_mono # By Aaron Robinson (10) and others # Via GitHub * upstream/main: (108 commits) [mbr] Add Apple sample (dotnet#50740) make EstablishProxyTunnelAsync throw on failure status code from proxy (dotnet#50763) Improve RGB Min Max evaluation performance by using 2 or 3 comparison… (dotnet#50622) [mono] More domain cleanups (dotnet#50479) Fix Crossgen2 of PlatformDefaultMemberFunction methods and calls. (dotnet#50754) Disable EventSource generator in design-time builds (dotnet#50741) Fix X509 test failures on Android (dotnet#50301) Do not confuse fgDispBasicBlocks in fgMorphBlocks (dotnet#50703) Enforce 64KB event payload size limit on EventPipe (dotnet#50600) Reorganize CoreCLR native build to reduce CMake reconfigures when the build system is untouched (dotnet#49906) [mbr] Turn on hot reload for iOS, tvOS and MacCatalyst (dotnet#50458) improve connection scavenge logic by doing zero-byte read (dotnet#50545) Resolve call mdtokens when making tier 1 inline observations (dotnet#50675) Annotate APIs in System.Private.Xml (dotnet#49682) Support compiling against OpenSSL 3 headers Change Configuration.Json to use a regular Dictionary. (dotnet#50611) Remove unused BigNumFromBinary P/Invoke (dotnet#50670) Make Ninja the default CMake generator on Windows for the repo (dotnet#49715) [AppleAppBuilder] Entitlements to run tests on catalyst using the JIT (dotnet#50637) [mono] Fix delegate invokes to dynamic methods in mixed mode. (dotnet#50547) ... # Conflicts: # src/mono/dlls/mscordbi/CMakeLists.txt

sywhang · 2021-04-01T18:46:21Z

ETW has a maximum of 64KB event payload size limit that is not enforced on EventPipe. This causes issues like #50515 to occur where an event that's greater than 100KB can cause the flushing thread's event block to be too small to fit a single event payload, which crashes the application.

josalem · 2021-04-01T19:05:21Z

 	ep_return_false_if_nok (ep_event_is_enabled (ep_event));

+	// Check that the payload size is less than 64 KB (max size for ETW events)
+	ep_return_false_if_nok (ep_event_payload_get_size (payload) <= 64 * 1024);


ETW might limit its payloads to 64KB, but does EventPipe need to? EventSource doesn't explicitly prevent you from having payloads this large, e.g., your test.

It doesn't. But ETW will drop the event if it's greater than 64KB and I'd like to enforce the same behavior here since:
We need some kind of limit regardless. Currently >100KB payloads crash the app. We could enforce a 100KB limit instead of 64KB.
But if we do enforce 100KB limit, that leads to a discrepancy between ETW/EventPipe behavior which we then need to go and figure out how customers can find out whether big events are getting dropped on ETW but not on EventPipe and document why such behavior differences exist.
Another approach is to enforce 1MB payload (which we currently do - just not explicitly but because we just fail to allocate buffers greater than that) and increase the max flushing block size to 1MB but that's a pretty inefficient use of space for sessions listening on much smaller-sized payloads (i.e. counters).

Fair point. Let me mull this over, but I think 64KB is fine. Should we remove the DebugBreak in the EP_UNLIKELY macro expansion?

I'd actually like to keep that. It lets us catch exceptions like this rather than silently failing and us never noticing the failures.

noahfalk

A few comments inline, looks good otherwise : )

sywhang · 2021-04-05T23:27:36Z

I'm merging this since CI failure is due to #50753.

…shim_mono # By Aaron Robinson (10) and others # Via GitHub * upstream/main: (108 commits) [mbr] Add Apple sample (dotnet#50740) make EstablishProxyTunnelAsync throw on failure status code from proxy (dotnet#50763) Improve RGB Min Max evaluation performance by using 2 or 3 comparison… (dotnet#50622) [mono] More domain cleanups (dotnet#50479) Fix Crossgen2 of PlatformDefaultMemberFunction methods and calls. (dotnet#50754) Disable EventSource generator in design-time builds (dotnet#50741) Fix X509 test failures on Android (dotnet#50301) Do not confuse fgDispBasicBlocks in fgMorphBlocks (dotnet#50703) Enforce 64KB event payload size limit on EventPipe (dotnet#50600) Reorganize CoreCLR native build to reduce CMake reconfigures when the build system is untouched (dotnet#49906) [mbr] Turn on hot reload for iOS, tvOS and MacCatalyst (dotnet#50458) improve connection scavenge logic by doing zero-byte read (dotnet#50545) Resolve call mdtokens when making tier 1 inline observations (dotnet#50675) Annotate APIs in System.Private.Xml (dotnet#49682) Support compiling against OpenSSL 3 headers Change Configuration.Json to use a regular Dictionary. (dotnet#50611) Remove unused BigNumFromBinary P/Invoke (dotnet#50670) Make Ninja the default CMake generator on Windows for the repo (dotnet#49715) [AppleAppBuilder] Entitlements to run tests on catalyst using the JIT (dotnet#50637) [mono] Fix delegate invokes to dynamic methods in mixed mode. (dotnet#50547) ... # Conflicts: # src/mono/dlls/mscordbi/CMakeLists.txt

sywhang added 2 commits March 30, 2021 23:13

allow >100KB events to be written
972601a

prevent events > 64KB from being written
77dd874

sywhang requested review from josalem and noahfalk April 1, 2021 18:46

ghost added the area-Tracing-coreclr label Apr 1, 2021

josalem reviewed Apr 1, 2021
View reviewed changes

iSazonov reviewed Apr 2, 2021
View reviewed changes

Comment threadsrc/tests/tracing/eventpipe/bigevent/bigevent.cs

noahfalk approved these changes Apr 3, 2021
View reviewed changes

Comment threadsrc/native/eventpipe/ep-file.c Outdated
Comment threadsrc/native/eventpipe/ep-buffer-manager.c Outdated

sywhang added 5 commits April 5, 2021 09:48

code review feedback
8e33754

increment seq num
60339ee

fix build
2efcc89

fix build on clang
baf67b5

fix mono build
a38660c

josalem approved these changes Apr 5, 2021
View reviewed changes

Comment threadsrc/native/eventpipe/ep-buffer-manager.c

sywhang merged commit bdd7ca0 into dotnet:mainApr 5, 2021

sywhang deleted the dev/suwhang/big-event branch April 5, 2021 23:27

sywhang mentioned this pull request Apr 5, 2021
Sending large payload crashes EventPipe #50515
Closed

ghost locked as resolved and limited conversation to collaborators May 6, 2021

karelz added this to the 6.0.0 milestone May 20, 2021

sywhang added 2 commits March 30, 2021 23:13

allow >100KB events to be written
972601a

prevent events > 64KB from being written
77dd874

sywhang requested review from josalem and noahfalk April 1, 2021 18:46

ghost added the area-Tracing-coreclr label Apr 1, 2021

josalem reviewed Apr 1, 2021
View reviewed changes

iSazonov reviewed Apr 2, 2021
View reviewed changes

Comment threadsrc/tests/tracing/eventpipe/bigevent/bigevent.cs

noahfalk approved these changes Apr 3, 2021
View reviewed changes

Comment threadsrc/native/eventpipe/ep-file.c Outdated
Comment threadsrc/native/eventpipe/ep-buffer-manager.c Outdated

sywhang added 5 commits April 5, 2021 09:48

code review feedback
8e33754

increment seq num
60339ee

fix build
2efcc89

fix build on clang
baf67b5

fix mono build
a38660c

josalem approved these changes Apr 5, 2021
View reviewed changes

Comment threadsrc/native/eventpipe/ep-buffer-manager.c

sywhang merged commit bdd7ca0 into dotnet:mainApr 5, 2021

sywhang deleted the dev/suwhang/big-event branch April 5, 2021 23:27

sywhang mentioned this pull request Apr 5, 2021
Sending large payload crashes EventPipe #50515
Closed

ghost locked as resolved and limited conversation to collaborators May 6, 2021

karelz added this to the 6.0.0 milestone May 20, 2021

sywhang added 2 commits March 30, 2021 23:13

allow >100KB events to be written
972601a

prevent events > 64KB from being written
77dd874

sywhang requested review from josalem and noahfalk April 1, 2021 18:46

ghost added the area-Tracing-coreclr label Apr 1, 2021

josalem reviewed Apr 1, 2021
View reviewed changes

iSazonov reviewed Apr 2, 2021
View reviewed changes

Comment threadsrc/tests/tracing/eventpipe/bigevent/bigevent.cs

noahfalk approved these changes Apr 3, 2021
View reviewed changes

Comment threadsrc/native/eventpipe/ep-file.c Outdated
Comment threadsrc/native/eventpipe/ep-buffer-manager.c Outdated

sywhang added 5 commits April 5, 2021 09:48

code review feedback
8e33754

increment seq num
60339ee

fix build
2efcc89

fix build on clang
baf67b5

fix mono build
a38660c

josalem approved these changes Apr 5, 2021
View reviewed changes

Comment threadsrc/native/eventpipe/ep-buffer-manager.c

sywhang merged commit bdd7ca0 into dotnet:mainApr 5, 2021

sywhang deleted the dev/suwhang/big-event branch April 5, 2021 23:27

sywhang mentioned this pull request Apr 5, 2021
Sending large payload crashes EventPipe #50515
Closed

ghost locked as resolved and limited conversation to collaborators May 6, 2021

karelz added this to the 6.0.0 milestone May 20, 2021

sywhang added 2 commits March 30, 2021 23:13

allow >100KB events to be written
972601a

prevent events > 64KB from being written
77dd874

sywhang requested review from josalem and noahfalk April 1, 2021 18:46

ghost added the area-Tracing-coreclr label Apr 1, 2021

josalem reviewed Apr 1, 2021
View reviewed changes

iSazonov reviewed Apr 2, 2021
View reviewed changes

Comment threadsrc/tests/tracing/eventpipe/bigevent/bigevent.cs

noahfalk approved these changes Apr 3, 2021
View reviewed changes

Comment threadsrc/native/eventpipe/ep-file.c Outdated
Comment threadsrc/native/eventpipe/ep-buffer-manager.c Outdated

sywhang added 5 commits April 5, 2021 09:48

code review feedback
8e33754

increment seq num
60339ee

fix build
2efcc89

fix build on clang
baf67b5

fix mono build
a38660c

josalem approved these changes Apr 5, 2021
View reviewed changes

Comment threadsrc/native/eventpipe/ep-buffer-manager.c

sywhang merged commit bdd7ca0 into dotnet:mainApr 5, 2021

sywhang deleted the dev/suwhang/big-event branch April 5, 2021 23:27

sywhang mentioned this pull request Apr 5, 2021
Sending large payload crashes EventPipe #50515
Closed

ghost locked as resolved and limited conversation to collaborators May 6, 2021

karelz added this to the 6.0.0 milestone May 20, 2021

sywhang added 2 commits March 30, 2021 23:13

allow >100KB events to be written
972601a

prevent events > 64KB from being written
77dd874

sywhang requested review from josalem and noahfalk April 1, 2021 18:46

ghost added the area-Tracing-coreclr label Apr 1, 2021

josalem reviewed Apr 1, 2021
View reviewed changes

iSazonov reviewed Apr 2, 2021
View reviewed changes

Comment threadsrc/tests/tracing/eventpipe/bigevent/bigevent.cs

noahfalk approved these changes Apr 3, 2021
View reviewed changes

Comment threadsrc/native/eventpipe/ep-file.c Outdated
Comment threadsrc/native/eventpipe/ep-buffer-manager.c Outdated

sywhang added 5 commits April 5, 2021 09:48

code review feedback
8e33754

increment seq num
60339ee

fix build
2efcc89

fix build on clang
baf67b5

fix mono build
a38660c

josalem approved these changes Apr 5, 2021
View reviewed changes

Comment threadsrc/native/eventpipe/ep-buffer-manager.c

sywhang merged commit bdd7ca0 into dotnet:mainApr 5, 2021

sywhang deleted the dev/suwhang/big-event branch April 5, 2021 23:27

sywhang mentioned this pull request Apr 5, 2021
Sending large payload crashes EventPipe #50515
Closed

ghost locked as resolved and limited conversation to collaborators May 6, 2021

karelz added this to the 6.0.0 milestone May 20, 2021

sywhang added 2 commits March 30, 2021 23:13

allow >100KB events to be written
972601a

prevent events > 64KB from being written
77dd874

sywhang requested review from josalem and noahfalk April 1, 2021 18:46

ghost added the area-Tracing-coreclr label Apr 1, 2021

josalem reviewed Apr 1, 2021
View reviewed changes

iSazonov reviewed Apr 2, 2021
View reviewed changes

Comment threadsrc/tests/tracing/eventpipe/bigevent/bigevent.cs

noahfalk approved these changes Apr 3, 2021
View reviewed changes

Comment threadsrc/native/eventpipe/ep-file.c Outdated
Comment threadsrc/native/eventpipe/ep-buffer-manager.c Outdated

sywhang added 5 commits April 5, 2021 09:48

code review feedback
8e33754

increment seq num
60339ee

fix build
2efcc89

fix build on clang
baf67b5

fix mono build
a38660c

josalem approved these changes Apr 5, 2021
View reviewed changes

Comment threadsrc/native/eventpipe/ep-buffer-manager.c

sywhang merged commit bdd7ca0 into dotnet:mainApr 5, 2021

sywhang deleted the dev/suwhang/big-event branch April 5, 2021 23:27

sywhang mentioned this pull request Apr 5, 2021
Sending large payload crashes EventPipe #50515
Closed

ghost locked as resolved and limited conversation to collaborators May 6, 2021

karelz added this to the 6.0.0 milestone May 20, 2021

sywhang added 2 commits March 30, 2021 23:13

allow >100KB events to be written
972601a

prevent events > 64KB from being written
77dd874

sywhang requested review from josalem and noahfalk April 1, 2021 18:46

ghost added the area-Tracing-coreclr label Apr 1, 2021

josalem reviewed Apr 1, 2021
View reviewed changes

iSazonov reviewed Apr 2, 2021
View reviewed changes

Comment threadsrc/tests/tracing/eventpipe/bigevent/bigevent.cs

noahfalk approved these changes Apr 3, 2021
View reviewed changes

Comment threadsrc/native/eventpipe/ep-file.c Outdated
Comment threadsrc/native/eventpipe/ep-buffer-manager.c Outdated

sywhang added 5 commits April 5, 2021 09:48

code review feedback
8e33754

increment seq num
60339ee

fix build
2efcc89

fix build on clang
baf67b5

fix mono build
a38660c

josalem approved these changes Apr 5, 2021
View reviewed changes

Comment threadsrc/native/eventpipe/ep-buffer-manager.c

sywhang merged commit bdd7ca0 into dotnet:mainApr 5, 2021

sywhang deleted the dev/suwhang/big-event branch April 5, 2021 23:27

sywhang mentioned this pull request Apr 5, 2021
Sending large payload crashes EventPipe #50515
Closed

ghost locked as resolved and limited conversation to collaborators May 6, 2021

karelz added this to the 6.0.0 milestone May 20, 2021

sywhang added 2 commits March 30, 2021 23:13

allow >100KB events to be written
972601a

prevent events > 64KB from being written
77dd874

sywhang requested review from josalem and noahfalk April 1, 2021 18:46

ghost added the area-Tracing-coreclr label Apr 1, 2021

josalem reviewed Apr 1, 2021
View reviewed changes

iSazonov reviewed Apr 2, 2021
View reviewed changes

Comment threadsrc/tests/tracing/eventpipe/bigevent/bigevent.cs

noahfalk approved these changes Apr 3, 2021
View reviewed changes

Comment threadsrc/native/eventpipe/ep-file.c Outdated
Comment threadsrc/native/eventpipe/ep-buffer-manager.c Outdated

sywhang added 5 commits April 5, 2021 09:48

code review feedback
8e33754

increment seq num
60339ee

fix build
2efcc89

fix build on clang
baf67b5

fix mono build
a38660c

josalem approved these changes Apr 5, 2021
View reviewed changes

Comment threadsrc/native/eventpipe/ep-buffer-manager.c

sywhang merged commit bdd7ca0 into dotnet:mainApr 5, 2021

sywhang deleted the dev/suwhang/big-event branch April 5, 2021 23:27

sywhang mentioned this pull request Apr 5, 2021
Sending large payload crashes EventPipe #50515
Closed

ghost locked as resolved and limited conversation to collaborators May 6, 2021

karelz added this to the 6.0.0 milestone May 20, 2021

Uh oh!

Conversation

sywhang commented Apr 1, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sywhangApr 1, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

noahfalk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sywhang commented Apr 5, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

sywhang commented Apr 1, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sywhangApr 1, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

noahfalk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sywhang commented Apr 5, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

sywhang commented Apr 1, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sywhangApr 1, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

noahfalk left a comment

Choose a reason for hiding this comment

Uh oh!

sywhangApr 1, 2021•
edited
Loading

sywhangApr 1, 2021•
edited
Loading

sywhangApr 1, 2021•
edited
Loading

sywhangApr 1, 2021•
edited
Loading