GC Bridge in NativeAOT is flakey

[jonpryor]

Android framework version

net10.0-android (Preview)

Affected platform version

.NET 10 RC2

Description

The Java GC Bridge under NativeAOT is flakey, which may result in apps crashing.

(Shock, a nearly unmentionable preview feature is flakey! Where are my smelling salts?)

Steps to Reproduce

This was observed with the unoplatform/uno.chefs sample app, built for NativeAOT:

git clone https://github.com/unoplatform/uno.chefs.git
git checkout f8c5943efa4eb702a066d09c6e3ef389ade3b973
cat >patch <<EOF
diff --git a/global.json b/global.json
index 7f65a2cc..d6120d9e 100644
--- a/global.json
+++ b/global.json
@@ -1,8 +1,8 @@
 {
 "msbuild-sdks": {
- "Uno.Sdk": "6.3.28"
+ "Uno.Sdk": "6.4.0-dev.57"
 },
 "sdk": {
- "allowPrerelease": false
+ "allowPrerelease": true
 }
 }
EOF
git apply < patch

dotnet publish -c Release -f net10.0-android -r android-x64 -p:TargetFrameworkOverride=net10.0-android \
 -p:PublishAot=true -p:UseSkiaRendering=false \
 -p:AndroidPackageFormat=apk -bl Chefs/Chefs.csproj

adb install Chefs/bin/Release/net10.0-android/android-x64/publish/uno.platform.chefs.skia-Signed.apk

Get your adb logcat log going:

adb logcat > log.txt &
less log.txt

Then repeat the following commands in a loop until you see it crash:

adb shell am start uno.platform.chefs.skia/crc6413a18e1a0e43bcb3.MainActivity
# wait ~10 seconds
adb shell am force-stop uno.platform.chefs.skia

When I have observed the crash, no app interaction was required. In limited testing, crash happened one in 5-10 app runs.

Did you find any workaround?

NativeAOT is not even Preview! (Who knew?)

Don't use NativeAOT in a production capacity.

Relevant log output

W form.chefs.ski: Unexpected CPU variant for X86 using defaults: x86_64
E form.chefs.ski: Not starting debugger since process cannot load the jdwp agent.
D NativeAotRuntimeProvider: NativeAotRuntimeProvider()
D NativeAotRuntimeProvider: NativeAotRuntimeProvider.attachInfo(): calling JavaInteropRuntime.init()…
D JavaInteropRuntime: Loading Chefs.so...
I JavaInteropRuntime: JNI_OnLoad()
D DOTNET : AndroidCryptoNative_InitLibraryOnLoad: jint AndroidCryptoNative_InitLibraryOnLoad(JavaVM *, void *) in /__w/1/s/src/runtime/src/native/libs/System.Security.Cryptography.Native.Android/pal_jni.c
I DOTNET : GetOptionalMethod: optional method setApplicationProtocols ([Ljava/lang/String;)V was not found
D DOTNET : GetOptionalClassGRef: optional class com/android/org/conscrypt/OpenSSLEngineImpl was not found
I DOTNET : GetOptionalMethod: optional method getApplicationProtocol ()Ljava/lang/String; was not found
D NativeAotRuntimeProvider: NativeAotRuntimeProvider.onCreate()
D AppCompatDelegate: Checking for metadata for AppLocalesMetadataHolderService : Service not found
D OpenGLRenderer: Skia GL Pipeline
F form.chefs.ski: indirect_reference_table.cc:59] JNI ERROR (app bug): accessed deleted Global 0x367a
F form.chefs.ski: runtime.cc:558] Runtime aborting...
F form.chefs.ski: runtime.cc:558] Dumping all threads without appropriate locks held: thread list lock
F form.chefs.ski: runtime.cc:558] All threads:
F form.chefs.ski: runtime.cc:558] DALVIK THREADS (21):
F form.chefs.ski: runtime.cc:558] "main" prio=10 tid=1 Runnable
F form.chefs.ski: runtime.cc:558] | group="" sCount=0 dsCount=0 flags=2 obj=0x75c45a48 self=0x717b96c14c00
F form.chefs.ski: runtime.cc:558] | sysTid=5947 nice=-10 cgrp=default sched=0/0 handle=0x717c1c452548
F form.chefs.ski: runtime.cc:558] | state=R schedstat=( 683883508 100769853 782 ) utm=59 stm=9 core=0 HZ=100
F form.chefs.ski: runtime.cc:558] | stack=0x7ffe6ddbd000-0x7ffe6ddbf000 stackSize=8MB
F form.chefs.ski: runtime.cc:558] | held mutexes= "abort lock" "mutator lock"(shared held)
F form.chefs.ski: runtime.cc:558] native: #00 pc 0000000000403e33 /system/lib64/libart.so (art::DumpNativeStack(std::__1::basic_ostream<char, std::__1::char_traits<char>>&, int, BacktraceMap*, char const*, art::ArtMethod*, void*, bool)+227)
F form.chefs.ski: runtime.cc:558] native: #01 pc 00000000004ee1f7 /system/lib64/libart.so (art::Thread::DumpStack(std::__1::basic_ostream<char, std::__1::char_traits<char>>&, bool, BacktraceMap*, bool) const+343)
F form.chefs.ski: runtime.cc:558] native: #02 pc 0000000000509d8e /system/lib64/libart.so (art::DumpCheckpoint::Run(art::Thread*)+846)
F form.chefs.ski: runtime.cc:558] native: #03 pc 00000000005020f9 /system/lib64/libart.so (art::ThreadList::RunCheckpoint(art::Closure*, art::Closure*)+489)
F form.chefs.ski: runtime.cc:558] native: #04 pc 00000000005012c4 /system/lib64/libart.so (art::ThreadList::Dump(std::__1::basic_ostream<char, std::__1::char_traits<char>>&, bool)+580)
F form.chefs.ski: runtime.cc:558] native: #05 pc 00000000004bf6ea /system/lib64/libart.so (art::Runtime::Abort(char const*)+282)
F form.chefs.ski: runtime.cc:558] native: #06 pc 0000000000008cdc /system/lib64/libbase.so (android::base::LogMessage::~LogMessage()+716)
F form.chefs.ski: runtime.cc:558] native: #07 pc 000000000025b863 /system/lib64/libart.so (art::IndirectReferenceTable::AbortIfNoCheckJNI(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&)+243)
F form.chefs.ski: runtime.cc:558] native: #08 pc 00000000003252f2 /system/lib64/libart.so (art::IndirectReferenceTable::GetChecked(void*) const+466)
F form.chefs.ski: runtime.cc:558] native: #09 pc 000000000031fc32 /system/lib64/libart.so (art::JavaVMExt::DecodeGlobal(void*)+18)
F form.chefs.ski: runtime.cc:558] native: #10 pc 00000000004f41c5 /system/lib64/libart.so (art::Thread::DecodeJObject(_jobject*) const+165)
F form.chefs.ski: runtime.cc:558] native: #11 pc 00000000003c507f /system/lib64/libart.so (_ZN3art3JNI23DecodeAndCheckArrayTypeIP12_jfloatArrayfNS_6mirror14PrimitiveArrayIfEEEEPT1_RNS_18ScopedObjectAccessET_PKcSD_+47)
F form.chefs.ski: runtime.cc:558] native: #12 pc 00000000003b9640 /system/lib64/libart.so (art::JNI::GetFloatArrayRegion(_JNIEnv*, _jfloatArray*, int, int, float*)+800)
F form.chefs.ski: runtime.cc:558] native: #13 pc 0000000003332767 /data/app/uno.platform.chefs.skia-Hn38_Q2Ao0LTHQhJNMH6_A==/lib/x86_64/libChefs.so (offset 2f06000) (???)
F form.chefs.ski: runtime.cc:558] native: #14 pc 0000000003abde5c /data/app/uno.platform.chefs.skia-Hn38_Q2Ao0LTHQhJNMH6_A==/lib/x86_64/libChefs.so (offset 2f06000) (???)
F form.chefs.ski: runtime.cc:558] native: #15 pc 00000000000147f7 <anonymous:71798f400000> (???)
F form.chefs.ski: runtime.cc:558] at crc64fe881b1858fb9a5d.SimpleOrientationSensor_OrientationListener.n_onSensorChanged(Native method)
F form.chefs.ski: runtime.cc:558] at crc64fe881b1858fb9a5d.SimpleOrientationSensor_OrientationListener.onSensorChanged(SimpleOrientationSensor_OrientationListener.java:37)
F form.chefs.ski: runtime.cc:558] at android.hardware.SystemSensorManager$SensorEventQueue.dispatchSensorEvent(SystemSensorManager.java:833)
F form.chefs.ski: runtime.cc:558] at android.os.MessageQueue.nativePollOnce(Native method)
F form.chefs.ski: runtime.cc:558] at android.os.MessageQueue.next(MessageQueue.java:326)
F form.chefs.ski: runtime.cc:558] at android.os.Looper.loop(Looper.java:160)
F form.chefs.ski: runtime.cc:558] at android.app.ActivityThread.main(ActivityThread.java:6669)
F form.chefs.ski: runtime.cc:558] at java.lang.reflect.Method.invoke(Native method)
F form.chefs.ski: runtime.cc:558] at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
F form.chefs.ski: runtime.cc:558] at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)
F form.chefs.ski: runtime.cc:558] 
F form.chefs.ski: runtime.cc:558] "Jit thread pool worker thread 0" prio=10 tid=2 Native
F form.chefs.ski: runtime.cc:558] | group="" sCount=1 dsCount=0 flags=1 obj=0x12cc0300 self=0x717b9080e000
F form.chefs.ski: runtime.cc:558] | sysTid=5950 nice=9 cgrp=default sched=0/0 handle=0x717b90b274f0
F form.chefs.ski: runtime.cc:558] | state=S schedstat=( 3308423 2428535 10 ) utm=0 stm=0 core=1 HZ=100
F form.chefs.ski: runtime.cc:558] | stack=0x717b90a29000-0x717b90a2b000 stackSize=1021KB
F form.chefs.ski: runtime.cc:558] | held mutexes=
F form.chefs.ski: runtime.cc:558] kernel: (couldn't read /proc/self/task/5950/stack)
F form.chefs.ski: runtime.cc:558] native: #00 pc 0000000000026b96 /system/lib64/libc.so (syscall+22)
F form.chefs.ski: runtime.cc:558] native: #01 pc 00000000000d70bc /system/lib64/libart.so (art::ConditionVariable::WaitHoldingLocks(art::Thread*)+108)
F form.chefs.ski: runtime.cc:558] native: #02 pc 000000000050b82c /system/lib64/libart.so (art::ThreadPool::GetTask(art::Thread*)+316)
F form.chefs.ski: runtime.cc:558] native: #03 pc 000000000050ac11 /system/lib64/libart.so (art::ThreadPoolWorker::Run()+97)
F form.chefs.ski: runtime.cc:558] native: #04 pc 000000000050a6a3 /system/lib64/libart.so (art::ThreadPoolWorker::Callback(void*)+131)
F form.chefs.ski: runtime.cc:558] native: #05 pc 0000000000092bab /system/lib64/libc.so (__pthread_start(void*)+27)
F form.chefs.ski: runtime.cc:558] native: #06 pc 000000000002af2d /system/lib64/libc.so (__start_thread+61)
F form.chefs.ski: runtime.cc:558] (no managed stack frames)
F form.chefs.ski: runtime.cc:558] 
F form.chefs.ski: runtime.cc:558] "Signal Catcher" prio=10 tid=3 WaitingInMainSignalCatcherLoop
F form.chefs.ski: runtime.cc:558] | group="" sCount=1 dsCount=0 flags=1 obj=0x12cc0388 self=0x717b96c16400
F form.chefs.ski: runtime.cc:558] | sysTid=5953 nice=0 cgrp=default sched=0/0 handle=0x717b907ff4f0
F form.chefs.ski: runtime.cc:558] | state=S schedstat=( 248746 523019 2 ) utm=0 stm=0 core=0 HZ=100
F form.chefs.ski: runtime.cc:558] | stack=0x717b90704000-0x717b90706000 stackSize=1009KB
F form.chefs.ski: runtime.cc:558] | held mutexes=
F form.chefs.ski: runtime.cc:558] kernel: (couldn't read /proc/self/task/5953/stack)
F form.chefs.ski: runtime.cc:558] native: #00 pc 000000000007e45a /system/lib64/libc.so (__rt_sigtimedwait+10)
F form.chefs.ski: runtime.cc:558] native: #01 pc 0000000000035622 /system/lib64/libc.so (sigwait+66)
F form.chefs.ski: runtime.cc:558] native: #02 pc 00000000004dab1a /system/lib64/libart.so (art::SignalCatcher::WaitForSignal(art::Thread*, art::SignalSet&)+250)
F form.chefs.ski: runtime.cc:558] native: #03 pc 00000000004d9377 /system/lib64/libart.so (art::SignalCatcher::Run(void*)+279)
F form.chefs.ski: runtime.cc:558] native: #04 pc 0000000000092bab /system/lib64/libc.so (__pthread_start(void*)+27)
F form.chefs.ski: runtime.cc:558] native: #05 pc 000000000002af2d /system/lib64/libc.so (__start_thread+61)
F form.chefs.ski: runtime.cc:558] (no managed stack frames)
F form.chefs.ski: runtime.cc:558] 
F form.chefs.ski: runtime.cc:558] "FinalizerDaemon" prio=10 tid=4 Waiting
F form.chefs.ski: runtime.cc:558] | group="" sCount=1 dsCount=0 flags=1 obj=0x12cc0410 self=0x717b96dc0c00
F form.chefs.ski: runtime.cc:558] | sysTid=5955 nice=4 cgrp=default sched=0/0 handle=0x717b82ce04f0
F form.chefs.ski: runtime.cc:558] | state=S schedstat=( 2950721 1635705 15 ) utm=0 stm=0 core=1 HZ=100
F form.chefs.ski: runtime.cc:558] | stack=0x717b82bdd000-0x717b82bdf000 stackSize=1041KB
F form.chefs.ski: runtime.cc:558] | held mutexes=
F form.chefs.ski: runtime.cc:558] kernel: (couldn't read /proc/self/task/5955/stack)
F form.chefs.ski: runtime.cc:558] native: #00 pc 0000000000026b96 /system/lib64/libc.so (syscall+22)
F form.chefs.ski: runtime.cc:558] native: #01 pc 00000000000d70bc /system/lib64/libart.so (art::ConditionVariable::WaitHoldingLocks(art::Thread*)+108)
F form.chefs.ski: runtime.cc:558] native: #02 pc 00000000003ff20f /system/lib64/libart.so (art::Monitor::Wait(art::Thread*, long, int, bool, art::ThreadState)+655)
F form.chefs.ski: runtime.cc:558] native: #03 pc 0000000000400db1 /system/lib64/libart.so (art::Monitor::Wait(art::Thread*, art::mirror::Object*, long, int, bool, art::ThreadState)+449)
F form.chefs.ski: runtime.cc:558] at java.lang.Object.wait(Native method)
F form.chefs.ski: runtime.cc:558] - waiting on <0x0b94c8fe> (a java.lang.Object)
F form.chefs.ski: runtime.cc:558] at java.lang.Object.wait(Object.java:422)
F form.chefs.ski: runtime.cc:558] at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:188)
F form.chefs.ski: runtime.cc:558] - locked <0x0b94c8fe> (a java.lang.Object)
F form.chefs.ski: runtime.cc:558] at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:209)
F form.chefs.ski: runtime.cc:558] at java.lang.Daemons$FinalizerDaemon.runInternal(Daemons.java:232)
F form.chefs.ski: runtime.cc:558] at java.lang.Daemons$Daemon.run(Daemons.java:103)
F form.chefs.ski: runtime.cc:558] at java.lang.Thread.run(Thread.java:764)
F form.chefs.ski: runtime.cc:558] 
F form.chefs.ski: runtime.cc:558] "ReferenceQueueDaemon" prio=10 tid=5 Waiting
F form.chefs.ski: runtime.cc:558] | group="" sCount=1 dsCount=0 flags=1 obj=0x12cc0498 self=0x717b96dc0000
F form.chefs.ski: runtime.cc:558] | sysTid=5954 nice=4 cgrp=default sched=0/0 handle=0x717b82de64f0
F form.chefs.ski: runtime.cc:558] | state=S schedstat=( 2633205 1295966 15 ) utm=0 stm=0 core=0 HZ=100
F form.chefs.ski: runtime.cc:558] | stack=0x717b82ce3000-0x717b82ce5000 stackSize=1041KB
F form.chefs.ski: runtime.cc:558] | held mutexes=
F form.chefs.ski: runtime.cc:558] kernel: (couldn't read /proc/self/task/5954/stack)
F form.chefs.ski: runtime.cc:558] native: #00 pc 0000000000026b96 /system/lib64/libc.so (syscall+22)
F form.chefs.ski: runtime.cc:558] native: #01 pc 00000000000d70bc /system/lib64/libart.so (art::ConditionVariable::WaitHoldingLocks(art::Thread*)+108)
F form.chefs.ski: runtime.cc:558] native: #02 pc 00000000003ff20f /system/lib64/libart.so (art::Monitor::Wait(art::Thread*, long, int, bool, art::ThreadState)+655)
F form.chefs.ski: runtime.cc:558] native: #03 pc 0000000000400db1 /system/lib64/libart.so (art::Monitor::Wait(art::Thread*, art::mirror::Object*, long, int, bool, art::ThreadState)+449)
F form.chefs.ski: runtime.cc:558] at java.lang.Object.wait(Native method)
F form.chefs.ski: runtime.cc:558] - waiting on <0x060b285f> (a java.lang.Class<java.lang.ref.ReferenceQueue>)
F form.chefs.ski: runtime.cc:558] at java.lang.Daemons$ReferenceQueueDaemon.runInternal(Daemons.java:178)
F form.chefs.ski: runtime.cc:558] - locked <0x060b285f> (a java.lang.Class<java.lang.ref.ReferenceQueue>)
F form.chefs.ski: runtime.cc:558] at java.lang.Daemons$Daemon.run(Daemons.java:103)
F form.chefs.ski: runtime.cc:558] at java.lang.Thread.run(Thread.java:764)
F form.chefs.ski: runtime.cc:558] 
F form.chefs.ski: runtime.cc:558] "FinalizerWatchdogDaemon" prio=10 tid=6 Sleeping
F form.chefs.ski: runtime.cc:558] | group="" sCount=1 dsCount=0 flags=1 obj=0x12cc0520 self=0x717b8efc6000
F form.chefs.ski: runtime.cc:558] | sysTid=5956 nice=4 cgrp=default sched=0/0 handle=0x717b82bda4f0
F form.chefs.ski: runtime.cc:558] | state=S schedstat=( 489751 33878 2 ) utm=0 stm=0 core=2 HZ=100
F form.chefs.ski: runtime.cc:558] | stack=0x717b82ad7000-0x717b82ad9000 stackSize=1041KB
F form.chefs.ski: runtime.cc:558] | held mutexes=
F form.chefs.ski: runtime.cc:558] kernel: (couldn't read /proc/self/task/5956/stack)
F form.chefs.ski: runtime.cc:558] native: #00 pc 0000000000026b98 /system/lib64/libc.so (syscall+24)
F form.chefs.ski: runtime.cc:558] native: #01 pc 00000000000d750a /system/lib64/libart.so (art::ConditionVariable::TimedWait(art::Thread*, long, int)+138)
F form.chefs.ski: runtime.cc:558] native: #02 pc 00000000003ff21f /system/lib64/libart.so (art::Monitor::Wait(art::Thread*, long, int, bool, art::ThreadState)+671)
F form.chefs.ski: runtime.cc:558] native: #03 pc 0000000000400db1 /system/lib64/libart.so (art::Monitor::Wait(art::Thread*, art::mirror::Object*, long, int, bool, art::ThreadState)+449)
F form.chefs.ski: runtime.cc:558] at java.lang.Thread.sleep(Native method)
F form.chefs.ski: runtime.cc:558] - sleeping on <0x018752ac> (a java.lang.Object)
F form.chefs.ski: runtime.cc:558] at java.lang.Thread.sleep(Thread.java:373)
F form.chefs.ski: runtime.cc:558] - locked <0x018752ac> (a java.lang.Object)
F form.chefs.ski: runtime.cc:558] at java.lang.Thread.sleep(Thread.java:314)
F form.chefs.ski: runtime.cc:558] at java.lang.Daemons$FinalizerWatchdogDaemon.sleepFor(Daemons.java:342)
F form.chefs.ski: runtime.cc:558] at java.lang.Daemons$FinalizerWatchdogDaemon.waitForFinalization(Daemons.java:364)
F form.chefs.ski: runtime.cc:558] at java.lang.Daemons$FinalizerWatchdogDaemon.runInternal(Daemons.java:281)
F form.chefs.ski: runtime.cc:558] at java.lang.Daemons$Daemon.run(Daemons.java:103)
F form.chefs.ski: runtime.cc:558] at java.lang.Thread.run(Thread.java:764)
F form.chefs.ski: runtime.cc:558] 
F form.chefs.ski: runtime.cc:558] "HeapTaskDaemon" prio=10 tid=7 WaitingForTaskProcessor
F form.chefs.ski: runtime.cc:558] | group="" sCount=1 dsCount=0 flags=1 obj=0x12cc05b0 self=0x717b8efc6c00
F form.chefs.ski: runtime.cc:558] | sysTid=5957 nice=4 cgrp=default sched=0/0 handle=0x717b82ad44f0
F form.chefs.ski: runtime.cc:558] | state=S schedstat=( 654995 0 5 ) utm=0 stm=0 core=3 HZ=100
F form.chefs.ski: runtime.cc:558] | stack=0x717b829d1000-0x717b829d3000 stackSize=1041KB
F form.chefs.ski: runtime.cc:558] | held mutexes=
F form.chefs.ski: runtime.cc:558] kernel: (couldn't read /proc/self/task/5957/stack)
F form.chefs.ski: runtime.cc:558] native: #00 pc 0000000000026b98 /system/lib64/libc.so (syscall+24)
F form.chefs.ski: runtime.cc:558] native: #01 pc 00000000000d750a /system/lib64/libart.so (art::ConditionVariable::TimedWait(art::Thread*, long, int)+138)
F form.chefs.ski: runtime.cc:558] native: #02 pc 00000000002477bc /system/lib64/libart.so (art::gc::TaskProcessor::GetTask(art::Thread*)+300)
F form.chefs.ski: runtime.cc:558] native: #03 pc 000000000024807f /system/lib64/libart.so (art::gc::TaskProcessor::RunAllTasks(art::Thread*)+63)
F form.chefs.ski: runtime.cc:558] at dalvik.system.VMRuntime.runHeapTasks(Native method)
F form.chefs.ski: runtime.cc:558] at java.lang.Daemons$HeapTaskDaemon.runInternal(Daemons.java:475)
F form.chefs.ski: runtime.cc:558] at java.lang.Daemons$Daemon.run(Daemons.java:103)
F form.chefs.ski: runtime.cc:558] at java.lang.Thread.run(Thread.java:764)
F form.chefs.ski: runtime.cc:558] 
F form.chefs.ski: runtime.cc:558] "Binder:5947_1" prio=10 tid=8 Native
F form.chefs.ski: runtime.cc:558] | group="" sCount=1 dsCount=0 flags=1 obj=0x12cc0638 self=0x717b90815800
F form.chefs.ski: runtime.cc:558] | sysTid=5958 nice=0 cgrp=default sched=0/0 handle=0x717b828d04f0
F form.chefs.ski: runtime.cc:558] | state=S schedstat=( 1203152 405942 8 ) utm=0 stm=0 core=3 HZ=100
F form.chefs.ski: runtime.cc:558] | stack=0x717b827d5000-0x717b827d7000 stackSize=1009KB
F form.chefs.ski: runtime.cc:558] | held mutexes=
F form.chefs.ski: runtime.cc:558] kernel: (couldn't read /proc/self/task/5958/stack)
F form.chefs.ski: runtime.cc:558] native: #00 pc 000000000007e315 /system/lib64/libc.so (__ioctl+5)
F form.chefs.ski: runtime.cc:558] native: #01 pc 0000000000030cde /system/lib64/libc.so (ioctl+206)
F form.chefs.ski: runtime.cc:558] native: #02 pc 000000000005b40e /system/lib64/libbinder.so (android::IPCThreadState::talkWithDriver(bool)+270)
F form.chefs.ski: runtime.cc:558] native: #03 pc 000000000005b5f0 /system/lib64/libbinder.so (android::IPCThreadState::getAndExecuteCommand()+16)
F form.chefs.ski: runtime.cc:558] native: #04 pc 000000000005bdd1 /system/lib64/libbinder.so (android::IPCThreadState::joinThreadPool(bool)+49)
F form.chefs.ski: runtime.cc:558] native: #05 pc 0000000000082277 /system/lib64/libbinder.so (android::PoolThread::threadLoop()+23)
F form.chefs.ski: runtime.cc:558] native: #06 pc 000000000000f79a /system/lib64/libutils.so (android::Thread::_threadLoop(void*)+314)
F form.chefs.ski: runtime.cc:558] native: #07 pc 00000000000b5573 /system/lib64/libandroid_runtime.so (android::AndroidRuntime::javaThreadShell(void*)+131)
F form.chefs.ski: runtime.cc:558] native: #08 pc 0000000000092bab /system/lib64/libc.so (__pthread_start(void*)+27)
F form.chefs.ski: runtime.cc:558] native: #09 pc 000000000002af2d /system/lib64/libc.so (__start_thread+61)
F form.chefs.ski: runtime.cc:558] (no managed stack frames)
F form.chefs.ski: runtime.cc:558] 
F form.chefs.ski: runtime.cc:558] "Binder:5947_2" prio=10 tid=9 Native
F form.chefs.ski: runtime.cc:558] | group="" sCount=1 dsCount=0 flags=1 obj=0x12cc06c0 self=0x717b8ee73800
F form.chefs.ski: runtime.cc:558] | sysTid=5959 nice=0 cgrp=default sched=0/0 handle=0x717b827d24f0
F form.chefs.ski: runtime.cc:558] | state=S schedstat=( 3916259 1868214 11 ) utm=0 stm=0 core=1 HZ=100
F form.chefs.ski: runtime.cc:558] | stack=0x717b826d7000-0x717b826d9000 stackSize=1009KB
F form.chefs.ski: runtime.cc:558] | held mutexes=
F form.chefs.ski: runtime.cc:558] kernel: (couldn't read /proc/self/task/5959/stack)
F form.chefs.ski: runtime.cc:558] native: #00 pc 000000000007e315 /system/lib64/libc.so (__ioctl+5)
F form.chefs.ski: runtime.cc:558] native: #01 pc 0000000000030cde /system/lib64/libc.so (ioctl+206)
F form.chefs.ski: runtime.cc:558] native: #02 pc 000000000005b40e /system/lib64/libbinder.so (android::IPCThreadState::talkWithDriver(bool)+270)
F form.chefs.ski: runtime.cc:558] native: #03 pc 000000000005b5f0 /system/lib64/libbinder.so (android::IPCThreadState::getAndExecuteCommand()+16)
F form.chefs.ski: runtime.cc:558] native: #04 pc 000000000005bdd1 /system/lib64/libbinder.so (android::IPCThreadState::joinThreadPool(bool)+49)
F form.chefs.ski: runtime.cc:558] native: #05 pc 0000000000082277 /system/lib64/libbinder.so (android::PoolThread::threadLoop()+23)
F form.chefs.ski: runtime.cc:558] native: #06 pc 000000000000f79a /system/lib64/libutils.so (android::Thread::_threadLoop(void*)+314)
F form.chefs.ski: runtime.cc:558] native: #07 pc 00000000000b5573 /system/lib64/libandroid_runtime.so (android::AndroidRuntime::javaThreadShell(void*)+131)
F form.chefs.ski: runtime.cc:558] native: #08 pc 0000000000092bab /system/lib64/libc.so (__pthread_start(void*)+27)
F form.chefs.ski: runtime.cc:558] native: #09 pc 000000000002af2d /system/lib64/libc.so (__start_thread+61)
F form.chefs.ski: runtime.cc:558] (no managed stack frames)
F form.chefs.ski: runtime.cc:558] 
F form.chefs.ski: runtime.cc:558] "Profile Saver" prio=10 tid=10 Native
F form.chefs.ski: runtime.cc:558] | group="" sCount=1 dsCount=0 flags=1 obj=0x12cc0748 self=0x717b9083c000
F form.chefs.ski: runtime.cc:558] | sysTid=5960 nice=9 cgrp=default sched=0/0 handle=0x717b81ace4f0
F form.chefs.ski: runtime.cc:558] | state=S schedstat=( 206762 58560 1 ) utm=0 stm=0 core=1 HZ=100
F form.chefs.ski: runtime.cc:558] | stack=0x717b819d3000-0x717b819d5000 stackSize=1009KB
F form.chefs.ski: runtime.cc:558] | held mutexes=
F form.chefs.ski: runtime.cc:558] kernel: (couldn't read /proc/self/task/5960/stack)
F form.chefs.ski: runtime.cc:558] native: #00 pc 0000000000026b98 /system/lib64/libc.so (syscall+24)
F form.chefs.ski: runtime.cc:558] native: #01 pc 00000000000d750a /system/lib64/libart.so (art::ConditionVariable::TimedWait(art::Thread*, long, int)+138)
F form.chefs.ski: runtime.cc:558] native: #02 pc 000000000035ee8c /system/lib64/libart.so (art::ProfileSaver::Run()+156)
F form.chefs.ski: runtime.cc:558] native: #03 pc 0000000000362cd7 /system/lib64/libart.so (art::ProfileSaver::RunProfileSaverThread(void*)+87)
F form.chefs.ski: runtime.cc:558] native: #04 pc 0000000000092bab /system/lib64/libc.so (__pthread_start(void*)+27)
F form.chefs.ski: runtime.cc:558] native: #05 pc 000000000002af2d /system/lib64/libc.so (__start_thread+61)
F form.chefs.ski: runtime.cc:558] (no managed stack frames)
F form.chefs.ski: runtime.cc:558] 
F form.chefs.ski: runtime.cc:558] "Thread-2" prio=10 tid=11 WaitingForCheckPointsToRun
F form.chefs.ski: runtime.cc:558] | group="" sCount=1 dsCount=0 flags=1 obj=0x12cc1058 self=0x717b90843800
F form.chefs.ski: runtime.cc:558] | sysTid=5962 nice=-10 cgrp=default sched=0/0 handle=0x71798868e4f0
F form.chefs.ski: runtime.cc:558] | state=S schedstat=( 41545012 5207359 38 ) utm=4 stm=0 core=0 HZ=100
F form.chefs.ski: runtime.cc:558] | stack=0x717988593000-0x717988595000 stackSize=1009KB
F form.chefs.ski: runtime.cc:558] | held mutexes=
F form.chefs.ski: runtime.cc:558] kernel: (couldn't read /proc/self/task/5962/stack)
F form.chefs.ski: runtime.cc:558] native: #00 pc 0000000000026b96 /system/lib64/libc.so (syscall+22)
F form.chefs.ski: runtime.cc:558] native: #01 pc 00000000000d70bc /system/lib64/libart.so (art::ConditionVariable::WaitHoldingLocks(art::Thread*)+108)
F form.chefs.ski: runtime.cc:558] native: #02 pc 00000000000d2196 /system/lib64/libart.so (_ZN3art7Barrier9IncrementILNS0_12LockHandlingE1EEEvPNS_6ThreadEi+70)
F form.chefs.ski: runtime.cc:558] native: #03 pc 00000000001ccf1e /system/lib64/libart.so (art::gc::collector::ConcurrentCopying::ActivateReadBarrierEntrypoints()+350)
F form.chefs.ski: runtime.cc:558] native: #04 pc 00000000001cc6ad /system/lib64/libart.so (art::gc::collector::ConcurrentCopying::RunPhases()+525)
F form.chefs.ski: runtime.cc:558] native: #05 pc 00000000001e44a9 /system/lib64/libart.so (art::gc::collector::GarbageCollector::Run(art::gc::GcCause, bool)+345)
F form.chefs.ski: runtime.cc:558] native: #06 pc 000000000020a01e /system/lib64/libart.so (art::gc::Heap::CollectGarbageInternal(art::gc::collector::GcType, art::gc::GcCause, bool)+3374)
F form.chefs.ski: runtime.cc:558] native: #07 pc 0000000000003c7e /system/lib64/libopenjdkjvm.so (JVM_GC+94)
F form.chefs.ski: runtime.cc:558] at java.lang.Runtime.gc(Native method)
F form.chefs.ski: runtime.cc:558] 
F form.chefs.ski: runtime.cc:558] "Thread-3" prio=10 tid=12 Native
F form.chefs.ski: runtime.cc:558] | group="" sCount=1 dsCount=0 flags=1 obj=0x12cc07d0 self=0x717b8ee6f000
F form.chefs.ski: runtime.cc:558] | sysTid=5961 nice=-10 cgrp=default sched=0/0 handle=0x717b7a9d84f0
F form.chefs.ski: runtime.cc:558] | state=S schedstat=( 4085964 0 22 ) utm=0 stm=0 core=1 HZ=100
F form.chefs.ski: runtime.cc:558] | stack=0x717b7a8dd000-0x717b7a8df000 stackSize=1009KB
F form.chefs.ski: runtime.cc:558] | held mutexes=
F form.chefs.ski: runtime.cc:558] kernel: (couldn't read /proc/self/task/5961/stack)
F form.chefs.ski: runtime.cc:558] native: #00 pc 0000000000026b96 /system/lib64/libc.so (syscall+22)
F form.chefs.ski: runtime.cc:558] native: #01 pc 0000000000029cd5 /system/lib64/libc.so (__futex_wait_ex(void volatile*, bool, int, bool, timespec const*)+133)
F form.chefs.ski: runtime.cc:558] native: #02 pc 000000000009219d /system/lib64/libc.so (pthread_cond_wait+45)
F form.chefs.ski: runtime.cc:558] native: #03 pc 00000000030554c2 /data/app/uno.platform.chefs.skia-Hn38_Q2Ao0LTHQhJNMH6_A==/lib/x86_64/libChefs.so (offset 2f06000) (???)
F form.chefs.ski: runtime.cc:558] native: #04 pc 0000000003011e3e /data/app/uno.platform.chefs.skia-Hn38_Q2Ao0LTHQhJNMH6_A==/lib/x86_64/libChefs.so (offset 2f06000) (???)
F form.chefs.ski: runtime.cc:558] (no managed stack frames)
F form.chefs.ski: runtime.cc:558] 
F form.chefs.ski: runtime.cc:558] "Thread-4" prio=10 tid=13 Native
F form.chefs.ski: runtime.cc:558] | group="" sCount=1 dsCount=0 flags=1 obj=0x12cc0858 self=0x7179883af400
F form.chefs.ski: runtime.cc:558] | sysTid=5967 nice=-10 cgrp=default sched=0/0 handle=0x7179875144f0
F form.chefs.ski: runtime.cc:558] | state=S schedstat=( 82689275 15780805 138 ) utm=4 stm=4 core=2 HZ=100
F form.chefs.ski: runtime.cc:558] | stack=0x717987419000-0x71798741b000 stackSize=1009KB
F form.chefs.ski: runtime.cc:558] | held mutexes=
F form.chefs.ski: runtime.cc:558] kernel: (couldn't read /proc/self/task/5967/stack)
F form.chefs.ski: runtime.cc:558] native: #00 pc 0000000000026b98 /system/lib64/libc.so (syscall+24)
F form.chefs.ski: runtime.cc:558] native: #01 pc 0000000000029cd5 /system/lib64/libc.so (__futex_wait_ex(void volatile*, bool, int, bool, timespec const*)+133)
F form.chefs.ski: runtime.cc:558] native: #02 pc 0000000000092215 /system/lib64/libc.so (pthread_cond_timedwait+85)
F form.chefs.ski: runtime.cc:558] native: #03 pc 0000000002ff1477 /data/app/uno.platform.chefs.skia-Hn38_Q2Ao0LTHQhJNMH6_A==/lib/x86_64/libChefs.so (offset 2f06000) (???)
F form.chefs.ski: runtime.cc:558] native: #04 pc 00000000038e9a31 /data/app/uno.platform.chefs.skia-Hn38_Q2Ao0LTHQhJNMH6_A==/lib/x86_64/libChefs.so (offset 2f06000) (???)
F form.chefs.ski: runtime.cc:558] native: #05 pc 00004e2000007179 ???
F form.chefs.ski: runtime.cc:558] (no managed stack frames)
F form.chefs.ski: runtime.cc:558] 
F form.chefs.ski: runtime.cc:558] "GLThread 294" prio=10 tid=14 Waiting
F form.chefs.ski: runtime.cc:558] | group="" sCount=1 dsCount=0 flags=1 obj=0x12cc08e0 self=0x7179883ad000
F form.chefs.ski: runtime.cc:558] | sysTid=5971 nice=0 cgrp=default sched=0/0 handle=0x717986c034f0
F form.chefs.ski: runtime.cc:558] | state=S schedstat=( 103682508 50152834 295 ) utm=0 stm=10 core=2 HZ=100
F form.chefs.ski: runtime.cc:558] | stack=0x717986b00000-0x717986b02000 stackSize=1041KB
F form.chefs.ski: runtime.cc:558] | held mutexes=
F form.chefs.ski: runtime.cc:558] kernel: (couldn't read /proc/self/task/5971/stack)
F form.chefs.ski: runtime.cc:558] native: #00 pc 0000000000026b96 /system/lib64/libc.so (syscall+22)
F form.chefs.ski: runtime.cc:558] native: #01 pc 00000000000d70bc /system/lib64/libart.so (art::ConditionVariable::WaitHoldingLocks(art::Thread*)+108)
F form.chefs.ski: runtime.cc:558] native: #02 pc 00000000003ff20f /system/lib64/libart.so (art::Monitor::Wait(art::Thread*, long, int, bool, art::ThreadState)+655)
F form.chefs.ski: runtime.cc:558] native: #03 pc 0000000000400db1 /system/lib64/libart.so (art::Monitor::Wait(art::Thread*, art::mirror::Object*, long, int, bool, art::ThreadState)+449)
F form.chefs.ski: runtime.cc:558] at java.lang.Object.wait(Native method)
F form.chefs.ski: runtime.cc:558] - waiting on <0x0ed6cb75> (a android.opengl.GLSurfaceView$GLThreadManager)
F form.chefs.ski: runtime.cc:558] at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1495)
F form.chefs.ski: runtime.cc:558] - locked <0x0ed6cb75> (a android.opengl.GLSurfaceView$GLThreadManager)
F form.chefs.ski: runtime.cc:558] at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1270)
F form.chefs.ski: runtime.cc:558] 
F form.chefs.ski: runtime.cc:558] "RenderThread" prio=10 tid=15 Native
F form.chefs.ski: runtime.cc:558] | group="" sCount=1 dsCount=0 flags=1 obj=0x12cc0ca0 self=0x717b90870000
F form.chefs.ski: runtime.cc:558] | sysTid=5972 nice=-10 cgrp=default sched=0/0 handle=0x717986afd4f0
F form.chefs.ski: runtime.cc:558] | state=S schedstat=( 58714370 9426563 165 ) utm=3 stm=2 core=3 HZ=100
F form.chefs.ski: runtime.cc:558] | stack=0x717986a02000-0x717986a04000 stackSize=1009KB
F form.chefs.ski: runtime.cc:558] | held mutexes=
F form.chefs.ski: runtime.cc:558] kernel: (couldn't read /proc/self/task/5972/stack)
F form.chefs.ski: runtime.cc:558] native: #00 pc 000000000007e1da /system/lib64/libc.so (__epoll_pwait+10)
F form.chefs.ski: runtime.cc:558] native: #01 pc 0000000000013d12 /system/lib64/libutils.so (android::Looper::pollInner(int)+162)
F form.chefs.ski: runtime.cc:558] native: #02 pc 0000000000013bc9 /system/lib64/libutils.so (android::Looper::pollOnce(int, int*, int*, void**)+41)
F form.chefs.ski: runtime.cc:558] native: #03 pc 0000000000119d1b /system/lib64/libhwui.so (android::uirenderer::renderthread::RenderThread::threadLoop()+251)
F form.chefs.ski: runtime.cc:558] native: #04 pc 000000000000f79a /system/lib64/libutils.so (android::Thread::_threadLoop(void*)+314)
F form.chefs.ski: runtime.cc:558] native: #05 pc 0000000000092bab /system/lib64/libc.so (__pthread_start(void*)+27)
F form.chefs.ski: runtime.cc:558] native: #06 pc 000000000002af2d /system/lib64/libc.so (__start_thread+61)
F form.chefs.ski: runtime.cc:558] (no managed stack frames)
F form.chefs.ski: runtime.cc:558] 
F form.chefs.ski: runtime.cc:558] "Thread-6" prio=10 tid=16 Native
F form.chefs.ski: runtime.cc:558] | group="" sCount=1 dsCount=0 flags=1 obj=0x12cc0d28 self=0x7179871fbc00
F form.chefs.ski: runtime.cc:558] | sysTid=5969 nice=-10 cgrp=default sched=0/0 handle=0x717986dff4f0
F form.chefs.ski: runtime.cc:558] | state=S schedstat=( 46114863 3169360 54 ) utm=4 stm=0 core=2 HZ=100
F form.chefs.ski: runtime.cc:558] | stack=0x717986d04000-0x717986d06000 stackSize=1009KB
F form.chefs.ski: runtime.cc:558] | held mutexes=
F form.chefs.ski: runtime.cc:558] kernel: (couldn't read /proc/self/task/5969/stack)
F form.chefs.ski: runtime.cc:558] native: #00 pc 0000000000026b98 /system/lib64/libc.so (syscall+24)
F form.chefs.ski: runtime.cc:558] native: #01 pc 0000000000029cd5 /system/lib64/libc.so (__futex_wait_ex(void volatile*, bool, int, bool, timespec const*)+133)
F form.chefs.ski: runtime.cc:558] native: #02 pc 0000000000092215 /system/lib64/libc.so (pthread_cond_timedwait+85)
F form.chefs.ski: runtime.cc:558] native: #03 pc 0000000002ff1477 /data/app/uno.platform.chefs.skia-Hn38_Q2Ao0LTHQhJNMH6_A==/lib/x86_64/libChefs.so (offset 2f06000) (???)
F form.chefs.ski: runtime.cc:558] native: #04 pc 00000000038e9a31 /data/app/uno.platform.chefs.skia-Hn38_Q2Ao0LTHQhJNMH6_A==/lib/x86_64/libChefs.so (offset 2f06000) (???)
F form.chefs.ski: runtime.cc:558] native: #05 pc 00004e2000007179 ???
F form.chefs.ski: runtime.cc:558] (no managed stack frames)
F form.chefs.ski: runtime.cc:558] 
F form.chefs.ski: runtime.cc:558] "Binder:5947_3" prio=10 tid=17 Native
F form.chefs.ski: runtime.cc:558] | group="" sCount=1 dsCount=0 flags=1 obj=0x12cc0db0 self=0x71798820f400
F form.chefs.ski: runtime.cc:558] | sysTid=5973 nice=0 cgrp=default sched=0/0 handle=0x7179865ff4f0
F form.chefs.ski: runtime.cc:558] | state=S schedstat=( 1344899 214088 6 ) utm=0 stm=0 core=1 HZ=100
F form.chefs.ski: runtime.cc:558] | stack=0x717986504000-0x717986506000 stackSize=1009KB
F form.chefs.ski: runtime.cc:558] | held mutexes=
F form.chefs.ski: runtime.cc:558] kernel: (couldn't read /proc/self/task/5973/stack)
F form.chefs.ski: runtime.cc:558] native: #00 pc 000000000007e315 /system/lib64/libc.so (__ioctl+5)
F form.chefs.ski: runtime.cc:558] native: #01 pc 0000000000030cde /system/lib64/libc.so (ioctl+206)
F form.chefs.ski: runtime.cc:558] native: #02 pc 000000000005b40e /system/lib64/libbinder.so (android::IPCThreadState::talkWithDriver(bool)+270)
F form.chefs.ski: runtime.cc:558] native: #03 pc 000000000005b5f0 /system/lib64/libbinder.so (android::IPCThreadState::getAndExecuteCommand()+16)
F form.chefs.ski: runtime.cc:558] native: #04 pc 000000000005bdd1 /system/lib64/libbinder.so (android::IPCThreadState::joinThreadPool(bool)+49)
F form.chefs.ski: runtime.cc:558] native: #05 pc 0000000000082277 /system/lib64/libbinder.so (android::PoolThread::threadLoop()+23)
F form.chefs.ski: runtime.cc:558] native: #06 pc 000000000000f79a /system/lib64/libutils.so (android::Thread::_threadLoop(void*)+314)
F form.chefs.ski: runtime.cc:558] native: #07 pc 00000000000b5573 /system/lib64/libandroid_runtime.so (android::AndroidRuntime::javaThreadShell(void*)+131)
F form.chefs.ski: runtime.cc:558] native: #08 pc 0000000000092bab /system/lib64/libc.so (__pthread_start(void*)+27)
F form.chefs.ski: runtime.cc:558] native: #09 pc 000000000002af2d /system/lib64/libc.so (__start_thread+61)
F form.chefs.ski: runtime.cc:558] (no managed stack frames)
F form.chefs.ski: runtime.cc:558] 
F form.chefs.ski: runtime.cc:558] "hwuiTask1" prio=10 tid=18 Native
F form.chefs.ski: runtime.cc:558] | group="" sCount=1 dsCount=0 flags=1 obj=0x12cc0e38 self=0x717b90877800
F form.chefs.ski: runtime.cc:558] | sysTid=5974 nice=-2 cgrp=default sched=0/0 handle=0x7179855ff4f0
F form.chefs.ski: runtime.cc:558] | state=S schedstat=( 2361382 2475967 19 ) utm=0 stm=0 core=0 HZ=100
F form.chefs.ski: runtime.cc:558] | stack=0x717985504000-0x717985506000 stackSize=1009KB
F form.chefs.ski: runtime.cc:558] | held mutexes=
F form.chefs.ski: runtime.cc:558] kernel: (couldn't read /proc/self/task/5974/stack)
F form.chefs.ski: runtime.cc:558] native: #00 pc 0000000000026b96 /system/lib64/libc.so (syscall+22)
F form.chefs.ski: runtime.cc:558] native: #01 pc 0000000000029cd5 /system/lib64/libc.so (__futex_wait_ex(void volatile*, bool, int, bool, timespec const*)+133)
F form.chefs.ski: runtime.cc:558] native: #02 pc 000000000009219d /system/lib64/libc.so (pthread_cond_wait+45)
F form.chefs.ski: runtime.cc:558] native: #03 pc 00000000004b5978 /system/lib64/libhwui.so (android::uirenderer::TaskManager::WorkerThread::threadLoop()+328)
F form.chefs.ski: runtime.cc:558] native: #04 pc 000000000000f73e /system/lib64/libutils.so (android::Thread::_threadLoop(void*)+222)
F form.chefs.ski: runtime.cc:558] native: #05 pc 00000000000b5573 /system/lib64/libandroid_runtime.so (android::AndroidRuntime::javaThreadShell(void*)+131)
F form.chefs.ski: runtime.cc:558] native: #06 pc 0000000000092bab /system/lib64/libc.so (__pthread_start(void*)+27)
F form.chefs.ski: runtime.cc:558] native: #07 pc 000000000002af2d /system/lib64/libc.so (__start_thread+61)
F form.chefs.ski: runtime.cc:558] (no managed stack frames)
F form.chefs.ski: runtime.cc:558] 
F form.chefs.ski: runtime.cc:558] "Thread-7" prio=10 tid=19 Native
F form.chefs.ski: runtime.cc:558] | group="" sCount=1 dsCount=0 flags=1 obj=0x12cc0ec0 self=0x7179883ae800
F form.chefs.ski: runtime.cc:558] | sysTid=5976 nice=-10 cgrp=default sched=0/0 handle=0x717984d2f4f0
F form.chefs.ski: runtime.cc:558] | state=S schedstat=( 37152175 16123358 53 ) utm=2 stm=1 core=3 HZ=100
F form.chefs.ski: runtime.cc:558] | stack=0x717984c34000-0x717984c36000 stackSize=1009KB
F form.chefs.ski: runtime.cc:558] | held mutexes=
F form.chefs.ski: runtime.cc:558] kernel: (couldn't read /proc/self/task/5976/stack)
F form.chefs.ski: runtime.cc:558] native: #00 pc 0000000000026b98 /system/lib64/libc.so (syscall+24)
F form.chefs.ski: runtime.cc:558] native: #01 pc 0000000000029cd5 /system/lib64/libc.so (__futex_wait_ex(void volatile*, bool, int, bool, timespec const*)+133)
F form.chefs.ski: runtime.cc:558] native: #02 pc 0000000000092215 /system/lib64/libc.so (pthread_cond_timedwait+85)
F form.chefs.ski: runtime.cc:558] native: #03 pc 0000000002ff1477 /data/app/uno.platform.chefs.skia-Hn38_Q2Ao0LTHQhJNMH6_A==/lib/x86_64/libChefs.so (offset 2f06000) (???)
F form.chefs.ski: runtime.cc:558] native: #04 pc 00000000038e9a31 /data/app/uno.platform.chefs.skia-Hn38_Q2Ao0LTHQhJNMH6_A==/lib/x86_64/libChefs.so (offset 2f06000) (???)
F form.chefs.ski: runtime.cc:558] native: #05 pc 00004e2000007179 ???
F form.chefs.ski: runtime.cc:558] (no managed stack frames)
F form.chefs.ski: runtime.cc:558] 
F form.chefs.ski: runtime.cc:558] "Thread-8" prio=10 tid=20 Native
F form.chefs.ski: runtime.cc:558] | group="" sCount=1 dsCount=0 flags=1 obj=0x12cc0f48 self=0x717b90962800
F form.chefs.ski: runtime.cc:558] | sysTid=5977 nice=-10 cgrp=default sched=0/0 handle=0x71798422d4f0
F form.chefs.ski: runtime.cc:558] | state=S schedstat=( 46688421 42540672 60 ) utm=1 stm=3 core=3 HZ=100
F form.chefs.ski: runtime.cc:558] | stack=0x717984132000-0x717984134000 stackSize=1009KB
F form.chefs.ski: runtime.cc:558] | held mutexes=
F form.chefs.ski: runtime.cc:558] kernel: (couldn't read /proc/self/task/5977/stack)
F form.chefs.ski: runtime.cc:558] native: #00 pc 0000000000026b98 /system/lib64/libc.so (syscall+24)
F form.chefs.ski: runtime.cc:558] native: #01 pc 0000000000029cd5 /system/lib64/libc.so (__futex_wait_ex(void volatile*, bool, int, bool, timespec const*)+133)
F form.chefs.ski: runtime.cc:558] native: #02 pc 0000000000092215 /system/lib64/libc.so (pthread_cond_timedwait+85)
F form.chefs.ski: runtime.cc:558] native: #03 pc 0000000002ff1477 /data/app/uno.platform.chefs.skia-Hn38_Q2Ao0LTHQhJNMH6_A==/lib/x86_64/libChefs.so (offset 2f06000) (???)
F form.chefs.ski: runtime.cc:558] native: #04 pc 00000000038e9a31 /data/app/uno.platform.chefs.skia-Hn38_Q2Ao0LTHQhJNMH6_A==/lib/x86_64/libChefs.so (offset 2f06000) (???)
F form.chefs.ski: runtime.cc:558] native: #05 pc 00004e2000007179 ???
F form.chefs.ski: runtime.cc:558] (no managed stack frames)
F form.chefs.ski: runtime.cc:558] 
F form.chefs.ski: runtime.cc:558] "Thread-9" prio=10 tid=21 Native
F form.chefs.ski: runtime.cc:558] | group="" sCount=1 dsCount=0 flags=1 obj=0x12cc0fd0 self=0x717984b4d000
F form.chefs.ski: runtime.cc:558] | sysTid=5978 nice=-10 cgrp=default sched=0/0 handle=0x71798412f4f0
F form.chefs.ski: runtime.cc:558] | state=S schedstat=( 11902223 24562907 19 ) utm=1 stm=0 core=1 HZ=100
F form.chefs.ski: runtime.cc:558] | stack=0x717984034000-0x717984036000 stackSize=1009KB
F form.chefs.ski: runtime.cc:558] | held mutexes=
F form.chefs.ski: runtime.cc:558] kernel: (couldn't read /proc/self/task/5978/stack)
F form.chefs.ski: runtime.cc:558] native: #00 pc 0000000000026b98 /system/lib64/libc.so (syscall+24)
F form.chefs.ski: runtime.cc:558] native: #01 pc 0000000000029cd5 /system/lib64/libc.so (__futex_wait_ex(void volatile*, bool, int, bool, timespec const*)+133)
F form.chefs.ski: runtime.cc:558] native: #02 pc 0000000000092215 /system/lib64/libc.so (pthread_cond_timedwait+85)
F form.chefs.ski: runtime.cc:558] native: #03 pc 0000000002ff1477 /data/app/uno.platform.chefs.skia-Hn38_Q2Ao0LTHQhJNMH6_A==/lib/x86_64/libChefs.so (offset 2f06000) (???)
F form.chefs.ski: runtime.cc:558] native: #04 pc 00000000038e9a31 /data/app/uno.platform.chefs.skia-Hn38_Q2Ao0LTHQhJNMH6_A==/lib/x86_64/libChefs.so (offset 2f06000) (???)
F form.chefs.ski: runtime.cc:558] native: #05 pc 00004e200000717b ???
F form.chefs.ski: runtime.cc:558] (no managed stack frames)
F form.chefs.ski: runtime.cc:558] 
F form.chefs.ski: runtime.cc:558] Aborting thread:
F form.chefs.ski: runtime.cc:558] "main" prio=10 tid=1 Runnable
F form.chefs.ski: runtime.cc:558] | group="" sCount=1 dsCount=0 flags=9 obj=0x75c45a48 self=0x717b96c14c00
F form.chefs.ski: runtime.cc:558] | sysTid=5947 nice=-10 cgrp=default sched=0/0 handle=0x717c1c452548
F form.chefs.ski: runtime.cc:558] | state=R schedstat=( 717226410 100972163 836 ) utm=62 stm=9 core=2 HZ=100
F form.chefs.ski: runtime.cc:558] | stack=0x7ffe6ddbd000-0x7ffe6ddbf000 stackSize=8MB
F form.chefs.ski: runtime.cc:558] | held mutexes= "abort lock" "mutator lock"(shared held)
F form.chefs.ski: runtime.cc:558] native: #00 pc 0000000000403e33 /system/lib64/libart.so (art::DumpNativeStack(std::__1::basic_ostream<char, std::__1::char_traits<char>>&, int, BacktraceMap*, char const*, art::ArtMethod*, void*, bool)+227)
F form.chefs.ski: runtime.cc:558] native: #01 pc 00000000004ee1f7 /system/lib64/libart.so (art::Thread::DumpStack(std::__1::basic_ostream<char, std::__1::char_traits<char>>&, bool, BacktraceMap*, bool) const+343)
F form.chefs.ski: runtime.cc:558] native: #02 pc 00000000004d0b8f /system/lib64/libart.so (art::AbortState::DumpThread(std::__1::basic_ostream<char, std::__1::char_traits<char>>&, art::Thread*) const+47)
F form.chefs.ski: runtime.cc:558] native: #03 pc 00000000004bfa4d /system/lib64/libart.so (art::Runtime::Abort(char const*)+1149)
F form.chefs.ski: runtime.cc:558] native: #04 pc 0000000000008cdc /system/lib64/libbase.so (android::base::LogMessage::~LogMessage()+716)
F form.chefs.ski: runtime.cc:558] native: #05 pc 000000000025b863 /system/lib64/libart.so (art::IndirectReferenceTable::AbortIfNoCheckJNI(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&)+243)
F form.chefs.ski: runtime.cc:558] native: #06 pc 00000000003252f2 /system/lib64/libart.so (art::IndirectReferenceTable::GetChecked(void*) const+466)
F form.chefs.ski: runtime.cc:558] native: #07 pc 000000000031fc32 /system/lib64/libart.so (art::JavaVMExt::DecodeGlobal(void*)+18)
F form.chefs.ski: runtime.cc:558] native: #08 pc 00000000004f41c5 /system/lib64/libart.so (art::Thread::DecodeJObject(_jobject*) const+165)
F form.chefs.ski: runtime.cc:558] native: #09 pc 00000000003c507f /system/lib64/libart.so (_ZN3art3JNI23DecodeAndCheckArrayTypeIP12_jfloatArrayfNS_6mirror14PrimitiveArrayIfEEEEPT1_RNS_18ScopedObjectAccessET_PKcSD_+47)
F form.chefs.ski: runtime.cc:558] native: #10 pc 00000000003b9640 /system/lib64/libart.so (art::JNI::GetFloatArrayRegion(_JNIEnv*, _jfloatArray*, int, int, float*)+800)
F form.chefs.ski: runtime.cc:558] native: #11 pc 0000000003332767 /data/app/uno.platform.chefs.skia-Hn38_Q2Ao0LTHQhJNMH6_A==/lib/x86_64/libChefs.so (offset 2f06000) (???)
F form.chefs.ski: runtime.cc:558] native: #12 pc 0000000003abde5c /data/app/uno.platform.chefs.skia-Hn38_Q2Ao0LTHQhJNMH6_A==/lib/x86_64/libChefs.so (offset 2f06000) (???)
F form.chefs.ski: runtime.cc:558] native: #13 pc 00000000000147f7 <anonymous:71798f400000> (???)
F form.chefs.ski: runtime.cc:558] at crc64fe881b1858fb9a5d.SimpleOrientationSensor_OrientationListener.n_onSensorChanged(Native method)
F form.chefs.ski: runtime.cc:558] at crc64fe881b1858fb9a5d.SimpleOrientationSensor_OrientationListener.onSensorChanged(SimpleOrientationSensor_OrientationListener.java:37)
F form.chefs.ski: runtime.cc:558] at android.hardware.SystemSensorManager$SensorEventQueue.dispatchSensorEvent(SystemSensorManager.java:833)
F form.chefs.ski: runtime.cc:558] at android.os.MessageQueue.nativePollOnce(Native method)
F form.chefs.ski: runtime.cc:558] at android.os.MessageQueue.next(MessageQueue.java:326)
F form.chefs.ski: runtime.cc:558] at android.os.Looper.loop(Looper.java:160)
F form.chefs.ski: runtime.cc:558] at android.app.ActivityThread.main(ActivityThread.java:6669)
F form.chefs.ski: runtime.cc:558] at java.lang.reflect.Method.invoke(Native method)
F form.chefs.ski: runtime.cc:558] at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
F form.chefs.ski: runtime.cc:558] at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)
F form.chefs.ski: runtime.cc:558] 
F libc : Fatal signal 6 (SIGABRT), code -6 (SI_TKILL) in tid 5947 (form.chefs.skia), pid 5947 (form.chefs.skia)

[jonpryor]

"So," thinks I, "what about GREF logging?"

adb shell setprop debug.dotnet.log gref

That produces no output. Doh!

adb shell setprop debug.mono.log gref

Does produce output, but it's quite minimal:

I monodroid-gref: +g+ grefc 1 gwrefc 0 obj-handle 0x7fff0d1ccae8/* -> new-handle 0x29e6/* from thread '<null>'(0)

Where are my stack traces at?

Next I try:

adb shell setprop debug.mono.log gref+

This immediately crashes:

F DEBUG : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
F DEBUG : Build fingerprint: 'google/sdk_gphone_x86_64/generic_x86_64:9/PSR1.180720.075/5124027:user/release-keys'
F DEBUG : Revision: '0'
F DEBUG : ABI: 'x86_64'
F DEBUG : pid: 9081, tid: 9081, name: form.chefs.skia >>> uno.platform.chefs.skia <<<
F DEBUG : signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0
F DEBUG : Cause: null pointer dereference
F DEBUG : rax 0000000000000000 rbx 0000000000000001 rcx 0000000000000006 rdx 00007aa40c400000
F DEBUG : r8 fffffffffffffff0 r9 0000000000000000 r10 00007aa48cb68370 r11 0000000000000206
F DEBUG : r12 0000000000000000 r13 0000000000000001 r14 00007aa3f6f67fb0 r15 00000000000029e6
F DEBUG : rdi 00007aa40c427208 rsi 00007aa40c5c4e00
F DEBUG : rbp 00007fff0d1cc7e0 rsp 00007fff0d1cc540 rip 00007aa3f319f5c0
F DEBUG : 
F DEBUG : backtrace:
F DEBUG : #00 pc 00000000030d05c0 /data/app/uno.platform.chefs.skia-Iqzt05HN0djhiaKkWjWDww==/lib/x86_64/libChefs.so (offset 0x2f06000)
F DEBUG : #01 pc 00000000035ddef4 /data/app/uno.platform.chefs.skia-Iqzt05HN0djhiaKkWjWDww==/lib/x86_64/libChefs.so (offset 0x2f06000)
F DEBUG : #02 pc 0000000000000001 <unknown>

Oof.

File output?

adb shell setprop debug.mono.log 'gref=/data/local/tmp/g.txt' 

That works (in that the app doesn't immediately crash at startup).

Unfortunately, it also doesn't work, in that /data/local/tmp/g.txt is not created.

No stack traces for me!

Also unsurprisingly, enabling GREF logging -- however minimal -- "fixes" the GREF crash. (Fun!)

I'm not sure how to provide more info about the crash. ☹️

[jonpryor]

I should also note that I observed this on an emulator created with:

#!/bin/sh -ex

TMP="$HOME/Developer/tmp"
export ANDROID_HOME="$HOME/android-toolchain/sdk"
export ANDROID_PREFS_ROOT="${ANDROID_PREFS_ROOT:=$HOME}"

AVD_NAME="xamarin_android_emulator"
AVD_ABI=x86_64

AVDMANAGER="$ANDROID_HOME/cmdline-tools/12.0/bin/avdmanager"

if "$AVDMANAGER" list avd | grep "Name: $AVD_NAME" >/dev/null 2>&1; then
 "$AVDMANAGER" delete avd -n "$AVD_NAME"
fi

echo "no" | "$ANDROID_HOME/cmdline-tools/12.0/bin/avdmanager" create avd \
 -n "$AVD_NAME" --abi "$AVD_ABI" \
 -k "system-images;android-28;google_apis_playstore;x86_64" \
 --sdcard 128M

exec "$ANDROID_HOME/emulator/emulator" -avd "$AVD_NAME" \
 -skin 1280x800 -memory 4096 -gpu swiftshader_indirect -no-snapshot \
 -no-boot-anim -noaudio -prop ro.debuggable=1 \
 > "$TMP/loge.txt" 2>&1 

[jonpryor]

And another added observed wrinkle: crashes (for me) happen more reliably if debug.mono.log is not set, i.e. immediately after launching the emulator, the first three launches crashed. Fourth launch was "fine"; fifth launch crashed, sixth launch was "fine."

Set debug.mono.log=gref, and the next 5 launches did not crash.

"Clear" debug.mono.log ¹, first launch crashed (this time; previous attempt I didn't see a crash until I recreated the emulator), and next 5 launches did not crash.

Footnotes

1. adb shell setprop debug.mono.log '""' ↩

[grendello]

"So," thinks I, "what about GREF logging?"

adb shell setprop debug.dotnet.log gref

That produces no output. Doh!

Adding "aliases" to all the debug.mono.* system properties would double the number of calls to get system properties at startup, and while the debug.dotnet.* aliases might be nice, I think it's not very practical given years of dotnet.mono.* use and the overhead involved.

adb shell setprop debug.mono.log gref

Does produce output, but it's quite minimal:

I monodroid-gref: +g+ grefc 1 gwrefc 0 obj-handle 0x7fff0d1ccae8/* -> new-handle 0x29e6/* from thread '<null>'(0)

Where are my stack traces at?

Output for debug.mono.log gref is identical to what the MonoVM-based runtime produces, what stack traces do you mean? gref+ works identically between MonoVM and CoreCLR but it does crash immediately with NativeAOT, I'm working on figuring out where it crashes. As it stands, your crash has more info than mine:

10-31 10:32:49.694 12422 12422 F DEBUG : Cause: null pointer dereference
10-31 10:32:49.694 12422 12422 F DEBUG : x0 000000718a3e1dc4 x1 0000007fc5758494 x2 0000007093fd3470 x3 0000007fc5758498
10-31 10:32:49.694 12422 12422 F DEBUG : x4 0000000000000060 x5 000000000005691e x6 1f646b636d60672c x7 7f7f7f7f7f7f7f7f
10-31 10:32:49.694 12422 12422 F DEBUG : x8 0000000000000000 x9 0000000000000746 x10 0000006e48f8a9c8 x11 000000001a115128
10-31 10:32:49.694 12422 12422 F DEBUG : x12 0000000000060006 x13 000000007fffffff x14 000000000005691e x15 000000082bfbe0b0
10-31 10:32:49.694 12422 12422 F DEBUG : x16 0000006e48ed8a60 x17 000000718a3374e0 x18 00000071a1104000 x19 0000000000000001
10-31 10:32:49.694 12422 12422 F DEBUG : x20 0000007fc5758559 x21 0000006e48f8a000 x22 000000000000002a x23 0000006e48b68816
10-31 10:32:49.694 12422 12422 F DEBUG : x24 0000007fc5758660 x25 0000007fc5758760 x26 0000000000000000 x27 0000007fc5758768
10-31 10:32:49.694 12422 12422 F DEBUG : x28 000000000000000c x29 0000007fc57587a0
10-31 10:32:49.694 12422 12422 F DEBUG : lr 0005756e48ca8c20 sp 0000007fc57584d0 pc 0000006e48ca8c88 pst 0000000080001000
10-31 10:32:49.694 12422 12422 F DEBUG : esr 0000000092000006
10-31 10:32:49.694 12422 12422 F DEBUG : 1 total frames
10-31 10:32:49.694 12422 12422 F DEBUG : backtrace:
10-31 10:32:49.694 12422 12422 F DEBUG : #00 pc 0000000000071c88 /data/app/~~eoNQbwNv4NZ6P4nszJV0MQ==/com.xamarin.XAPerfTest.net10-ZJyDWnNfe44UQmOqpMkRRA==/split_config.arm64_v8a.apk (offset 0x201000)

Oh well :)

[grendello]

@jonpryor can you try adding -p:StripSymbols=false to your build command line? Maybe it'll give us symbols in your stack trace

[grendello]

So the crash is because the from string (which should contain the stack trace) is null when calling _monodroid_gref_log_new and we didn't expect that. The root cause, however, appears to be that NativeAOT doesn't see gref logging as enabled in this method, and so it never actually captures the trace. On MonoVM and CoreCLR when gref+ logging is enabled, we never pass null for from and thus it happened to work.

I'll open a PR later today which will address this issue.

[vyacheslav-volkov]

@grendello Is there any chance this fix could be included (or backported) into an .NET 10 Android service release?
I'm testing NativeAOT on a fairly large application and, in general, everything works great — but about once every 5–10 launches the app crashes with a null pointer dereference that looks related to this issue.
If it weren’t for this crash, NativeAOT would be very stable in our scenario, as I haven't noticed any other problems so far.

[grendello]

@vyacheslav-volkov I'm not opposed to including it in .NET10 servicing, I think it would be just fine since the fix doesn't affect MonoVM. However, the decision isn't mine.

/cc @jonathanpeppers