Uh oh!
There was an error while loading. Please reload this page.
TEST: Ignone - test CI - #5
Closed
mengdonglin wants to merge 2 commits into
Closed
Conversation
mengdonglin
commented
Jun 21, 2018
CollaboratorAuthor
Don't merge. For CI test. |
RDharageswari pushed a commit
to RDharageswari/sof
that referenced
this pull request
Jun 6, 2020
This BUG halt was reported a while back, but the patch somehow got missed: PID: 2879 TASK: c16adaa0 CPU: 1 COMMAND: "sctpn" #0 [f418dd28] crash_kexec at c04a7d8c thesofproject#1 [f418dd7c] oops_end at c0863e02 thesofproject#2 [f418dd90] do_invalid_op at c040aaca thesofproject#3 [f418de28] error_code (via invalid_op) at c08631a5 EAX: f34baac0 EBX: 00000090 ECX: f418deb0 EDX: f5542950 EBP: 00000000 DS: 007b ESI: f34ba800 ES: 007b EDI: f418dea0 GS: 00e0 CS: 0060 EIP: c046fa5e ERR: ffffffff EFLAGS: 00010286 thesofproject#4 [f418de5c] add_timer at c046fa5e thesofproject#5 [f418de68] sctp_do_sm at f8db8c77 [sctp] thesofproject#6 [f418df30] sctp_primitive_SHUTDOWN at f8dcc1b5 [sctp] thesofproject#7 [f418df48] inet_shutdown at c080baf9 thesofproject#8 [f418df5c] sys_shutdown at c079eedf thesofproject#9 [f418df70] sys_socketcall at c079fe88 EAX: ffffffda EBX: 0000000d ECX: bfceea90 EDX: 0937af98 DS: 007b ESI: 0000000c ES: 007b EDI: b7150ae4 SS: 007b ESP: bfceea7c EBP: bfceeaa8 GS: 0033 CS: 0073 EIP: b775c424 ERR: 00000066 EFLAGS: 00000282 It appears that the side effect that starts the shutdown timer was processed multiple times, which can happen as multiple paths can trigger it. This of course leads to the BUG halt in add_timer getting called. Fix seems pretty straightforward, just check before the timer is added if its already been started. If it has mod the timer instead to min(current expiration, new expiration) Its been tested but not confirmed to fix the problem, as the issue has only occured in production environments where test kernels are enjoined from being installed. It appears to be a sane fix to me though. Also, recentely, Jere found a reproducer posted on list to confirm that this resolves the issues Signed-off-by: Neil Horman <nhorman@tuxdriver.com> CC: Vlad Yasevich <vyasevich@gmail.com> CC: "David S. Miller" <davem@davemloft.net> CC: jere.leppanen@nokia.com CC: marcelo.leitner@gmail.com CC: netdev@vger.kernel.org Acked-by: Marcelo Ricardo Leitner <marcelo.leitner@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
marc-hb added a commit
to marc-hb/sof
that referenced
this pull request
Feb 27, 2024
As discussed in the alternative approach zephyrproject-rtos/zephyr#68494, k_panic() in POSIX mode has various shortcomings that do not provide a useful trace. Useless pointers to signal handlers or other cleanup routines are printed instead. Leverage our already existing k_sys_fatal_error_handler() and dereference a NULL pointer there when in POSIX mode. This "fails fast" and provides a complete and relevant stack trace in CI when fuzzing or when using some other static analyzer. Example of how fuzzing failure thesofproject#8832 would have looked like in CI results thanks to this commit: ``` ./build-fuzz/zephyr/zephyr.exe: Running 1 inputs 1 time(s) each. Running: ./rballoc_align_fuzz_crash *** Booting Zephyr OS build zephyr-v3.5.0-3971-ge07de4e0a167 *** [00:00:00.000,000] <inf> main: SOF on native_posix [00:00:00.000,000] <inf> main: SOF initialized @ WEST_TOPDIR/sof/zephyr/lib/alloc.c:391 [00:00:00.000,000] <err> os: >>> ZEPHYR FATAL ERROR 4: Kernel panic [00:00:00.000,000] <err> os: Current thread: 0x891f8a0 (unknown) [00:00:00.000,000] <err> zephyr: Halting emulation AddressSanitizer:DEADLYSIGNAL ================================================================= ==1784402==ERROR: AddressSanitizer: SEGV on unknown address 0x00000000 ==1784402==The signal is caused by a WRITE memory access. ==1784402==Hint: address points to the zero page. #0 0x829a77d in k_sys_fatal_error_handler zephyr/wrapper.c:352:19 #1 0x829b8c0 in rballoc_align zephyr/lib/alloc.c:391:3 thesofproject#2 0x8281438 in buffer_alloc src/audio/buffer.c:58:16 thesofproject#3 0x826a60a in buffer_new src/ipc/ipc-helper.c:48:11 thesofproject#4 0x8262107 in ipc_buffer_new src/ipc/ipc3/helper.c:459:11 thesofproject#5 0x825944d in ipc_glb_tplg_buffer_new src/ipc/ipc3/handler.c:1305:8 thesofproject#6 0x8257036 in ipc_cmd src/ipc/ipc3/handler.c:1651:9 thesofproject#7 0x8272e59 in ipc_platform_do_cmd src/platform/posix/ipc.c:162:2 thesofproject#8 0x826a2ac in ipc_do_cmd src/ipc/ipc-common.c:328:9 thesofproject#9 0x829b0ab in task_run zephyr/include/rtos/task.h:94:9 thesofproject#10 0x829abd8 in edf_work_handler zephyr/edf_schedule.c:32:16 thesofproject#11 0x83560f7 in work_queue_main zephyr/kernel/work.c:688:3 thesofproject#12 0x82244c2 in z_thread_entry zephyr/lib/os/thread_entry.c:48:2 ``` Signed-off-by: Marc Herbert <marc.herbert@intel.com>
marc-hb added a commit
to marc-hb/sof
that referenced
this pull request
Feb 27, 2024
As discussed in the alternative approach zephyrproject-rtos/zephyr#68494, k_panic() in POSIX mode has various shortcomings that do not provide a useful trace. Useless pointers to signal handlers or other cleanup routines are printed instead. Leverage our already existing k_sys_fatal_error_handler() and dereference a NULL pointer there when in POSIX mode. This "fails fast" and provides a complete and relevant stack trace in CI when fuzzing or when using some other static analyzer. Example of how fuzzing failure thesofproject#8832 would have looked like in CI results thanks to this commit: ``` ./build-fuzz/zephyr/zephyr.exe: Running 1 inputs 1 time(s) each. Running: ./rballoc_align_fuzz_crash *** Booting Zephyr OS build zephyr-v3.5.0-3971-ge07de4e0a167 *** [00:00:00.000,000] <inf> main: SOF on native_posix [00:00:00.000,000] <inf> main: SOF initialized @ WEST_TOPDIR/sof/zephyr/lib/alloc.c:391 [00:00:00.000,000] <err> os: >>> ZEPHYR FATAL ERROR 4: Kernel panic [00:00:00.000,000] <err> os: Current thread: 0x891f8a0 (unknown) [00:00:00.000,000] <err> zephyr: Halting emulation AddressSanitizer:DEADLYSIGNAL ================================================================= ==1784402==ERROR: AddressSanitizer: SEGV on unknown address 0x00000000 ==1784402==The signal is caused by a WRITE memory access. ==1784402==Hint: address points to the zero page. #0 0x829a77d in k_sys_fatal_error_handler zephyr/wrapper.c:352:19 #1 0x829b8c0 in rballoc_align zephyr/lib/alloc.c:391:3 thesofproject#2 0x8281438 in buffer_alloc src/audio/buffer.c:58:16 thesofproject#3 0x826a60a in buffer_new src/ipc/ipc-helper.c:48:11 thesofproject#4 0x8262107 in ipc_buffer_new src/ipc/ipc3/helper.c:459:11 thesofproject#5 0x825944d in ipc_glb_tplg_buffer_new src/ipc/ipc3/handler.c:1305:8 thesofproject#6 0x8257036 in ipc_cmd src/ipc/ipc3/handler.c:1651:9 thesofproject#7 0x8272e59 in ipc_platform_do_cmd src/platform/posix/ipc.c:162:2 thesofproject#8 0x826a2ac in ipc_do_cmd src/ipc/ipc-common.c:328:9 thesofproject#9 0x829b0ab in task_run zephyr/include/rtos/task.h:94:9 thesofproject#10 0x829abd8 in edf_work_handler zephyr/edf_schedule.c:32:16 thesofproject#11 0x83560f7 in work_queue_main zephyr/kernel/work.c:688:3 thesofproject#12 0x82244c2 in z_thread_entry zephyr/lib/os/thread_entry.c:48:2 ``` Signed-off-by: Marc Herbert <marc.herbert@intel.com>
lgirdwood pushed a commit
that referenced
this pull request
Feb 28, 2024
As discussed in the alternative approach zephyrproject-rtos/zephyr#68494, k_panic() in POSIX mode has various shortcomings that do not provide a useful trace. Useless pointers to signal handlers or other cleanup routines are printed instead. Leverage our already existing k_sys_fatal_error_handler() and dereference a NULL pointer there when in POSIX mode. This "fails fast" and provides a complete and relevant stack trace in CI when fuzzing or when using some other static analyzer. Example of how fuzzing failure #8832 would have looked like in CI results thanks to this commit: ``` ./build-fuzz/zephyr/zephyr.exe: Running 1 inputs 1 time(s) each. Running: ./rballoc_align_fuzz_crash *** Booting Zephyr OS build zephyr-v3.5.0-3971-ge07de4e0a167 *** [00:00:00.000,000] <inf> main: SOF on native_posix [00:00:00.000,000] <inf> main: SOF initialized @ WEST_TOPDIR/sof/zephyr/lib/alloc.c:391 [00:00:00.000,000] <err> os: >>> ZEPHYR FATAL ERROR 4: Kernel panic [00:00:00.000,000] <err> os: Current thread: 0x891f8a0 (unknown) [00:00:00.000,000] <err> zephyr: Halting emulation AddressSanitizer:DEADLYSIGNAL ================================================================= ==1784402==ERROR: AddressSanitizer: SEGV on unknown address 0x00000000 ==1784402==The signal is caused by a WRITE memory access. ==1784402==Hint: address points to the zero page. #0 0x829a77d in k_sys_fatal_error_handler zephyr/wrapper.c:352:19 #1 0x829b8c0 in rballoc_align zephyr/lib/alloc.c:391:3 #2 0x8281438 in buffer_alloc src/audio/buffer.c:58:16 #3 0x826a60a in buffer_new src/ipc/ipc-helper.c:48:11 #4 0x8262107 in ipc_buffer_new src/ipc/ipc3/helper.c:459:11 #5 0x825944d in ipc_glb_tplg_buffer_new src/ipc/ipc3/handler.c:1305:8 #6 0x8257036 in ipc_cmd src/ipc/ipc3/handler.c:1651:9 #7 0x8272e59 in ipc_platform_do_cmd src/platform/posix/ipc.c:162:2 #8 0x826a2ac in ipc_do_cmd src/ipc/ipc-common.c:328:9 #9 0x829b0ab in task_run zephyr/include/rtos/task.h:94:9 #10 0x829abd8 in edf_work_handler zephyr/edf_schedule.c:32:16 #11 0x83560f7 in work_queue_main zephyr/kernel/work.c:688:3 #12 0x82244c2 in z_thread_entry zephyr/lib/os/thread_entry.c:48:2 ``` Signed-off-by: Marc Herbert <marc.herbert@intel.com>
jsarha pushed a commit
to jsarha/sof
that referenced
this pull request
Oct 16, 2024
The fuzzer engine has produced crash caused by NULL pointer read that originated from ipc_stream_pcm_free(). The crash happens when the pipeline of the found comp_dev does not have a source_comp and pipeline_reset() is called for it. This commit simply adds a test for such a situation and bails out if it is found. Here is the call stack from the situation: #0 0x81e9317 in dev_comp_pipe_id sof/sof/src/include/sof/audio/component.h:646:25 thesofproject#1 0x81e8015 in pipeline_comp_reset sof/sof/src/audio/pipeline/pipeline-graph.c:326:22 thesofproject#2 0x81e7d1d in pipeline_reset sof/sof/src/audio/pipeline/pipeline-graph.c:393:8 thesofproject#3 0x820d7ea in ipc_stream_pcm_free sof/sof/src/ipc/ipc3/handler.c:398:8 thesofproject#4 0x8208969 in ipc_cmd sof/sof/src/ipc/ipc3/handler.c:1689:9 thesofproject#5 0x81cbed8 in ipc_platform_do_cmd sof/sof/src/platform/posix/ipc.c:162:2 thesofproject#6 0x81d10db in ipc_do_cmd sof/sof/src/ipc/ipc-common.c:330:9 thesofproject#7 0x81f87e9 in task_run sof/sof/zephyr/include/rtos/task.h:94:9 thesofproject#8 0x81f8308 in edf_work_handler sof/sof/zephyr/edf_schedule.c:31:16 thesofproject#9 0x82b4b32 in work_queue_main sof/zephyr/kernel/work.c:668:3 thesofproject#10 0x8193ec2 in z_thread_entry sof/zephyr/lib/os/thread_entry.c:36:2 thesofproject#11 0x815f639 in __asan::AsanThread::ThreadStart(unsigned long long) /src/llvm-project/compiler-rt/lib/asan/asan_thread.cpp:277:25 Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>
jsarha pushed a commit
to jsarha/sof
that referenced
this pull request
Oct 17, 2024
The fuzzer engine has produced crash caused by NULL pointer read that originated from ipc_stream_pcm_free(). The crash happens when the pipeline of the found comp_dev does not have a source_comp and pipeline_reset() is called for it. This commit simply adds a test for such a situation and bails out if it is found. Here is the call stack from the situation: #0 0x81e9317 in dev_comp_pipe_id sof/sof/src/include/sof/audio/component.h:646:25 thesofproject#1 0x81e8015 in pipeline_comp_reset sof/sof/src/audio/pipeline/pipeline-graph.c:326:22 thesofproject#2 0x81e7d1d in pipeline_reset sof/sof/src/audio/pipeline/pipeline-graph.c:393:8 thesofproject#3 0x820d7ea in ipc_stream_pcm_free sof/sof/src/ipc/ipc3/handler.c:398:8 thesofproject#4 0x8208969 in ipc_cmd sof/sof/src/ipc/ipc3/handler.c:1689:9 thesofproject#5 0x81cbed8 in ipc_platform_do_cmd sof/sof/src/platform/posix/ipc.c:162:2 thesofproject#6 0x81d10db in ipc_do_cmd sof/sof/src/ipc/ipc-common.c:330:9 thesofproject#7 0x81f87e9 in task_run sof/sof/zephyr/include/rtos/task.h:94:9 thesofproject#8 0x81f8308 in edf_work_handler sof/sof/zephyr/edf_schedule.c:31:16 thesofproject#9 0x82b4b32 in work_queue_main sof/zephyr/kernel/work.c:668:3 thesofproject#10 0x8193ec2 in z_thread_entry sof/zephyr/lib/os/thread_entry.c:36:2 thesofproject#11 0x815f639 in __asan::AsanThread::ThreadStart(unsigned long long) /src/llvm-project/compiler-rt/lib/asan/asan_thread.cpp:277:25 Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>
jsarha pushed a commit
to jsarha/sof
that referenced
this pull request
Oct 18, 2024
The fuzzer engine has produced crash caused by NULL pointer read that originated from ipc3 ipc_stream_pcm_free(). The crash happens when the pipeline of the found comp_dev does not have a source_comp and pipeline_reset() is called for it. This commit adds check to pipeline_comp_test() for this situation and bails out if it is found. Here is the call stack from the situation: #0 0x81e9317 in dev_comp_pipe_id sof/sof/src/include/sof/audio/component.h:646:25 thesofproject#1 0x81e8015 in pipeline_comp_reset sof/sof/src/audio/pipeline/pipeline-graph.c:326:22 thesofproject#2 0x81e7d1d in pipeline_reset sof/sof/src/audio/pipeline/pipeline-graph.c:393:8 thesofproject#3 0x820d7ea in ipc_stream_pcm_free sof/sof/src/ipc/ipc3/handler.c:398:8 thesofproject#4 0x8208969 in ipc_cmd sof/sof/src/ipc/ipc3/handler.c:1689:9 thesofproject#5 0x81cbed8 in ipc_platform_do_cmd sof/sof/src/platform/posix/ipc.c:162:2 thesofproject#6 0x81d10db in ipc_do_cmd sof/sof/src/ipc/ipc-common.c:330:9 thesofproject#7 0x81f87e9 in task_run sof/sof/zephyr/include/rtos/task.h:94:9 thesofproject#8 0x81f8308 in edf_work_handler sof/sof/zephyr/edf_schedule.c:31:16 thesofproject#9 0x82b4b32 in work_queue_main sof/zephyr/kernel/work.c:668:3 thesofproject#10 0x8193ec2 in z_thread_entry sof/zephyr/lib/os/thread_entry.c:36:2 thesofproject#11 0x815f639 in __asan::AsanThread::ThreadStart(unsigned long long) /src/llvm-project/compiler-rt/lib/asan/asan_thread.cpp:277:25 Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>
lgirdwood pushed a commit
that referenced
this pull request
Oct 21, 2024
The fuzzer engine has produced crash caused by NULL pointer read that originated from ipc3 ipc_stream_pcm_free(). The crash happens when the pipeline of the found comp_dev does not have a source_comp and pipeline_reset() is called for it. This commit adds check to pipeline_comp_test() for this situation and bails out if it is found. Here is the call stack from the situation: #0 0x81e9317 in dev_comp_pipe_id sof/sof/src/include/sof/audio/component.h:646:25 #1 0x81e8015 in pipeline_comp_reset sof/sof/src/audio/pipeline/pipeline-graph.c:326:22 #2 0x81e7d1d in pipeline_reset sof/sof/src/audio/pipeline/pipeline-graph.c:393:8 #3 0x820d7ea in ipc_stream_pcm_free sof/sof/src/ipc/ipc3/handler.c:398:8 #4 0x8208969 in ipc_cmd sof/sof/src/ipc/ipc3/handler.c:1689:9 #5 0x81cbed8 in ipc_platform_do_cmd sof/sof/src/platform/posix/ipc.c:162:2 #6 0x81d10db in ipc_do_cmd sof/sof/src/ipc/ipc-common.c:330:9 #7 0x81f87e9 in task_run sof/sof/zephyr/include/rtos/task.h:94:9 #8 0x81f8308 in edf_work_handler sof/sof/zephyr/edf_schedule.c:31:16 #9 0x82b4b32 in work_queue_main sof/zephyr/kernel/work.c:668:3 #10 0x8193ec2 in z_thread_entry sof/zephyr/lib/os/thread_entry.c:36:2 #11 0x815f639 in __asan::AsanThread::ThreadStart(unsigned long long) /src/llvm-project/compiler-rt/lib/asan/asan_thread.cpp:277:25 Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>
kv2019i pushed a commit
that referenced
this pull request
Jul 17, 2026
Fix a heap out-of-bounds access found by the IPC4 libFuzzer harness (native_sim/x86, ASAN-enabled build with CONFIG_SYS_HEAP_SANITIZER_ASAN). A single 173-byte crash input triggers a use-after-poison READ of size 4 in copier_dai_create(): ==ERROR: AddressSanitizer: use-after-poison on address 0x088cf1c4 READ of size 4 ... in copier_dai_create copier_dai.c:284 #1 copier_init copier.c:220 #2 module_init generic.c:123 #3 module_adapter_new_ext module_adapter.c:307 #4 comp_new_ipc4 ipc4/helper.c:198 #5 ipc4_init_module_instance handler-user.c:763 copier_dai.c:284 reads cd->config.gtw_cfg.config_length, which lands 8 bytes past the end of the under-allocated cd block. Fuzzer payload -------------- The harness frames the raw input as a 2-byte little-endian length prefix followed by an 8-byte IPC4 compact header (primary + extension word) and a payload that is mirrored into MAILBOX_HOSTBOX. The decoded message is an INIT_MODULE_INSTANCE (MODULE_MSG / INIT_INSTANCE) whose module_id resolves to the copier driver, so comp_new_ipc4() -> module_adapter_new() -> copier_init() runs on host-controlled config bytes. The copier's ipc4_copier_module_cfg carried (values from gdb): gtw_cfg.node_id.dw = 0x050012f5 dma_type = 18 (ipc4_alh_uaol_stream_link_ output_class, a non-host ALH/UAOL link gateway), v_index = 245 gtw_cfg.dma_buffer_size = 0xffffffff gtw_cfg.config_length = 0xffffffff <-- the trigger md->cfg.size (payload) = 888 bytes node_id != IPC4_INVALID_NODE_ID selects the gateway-copier path, and the UAOL link class routes into copier_dai_create(). The long run of 0xff bytes at the tail of the input is what supplies config_length = 0xffffffff. libFuzzer discovered the message by mutating the corpus; the crucial field is the all-ones config_length. Root cause ---------- copier_init() sizes the variable-length gateway blob that trails the config struct: if (copier->gtw_cfg.config_length > 1) { gtw_cfg_var_size += (copier->gtw_cfg.config_length - 1) << 2; cfg_total_size += gtw_cfg_var_size; } if (cfg_total_size > md->cfg.size) /* payload guard */ return -EINVAL; cd = mod_zalloc(mod, sizeof(*cd) + gtw_cfg_var_size); config_length is a uint32_t from the untrusted payload. The expression (config_length - 1) << 2 is evaluated in 32-bit arithmetic and wraps: with 0xffffffff it yields 0xfffffff8, i.e. -8 as a size_t. That poisons both dependent computations: * cfg_total_size wraps down to 76 (below even sizeof(*copier) = 84), so the "cfg_total_size > md->cfg.size" guard added earlier does not fire (76 < 888). * mod_zalloc() is called with sizeof(*cd) + (size_t)-8 = 452 - 8 = 444 bytes, while struct copier_data needs 452. gtw_cfg.config_length sits at offset 444, exactly past the truncated allocation. copier_dai_create() then reads that field (and the config_data pointer right after it) out of bounds. size_t is 32-bit on native_sim and on the real Xtensa DSP targets, so this is a genuine firmware bug, not just a host-fuzzer artifact. Fix --- Validate config_length before any size arithmetic is performed on it. The blob is expressed in dwords and must fit within the received init payload, so an upper bound of md->cfg.size / sizeof(uint32_t) both is correct and keeps (config_length - 1) << 2 from overflowing (the mailbox payload is bounded by SOF_IPC_MSG_MAX_SIZE): if (copier->gtw_cfg.config_length > md->cfg.size / sizeof(uint32_t)) { comp_err(dev, "copier_init(): invalid gtw_cfg.config_length %u", copier->gtw_cfg.config_length); return -EINVAL; } The existing precise "cfg_total_size > md->cfg.size" check is retained as a secondary guard for the non-overflowing mismatch case. Verification ------------ - The crash input now executes cleanly: config_length is rejected with -EINVAL, exit code 0, no ASAN report. The new bound branch is confirmed hit (config_length 0xffffffff vs bound 222). - Full IPC4 corpus replay: 92,780 inputs, 0 crashes (-runs=0), coverage unchanged (cov 1331 / ft 5270), confirming the check does not over-restrict legitimate copier init payloads. Found-by: IPC4 libFuzzer (ASAN, native_sim) Signed-off-by: Tomasz Leman <tomasz.m.leman@intel.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please don't merege. This is to test CI support for github pull request.