Skip to content

check-sof-logger: catch ERROR in firmware traces and fail - #799

Merged
marc-hb merged 1 commit into
thesofproject:mainfrom
marc-hb:catch-logger-error
Jul 28, 2022
Merged

check-sof-logger: catch ERROR in firmware traces and fail#799
marc-hb merged 1 commit into
thesofproject:mainfrom
marc-hb:catch-logger-error

Conversation

@marc-hb

Copy link
Copy Markdown
Collaborator

Add function find_logger_error() in case-lib so it can re-used by other
tests later.

Signed-off-by: Marc Herbert marc.herbert@intel.com

@marc-hb

Copy link
Copy Markdown
CollaboratorAuthor

https://sof-ci.01.org/softestpr/PR799/build917/devicetest/?model=BDW_WSB_RT286&testcase=check-sof-logger has a very interesting:

 TIMESTAMP (us) DELTA C# COMPONENT LOCATION CONTENT	ktime=117.383s @ 2021-11-01 23:27:33 +0000 UTC
[ 1852324251.812] ( 0.000) c0 dma-trace src/trace/dma-trace.c:422 INFO SHM: FW ABI 0x3014000 DBG ABI 0x5003000 tag v1.9-rc1-211-gb0bb21dc0ee6 src hash 0xa9c07289 (ldc hash 0xa9c07289)
[ 324686.862] ( 324686.875) c0 dma-trace src/trace/dma-trace.c:605 ERROR dtrace_add_event(): number of dropped logs = 1

@marc-hb

Copy link
Copy Markdown
CollaboratorAuthor

SOFCI TEST

@marc-hb
marc-hbforce-pushed the catch-logger-error branch from 9a8bab5 to 10ec30eCompareMay 3, 2022 01:09
@marc-hb
marc-hb requested a review from ujfalusiMay 3, 2022 01:31
@marc-hb

Copy link
Copy Markdown
CollaboratorAuthor

https://sof-ci.01.org/softestpr/PR799/build917/devicetest/?model=BDW_WSB_RT286&testcase=check-sof-logger has a very interesting:

https://sof-ci.01.org/softestpr/PR799/build16/devicetest has something similar on BYT (BDW is not in PR testing anywmore)

[ 178559514.207] ( 0.000) c0 dma-trace src/trace/dma-trace.c:347 INFO SHM: FW ABI 0x3016000 DBG ABI 0x5003000 tags SOF:v2.0-rc1-813-g5cfc37040bbe src hash 0xb72c3013 (ldc hash 0xb72c3013)
[ 27442.082] ( 27442.082) c0 dma-trace src/trace/dma-trace.c:654 ERROR dtrace_add_event(): number of dropped logs = 8

@marc-hb
marc-hbforce-pushed the catch-logger-error branch from 10ec30e to 31640e2CompareMay 26, 2022 04:21
@marc-hb
marc-hb marked this pull request as ready for review May 26, 2022 05:05
@marc-hb
marc-hb requested a review from a team as a code ownerMay 26, 2022 05:05
@ujfalusi

Copy link
Copy Markdown
Contributor

@marc-hb, is it a recent issue or it has been there?
It is very likely to happen if the firmware is generating 'enough' trace messages up to the point when sof-logger starts and opens the sof/trace debugfs file to start consuming the dtrace messages.

If you do not start the sof-logger at all then yeah, you are going to drop messages, but is this an error?

@marc-hb

Copy link
Copy Markdown
CollaboratorAuthor

Thanks @ujfalusi for the prompt review!

Interesting. So this reproduced systematically on BYT and BDW... because the buffer is smaller / too small? Or maybe because they have more interfaces and more to log.

So I can just downgrade this ERROR to a WARN, right?

@marc-hb
marc-hb marked this pull request as draft May 27, 2022 18:50
@ujfalusi

Copy link
Copy Markdown
Contributor

@marc-hb, I think it is WARN rather an ERR, yes (if you refer to the fw print about the dropped new entries. It only takes to not start sof-logger to eventually have them, which is fine?

It can explain 'blackouts' in logs

@marc-hb
marc-hbforce-pushed the catch-logger-error branch from 31640e2 to c572179CompareJuly 27, 2022 01:02
@marc-hb

Copy link
Copy Markdown
CollaboratorAuthor

https://sof-ci.01.org/softestpr/PR799/build112/devicetest is all green. I recommend we merge this "as is" and if this error message become too frequent we can downgrade it.

@marc-hb
marc-hb marked this pull request as ready for review July 27, 2022 03:53
Comment threadtest-case/check-sof-logger.sh
Add function check_logger_error() in case-lib so it can re-used by other
tests later.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
@marc-hb
marc-hbforce-pushed the catch-logger-error branch from c572179 to 87aeb4dCompareJuly 27, 2022 23:28
@marc-hb

Copy link
Copy Markdown
CollaboratorAuthor

https://sof-ci.01.org/softestpr/PR799/build113/devicetest/ has one usual WHL UPX suspend/resume TIMEOUT, everything else is green.

@keqiaozhangkeqiaozhang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@marc-hb

Copy link
Copy Markdown
CollaboratorAuthor

Thanks @keqiaozhang

@marc-hb
marc-hb merged commit 6544bc1 into thesofproject:mainJul 28, 2022
@marc-hb
marc-hb deleted the catch-logger-error branch July 28, 2022 02:18
@marc-hb

marc-hb commented Jul 29, 2022

Copy link
Copy Markdown
CollaboratorAuthor

Pattern matching fix submitted in

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:logsLog and results collection, storage, etc.False Pass / green failure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@marc-hb@ujfalusi@keqiaozhang