Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -2,7 +2,7 @@ name: CI

on:
push:
branches: [ main, master, 'feature/**' ]
branches: [ '**' ]
tags: ['v*']
pull_request:
workflow_dispatch:
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/lint.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -4,8 +4,7 @@ on:
pull_request:
push:
branches:
- main
- feature/**
- '**'

permissions:
contents: read
Expand Down
12 changes: 8 additions & 4 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -10,7 +10,7 @@ Trace helps you collect structured runtime logs in Arduino ESP32 projects with b

## Why use Trace?

* **Bounded memory** - recent history, realtime delivery, pending flush logs, and payload lengths have separate configured limits.
* **Bounded memory** - recent history, realtime delivery, pending flush logs, and payload lengths use fixed-capacity storage.
* **Structured output** - log records keep level, tag, message, formatted text, sequence, and uptime.
* **Task-side callbacks** - realtime observation and persistence callbacks run from the internal Trace task.
* **ESP32 task control** - configure byte stack size, priority, core affinity, and stack memory preference.
Expand DownExpand Up@@ -80,15 +80,18 @@ void loop() {
* `maxRecentLogs` controls queryable in-RAM history only.
* `maxRealtimeLogs` controls the realtime delivery queue used by `onLog()` and stream output.
* `maxPendingLogs` controls unsaved logs waiting for flush.
* Queue-count `0` disables that queue. Payload-cap `0` means unlimited.
* Queue-count `0` disables that queue. Payload-cap `0` uses the compiled maximum for that payload type.
* `setStream()` writes formatted realtime logs to any Arduino `Print` stream such as `Serial`, `Serial1`, `WiFiClient`, or a custom sink.
* Stream output uses ANSI colors by default. Callback, flush, and query `TraceLog::formatted` values stay plain text.
* After `Trace::init()`, accepted direct C-string log calls do not allocate on the internal enqueue path when all target queues have capacity and no output-boundary conversion is triggered.
* Query APIs, flush batch conversion, `onLog()` conversion, stream implementations, and user-created `std::string` values may still allocate.
* `onLog()` is for realtime observation; `onFlush()` is for persistence.
* Callbacks should avoid long blocking work and should not recursively call Trace logging methods.
* Trace does not own attached `Print` or `Tempo` instances. Keep them alive until `Trace::end()` completes.
* Detaching or replacing `Print` or `Tempo` while Trace is active does not synchronize already snapshotted worker use.
* Stack sizes are FreeRTOS byte sizes on ESP32 and must be at least 1024 bytes.
* `TraceStackType::Auto` prefers PSRAM task stacks when supported and falls back to internal RAM.
* `TraceStorageMemory::PreferPsram` opts recent and pending log buffers into PSRAM with internal fallback. Realtime delivery stays internal.

## Examples

Expand DownExpand Up@@ -149,16 +152,17 @@ For the full API, see [`docs/api.md`](docs/api.md).
| Platform | `espressif32` |
| Language | C++20 |
| Filesystem | none |
| PSRAM | Optional for task stacks when ESP-IDF support is available |
| PSRAM | Optional for task stacks and opt-in recent/pending log storage |
| Dependencies | `bblanchon/ArduinoJson >= 7.0.0` |
| Exceptions | Not used |
| Status | Early-stage `0.1.0` |
| Status | Release `0.2.0` |

## Configuration

```cpp
TraceConfig config;
config.stackSize = 4096;
config.storageMemory = TraceStorageMemory::Internal;
config.maxRecentLogs = 100;
config.maxRealtimeLogs = 100;
config.maxPendingLogs = 50;
Expand Down
4 changes: 3 additions & 1 deletion docs/api.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,7 +12,7 @@

`TraceFlushResult` values are `Ok`, `Failed`, and `Retry`.

`Ok` removes flushed pending logs. `Failed` is terminal for the current `flushAndWait()` call but retains pending logs. `Retry` retains pending logs, schedules another attempt using `TraceConfig::retryIntervalMs`, and keeps `flushAndWait()` waiting until `Ok`, `Failed`, or timeout.
`Ok` removes flushed pending logs. `Failed` is terminal for the current `flushAndWait()` call but retains pending logs. `Retry` retains pending logs, schedules another attempt using `TraceConfig::retryIntervalMs`, and keeps `flushAndWait()` waiting until `Ok`, `Failed`, or timeout. Normal flush requests do not bypass the retry deadline; urgent error and fatal flush requests may bypass it.

## Main methods

Expand DownExpand Up@@ -67,6 +67,8 @@ std::vector<TraceLog> getLastLogs(size_t count);
std::vector<TraceLog> getLogsByTag(const char *tag);
```

`TraceDiag` includes queue counts, drop and flush counters, task stack information, queue allocation byte counts, and PSRAM placement flags for recent, realtime, and pending queues.

## TraceLog

`TraceLog` stores `sequence`, `level`, `tag`, `message`, `formatted`, `timeText`, `uptimeMs`, and `truncated`.
Expand Down
34 changes: 30 additions & 4 deletions docs/configuration.md
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,14 @@
# Configuration

`TraceConfig` controls the internal task, log limits, flush timing, overflow behavior, JSON formatting, level filtering, and stream colors.
`TraceConfig` controls the internal task, log storage memory, log limits, flush timing, overflow behavior, JSON formatting, level filtering, and stream colors.

```cpp
TraceConfig config;
config.stackSize = 4096;
config.priority = 1;
config.coreId = tskNO_AFFINITY;
config.stackType = TraceStackType::Auto;
config.storageMemory = TraceStorageMemory::Internal;
config.maxRecentLogs = 100;
config.maxRealtimeLogs = 100;
config.maxPendingLogs = 50;
Expand DownExpand Up@@ -35,7 +36,21 @@ config.maxFormattedLength = 384;

`maxPendingLogs = 0` disables persistence buffering. Logs are still accepted for recent history and realtime delivery, but they are counted as dropped for persistence.

Queue-count `0` means disabled. Payload-cap `0` means unlimited.
Queue-count `0` means disabled. Payload-cap `0` means use the compiled maximum for that payload type.

## Storage memory

Trace stores recent, realtime, and pending logs in fixed-capacity ring buffers. Each enabled queue is allocated once during `init()`.

After `Trace::init()`, accepted direct C-string log calls do not allocate on the internal enqueue path when all target queues have capacity and no output-boundary conversion is triggered.

This guarantee is intentionally narrow. Query APIs allocate `std::vector<TraceLog>`, flush batch conversion and `onLog()` conversion can allocate because public `TraceLog` contains `std::string`, stream implementations may allocate internally, and user code may allocate before passing `std::string` values to Trace.

`TraceStorageMemory::Internal` is the deterministic default. Recent, realtime, and pending queues use internal-capable memory.

`TraceStorageMemory::PreferPsram` uses PSRAM for recent and pending queues when PSRAM is available, otherwise it falls back to internal-capable memory. The realtime queue remains internal.

`TraceStorageMemory::RequirePsram` requires recent and pending queues to allocate in PSRAM. `init()` returns `TraceStatus::OutOfMemory` if PSRAM is unavailable or allocation fails. The realtime queue remains internal.

## Payload limits

Expand All@@ -45,6 +60,17 @@ Queue-count `0` means disabled. Payload-cap `0` means unlimited.

`maxFormattedLength` applies to `printf`-style and JSON-formatted input before it becomes `TraceLog::message`.

Runtime payload limits are bounded by compile-time caps:

```cpp
TRACE_RECORD_MAX_TAG_LENGTH
TRACE_RECORD_MAX_MESSAGE_LENGTH
TRACE_FORMATTED_BUFFER_LENGTH
TRACE_TIME_TEXT_BUFFER_LENGTH
```

Setting a runtime limit above the compiled cap clamps to the compiled cap. Setting a runtime payload limit to `0` uses the compiled cap.

When Trace truncates a log, `TraceLog::truncated` is set. `TraceDiag::truncatedLogCount` increments once per log record, even if both tag and message were truncated.

## Flush triggers
Expand All@@ -56,7 +82,7 @@ Trace flushes pending logs when:
* `flushIntervalMs` elapses with pending logs.
* `flushOnError` is enabled and an `Error` or `Fatal` log is queued.

`retryIntervalMs` controls the delay after `TraceFlushResult::Retry`. Values smaller than the worker poll interval are clamped so retry cannot spin in a tight loop.
`retryIntervalMs` controls the delay after `TraceFlushResult::Retry`. Values smaller than the worker poll interval are clamped so retry cannot spin in a tight loop. Normal flush requests set `flushRequested` but do not clear or bypass the retry deadline. Urgent error and fatal flush requests may bypass the retry deadline.

## Flush results

Expand All@@ -74,7 +100,7 @@ Trace flushes pending logs when:

`BlockCaller` requests a flush and waits up to `blockCallerTimeoutMs` for pending space.

`FlushImmediately` requests a flush immediately and retries until space appears or `blockCallerTimeoutMs` expires.
`FlushImmediately` requests a flush immediately and queues the new record only if pending space becomes available before `blockCallerTimeoutMs` expires.

## Stack policy

Expand Down
2 changes: 1 addition & 1 deletion library.json
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
{
"name": "Trace",
"version": "0.1.0",
"version": "0.2.0",
"description": "Logging and diagnostics library for ESP32 devices with bounded buffers and task-side flushing.",
"keywords": [
"esp32",
Expand Down
2 changes: 1 addition & 1 deletion library.properties
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
name=Trace
version=0.1.0
version=0.2.0
author=zekageri
maintainer=zekageri
sentence=Logging and diagnostics library for ESP32 devices.
Expand Down
51 changes: 46 additions & 5 deletions src/Trace.h
Original file line numberDiff line numberDiff line change
Expand Up@@ -13,6 +13,22 @@
#include <freertos/FreeRTOS.h>
#include <freertos/task.h>

#ifndef TRACE_RECORD_MAX_TAG_LENGTH
#define TRACE_RECORD_MAX_TAG_LENGTH 32
#endif

#ifndef TRACE_RECORD_MAX_MESSAGE_LENGTH
#define TRACE_RECORD_MAX_MESSAGE_LENGTH 256
#endif

#ifndef TRACE_FORMATTED_BUFFER_LENGTH
#define TRACE_FORMATTED_BUFFER_LENGTH 384
#endif

#ifndef TRACE_TIME_TEXT_BUFFER_LENGTH
#define TRACE_TIME_TEXT_BUFFER_LENGTH 48
#endif

class Tempo;
struct TraceImpl;

Expand DownExpand Up@@ -43,6 +59,12 @@ enum class TraceStackType : uint8_t {
Psram,
};

enum class TraceStorageMemory : uint8_t {
Internal,
PreferPsram,
RequirePsram,
};

enum class TraceOverflowPolicy : uint8_t {
DropOldestPending,
DropNewest,
Expand DownExpand Up@@ -89,6 +111,7 @@ struct TraceConfig {
UBaseType_t priority = 1;
BaseType_t coreId = tskNO_AFFINITY;
TraceStackType stackType = TraceStackType::Auto;
TraceStorageMemory storageMemory = TraceStorageMemory::Internal;
size_t maxRecentLogs = 100;
size_t maxRealtimeLogs = 100;
size_t maxPendingLogs = 50;
Expand DownExpand Up@@ -146,6 +169,12 @@ struct TraceDiag {
size_t stackHighWaterMarkBytes = 0;
TraceStackType requestedStackType = TraceStackType::Auto;
TraceStackType actualStackType = TraceStackType::Internal;
size_t recentAllocatedBytes = 0;
size_t realtimeAllocatedBytes = 0;
size_t pendingAllocatedBytes = 0;
bool recentLogsInPsram = false;
bool realtimeLogsInPsram = false;
bool pendingLogsInPsram = false;
};

using TraceTimeFormatter = bool (*)(const Tempo &tempo, char *buffer, size_t bufferSize);
Expand DownExpand Up@@ -242,6 +271,14 @@ class Trace {

private:
TraceResult log(TraceLevel level, const char *tag, const std::string &message);
TraceResult logRaw(
TraceLevel level,
const char *tag,
size_t tagLen,
const char *message,
size_t messageLen,
bool alreadyTruncated
);
TraceResult logJson(TraceLevel level, const char *tag, const JsonDocument &doc);
TraceResult logVPrintf(TraceLevel level, const char *tag, const char *format, va_list args);

Expand All@@ -260,14 +297,18 @@ class Trace {
return TraceResult::failure(TraceStatus::InvalidArgument, "format failed");
}
const size_t limit = getMaxFormattedLength();
const bool unlimited = limit == 0;
const size_t outputLength = static_cast<size_t>(needed);
const size_t boundedLength = unlimited ? outputLength : std::min(outputLength, limit);
std::vector<char> buffer(boundedLength + 1);
snprintf(buffer.data(), buffer.size(), format, args...);
return log(level, tag, std::string(buffer.data()), !unlimited && outputLength > limit);
const size_t boundedLength = std::min(outputLength, limit);
char buffer[TRACE_FORMATTED_BUFFER_LENGTH + 1] = {};
snprintf(buffer, boundedLength + 1, format, args...);
const size_t tagLimit = getMaxTagLength();
const size_t tagLen = boundedStrLen(tag, tagLimit + 1);
return logRaw(level, tag, tagLen, buffer, boundedLength, outputLength > limit);
}

static size_t boundedStrLen(const char *value, size_t maxLen);
size_t getMaxTagLength() const;
size_t getMaxMessageLength() const;
size_t getMaxFormattedLength() const;
TraceResult log(
TraceLevel level,
Expand Down
30 changes: 18 additions & 12 deletions src/TraceFlush.cpp
Original file line numberDiff line numberDiff line change
Expand Up@@ -17,8 +17,14 @@ void TraceImpl::performFlush() {
return;
}
callback = onFlush;
batch.logs = pendingLogs;
batch.createdAtUptimeMs = millis();
batch.logs.reserve(pendingLogs.size());
for (size_t i = 0; i < pendingLogs.size(); ++i) {
TraceRecord record;
if (pendingLogs.peek(i, record)) {
batch.logs.push_back(toPublicLog(record));
}
}
if (!batch.logs.empty()) {
maxSequence = batch.logs.back().sequence;
}
Expand DownExpand Up@@ -46,16 +52,10 @@ void TraceImpl::performFlush() {
if (flushResult == TraceFlushResult::Ok) {
nextFlushAttemptMs = 0;
if (maxSequence > 0) {
pendingLogs.erase(
std::remove_if(
pendingLogs.begin(),
pendingLogs.end(),
[maxSequence](const TraceLog &log) {
return log.sequence <= maxSequence;
}
),
pendingLogs.end()
);
TraceRecord record;
while (pendingLogs.peek(0, record) && record.sequence <= maxSequence) {
pendingLogs.pop(record);
}
}
flushSuccessCount++;
} else if (flushResult == TraceFlushResult::Retry) {
Expand All@@ -76,13 +76,19 @@ bool TraceImpl::shouldFlushNow() {
if (!lock) {
return false;
}
const uint64_t nowMs = millis();
if (
!pendingLogs.empty() && nextFlushAttemptMs > 0 && nowMs < nextFlushAttemptMs &&
!urgentFlushRequested
) {
return false;
}
if (flushRequested || urgentFlushRequested) {
return true;
}
if (pendingLogs.empty()) {
return false;
}
const uint64_t nowMs = millis();
if (nextFlushAttemptMs > 0) {
return nowMs >= nextFlushAttemptMs;
}
Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -2,7 +2,7 @@ name: CI

on:
push:
branches: [ main, master, 'feature/**' ]
branches: [ '**' ]
tags: ['v*']
pull_request:
workflow_dispatch:
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/lint.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -4,8 +4,7 @@ on:
pull_request:
push:
branches:
- main
- feature/**
- '**'

permissions:
contents: read
Expand Down
12 changes: 8 additions & 4 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -10,7 +10,7 @@ Trace helps you collect structured runtime logs in Arduino ESP32 projects with b

## Why use Trace?

* **Bounded memory** - recent history, realtime delivery, pending flush logs, and payload lengths have separate configured limits.
* **Bounded memory** - recent history, realtime delivery, pending flush logs, and payload lengths use fixed-capacity storage.
* **Structured output** - log records keep level, tag, message, formatted text, sequence, and uptime.
* **Task-side callbacks** - realtime observation and persistence callbacks run from the internal Trace task.
* **ESP32 task control** - configure byte stack size, priority, core affinity, and stack memory preference.
Expand DownExpand Up@@ -80,15 +80,18 @@ void loop() {
* `maxRecentLogs` controls queryable in-RAM history only.
* `maxRealtimeLogs` controls the realtime delivery queue used by `onLog()` and stream output.
* `maxPendingLogs` controls unsaved logs waiting for flush.
* Queue-count `0` disables that queue. Payload-cap `0` means unlimited.
* Queue-count `0` disables that queue. Payload-cap `0` uses the compiled maximum for that payload type.
* `setStream()` writes formatted realtime logs to any Arduino `Print` stream such as `Serial`, `Serial1`, `WiFiClient`, or a custom sink.
* Stream output uses ANSI colors by default. Callback, flush, and query `TraceLog::formatted` values stay plain text.
* After `Trace::init()`, accepted direct C-string log calls do not allocate on the internal enqueue path when all target queues have capacity and no output-boundary conversion is triggered.
* Query APIs, flush batch conversion, `onLog()` conversion, stream implementations, and user-created `std::string` values may still allocate.
* `onLog()` is for realtime observation; `onFlush()` is for persistence.
* Callbacks should avoid long blocking work and should not recursively call Trace logging methods.
* Trace does not own attached `Print` or `Tempo` instances. Keep them alive until `Trace::end()` completes.
* Detaching or replacing `Print` or `Tempo` while Trace is active does not synchronize already snapshotted worker use.
* Stack sizes are FreeRTOS byte sizes on ESP32 and must be at least 1024 bytes.
* `TraceStackType::Auto` prefers PSRAM task stacks when supported and falls back to internal RAM.
* `TraceStorageMemory::PreferPsram` opts recent and pending log buffers into PSRAM with internal fallback. Realtime delivery stays internal.

## Examples

Expand DownExpand Up@@ -149,16 +152,17 @@ For the full API, see [`docs/api.md`](docs/api.md).
| Platform | `espressif32` |
| Language | C++20 |
| Filesystem | none |
| PSRAM | Optional for task stacks when ESP-IDF support is available |
| PSRAM | Optional for task stacks and opt-in recent/pending log storage |
| Dependencies | `bblanchon/ArduinoJson >= 7.0.0` |
| Exceptions | Not used |
| Status | Early-stage `0.1.0` |
| Status | Release `0.2.0` |

## Configuration

```cpp
TraceConfig config;
config.stackSize = 4096;
config.storageMemory = TraceStorageMemory::Internal;
config.maxRecentLogs = 100;
config.maxRealtimeLogs = 100;
config.maxPendingLogs = 50;
Expand Down
4 changes: 3 additions & 1 deletion docs/api.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,7 +12,7 @@

`TraceFlushResult` values are `Ok`, `Failed`, and `Retry`.

`Ok` removes flushed pending logs. `Failed` is terminal for the current `flushAndWait()` call but retains pending logs. `Retry` retains pending logs, schedules another attempt using `TraceConfig::retryIntervalMs`, and keeps `flushAndWait()` waiting until `Ok`, `Failed`, or timeout.
`Ok` removes flushed pending logs. `Failed` is terminal for the current `flushAndWait()` call but retains pending logs. `Retry` retains pending logs, schedules another attempt using `TraceConfig::retryIntervalMs`, and keeps `flushAndWait()` waiting until `Ok`, `Failed`, or timeout. Normal flush requests do not bypass the retry deadline; urgent error and fatal flush requests may bypass it.

## Main methods

Expand DownExpand Up@@ -67,6 +67,8 @@ std::vector<TraceLog> getLastLogs(size_t count);
std::vector<TraceLog> getLogsByTag(const char *tag);
```

`TraceDiag` includes queue counts, drop and flush counters, task stack information, queue allocation byte counts, and PSRAM placement flags for recent, realtime, and pending queues.

## TraceLog

`TraceLog` stores `sequence`, `level`, `tag`, `message`, `formatted`, `timeText`, `uptimeMs`, and `truncated`.
Expand Down
34 changes: 30 additions & 4 deletions docs/configuration.md
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,14 @@
# Configuration

`TraceConfig` controls the internal task, log limits, flush timing, overflow behavior, JSON formatting, level filtering, and stream colors.
`TraceConfig` controls the internal task, log storage memory, log limits, flush timing, overflow behavior, JSON formatting, level filtering, and stream colors.

```cpp
TraceConfig config;
config.stackSize = 4096;
config.priority = 1;
config.coreId = tskNO_AFFINITY;
config.stackType = TraceStackType::Auto;
config.storageMemory = TraceStorageMemory::Internal;
config.maxRecentLogs = 100;
config.maxRealtimeLogs = 100;
config.maxPendingLogs = 50;
Expand DownExpand Up@@ -35,7 +36,21 @@ config.maxFormattedLength = 384;

`maxPendingLogs = 0` disables persistence buffering. Logs are still accepted for recent history and realtime delivery, but they are counted as dropped for persistence.

Queue-count `0` means disabled. Payload-cap `0` means unlimited.
Queue-count `0` means disabled. Payload-cap `0` means use the compiled maximum for that payload type.

## Storage memory

Trace stores recent, realtime, and pending logs in fixed-capacity ring buffers. Each enabled queue is allocated once during `init()`.

After `Trace::init()`, accepted direct C-string log calls do not allocate on the internal enqueue path when all target queues have capacity and no output-boundary conversion is triggered.

This guarantee is intentionally narrow. Query APIs allocate `std::vector<TraceLog>`, flush batch conversion and `onLog()` conversion can allocate because public `TraceLog` contains `std::string`, stream implementations may allocate internally, and user code may allocate before passing `std::string` values to Trace.

`TraceStorageMemory::Internal` is the deterministic default. Recent, realtime, and pending queues use internal-capable memory.

`TraceStorageMemory::PreferPsram` uses PSRAM for recent and pending queues when PSRAM is available, otherwise it falls back to internal-capable memory. The realtime queue remains internal.

`TraceStorageMemory::RequirePsram` requires recent and pending queues to allocate in PSRAM. `init()` returns `TraceStatus::OutOfMemory` if PSRAM is unavailable or allocation fails. The realtime queue remains internal.

## Payload limits

Expand All@@ -45,6 +60,17 @@ Queue-count `0` means disabled. Payload-cap `0` means unlimited.

`maxFormattedLength` applies to `printf`-style and JSON-formatted input before it becomes `TraceLog::message`.

Runtime payload limits are bounded by compile-time caps:

```cpp
TRACE_RECORD_MAX_TAG_LENGTH
TRACE_RECORD_MAX_MESSAGE_LENGTH
TRACE_FORMATTED_BUFFER_LENGTH
TRACE_TIME_TEXT_BUFFER_LENGTH
```

Setting a runtime limit above the compiled cap clamps to the compiled cap. Setting a runtime payload limit to `0` uses the compiled cap.

When Trace truncates a log, `TraceLog::truncated` is set. `TraceDiag::truncatedLogCount` increments once per log record, even if both tag and message were truncated.

## Flush triggers
Expand All@@ -56,7 +82,7 @@ Trace flushes pending logs when:
* `flushIntervalMs` elapses with pending logs.
* `flushOnError` is enabled and an `Error` or `Fatal` log is queued.

`retryIntervalMs` controls the delay after `TraceFlushResult::Retry`. Values smaller than the worker poll interval are clamped so retry cannot spin in a tight loop.
`retryIntervalMs` controls the delay after `TraceFlushResult::Retry`. Values smaller than the worker poll interval are clamped so retry cannot spin in a tight loop. Normal flush requests set `flushRequested` but do not clear or bypass the retry deadline. Urgent error and fatal flush requests may bypass the retry deadline.

## Flush results

Expand All@@ -74,7 +100,7 @@ Trace flushes pending logs when:

`BlockCaller` requests a flush and waits up to `blockCallerTimeoutMs` for pending space.

`FlushImmediately` requests a flush immediately and retries until space appears or `blockCallerTimeoutMs` expires.
`FlushImmediately` requests a flush immediately and queues the new record only if pending space becomes available before `blockCallerTimeoutMs` expires.

## Stack policy

Expand Down
2 changes: 1 addition & 1 deletion library.json
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
{
"name": "Trace",
"version": "0.1.0",
"version": "0.2.0",
"description": "Logging and diagnostics library for ESP32 devices with bounded buffers and task-side flushing.",
"keywords": [
"esp32",
Expand Down
2 changes: 1 addition & 1 deletion library.properties
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
name=Trace
version=0.1.0
version=0.2.0
author=zekageri
maintainer=zekageri
sentence=Logging and diagnostics library for ESP32 devices.
Expand Down
51 changes: 46 additions & 5 deletions src/Trace.h
Original file line numberDiff line numberDiff line change
Expand Up@@ -13,6 +13,22 @@
#include <freertos/FreeRTOS.h>
#include <freertos/task.h>

#ifndef TRACE_RECORD_MAX_TAG_LENGTH
#define TRACE_RECORD_MAX_TAG_LENGTH 32
#endif

#ifndef TRACE_RECORD_MAX_MESSAGE_LENGTH
#define TRACE_RECORD_MAX_MESSAGE_LENGTH 256
#endif

#ifndef TRACE_FORMATTED_BUFFER_LENGTH
#define TRACE_FORMATTED_BUFFER_LENGTH 384
#endif

#ifndef TRACE_TIME_TEXT_BUFFER_LENGTH
#define TRACE_TIME_TEXT_BUFFER_LENGTH 48
#endif

class Tempo;
struct TraceImpl;

Expand DownExpand Up@@ -43,6 +59,12 @@ enum class TraceStackType : uint8_t {
Psram,
};

enum class TraceStorageMemory : uint8_t {
Internal,
PreferPsram,
RequirePsram,
};

enum class TraceOverflowPolicy : uint8_t {
DropOldestPending,
DropNewest,
Expand DownExpand Up@@ -89,6 +111,7 @@ struct TraceConfig {
UBaseType_t priority = 1;
BaseType_t coreId = tskNO_AFFINITY;
TraceStackType stackType = TraceStackType::Auto;
TraceStorageMemory storageMemory = TraceStorageMemory::Internal;
size_t maxRecentLogs = 100;
size_t maxRealtimeLogs = 100;
size_t maxPendingLogs = 50;
Expand DownExpand Up@@ -146,6 +169,12 @@ struct TraceDiag {
size_t stackHighWaterMarkBytes = 0;
TraceStackType requestedStackType = TraceStackType::Auto;
TraceStackType actualStackType = TraceStackType::Internal;
size_t recentAllocatedBytes = 0;
size_t realtimeAllocatedBytes = 0;
size_t pendingAllocatedBytes = 0;
bool recentLogsInPsram = false;
bool realtimeLogsInPsram = false;
bool pendingLogsInPsram = false;
};

using TraceTimeFormatter = bool (*)(const Tempo &tempo, char *buffer, size_t bufferSize);
Expand DownExpand Up@@ -242,6 +271,14 @@ class Trace {

private:
TraceResult log(TraceLevel level, const char *tag, const std::string &message);
TraceResult logRaw(
TraceLevel level,
const char *tag,
size_t tagLen,
const char *message,
size_t messageLen,
bool alreadyTruncated
);
TraceResult logJson(TraceLevel level, const char *tag, const JsonDocument &doc);
TraceResult logVPrintf(TraceLevel level, const char *tag, const char *format, va_list args);

Expand All@@ -260,14 +297,18 @@ class Trace {
return TraceResult::failure(TraceStatus::InvalidArgument, "format failed");
}
const size_t limit = getMaxFormattedLength();
const bool unlimited = limit == 0;
const size_t outputLength = static_cast<size_t>(needed);
const size_t boundedLength = unlimited ? outputLength : std::min(outputLength, limit);
std::vector<char> buffer(boundedLength + 1);
snprintf(buffer.data(), buffer.size(), format, args...);
return log(level, tag, std::string(buffer.data()), !unlimited && outputLength > limit);
const size_t boundedLength = std::min(outputLength, limit);
char buffer[TRACE_FORMATTED_BUFFER_LENGTH + 1] = {};
snprintf(buffer, boundedLength + 1, format, args...);
const size_t tagLimit = getMaxTagLength();
const size_t tagLen = boundedStrLen(tag, tagLimit + 1);
return logRaw(level, tag, tagLen, buffer, boundedLength, outputLength > limit);
}

static size_t boundedStrLen(const char *value, size_t maxLen);
size_t getMaxTagLength() const;
size_t getMaxMessageLength() const;
size_t getMaxFormattedLength() const;
TraceResult log(
TraceLevel level,
Expand Down
30 changes: 18 additions & 12 deletions src/TraceFlush.cpp
Original file line numberDiff line numberDiff line change
Expand Up@@ -17,8 +17,14 @@ void TraceImpl::performFlush() {
return;
}
callback = onFlush;
batch.logs = pendingLogs;
batch.createdAtUptimeMs = millis();
batch.logs.reserve(pendingLogs.size());
for (size_t i = 0; i < pendingLogs.size(); ++i) {
TraceRecord record;
if (pendingLogs.peek(i, record)) {
batch.logs.push_back(toPublicLog(record));
}
}
if (!batch.logs.empty()) {
maxSequence = batch.logs.back().sequence;
}
Expand DownExpand Up@@ -46,16 +52,10 @@ void TraceImpl::performFlush() {
if (flushResult == TraceFlushResult::Ok) {
nextFlushAttemptMs = 0;
if (maxSequence > 0) {
pendingLogs.erase(
std::remove_if(
pendingLogs.begin(),
pendingLogs.end(),
[maxSequence](const TraceLog &log) {
return log.sequence <= maxSequence;
}
),
pendingLogs.end()
);
TraceRecord record;
while (pendingLogs.peek(0, record) && record.sequence <= maxSequence) {
pendingLogs.pop(record);
}
}
flushSuccessCount++;
} else if (flushResult == TraceFlushResult::Retry) {
Expand All@@ -76,13 +76,19 @@ bool TraceImpl::shouldFlushNow() {
if (!lock) {
return false;
}
const uint64_t nowMs = millis();
if (
!pendingLogs.empty() && nextFlushAttemptMs > 0 && nowMs < nextFlushAttemptMs &&
!urgentFlushRequested
) {
return false;
}
if (flushRequested || urgentFlushRequested) {
return true;
}
if (pendingLogs.empty()) {
return false;
}
const uint64_t nowMs = millis();
if (nextFlushAttemptMs > 0) {
return nowMs >= nextFlushAttemptMs;
}
Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -2,7 +2,7 @@ name: CI

on:
push:
branches: [ main, master, 'feature/**' ]
branches: [ '**' ]
tags: ['v*']
pull_request:
workflow_dispatch:
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/lint.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -4,8 +4,7 @@ on:
pull_request:
push:
branches:
- main
- feature/**
- '**'

permissions:
contents: read
Expand Down
12 changes: 8 additions & 4 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -10,7 +10,7 @@ Trace helps you collect structured runtime logs in Arduino ESP32 projects with b

## Why use Trace?

* **Bounded memory** - recent history, realtime delivery, pending flush logs, and payload lengths have separate configured limits.
* **Bounded memory** - recent history, realtime delivery, pending flush logs, and payload lengths use fixed-capacity storage.
* **Structured output** - log records keep level, tag, message, formatted text, sequence, and uptime.
* **Task-side callbacks** - realtime observation and persistence callbacks run from the internal Trace task.
* **ESP32 task control** - configure byte stack size, priority, core affinity, and stack memory preference.
Expand DownExpand Up@@ -80,15 +80,18 @@ void loop() {
* `maxRecentLogs` controls queryable in-RAM history only.
* `maxRealtimeLogs` controls the realtime delivery queue used by `onLog()` and stream output.
* `maxPendingLogs` controls unsaved logs waiting for flush.
* Queue-count `0` disables that queue. Payload-cap `0` means unlimited.
* Queue-count `0` disables that queue. Payload-cap `0` uses the compiled maximum for that payload type.
* `setStream()` writes formatted realtime logs to any Arduino `Print` stream such as `Serial`, `Serial1`, `WiFiClient`, or a custom sink.
* Stream output uses ANSI colors by default. Callback, flush, and query `TraceLog::formatted` values stay plain text.
* After `Trace::init()`, accepted direct C-string log calls do not allocate on the internal enqueue path when all target queues have capacity and no output-boundary conversion is triggered.
* Query APIs, flush batch conversion, `onLog()` conversion, stream implementations, and user-created `std::string` values may still allocate.
* `onLog()` is for realtime observation; `onFlush()` is for persistence.
* Callbacks should avoid long blocking work and should not recursively call Trace logging methods.
* Trace does not own attached `Print` or `Tempo` instances. Keep them alive until `Trace::end()` completes.
* Detaching or replacing `Print` or `Tempo` while Trace is active does not synchronize already snapshotted worker use.
* Stack sizes are FreeRTOS byte sizes on ESP32 and must be at least 1024 bytes.
* `TraceStackType::Auto` prefers PSRAM task stacks when supported and falls back to internal RAM.
* `TraceStorageMemory::PreferPsram` opts recent and pending log buffers into PSRAM with internal fallback. Realtime delivery stays internal.

## Examples

Expand DownExpand Up@@ -149,16 +152,17 @@ For the full API, see [`docs/api.md`](docs/api.md).
| Platform | `espressif32` |
| Language | C++20 |
| Filesystem | none |
| PSRAM | Optional for task stacks when ESP-IDF support is available |
| PSRAM | Optional for task stacks and opt-in recent/pending log storage |
| Dependencies | `bblanchon/ArduinoJson >= 7.0.0` |
| Exceptions | Not used |
| Status | Early-stage `0.1.0` |
| Status | Release `0.2.0` |

## Configuration

```cpp
TraceConfig config;
config.stackSize = 4096;
config.storageMemory = TraceStorageMemory::Internal;
config.maxRecentLogs = 100;
config.maxRealtimeLogs = 100;
config.maxPendingLogs = 50;
Expand Down
4 changes: 3 additions & 1 deletion docs/api.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,7 +12,7 @@

`TraceFlushResult` values are `Ok`, `Failed`, and `Retry`.

`Ok` removes flushed pending logs. `Failed` is terminal for the current `flushAndWait()` call but retains pending logs. `Retry` retains pending logs, schedules another attempt using `TraceConfig::retryIntervalMs`, and keeps `flushAndWait()` waiting until `Ok`, `Failed`, or timeout.
`Ok` removes flushed pending logs. `Failed` is terminal for the current `flushAndWait()` call but retains pending logs. `Retry` retains pending logs, schedules another attempt using `TraceConfig::retryIntervalMs`, and keeps `flushAndWait()` waiting until `Ok`, `Failed`, or timeout. Normal flush requests do not bypass the retry deadline; urgent error and fatal flush requests may bypass it.

## Main methods

Expand DownExpand Up@@ -67,6 +67,8 @@ std::vector<TraceLog> getLastLogs(size_t count);
std::vector<TraceLog> getLogsByTag(const char *tag);
```

`TraceDiag` includes queue counts, drop and flush counters, task stack information, queue allocation byte counts, and PSRAM placement flags for recent, realtime, and pending queues.

## TraceLog

`TraceLog` stores `sequence`, `level`, `tag`, `message`, `formatted`, `timeText`, `uptimeMs`, and `truncated`.
Expand Down
34 changes: 30 additions & 4 deletions docs/configuration.md
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,14 @@
# Configuration

`TraceConfig` controls the internal task, log limits, flush timing, overflow behavior, JSON formatting, level filtering, and stream colors.
`TraceConfig` controls the internal task, log storage memory, log limits, flush timing, overflow behavior, JSON formatting, level filtering, and stream colors.

```cpp
TraceConfig config;
config.stackSize = 4096;
config.priority = 1;
config.coreId = tskNO_AFFINITY;
config.stackType = TraceStackType::Auto;
config.storageMemory = TraceStorageMemory::Internal;
config.maxRecentLogs = 100;
config.maxRealtimeLogs = 100;
config.maxPendingLogs = 50;
Expand DownExpand Up@@ -35,7 +36,21 @@ config.maxFormattedLength = 384;

`maxPendingLogs = 0` disables persistence buffering. Logs are still accepted for recent history and realtime delivery, but they are counted as dropped for persistence.

Queue-count `0` means disabled. Payload-cap `0` means unlimited.
Queue-count `0` means disabled. Payload-cap `0` means use the compiled maximum for that payload type.

## Storage memory

Trace stores recent, realtime, and pending logs in fixed-capacity ring buffers. Each enabled queue is allocated once during `init()`.

After `Trace::init()`, accepted direct C-string log calls do not allocate on the internal enqueue path when all target queues have capacity and no output-boundary conversion is triggered.

This guarantee is intentionally narrow. Query APIs allocate `std::vector<TraceLog>`, flush batch conversion and `onLog()` conversion can allocate because public `TraceLog` contains `std::string`, stream implementations may allocate internally, and user code may allocate before passing `std::string` values to Trace.

`TraceStorageMemory::Internal` is the deterministic default. Recent, realtime, and pending queues use internal-capable memory.

`TraceStorageMemory::PreferPsram` uses PSRAM for recent and pending queues when PSRAM is available, otherwise it falls back to internal-capable memory. The realtime queue remains internal.

`TraceStorageMemory::RequirePsram` requires recent and pending queues to allocate in PSRAM. `init()` returns `TraceStatus::OutOfMemory` if PSRAM is unavailable or allocation fails. The realtime queue remains internal.

## Payload limits

Expand All@@ -45,6 +60,17 @@ Queue-count `0` means disabled. Payload-cap `0` means unlimited.

`maxFormattedLength` applies to `printf`-style and JSON-formatted input before it becomes `TraceLog::message`.

Runtime payload limits are bounded by compile-time caps:

```cpp
TRACE_RECORD_MAX_TAG_LENGTH
TRACE_RECORD_MAX_MESSAGE_LENGTH
TRACE_FORMATTED_BUFFER_LENGTH
TRACE_TIME_TEXT_BUFFER_LENGTH
```

Setting a runtime limit above the compiled cap clamps to the compiled cap. Setting a runtime payload limit to `0` uses the compiled cap.

When Trace truncates a log, `TraceLog::truncated` is set. `TraceDiag::truncatedLogCount` increments once per log record, even if both tag and message were truncated.

## Flush triggers
Expand All@@ -56,7 +82,7 @@ Trace flushes pending logs when:
* `flushIntervalMs` elapses with pending logs.
* `flushOnError` is enabled and an `Error` or `Fatal` log is queued.

`retryIntervalMs` controls the delay after `TraceFlushResult::Retry`. Values smaller than the worker poll interval are clamped so retry cannot spin in a tight loop.
`retryIntervalMs` controls the delay after `TraceFlushResult::Retry`. Values smaller than the worker poll interval are clamped so retry cannot spin in a tight loop. Normal flush requests set `flushRequested` but do not clear or bypass the retry deadline. Urgent error and fatal flush requests may bypass the retry deadline.

## Flush results

Expand All@@ -74,7 +100,7 @@ Trace flushes pending logs when:

`BlockCaller` requests a flush and waits up to `blockCallerTimeoutMs` for pending space.

`FlushImmediately` requests a flush immediately and retries until space appears or `blockCallerTimeoutMs` expires.
`FlushImmediately` requests a flush immediately and queues the new record only if pending space becomes available before `blockCallerTimeoutMs` expires.

## Stack policy

Expand Down
2 changes: 1 addition & 1 deletion library.json
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
{
"name": "Trace",
"version": "0.1.0",
"version": "0.2.0",
"description": "Logging and diagnostics library for ESP32 devices with bounded buffers and task-side flushing.",
"keywords": [
"esp32",
Expand Down
2 changes: 1 addition & 1 deletion library.properties
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
name=Trace
version=0.1.0
version=0.2.0
author=zekageri
maintainer=zekageri
sentence=Logging and diagnostics library for ESP32 devices.
Expand Down
51 changes: 46 additions & 5 deletions src/Trace.h
Original file line numberDiff line numberDiff line change
Expand Up@@ -13,6 +13,22 @@
#include <freertos/FreeRTOS.h>
#include <freertos/task.h>

#ifndef TRACE_RECORD_MAX_TAG_LENGTH
#define TRACE_RECORD_MAX_TAG_LENGTH 32
#endif

#ifndef TRACE_RECORD_MAX_MESSAGE_LENGTH
#define TRACE_RECORD_MAX_MESSAGE_LENGTH 256
#endif

#ifndef TRACE_FORMATTED_BUFFER_LENGTH
#define TRACE_FORMATTED_BUFFER_LENGTH 384
#endif

#ifndef TRACE_TIME_TEXT_BUFFER_LENGTH
#define TRACE_TIME_TEXT_BUFFER_LENGTH 48
#endif

class Tempo;
struct TraceImpl;

Expand DownExpand Up@@ -43,6 +59,12 @@ enum class TraceStackType : uint8_t {
Psram,
};

enum class TraceStorageMemory : uint8_t {
Internal,
PreferPsram,
RequirePsram,
};

enum class TraceOverflowPolicy : uint8_t {
DropOldestPending,
DropNewest,
Expand DownExpand Up@@ -89,6 +111,7 @@ struct TraceConfig {
UBaseType_t priority = 1;
BaseType_t coreId = tskNO_AFFINITY;
TraceStackType stackType = TraceStackType::Auto;
TraceStorageMemory storageMemory = TraceStorageMemory::Internal;
size_t maxRecentLogs = 100;
size_t maxRealtimeLogs = 100;
size_t maxPendingLogs = 50;
Expand DownExpand Up@@ -146,6 +169,12 @@ struct TraceDiag {
size_t stackHighWaterMarkBytes = 0;
TraceStackType requestedStackType = TraceStackType::Auto;
TraceStackType actualStackType = TraceStackType::Internal;
size_t recentAllocatedBytes = 0;
size_t realtimeAllocatedBytes = 0;
size_t pendingAllocatedBytes = 0;
bool recentLogsInPsram = false;
bool realtimeLogsInPsram = false;
bool pendingLogsInPsram = false;
};

using TraceTimeFormatter = bool (*)(const Tempo &tempo, char *buffer, size_t bufferSize);
Expand DownExpand Up@@ -242,6 +271,14 @@ class Trace {

private:
TraceResult log(TraceLevel level, const char *tag, const std::string &message);
TraceResult logRaw(
TraceLevel level,
const char *tag,
size_t tagLen,
const char *message,
size_t messageLen,
bool alreadyTruncated
);
TraceResult logJson(TraceLevel level, const char *tag, const JsonDocument &doc);
TraceResult logVPrintf(TraceLevel level, const char *tag, const char *format, va_list args);

Expand All@@ -260,14 +297,18 @@ class Trace {
return TraceResult::failure(TraceStatus::InvalidArgument, "format failed");
}
const size_t limit = getMaxFormattedLength();
const bool unlimited = limit == 0;
const size_t outputLength = static_cast<size_t>(needed);
const size_t boundedLength = unlimited ? outputLength : std::min(outputLength, limit);
std::vector<char> buffer(boundedLength + 1);
snprintf(buffer.data(), buffer.size(), format, args...);
return log(level, tag, std::string(buffer.data()), !unlimited && outputLength > limit);
const size_t boundedLength = std::min(outputLength, limit);
char buffer[TRACE_FORMATTED_BUFFER_LENGTH + 1] = {};
snprintf(buffer, boundedLength + 1, format, args...);
const size_t tagLimit = getMaxTagLength();
const size_t tagLen = boundedStrLen(tag, tagLimit + 1);
return logRaw(level, tag, tagLen, buffer, boundedLength, outputLength > limit);
}

static size_t boundedStrLen(const char *value, size_t maxLen);
size_t getMaxTagLength() const;
size_t getMaxMessageLength() const;
size_t getMaxFormattedLength() const;
TraceResult log(
TraceLevel level,
Expand Down
30 changes: 18 additions & 12 deletions src/TraceFlush.cpp
Original file line numberDiff line numberDiff line change
Expand Up@@ -17,8 +17,14 @@ void TraceImpl::performFlush() {
return;
}
callback = onFlush;
batch.logs = pendingLogs;
batch.createdAtUptimeMs = millis();
batch.logs.reserve(pendingLogs.size());
for (size_t i = 0; i < pendingLogs.size(); ++i) {
TraceRecord record;
if (pendingLogs.peek(i, record)) {
batch.logs.push_back(toPublicLog(record));
}
}
if (!batch.logs.empty()) {
maxSequence = batch.logs.back().sequence;
}
Expand DownExpand Up@@ -46,16 +52,10 @@ void TraceImpl::performFlush() {
if (flushResult == TraceFlushResult::Ok) {
nextFlushAttemptMs = 0;
if (maxSequence > 0) {
pendingLogs.erase(
std::remove_if(
pendingLogs.begin(),
pendingLogs.end(),
[maxSequence](const TraceLog &log) {
return log.sequence <= maxSequence;
}
),
pendingLogs.end()
);
TraceRecord record;
while (pendingLogs.peek(0, record) && record.sequence <= maxSequence) {
pendingLogs.pop(record);
}
}
flushSuccessCount++;
} else if (flushResult == TraceFlushResult::Retry) {
Expand All@@ -76,13 +76,19 @@ bool TraceImpl::shouldFlushNow() {
if (!lock) {
return false;
}
const uint64_t nowMs = millis();
if (
!pendingLogs.empty() && nextFlushAttemptMs > 0 && nowMs < nextFlushAttemptMs &&
!urgentFlushRequested
) {
return false;
}
if (flushRequested || urgentFlushRequested) {
return true;
}
if (pendingLogs.empty()) {
return false;
}
const uint64_t nowMs = millis();
if (nextFlushAttemptMs > 0) {
return nowMs >= nextFlushAttemptMs;
}
Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -2,7 +2,7 @@ name: CI

on:
push:
branches: [ main, master, 'feature/**' ]
branches: [ '**' ]
tags: ['v*']
pull_request:
workflow_dispatch:
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/lint.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -4,8 +4,7 @@ on:
pull_request:
push:
branches:
- main
- feature/**
- '**'

permissions:
contents: read
Expand Down
12 changes: 8 additions & 4 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -10,7 +10,7 @@ Trace helps you collect structured runtime logs in Arduino ESP32 projects with b

## Why use Trace?

* **Bounded memory** - recent history, realtime delivery, pending flush logs, and payload lengths have separate configured limits.
* **Bounded memory** - recent history, realtime delivery, pending flush logs, and payload lengths use fixed-capacity storage.
* **Structured output** - log records keep level, tag, message, formatted text, sequence, and uptime.
* **Task-side callbacks** - realtime observation and persistence callbacks run from the internal Trace task.
* **ESP32 task control** - configure byte stack size, priority, core affinity, and stack memory preference.
Expand DownExpand Up@@ -80,15 +80,18 @@ void loop() {
* `maxRecentLogs` controls queryable in-RAM history only.
* `maxRealtimeLogs` controls the realtime delivery queue used by `onLog()` and stream output.
* `maxPendingLogs` controls unsaved logs waiting for flush.
* Queue-count `0` disables that queue. Payload-cap `0` means unlimited.
* Queue-count `0` disables that queue. Payload-cap `0` uses the compiled maximum for that payload type.
* `setStream()` writes formatted realtime logs to any Arduino `Print` stream such as `Serial`, `Serial1`, `WiFiClient`, or a custom sink.
* Stream output uses ANSI colors by default. Callback, flush, and query `TraceLog::formatted` values stay plain text.
* After `Trace::init()`, accepted direct C-string log calls do not allocate on the internal enqueue path when all target queues have capacity and no output-boundary conversion is triggered.
* Query APIs, flush batch conversion, `onLog()` conversion, stream implementations, and user-created `std::string` values may still allocate.
* `onLog()` is for realtime observation; `onFlush()` is for persistence.
* Callbacks should avoid long blocking work and should not recursively call Trace logging methods.
* Trace does not own attached `Print` or `Tempo` instances. Keep them alive until `Trace::end()` completes.
* Detaching or replacing `Print` or `Tempo` while Trace is active does not synchronize already snapshotted worker use.
* Stack sizes are FreeRTOS byte sizes on ESP32 and must be at least 1024 bytes.
* `TraceStackType::Auto` prefers PSRAM task stacks when supported and falls back to internal RAM.
* `TraceStorageMemory::PreferPsram` opts recent and pending log buffers into PSRAM with internal fallback. Realtime delivery stays internal.

## Examples

Expand DownExpand Up@@ -149,16 +152,17 @@ For the full API, see [`docs/api.md`](docs/api.md).
| Platform | `espressif32` |
| Language | C++20 |
| Filesystem | none |
| PSRAM | Optional for task stacks when ESP-IDF support is available |
| PSRAM | Optional for task stacks and opt-in recent/pending log storage |
| Dependencies | `bblanchon/ArduinoJson >= 7.0.0` |
| Exceptions | Not used |
| Status | Early-stage `0.1.0` |
| Status | Release `0.2.0` |

## Configuration

```cpp
TraceConfig config;
config.stackSize = 4096;
config.storageMemory = TraceStorageMemory::Internal;
config.maxRecentLogs = 100;
config.maxRealtimeLogs = 100;
config.maxPendingLogs = 50;
Expand Down
4 changes: 3 additions & 1 deletion docs/api.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,7 +12,7 @@

`TraceFlushResult` values are `Ok`, `Failed`, and `Retry`.

`Ok` removes flushed pending logs. `Failed` is terminal for the current `flushAndWait()` call but retains pending logs. `Retry` retains pending logs, schedules another attempt using `TraceConfig::retryIntervalMs`, and keeps `flushAndWait()` waiting until `Ok`, `Failed`, or timeout.
`Ok` removes flushed pending logs. `Failed` is terminal for the current `flushAndWait()` call but retains pending logs. `Retry` retains pending logs, schedules another attempt using `TraceConfig::retryIntervalMs`, and keeps `flushAndWait()` waiting until `Ok`, `Failed`, or timeout. Normal flush requests do not bypass the retry deadline; urgent error and fatal flush requests may bypass it.

## Main methods

Expand DownExpand Up@@ -67,6 +67,8 @@ std::vector<TraceLog> getLastLogs(size_t count);
std::vector<TraceLog> getLogsByTag(const char *tag);
```

`TraceDiag` includes queue counts, drop and flush counters, task stack information, queue allocation byte counts, and PSRAM placement flags for recent, realtime, and pending queues.

## TraceLog

`TraceLog` stores `sequence`, `level`, `tag`, `message`, `formatted`, `timeText`, `uptimeMs`, and `truncated`.
Expand Down
34 changes: 30 additions & 4 deletions docs/configuration.md
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,14 @@
# Configuration

`TraceConfig` controls the internal task, log limits, flush timing, overflow behavior, JSON formatting, level filtering, and stream colors.
`TraceConfig` controls the internal task, log storage memory, log limits, flush timing, overflow behavior, JSON formatting, level filtering, and stream colors.

```cpp
TraceConfig config;
config.stackSize = 4096;
config.priority = 1;
config.coreId = tskNO_AFFINITY;
config.stackType = TraceStackType::Auto;
config.storageMemory = TraceStorageMemory::Internal;
config.maxRecentLogs = 100;
config.maxRealtimeLogs = 100;
config.maxPendingLogs = 50;
Expand DownExpand Up@@ -35,7 +36,21 @@ config.maxFormattedLength = 384;

`maxPendingLogs = 0` disables persistence buffering. Logs are still accepted for recent history and realtime delivery, but they are counted as dropped for persistence.

Queue-count `0` means disabled. Payload-cap `0` means unlimited.
Queue-count `0` means disabled. Payload-cap `0` means use the compiled maximum for that payload type.

## Storage memory

Trace stores recent, realtime, and pending logs in fixed-capacity ring buffers. Each enabled queue is allocated once during `init()`.

After `Trace::init()`, accepted direct C-string log calls do not allocate on the internal enqueue path when all target queues have capacity and no output-boundary conversion is triggered.

This guarantee is intentionally narrow. Query APIs allocate `std::vector<TraceLog>`, flush batch conversion and `onLog()` conversion can allocate because public `TraceLog` contains `std::string`, stream implementations may allocate internally, and user code may allocate before passing `std::string` values to Trace.

`TraceStorageMemory::Internal` is the deterministic default. Recent, realtime, and pending queues use internal-capable memory.

`TraceStorageMemory::PreferPsram` uses PSRAM for recent and pending queues when PSRAM is available, otherwise it falls back to internal-capable memory. The realtime queue remains internal.

`TraceStorageMemory::RequirePsram` requires recent and pending queues to allocate in PSRAM. `init()` returns `TraceStatus::OutOfMemory` if PSRAM is unavailable or allocation fails. The realtime queue remains internal.

## Payload limits

Expand All@@ -45,6 +60,17 @@ Queue-count `0` means disabled. Payload-cap `0` means unlimited.

`maxFormattedLength` applies to `printf`-style and JSON-formatted input before it becomes `TraceLog::message`.

Runtime payload limits are bounded by compile-time caps:

```cpp
TRACE_RECORD_MAX_TAG_LENGTH
TRACE_RECORD_MAX_MESSAGE_LENGTH
TRACE_FORMATTED_BUFFER_LENGTH
TRACE_TIME_TEXT_BUFFER_LENGTH
```

Setting a runtime limit above the compiled cap clamps to the compiled cap. Setting a runtime payload limit to `0` uses the compiled cap.

When Trace truncates a log, `TraceLog::truncated` is set. `TraceDiag::truncatedLogCount` increments once per log record, even if both tag and message were truncated.

## Flush triggers
Expand All@@ -56,7 +82,7 @@ Trace flushes pending logs when:
* `flushIntervalMs` elapses with pending logs.
* `flushOnError` is enabled and an `Error` or `Fatal` log is queued.

`retryIntervalMs` controls the delay after `TraceFlushResult::Retry`. Values smaller than the worker poll interval are clamped so retry cannot spin in a tight loop.
`retryIntervalMs` controls the delay after `TraceFlushResult::Retry`. Values smaller than the worker poll interval are clamped so retry cannot spin in a tight loop. Normal flush requests set `flushRequested` but do not clear or bypass the retry deadline. Urgent error and fatal flush requests may bypass the retry deadline.

## Flush results

Expand All@@ -74,7 +100,7 @@ Trace flushes pending logs when:

`BlockCaller` requests a flush and waits up to `blockCallerTimeoutMs` for pending space.

`FlushImmediately` requests a flush immediately and retries until space appears or `blockCallerTimeoutMs` expires.
`FlushImmediately` requests a flush immediately and queues the new record only if pending space becomes available before `blockCallerTimeoutMs` expires.

## Stack policy

Expand Down
2 changes: 1 addition & 1 deletion library.json
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
{
"name": "Trace",
"version": "0.1.0",
"version": "0.2.0",
"description": "Logging and diagnostics library for ESP32 devices with bounded buffers and task-side flushing.",
"keywords": [
"esp32",
Expand Down
2 changes: 1 addition & 1 deletion library.properties
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
name=Trace
version=0.1.0
version=0.2.0
author=zekageri
maintainer=zekageri
sentence=Logging and diagnostics library for ESP32 devices.
Expand Down
51 changes: 46 additions & 5 deletions src/Trace.h
Original file line numberDiff line numberDiff line change
Expand Up@@ -13,6 +13,22 @@
#include <freertos/FreeRTOS.h>
#include <freertos/task.h>

#ifndef TRACE_RECORD_MAX_TAG_LENGTH
#define TRACE_RECORD_MAX_TAG_LENGTH 32
#endif

#ifndef TRACE_RECORD_MAX_MESSAGE_LENGTH
#define TRACE_RECORD_MAX_MESSAGE_LENGTH 256
#endif

#ifndef TRACE_FORMATTED_BUFFER_LENGTH
#define TRACE_FORMATTED_BUFFER_LENGTH 384
#endif

#ifndef TRACE_TIME_TEXT_BUFFER_LENGTH
#define TRACE_TIME_TEXT_BUFFER_LENGTH 48
#endif

class Tempo;
struct TraceImpl;

Expand DownExpand Up@@ -43,6 +59,12 @@ enum class TraceStackType : uint8_t {
Psram,
};

enum class TraceStorageMemory : uint8_t {
Internal,
PreferPsram,
RequirePsram,
};

enum class TraceOverflowPolicy : uint8_t {
DropOldestPending,
DropNewest,
Expand DownExpand Up@@ -89,6 +111,7 @@ struct TraceConfig {
UBaseType_t priority = 1;
BaseType_t coreId = tskNO_AFFINITY;
TraceStackType stackType = TraceStackType::Auto;
TraceStorageMemory storageMemory = TraceStorageMemory::Internal;
size_t maxRecentLogs = 100;
size_t maxRealtimeLogs = 100;
size_t maxPendingLogs = 50;
Expand DownExpand Up@@ -146,6 +169,12 @@ struct TraceDiag {
size_t stackHighWaterMarkBytes = 0;
TraceStackType requestedStackType = TraceStackType::Auto;
TraceStackType actualStackType = TraceStackType::Internal;
size_t recentAllocatedBytes = 0;
size_t realtimeAllocatedBytes = 0;
size_t pendingAllocatedBytes = 0;
bool recentLogsInPsram = false;
bool realtimeLogsInPsram = false;
bool pendingLogsInPsram = false;
};

using TraceTimeFormatter = bool (*)(const Tempo &tempo, char *buffer, size_t bufferSize);
Expand DownExpand Up@@ -242,6 +271,14 @@ class Trace {

private:
TraceResult log(TraceLevel level, const char *tag, const std::string &message);
TraceResult logRaw(
TraceLevel level,
const char *tag,
size_t tagLen,
const char *message,
size_t messageLen,
bool alreadyTruncated
);
TraceResult logJson(TraceLevel level, const char *tag, const JsonDocument &doc);
TraceResult logVPrintf(TraceLevel level, const char *tag, const char *format, va_list args);

Expand All@@ -260,14 +297,18 @@ class Trace {
return TraceResult::failure(TraceStatus::InvalidArgument, "format failed");
}
const size_t limit = getMaxFormattedLength();
const bool unlimited = limit == 0;
const size_t outputLength = static_cast<size_t>(needed);
const size_t boundedLength = unlimited ? outputLength : std::min(outputLength, limit);
std::vector<char> buffer(boundedLength + 1);
snprintf(buffer.data(), buffer.size(), format, args...);
return log(level, tag, std::string(buffer.data()), !unlimited && outputLength > limit);
const size_t boundedLength = std::min(outputLength, limit);
char buffer[TRACE_FORMATTED_BUFFER_LENGTH + 1] = {};
snprintf(buffer, boundedLength + 1, format, args...);
const size_t tagLimit = getMaxTagLength();
const size_t tagLen = boundedStrLen(tag, tagLimit + 1);
return logRaw(level, tag, tagLen, buffer, boundedLength, outputLength > limit);
}

static size_t boundedStrLen(const char *value, size_t maxLen);
size_t getMaxTagLength() const;
size_t getMaxMessageLength() const;
size_t getMaxFormattedLength() const;
TraceResult log(
TraceLevel level,
Expand Down
30 changes: 18 additions & 12 deletions src/TraceFlush.cpp
Original file line numberDiff line numberDiff line change
Expand Up@@ -17,8 +17,14 @@ void TraceImpl::performFlush() {
return;
}
callback = onFlush;
batch.logs = pendingLogs;
batch.createdAtUptimeMs = millis();
batch.logs.reserve(pendingLogs.size());
for (size_t i = 0; i < pendingLogs.size(); ++i) {
TraceRecord record;
if (pendingLogs.peek(i, record)) {
batch.logs.push_back(toPublicLog(record));
}
}
if (!batch.logs.empty()) {
maxSequence = batch.logs.back().sequence;
}
Expand DownExpand Up@@ -46,16 +52,10 @@ void TraceImpl::performFlush() {
if (flushResult == TraceFlushResult::Ok) {
nextFlushAttemptMs = 0;
if (maxSequence > 0) {
pendingLogs.erase(
std::remove_if(
pendingLogs.begin(),
pendingLogs.end(),
[maxSequence](const TraceLog &log) {
return log.sequence <= maxSequence;
}
),
pendingLogs.end()
);
TraceRecord record;
while (pendingLogs.peek(0, record) && record.sequence <= maxSequence) {
pendingLogs.pop(record);
}
}
flushSuccessCount++;
} else if (flushResult == TraceFlushResult::Retry) {
Expand All@@ -76,13 +76,19 @@ bool TraceImpl::shouldFlushNow() {
if (!lock) {
return false;
}
const uint64_t nowMs = millis();
if (
!pendingLogs.empty() && nextFlushAttemptMs > 0 && nowMs < nextFlushAttemptMs &&
!urgentFlushRequested
) {
return false;
}
if (flushRequested || urgentFlushRequested) {
return true;
}
if (pendingLogs.empty()) {
return false;
}
const uint64_t nowMs = millis();
if (nextFlushAttemptMs > 0) {
return nowMs >= nextFlushAttemptMs;
}
Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -2,7 +2,7 @@ name: CI

on:
push:
branches: [ main, master, 'feature/**' ]
branches: [ '**' ]
tags: ['v*']
pull_request:
workflow_dispatch:
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/lint.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -4,8 +4,7 @@ on:
pull_request:
push:
branches:
- main
- feature/**
- '**'

permissions:
contents: read
Expand Down
12 changes: 8 additions & 4 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -10,7 +10,7 @@ Trace helps you collect structured runtime logs in Arduino ESP32 projects with b

## Why use Trace?

* **Bounded memory** - recent history, realtime delivery, pending flush logs, and payload lengths have separate configured limits.
* **Bounded memory** - recent history, realtime delivery, pending flush logs, and payload lengths use fixed-capacity storage.
* **Structured output** - log records keep level, tag, message, formatted text, sequence, and uptime.
* **Task-side callbacks** - realtime observation and persistence callbacks run from the internal Trace task.
* **ESP32 task control** - configure byte stack size, priority, core affinity, and stack memory preference.
Expand DownExpand Up@@ -80,15 +80,18 @@ void loop() {
* `maxRecentLogs` controls queryable in-RAM history only.
* `maxRealtimeLogs` controls the realtime delivery queue used by `onLog()` and stream output.
* `maxPendingLogs` controls unsaved logs waiting for flush.
* Queue-count `0` disables that queue. Payload-cap `0` means unlimited.
* Queue-count `0` disables that queue. Payload-cap `0` uses the compiled maximum for that payload type.
* `setStream()` writes formatted realtime logs to any Arduino `Print` stream such as `Serial`, `Serial1`, `WiFiClient`, or a custom sink.
* Stream output uses ANSI colors by default. Callback, flush, and query `TraceLog::formatted` values stay plain text.
* After `Trace::init()`, accepted direct C-string log calls do not allocate on the internal enqueue path when all target queues have capacity and no output-boundary conversion is triggered.
* Query APIs, flush batch conversion, `onLog()` conversion, stream implementations, and user-created `std::string` values may still allocate.
* `onLog()` is for realtime observation; `onFlush()` is for persistence.
* Callbacks should avoid long blocking work and should not recursively call Trace logging methods.
* Trace does not own attached `Print` or `Tempo` instances. Keep them alive until `Trace::end()` completes.
* Detaching or replacing `Print` or `Tempo` while Trace is active does not synchronize already snapshotted worker use.
* Stack sizes are FreeRTOS byte sizes on ESP32 and must be at least 1024 bytes.
* `TraceStackType::Auto` prefers PSRAM task stacks when supported and falls back to internal RAM.
* `TraceStorageMemory::PreferPsram` opts recent and pending log buffers into PSRAM with internal fallback. Realtime delivery stays internal.

## Examples

Expand DownExpand Up@@ -149,16 +152,17 @@ For the full API, see [`docs/api.md`](docs/api.md).
| Platform | `espressif32` |
| Language | C++20 |
| Filesystem | none |
| PSRAM | Optional for task stacks when ESP-IDF support is available |
| PSRAM | Optional for task stacks and opt-in recent/pending log storage |
| Dependencies | `bblanchon/ArduinoJson >= 7.0.0` |
| Exceptions | Not used |
| Status | Early-stage `0.1.0` |
| Status | Release `0.2.0` |

## Configuration

```cpp
TraceConfig config;
config.stackSize = 4096;
config.storageMemory = TraceStorageMemory::Internal;
config.maxRecentLogs = 100;
config.maxRealtimeLogs = 100;
config.maxPendingLogs = 50;
Expand Down
4 changes: 3 additions & 1 deletion docs/api.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,7 +12,7 @@

`TraceFlushResult` values are `Ok`, `Failed`, and `Retry`.

`Ok` removes flushed pending logs. `Failed` is terminal for the current `flushAndWait()` call but retains pending logs. `Retry` retains pending logs, schedules another attempt using `TraceConfig::retryIntervalMs`, and keeps `flushAndWait()` waiting until `Ok`, `Failed`, or timeout.
`Ok` removes flushed pending logs. `Failed` is terminal for the current `flushAndWait()` call but retains pending logs. `Retry` retains pending logs, schedules another attempt using `TraceConfig::retryIntervalMs`, and keeps `flushAndWait()` waiting until `Ok`, `Failed`, or timeout. Normal flush requests do not bypass the retry deadline; urgent error and fatal flush requests may bypass it.

## Main methods

Expand DownExpand Up@@ -67,6 +67,8 @@ std::vector<TraceLog> getLastLogs(size_t count);
std::vector<TraceLog> getLogsByTag(const char *tag);
```

`TraceDiag` includes queue counts, drop and flush counters, task stack information, queue allocation byte counts, and PSRAM placement flags for recent, realtime, and pending queues.

## TraceLog

`TraceLog` stores `sequence`, `level`, `tag`, `message`, `formatted`, `timeText`, `uptimeMs`, and `truncated`.
Expand Down
34 changes: 30 additions & 4 deletions docs/configuration.md
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,14 @@
# Configuration

`TraceConfig` controls the internal task, log limits, flush timing, overflow behavior, JSON formatting, level filtering, and stream colors.
`TraceConfig` controls the internal task, log storage memory, log limits, flush timing, overflow behavior, JSON formatting, level filtering, and stream colors.

```cpp
TraceConfig config;
config.stackSize = 4096;
config.priority = 1;
config.coreId = tskNO_AFFINITY;
config.stackType = TraceStackType::Auto;
config.storageMemory = TraceStorageMemory::Internal;
config.maxRecentLogs = 100;
config.maxRealtimeLogs = 100;
config.maxPendingLogs = 50;
Expand DownExpand Up@@ -35,7 +36,21 @@ config.maxFormattedLength = 384;

`maxPendingLogs = 0` disables persistence buffering. Logs are still accepted for recent history and realtime delivery, but they are counted as dropped for persistence.

Queue-count `0` means disabled. Payload-cap `0` means unlimited.
Queue-count `0` means disabled. Payload-cap `0` means use the compiled maximum for that payload type.

## Storage memory

Trace stores recent, realtime, and pending logs in fixed-capacity ring buffers. Each enabled queue is allocated once during `init()`.

After `Trace::init()`, accepted direct C-string log calls do not allocate on the internal enqueue path when all target queues have capacity and no output-boundary conversion is triggered.

This guarantee is intentionally narrow. Query APIs allocate `std::vector<TraceLog>`, flush batch conversion and `onLog()` conversion can allocate because public `TraceLog` contains `std::string`, stream implementations may allocate internally, and user code may allocate before passing `std::string` values to Trace.

`TraceStorageMemory::Internal` is the deterministic default. Recent, realtime, and pending queues use internal-capable memory.

`TraceStorageMemory::PreferPsram` uses PSRAM for recent and pending queues when PSRAM is available, otherwise it falls back to internal-capable memory. The realtime queue remains internal.

`TraceStorageMemory::RequirePsram` requires recent and pending queues to allocate in PSRAM. `init()` returns `TraceStatus::OutOfMemory` if PSRAM is unavailable or allocation fails. The realtime queue remains internal.

## Payload limits

Expand All@@ -45,6 +60,17 @@ Queue-count `0` means disabled. Payload-cap `0` means unlimited.

`maxFormattedLength` applies to `printf`-style and JSON-formatted input before it becomes `TraceLog::message`.

Runtime payload limits are bounded by compile-time caps:

```cpp
TRACE_RECORD_MAX_TAG_LENGTH
TRACE_RECORD_MAX_MESSAGE_LENGTH
TRACE_FORMATTED_BUFFER_LENGTH
TRACE_TIME_TEXT_BUFFER_LENGTH
```

Setting a runtime limit above the compiled cap clamps to the compiled cap. Setting a runtime payload limit to `0` uses the compiled cap.

When Trace truncates a log, `TraceLog::truncated` is set. `TraceDiag::truncatedLogCount` increments once per log record, even if both tag and message were truncated.

## Flush triggers
Expand All@@ -56,7 +82,7 @@ Trace flushes pending logs when:
* `flushIntervalMs` elapses with pending logs.
* `flushOnError` is enabled and an `Error` or `Fatal` log is queued.

`retryIntervalMs` controls the delay after `TraceFlushResult::Retry`. Values smaller than the worker poll interval are clamped so retry cannot spin in a tight loop.
`retryIntervalMs` controls the delay after `TraceFlushResult::Retry`. Values smaller than the worker poll interval are clamped so retry cannot spin in a tight loop. Normal flush requests set `flushRequested` but do not clear or bypass the retry deadline. Urgent error and fatal flush requests may bypass the retry deadline.

## Flush results

Expand All@@ -74,7 +100,7 @@ Trace flushes pending logs when:

`BlockCaller` requests a flush and waits up to `blockCallerTimeoutMs` for pending space.

`FlushImmediately` requests a flush immediately and retries until space appears or `blockCallerTimeoutMs` expires.
`FlushImmediately` requests a flush immediately and queues the new record only if pending space becomes available before `blockCallerTimeoutMs` expires.

## Stack policy

Expand Down
2 changes: 1 addition & 1 deletion library.json
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
{
"name": "Trace",
"version": "0.1.0",
"version": "0.2.0",
"description": "Logging and diagnostics library for ESP32 devices with bounded buffers and task-side flushing.",
"keywords": [
"esp32",
Expand Down
2 changes: 1 addition & 1 deletion library.properties
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
name=Trace
version=0.1.0
version=0.2.0
author=zekageri
maintainer=zekageri
sentence=Logging and diagnostics library for ESP32 devices.
Expand Down
51 changes: 46 additions & 5 deletions src/Trace.h
Original file line numberDiff line numberDiff line change
Expand Up@@ -13,6 +13,22 @@
#include <freertos/FreeRTOS.h>
#include <freertos/task.h>

#ifndef TRACE_RECORD_MAX_TAG_LENGTH
#define TRACE_RECORD_MAX_TAG_LENGTH 32
#endif

#ifndef TRACE_RECORD_MAX_MESSAGE_LENGTH
#define TRACE_RECORD_MAX_MESSAGE_LENGTH 256
#endif

#ifndef TRACE_FORMATTED_BUFFER_LENGTH
#define TRACE_FORMATTED_BUFFER_LENGTH 384
#endif

#ifndef TRACE_TIME_TEXT_BUFFER_LENGTH
#define TRACE_TIME_TEXT_BUFFER_LENGTH 48
#endif

class Tempo;
struct TraceImpl;

Expand DownExpand Up@@ -43,6 +59,12 @@ enum class TraceStackType : uint8_t {
Psram,
};

enum class TraceStorageMemory : uint8_t {
Internal,
PreferPsram,
RequirePsram,
};

enum class TraceOverflowPolicy : uint8_t {
DropOldestPending,
DropNewest,
Expand DownExpand Up@@ -89,6 +111,7 @@ struct TraceConfig {
UBaseType_t priority = 1;
BaseType_t coreId = tskNO_AFFINITY;
TraceStackType stackType = TraceStackType::Auto;
TraceStorageMemory storageMemory = TraceStorageMemory::Internal;
size_t maxRecentLogs = 100;
size_t maxRealtimeLogs = 100;
size_t maxPendingLogs = 50;
Expand DownExpand Up@@ -146,6 +169,12 @@ struct TraceDiag {
size_t stackHighWaterMarkBytes = 0;
TraceStackType requestedStackType = TraceStackType::Auto;
TraceStackType actualStackType = TraceStackType::Internal;
size_t recentAllocatedBytes = 0;
size_t realtimeAllocatedBytes = 0;
size_t pendingAllocatedBytes = 0;
bool recentLogsInPsram = false;
bool realtimeLogsInPsram = false;
bool pendingLogsInPsram = false;
};

using TraceTimeFormatter = bool (*)(const Tempo &tempo, char *buffer, size_t bufferSize);
Expand DownExpand Up@@ -242,6 +271,14 @@ class Trace {

private:
TraceResult log(TraceLevel level, const char *tag, const std::string &message);
TraceResult logRaw(
TraceLevel level,
const char *tag,
size_t tagLen,
const char *message,
size_t messageLen,
bool alreadyTruncated
);
TraceResult logJson(TraceLevel level, const char *tag, const JsonDocument &doc);
TraceResult logVPrintf(TraceLevel level, const char *tag, const char *format, va_list args);

Expand All@@ -260,14 +297,18 @@ class Trace {
return TraceResult::failure(TraceStatus::InvalidArgument, "format failed");
}
const size_t limit = getMaxFormattedLength();
const bool unlimited = limit == 0;
const size_t outputLength = static_cast<size_t>(needed);
const size_t boundedLength = unlimited ? outputLength : std::min(outputLength, limit);
std::vector<char> buffer(boundedLength + 1);
snprintf(buffer.data(), buffer.size(), format, args...);
return log(level, tag, std::string(buffer.data()), !unlimited && outputLength > limit);
const size_t boundedLength = std::min(outputLength, limit);
char buffer[TRACE_FORMATTED_BUFFER_LENGTH + 1] = {};
snprintf(buffer, boundedLength + 1, format, args...);
const size_t tagLimit = getMaxTagLength();
const size_t tagLen = boundedStrLen(tag, tagLimit + 1);
return logRaw(level, tag, tagLen, buffer, boundedLength, outputLength > limit);
}

static size_t boundedStrLen(const char *value, size_t maxLen);
size_t getMaxTagLength() const;
size_t getMaxMessageLength() const;
size_t getMaxFormattedLength() const;
TraceResult log(
TraceLevel level,
Expand Down
30 changes: 18 additions & 12 deletions src/TraceFlush.cpp
Original file line numberDiff line numberDiff line change
Expand Up@@ -17,8 +17,14 @@ void TraceImpl::performFlush() {
return;
}
callback = onFlush;
batch.logs = pendingLogs;
batch.createdAtUptimeMs = millis();
batch.logs.reserve(pendingLogs.size());
for (size_t i = 0; i < pendingLogs.size(); ++i) {
TraceRecord record;
if (pendingLogs.peek(i, record)) {
batch.logs.push_back(toPublicLog(record));
}
}
if (!batch.logs.empty()) {
maxSequence = batch.logs.back().sequence;
}
Expand DownExpand Up@@ -46,16 +52,10 @@ void TraceImpl::performFlush() {
if (flushResult == TraceFlushResult::Ok) {
nextFlushAttemptMs = 0;
if (maxSequence > 0) {
pendingLogs.erase(
std::remove_if(
pendingLogs.begin(),
pendingLogs.end(),
[maxSequence](const TraceLog &log) {
return log.sequence <= maxSequence;
}
),
pendingLogs.end()
);
TraceRecord record;
while (pendingLogs.peek(0, record) && record.sequence <= maxSequence) {
pendingLogs.pop(record);
}
}
flushSuccessCount++;
} else if (flushResult == TraceFlushResult::Retry) {
Expand All@@ -76,13 +76,19 @@ bool TraceImpl::shouldFlushNow() {
if (!lock) {
return false;
}
const uint64_t nowMs = millis();
if (
!pendingLogs.empty() && nextFlushAttemptMs > 0 && nowMs < nextFlushAttemptMs &&
!urgentFlushRequested
) {
return false;
}
if (flushRequested || urgentFlushRequested) {
return true;
}
if (pendingLogs.empty()) {
return false;
}
const uint64_t nowMs = millis();
if (nextFlushAttemptMs > 0) {
return nowMs >= nextFlushAttemptMs;
}
Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -2,7 +2,7 @@ name: CI

on:
push:
branches: [ main, master, 'feature/**' ]
branches: [ '**' ]
tags: ['v*']
pull_request:
workflow_dispatch:
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/lint.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -4,8 +4,7 @@ on:
pull_request:
push:
branches:
- main
- feature/**
- '**'

permissions:
contents: read
Expand Down
12 changes: 8 additions & 4 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -10,7 +10,7 @@ Trace helps you collect structured runtime logs in Arduino ESP32 projects with b

## Why use Trace?

* **Bounded memory** - recent history, realtime delivery, pending flush logs, and payload lengths have separate configured limits.
* **Bounded memory** - recent history, realtime delivery, pending flush logs, and payload lengths use fixed-capacity storage.
* **Structured output** - log records keep level, tag, message, formatted text, sequence, and uptime.
* **Task-side callbacks** - realtime observation and persistence callbacks run from the internal Trace task.
* **ESP32 task control** - configure byte stack size, priority, core affinity, and stack memory preference.
Expand DownExpand Up@@ -80,15 +80,18 @@ void loop() {
* `maxRecentLogs` controls queryable in-RAM history only.
* `maxRealtimeLogs` controls the realtime delivery queue used by `onLog()` and stream output.
* `maxPendingLogs` controls unsaved logs waiting for flush.
* Queue-count `0` disables that queue. Payload-cap `0` means unlimited.
* Queue-count `0` disables that queue. Payload-cap `0` uses the compiled maximum for that payload type.
* `setStream()` writes formatted realtime logs to any Arduino `Print` stream such as `Serial`, `Serial1`, `WiFiClient`, or a custom sink.
* Stream output uses ANSI colors by default. Callback, flush, and query `TraceLog::formatted` values stay plain text.
* After `Trace::init()`, accepted direct C-string log calls do not allocate on the internal enqueue path when all target queues have capacity and no output-boundary conversion is triggered.
* Query APIs, flush batch conversion, `onLog()` conversion, stream implementations, and user-created `std::string` values may still allocate.
* `onLog()` is for realtime observation; `onFlush()` is for persistence.
* Callbacks should avoid long blocking work and should not recursively call Trace logging methods.
* Trace does not own attached `Print` or `Tempo` instances. Keep them alive until `Trace::end()` completes.
* Detaching or replacing `Print` or `Tempo` while Trace is active does not synchronize already snapshotted worker use.
* Stack sizes are FreeRTOS byte sizes on ESP32 and must be at least 1024 bytes.
* `TraceStackType::Auto` prefers PSRAM task stacks when supported and falls back to internal RAM.
* `TraceStorageMemory::PreferPsram` opts recent and pending log buffers into PSRAM with internal fallback. Realtime delivery stays internal.

## Examples

Expand DownExpand Up@@ -149,16 +152,17 @@ For the full API, see [`docs/api.md`](docs/api.md).
| Platform | `espressif32` |
| Language | C++20 |
| Filesystem | none |
| PSRAM | Optional for task stacks when ESP-IDF support is available |
| PSRAM | Optional for task stacks and opt-in recent/pending log storage |
| Dependencies | `bblanchon/ArduinoJson >= 7.0.0` |
| Exceptions | Not used |
| Status | Early-stage `0.1.0` |
| Status | Release `0.2.0` |

## Configuration

```cpp
TraceConfig config;
config.stackSize = 4096;
config.storageMemory = TraceStorageMemory::Internal;
config.maxRecentLogs = 100;
config.maxRealtimeLogs = 100;
config.maxPendingLogs = 50;
Expand Down
4 changes: 3 additions & 1 deletion docs/api.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,7 +12,7 @@

`TraceFlushResult` values are `Ok`, `Failed`, and `Retry`.

`Ok` removes flushed pending logs. `Failed` is terminal for the current `flushAndWait()` call but retains pending logs. `Retry` retains pending logs, schedules another attempt using `TraceConfig::retryIntervalMs`, and keeps `flushAndWait()` waiting until `Ok`, `Failed`, or timeout.
`Ok` removes flushed pending logs. `Failed` is terminal for the current `flushAndWait()` call but retains pending logs. `Retry` retains pending logs, schedules another attempt using `TraceConfig::retryIntervalMs`, and keeps `flushAndWait()` waiting until `Ok`, `Failed`, or timeout. Normal flush requests do not bypass the retry deadline; urgent error and fatal flush requests may bypass it.

## Main methods

Expand DownExpand Up@@ -67,6 +67,8 @@ std::vector<TraceLog> getLastLogs(size_t count);
std::vector<TraceLog> getLogsByTag(const char *tag);
```

`TraceDiag` includes queue counts, drop and flush counters, task stack information, queue allocation byte counts, and PSRAM placement flags for recent, realtime, and pending queues.

## TraceLog

`TraceLog` stores `sequence`, `level`, `tag`, `message`, `formatted`, `timeText`, `uptimeMs`, and `truncated`.
Expand Down
34 changes: 30 additions & 4 deletions docs/configuration.md
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,14 @@
# Configuration

`TraceConfig` controls the internal task, log limits, flush timing, overflow behavior, JSON formatting, level filtering, and stream colors.
`TraceConfig` controls the internal task, log storage memory, log limits, flush timing, overflow behavior, JSON formatting, level filtering, and stream colors.

```cpp
TraceConfig config;
config.stackSize = 4096;
config.priority = 1;
config.coreId = tskNO_AFFINITY;
config.stackType = TraceStackType::Auto;
config.storageMemory = TraceStorageMemory::Internal;
config.maxRecentLogs = 100;
config.maxRealtimeLogs = 100;
config.maxPendingLogs = 50;
Expand DownExpand Up@@ -35,7 +36,21 @@ config.maxFormattedLength = 384;

`maxPendingLogs = 0` disables persistence buffering. Logs are still accepted for recent history and realtime delivery, but they are counted as dropped for persistence.

Queue-count `0` means disabled. Payload-cap `0` means unlimited.
Queue-count `0` means disabled. Payload-cap `0` means use the compiled maximum for that payload type.

## Storage memory

Trace stores recent, realtime, and pending logs in fixed-capacity ring buffers. Each enabled queue is allocated once during `init()`.

After `Trace::init()`, accepted direct C-string log calls do not allocate on the internal enqueue path when all target queues have capacity and no output-boundary conversion is triggered.

This guarantee is intentionally narrow. Query APIs allocate `std::vector<TraceLog>`, flush batch conversion and `onLog()` conversion can allocate because public `TraceLog` contains `std::string`, stream implementations may allocate internally, and user code may allocate before passing `std::string` values to Trace.

`TraceStorageMemory::Internal` is the deterministic default. Recent, realtime, and pending queues use internal-capable memory.

`TraceStorageMemory::PreferPsram` uses PSRAM for recent and pending queues when PSRAM is available, otherwise it falls back to internal-capable memory. The realtime queue remains internal.

`TraceStorageMemory::RequirePsram` requires recent and pending queues to allocate in PSRAM. `init()` returns `TraceStatus::OutOfMemory` if PSRAM is unavailable or allocation fails. The realtime queue remains internal.

## Payload limits

Expand All@@ -45,6 +60,17 @@ Queue-count `0` means disabled. Payload-cap `0` means unlimited.

`maxFormattedLength` applies to `printf`-style and JSON-formatted input before it becomes `TraceLog::message`.

Runtime payload limits are bounded by compile-time caps:

```cpp
TRACE_RECORD_MAX_TAG_LENGTH
TRACE_RECORD_MAX_MESSAGE_LENGTH
TRACE_FORMATTED_BUFFER_LENGTH
TRACE_TIME_TEXT_BUFFER_LENGTH
```

Setting a runtime limit above the compiled cap clamps to the compiled cap. Setting a runtime payload limit to `0` uses the compiled cap.

When Trace truncates a log, `TraceLog::truncated` is set. `TraceDiag::truncatedLogCount` increments once per log record, even if both tag and message were truncated.

## Flush triggers
Expand All@@ -56,7 +82,7 @@ Trace flushes pending logs when:
* `flushIntervalMs` elapses with pending logs.
* `flushOnError` is enabled and an `Error` or `Fatal` log is queued.

`retryIntervalMs` controls the delay after `TraceFlushResult::Retry`. Values smaller than the worker poll interval are clamped so retry cannot spin in a tight loop.
`retryIntervalMs` controls the delay after `TraceFlushResult::Retry`. Values smaller than the worker poll interval are clamped so retry cannot spin in a tight loop. Normal flush requests set `flushRequested` but do not clear or bypass the retry deadline. Urgent error and fatal flush requests may bypass the retry deadline.

## Flush results

Expand All@@ -74,7 +100,7 @@ Trace flushes pending logs when:

`BlockCaller` requests a flush and waits up to `blockCallerTimeoutMs` for pending space.

`FlushImmediately` requests a flush immediately and retries until space appears or `blockCallerTimeoutMs` expires.
`FlushImmediately` requests a flush immediately and queues the new record only if pending space becomes available before `blockCallerTimeoutMs` expires.

## Stack policy

Expand Down
2 changes: 1 addition & 1 deletion library.json
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
{
"name": "Trace",
"version": "0.1.0",
"version": "0.2.0",
"description": "Logging and diagnostics library for ESP32 devices with bounded buffers and task-side flushing.",
"keywords": [
"esp32",
Expand Down
2 changes: 1 addition & 1 deletion library.properties
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
name=Trace
version=0.1.0
version=0.2.0
author=zekageri
maintainer=zekageri
sentence=Logging and diagnostics library for ESP32 devices.
Expand Down
51 changes: 46 additions & 5 deletions src/Trace.h
Original file line numberDiff line numberDiff line change
Expand Up@@ -13,6 +13,22 @@
#include <freertos/FreeRTOS.h>
#include <freertos/task.h>

#ifndef TRACE_RECORD_MAX_TAG_LENGTH
#define TRACE_RECORD_MAX_TAG_LENGTH 32
#endif

#ifndef TRACE_RECORD_MAX_MESSAGE_LENGTH
#define TRACE_RECORD_MAX_MESSAGE_LENGTH 256
#endif

#ifndef TRACE_FORMATTED_BUFFER_LENGTH
#define TRACE_FORMATTED_BUFFER_LENGTH 384
#endif

#ifndef TRACE_TIME_TEXT_BUFFER_LENGTH
#define TRACE_TIME_TEXT_BUFFER_LENGTH 48
#endif

class Tempo;
struct TraceImpl;

Expand DownExpand Up@@ -43,6 +59,12 @@ enum class TraceStackType : uint8_t {
Psram,
};

enum class TraceStorageMemory : uint8_t {
Internal,
PreferPsram,
RequirePsram,
};

enum class TraceOverflowPolicy : uint8_t {
DropOldestPending,
DropNewest,
Expand DownExpand Up@@ -89,6 +111,7 @@ struct TraceConfig {
UBaseType_t priority = 1;
BaseType_t coreId = tskNO_AFFINITY;
TraceStackType stackType = TraceStackType::Auto;
TraceStorageMemory storageMemory = TraceStorageMemory::Internal;
size_t maxRecentLogs = 100;
size_t maxRealtimeLogs = 100;
size_t maxPendingLogs = 50;
Expand DownExpand Up@@ -146,6 +169,12 @@ struct TraceDiag {
size_t stackHighWaterMarkBytes = 0;
TraceStackType requestedStackType = TraceStackType::Auto;
TraceStackType actualStackType = TraceStackType::Internal;
size_t recentAllocatedBytes = 0;
size_t realtimeAllocatedBytes = 0;
size_t pendingAllocatedBytes = 0;
bool recentLogsInPsram = false;
bool realtimeLogsInPsram = false;
bool pendingLogsInPsram = false;
};

using TraceTimeFormatter = bool (*)(const Tempo &tempo, char *buffer, size_t bufferSize);
Expand DownExpand Up@@ -242,6 +271,14 @@ class Trace {

private:
TraceResult log(TraceLevel level, const char *tag, const std::string &message);
TraceResult logRaw(
TraceLevel level,
const char *tag,
size_t tagLen,
const char *message,
size_t messageLen,
bool alreadyTruncated
);
TraceResult logJson(TraceLevel level, const char *tag, const JsonDocument &doc);
TraceResult logVPrintf(TraceLevel level, const char *tag, const char *format, va_list args);

Expand All@@ -260,14 +297,18 @@ class Trace {
return TraceResult::failure(TraceStatus::InvalidArgument, "format failed");
}
const size_t limit = getMaxFormattedLength();
const bool unlimited = limit == 0;
const size_t outputLength = static_cast<size_t>(needed);
const size_t boundedLength = unlimited ? outputLength : std::min(outputLength, limit);
std::vector<char> buffer(boundedLength + 1);
snprintf(buffer.data(), buffer.size(), format, args...);
return log(level, tag, std::string(buffer.data()), !unlimited && outputLength > limit);
const size_t boundedLength = std::min(outputLength, limit);
char buffer[TRACE_FORMATTED_BUFFER_LENGTH + 1] = {};
snprintf(buffer, boundedLength + 1, format, args...);
const size_t tagLimit = getMaxTagLength();
const size_t tagLen = boundedStrLen(tag, tagLimit + 1);
return logRaw(level, tag, tagLen, buffer, boundedLength, outputLength > limit);
}

static size_t boundedStrLen(const char *value, size_t maxLen);
size_t getMaxTagLength() const;
size_t getMaxMessageLength() const;
size_t getMaxFormattedLength() const;
TraceResult log(
TraceLevel level,
Expand Down
30 changes: 18 additions & 12 deletions src/TraceFlush.cpp
Original file line numberDiff line numberDiff line change
Expand Up@@ -17,8 +17,14 @@ void TraceImpl::performFlush() {
return;
}
callback = onFlush;
batch.logs = pendingLogs;
batch.createdAtUptimeMs = millis();
batch.logs.reserve(pendingLogs.size());
for (size_t i = 0; i < pendingLogs.size(); ++i) {
TraceRecord record;
if (pendingLogs.peek(i, record)) {
batch.logs.push_back(toPublicLog(record));
}
}
if (!batch.logs.empty()) {
maxSequence = batch.logs.back().sequence;
}
Expand DownExpand Up@@ -46,16 +52,10 @@ void TraceImpl::performFlush() {
if (flushResult == TraceFlushResult::Ok) {
nextFlushAttemptMs = 0;
if (maxSequence > 0) {
pendingLogs.erase(
std::remove_if(
pendingLogs.begin(),
pendingLogs.end(),
[maxSequence](const TraceLog &log) {
return log.sequence <= maxSequence;
}
),
pendingLogs.end()
);
TraceRecord record;
while (pendingLogs.peek(0, record) && record.sequence <= maxSequence) {
pendingLogs.pop(record);
}
}
flushSuccessCount++;
} else if (flushResult == TraceFlushResult::Retry) {
Expand All@@ -76,13 +76,19 @@ bool TraceImpl::shouldFlushNow() {
if (!lock) {
return false;
}
const uint64_t nowMs = millis();
if (
!pendingLogs.empty() && nextFlushAttemptMs > 0 && nowMs < nextFlushAttemptMs &&
!urgentFlushRequested
) {
return false;
}
if (flushRequested || urgentFlushRequested) {
return true;
}
if (pendingLogs.empty()) {
return false;
}
const uint64_t nowMs = millis();
if (nextFlushAttemptMs > 0) {
return nowMs >= nextFlushAttemptMs;
}
Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -2,7 +2,7 @@ name: CI

on:
push:
branches: [ main, master, 'feature/**' ]
branches: [ '**' ]
tags: ['v*']
pull_request:
workflow_dispatch:
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/lint.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -4,8 +4,7 @@ on:
pull_request:
push:
branches:
- main
- feature/**
- '**'

permissions:
contents: read
Expand Down
12 changes: 8 additions & 4 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -10,7 +10,7 @@ Trace helps you collect structured runtime logs in Arduino ESP32 projects with b

## Why use Trace?

* **Bounded memory** - recent history, realtime delivery, pending flush logs, and payload lengths have separate configured limits.
* **Bounded memory** - recent history, realtime delivery, pending flush logs, and payload lengths use fixed-capacity storage.
* **Structured output** - log records keep level, tag, message, formatted text, sequence, and uptime.
* **Task-side callbacks** - realtime observation and persistence callbacks run from the internal Trace task.
* **ESP32 task control** - configure byte stack size, priority, core affinity, and stack memory preference.
Expand DownExpand Up@@ -80,15 +80,18 @@ void loop() {
* `maxRecentLogs` controls queryable in-RAM history only.
* `maxRealtimeLogs` controls the realtime delivery queue used by `onLog()` and stream output.
* `maxPendingLogs` controls unsaved logs waiting for flush.
* Queue-count `0` disables that queue. Payload-cap `0` means unlimited.
* Queue-count `0` disables that queue. Payload-cap `0` uses the compiled maximum for that payload type.
* `setStream()` writes formatted realtime logs to any Arduino `Print` stream such as `Serial`, `Serial1`, `WiFiClient`, or a custom sink.
* Stream output uses ANSI colors by default. Callback, flush, and query `TraceLog::formatted` values stay plain text.
* After `Trace::init()`, accepted direct C-string log calls do not allocate on the internal enqueue path when all target queues have capacity and no output-boundary conversion is triggered.
* Query APIs, flush batch conversion, `onLog()` conversion, stream implementations, and user-created `std::string` values may still allocate.
* `onLog()` is for realtime observation; `onFlush()` is for persistence.
* Callbacks should avoid long blocking work and should not recursively call Trace logging methods.
* Trace does not own attached `Print` or `Tempo` instances. Keep them alive until `Trace::end()` completes.
* Detaching or replacing `Print` or `Tempo` while Trace is active does not synchronize already snapshotted worker use.
* Stack sizes are FreeRTOS byte sizes on ESP32 and must be at least 1024 bytes.
* `TraceStackType::Auto` prefers PSRAM task stacks when supported and falls back to internal RAM.
* `TraceStorageMemory::PreferPsram` opts recent and pending log buffers into PSRAM with internal fallback. Realtime delivery stays internal.

## Examples

Expand DownExpand Up@@ -149,16 +152,17 @@ For the full API, see [`docs/api.md`](docs/api.md).
| Platform | `espressif32` |
| Language | C++20 |
| Filesystem | none |
| PSRAM | Optional for task stacks when ESP-IDF support is available |
| PSRAM | Optional for task stacks and opt-in recent/pending log storage |
| Dependencies | `bblanchon/ArduinoJson >= 7.0.0` |
| Exceptions | Not used |
| Status | Early-stage `0.1.0` |
| Status | Release `0.2.0` |

## Configuration

```cpp
TraceConfig config;
config.stackSize = 4096;
config.storageMemory = TraceStorageMemory::Internal;
config.maxRecentLogs = 100;
config.maxRealtimeLogs = 100;
config.maxPendingLogs = 50;
Expand Down
4 changes: 3 additions & 1 deletion docs/api.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,7 +12,7 @@

`TraceFlushResult` values are `Ok`, `Failed`, and `Retry`.

`Ok` removes flushed pending logs. `Failed` is terminal for the current `flushAndWait()` call but retains pending logs. `Retry` retains pending logs, schedules another attempt using `TraceConfig::retryIntervalMs`, and keeps `flushAndWait()` waiting until `Ok`, `Failed`, or timeout.
`Ok` removes flushed pending logs. `Failed` is terminal for the current `flushAndWait()` call but retains pending logs. `Retry` retains pending logs, schedules another attempt using `TraceConfig::retryIntervalMs`, and keeps `flushAndWait()` waiting until `Ok`, `Failed`, or timeout. Normal flush requests do not bypass the retry deadline; urgent error and fatal flush requests may bypass it.

## Main methods

Expand DownExpand Up@@ -67,6 +67,8 @@ std::vector<TraceLog> getLastLogs(size_t count);
std::vector<TraceLog> getLogsByTag(const char *tag);
```

`TraceDiag` includes queue counts, drop and flush counters, task stack information, queue allocation byte counts, and PSRAM placement flags for recent, realtime, and pending queues.

## TraceLog

`TraceLog` stores `sequence`, `level`, `tag`, `message`, `formatted`, `timeText`, `uptimeMs`, and `truncated`.
Expand Down
34 changes: 30 additions & 4 deletions docs/configuration.md
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,14 @@
# Configuration

`TraceConfig` controls the internal task, log limits, flush timing, overflow behavior, JSON formatting, level filtering, and stream colors.
`TraceConfig` controls the internal task, log storage memory, log limits, flush timing, overflow behavior, JSON formatting, level filtering, and stream colors.

```cpp
TraceConfig config;
config.stackSize = 4096;
config.priority = 1;
config.coreId = tskNO_AFFINITY;
config.stackType = TraceStackType::Auto;
config.storageMemory = TraceStorageMemory::Internal;
config.maxRecentLogs = 100;
config.maxRealtimeLogs = 100;
config.maxPendingLogs = 50;
Expand DownExpand Up@@ -35,7 +36,21 @@ config.maxFormattedLength = 384;

`maxPendingLogs = 0` disables persistence buffering. Logs are still accepted for recent history and realtime delivery, but they are counted as dropped for persistence.

Queue-count `0` means disabled. Payload-cap `0` means unlimited.
Queue-count `0` means disabled. Payload-cap `0` means use the compiled maximum for that payload type.

## Storage memory

Trace stores recent, realtime, and pending logs in fixed-capacity ring buffers. Each enabled queue is allocated once during `init()`.

After `Trace::init()`, accepted direct C-string log calls do not allocate on the internal enqueue path when all target queues have capacity and no output-boundary conversion is triggered.

This guarantee is intentionally narrow. Query APIs allocate `std::vector<TraceLog>`, flush batch conversion and `onLog()` conversion can allocate because public `TraceLog` contains `std::string`, stream implementations may allocate internally, and user code may allocate before passing `std::string` values to Trace.

`TraceStorageMemory::Internal` is the deterministic default. Recent, realtime, and pending queues use internal-capable memory.

`TraceStorageMemory::PreferPsram` uses PSRAM for recent and pending queues when PSRAM is available, otherwise it falls back to internal-capable memory. The realtime queue remains internal.

`TraceStorageMemory::RequirePsram` requires recent and pending queues to allocate in PSRAM. `init()` returns `TraceStatus::OutOfMemory` if PSRAM is unavailable or allocation fails. The realtime queue remains internal.

## Payload limits

Expand All@@ -45,6 +60,17 @@ Queue-count `0` means disabled. Payload-cap `0` means unlimited.

`maxFormattedLength` applies to `printf`-style and JSON-formatted input before it becomes `TraceLog::message`.

Runtime payload limits are bounded by compile-time caps:

```cpp
TRACE_RECORD_MAX_TAG_LENGTH
TRACE_RECORD_MAX_MESSAGE_LENGTH
TRACE_FORMATTED_BUFFER_LENGTH
TRACE_TIME_TEXT_BUFFER_LENGTH
```

Setting a runtime limit above the compiled cap clamps to the compiled cap. Setting a runtime payload limit to `0` uses the compiled cap.

When Trace truncates a log, `TraceLog::truncated` is set. `TraceDiag::truncatedLogCount` increments once per log record, even if both tag and message were truncated.

## Flush triggers
Expand All@@ -56,7 +82,7 @@ Trace flushes pending logs when:
* `flushIntervalMs` elapses with pending logs.
* `flushOnError` is enabled and an `Error` or `Fatal` log is queued.

`retryIntervalMs` controls the delay after `TraceFlushResult::Retry`. Values smaller than the worker poll interval are clamped so retry cannot spin in a tight loop.
`retryIntervalMs` controls the delay after `TraceFlushResult::Retry`. Values smaller than the worker poll interval are clamped so retry cannot spin in a tight loop. Normal flush requests set `flushRequested` but do not clear or bypass the retry deadline. Urgent error and fatal flush requests may bypass the retry deadline.

## Flush results

Expand All@@ -74,7 +100,7 @@ Trace flushes pending logs when:

`BlockCaller` requests a flush and waits up to `blockCallerTimeoutMs` for pending space.

`FlushImmediately` requests a flush immediately and retries until space appears or `blockCallerTimeoutMs` expires.
`FlushImmediately` requests a flush immediately and queues the new record only if pending space becomes available before `blockCallerTimeoutMs` expires.

## Stack policy

Expand Down
2 changes: 1 addition & 1 deletion library.json
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
{
"name": "Trace",
"version": "0.1.0",
"version": "0.2.0",
"description": "Logging and diagnostics library for ESP32 devices with bounded buffers and task-side flushing.",
"keywords": [
"esp32",
Expand Down
2 changes: 1 addition & 1 deletion library.properties
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
name=Trace
version=0.1.0
version=0.2.0
author=zekageri
maintainer=zekageri
sentence=Logging and diagnostics library for ESP32 devices.
Expand Down
51 changes: 46 additions & 5 deletions src/Trace.h
Original file line numberDiff line numberDiff line change
Expand Up@@ -13,6 +13,22 @@
#include <freertos/FreeRTOS.h>
#include <freertos/task.h>

#ifndef TRACE_RECORD_MAX_TAG_LENGTH
#define TRACE_RECORD_MAX_TAG_LENGTH 32
#endif

#ifndef TRACE_RECORD_MAX_MESSAGE_LENGTH
#define TRACE_RECORD_MAX_MESSAGE_LENGTH 256
#endif

#ifndef TRACE_FORMATTED_BUFFER_LENGTH
#define TRACE_FORMATTED_BUFFER_LENGTH 384
#endif

#ifndef TRACE_TIME_TEXT_BUFFER_LENGTH
#define TRACE_TIME_TEXT_BUFFER_LENGTH 48
#endif

class Tempo;
struct TraceImpl;

Expand DownExpand Up@@ -43,6 +59,12 @@ enum class TraceStackType : uint8_t {
Psram,
};

enum class TraceStorageMemory : uint8_t {
Internal,
PreferPsram,
RequirePsram,
};

enum class TraceOverflowPolicy : uint8_t {
DropOldestPending,
DropNewest,
Expand DownExpand Up@@ -89,6 +111,7 @@ struct TraceConfig {
UBaseType_t priority = 1;
BaseType_t coreId = tskNO_AFFINITY;
TraceStackType stackType = TraceStackType::Auto;
TraceStorageMemory storageMemory = TraceStorageMemory::Internal;
size_t maxRecentLogs = 100;
size_t maxRealtimeLogs = 100;
size_t maxPendingLogs = 50;
Expand DownExpand Up@@ -146,6 +169,12 @@ struct TraceDiag {
size_t stackHighWaterMarkBytes = 0;
TraceStackType requestedStackType = TraceStackType::Auto;
TraceStackType actualStackType = TraceStackType::Internal;
size_t recentAllocatedBytes = 0;
size_t realtimeAllocatedBytes = 0;
size_t pendingAllocatedBytes = 0;
bool recentLogsInPsram = false;
bool realtimeLogsInPsram = false;
bool pendingLogsInPsram = false;
};

using TraceTimeFormatter = bool (*)(const Tempo &tempo, char *buffer, size_t bufferSize);
Expand DownExpand Up@@ -242,6 +271,14 @@ class Trace {

private:
TraceResult log(TraceLevel level, const char *tag, const std::string &message);
TraceResult logRaw(
TraceLevel level,
const char *tag,
size_t tagLen,
const char *message,
size_t messageLen,
bool alreadyTruncated
);
TraceResult logJson(TraceLevel level, const char *tag, const JsonDocument &doc);
TraceResult logVPrintf(TraceLevel level, const char *tag, const char *format, va_list args);

Expand All@@ -260,14 +297,18 @@ class Trace {
return TraceResult::failure(TraceStatus::InvalidArgument, "format failed");
}
const size_t limit = getMaxFormattedLength();
const bool unlimited = limit == 0;
const size_t outputLength = static_cast<size_t>(needed);
const size_t boundedLength = unlimited ? outputLength : std::min(outputLength, limit);
std::vector<char> buffer(boundedLength + 1);
snprintf(buffer.data(), buffer.size(), format, args...);
return log(level, tag, std::string(buffer.data()), !unlimited && outputLength > limit);
const size_t boundedLength = std::min(outputLength, limit);
char buffer[TRACE_FORMATTED_BUFFER_LENGTH + 1] = {};
snprintf(buffer, boundedLength + 1, format, args...);
const size_t tagLimit = getMaxTagLength();
const size_t tagLen = boundedStrLen(tag, tagLimit + 1);
return logRaw(level, tag, tagLen, buffer, boundedLength, outputLength > limit);
}

static size_t boundedStrLen(const char *value, size_t maxLen);
size_t getMaxTagLength() const;
size_t getMaxMessageLength() const;
size_t getMaxFormattedLength() const;
TraceResult log(
TraceLevel level,
Expand Down
30 changes: 18 additions & 12 deletions src/TraceFlush.cpp
Original file line numberDiff line numberDiff line change
Expand Up@@ -17,8 +17,14 @@ void TraceImpl::performFlush() {
return;
}
callback = onFlush;
batch.logs = pendingLogs;
batch.createdAtUptimeMs = millis();
batch.logs.reserve(pendingLogs.size());
for (size_t i = 0; i < pendingLogs.size(); ++i) {
TraceRecord record;
if (pendingLogs.peek(i, record)) {
batch.logs.push_back(toPublicLog(record));
}
}
if (!batch.logs.empty()) {
maxSequence = batch.logs.back().sequence;
}
Expand DownExpand Up@@ -46,16 +52,10 @@ void TraceImpl::performFlush() {
if (flushResult == TraceFlushResult::Ok) {
nextFlushAttemptMs = 0;
if (maxSequence > 0) {
pendingLogs.erase(
std::remove_if(
pendingLogs.begin(),
pendingLogs.end(),
[maxSequence](const TraceLog &log) {
return log.sequence <= maxSequence;
}
),
pendingLogs.end()
);
TraceRecord record;
while (pendingLogs.peek(0, record) && record.sequence <= maxSequence) {
pendingLogs.pop(record);
}
}
flushSuccessCount++;
} else if (flushResult == TraceFlushResult::Retry) {
Expand All@@ -76,13 +76,19 @@ bool TraceImpl::shouldFlushNow() {
if (!lock) {
return false;
}
const uint64_t nowMs = millis();
if (
!pendingLogs.empty() && nextFlushAttemptMs > 0 && nowMs < nextFlushAttemptMs &&
!urgentFlushRequested
) {
return false;
}
if (flushRequested || urgentFlushRequested) {
return true;
}
if (pendingLogs.empty()) {
return false;
}
const uint64_t nowMs = millis();
if (nextFlushAttemptMs > 0) {
return nowMs >= nextFlushAttemptMs;
}
Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -2,7 +2,7 @@ name: CI

on:
push:
branches: [ main, master, 'feature/**' ]
branches: [ '**' ]
tags: ['v*']
pull_request:
workflow_dispatch:
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/lint.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -4,8 +4,7 @@ on:
pull_request:
push:
branches:
- main
- feature/**
- '**'

permissions:
contents: read
Expand Down
12 changes: 8 additions & 4 deletions README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -10,7 +10,7 @@ Trace helps you collect structured runtime logs in Arduino ESP32 projects with b

## Why use Trace?

* **Bounded memory** - recent history, realtime delivery, pending flush logs, and payload lengths have separate configured limits.
* **Bounded memory** - recent history, realtime delivery, pending flush logs, and payload lengths use fixed-capacity storage.
* **Structured output** - log records keep level, tag, message, formatted text, sequence, and uptime.
* **Task-side callbacks** - realtime observation and persistence callbacks run from the internal Trace task.
* **ESP32 task control** - configure byte stack size, priority, core affinity, and stack memory preference.
Expand DownExpand Up@@ -80,15 +80,18 @@ void loop() {
* `maxRecentLogs` controls queryable in-RAM history only.
* `maxRealtimeLogs` controls the realtime delivery queue used by `onLog()` and stream output.
* `maxPendingLogs` controls unsaved logs waiting for flush.
* Queue-count `0` disables that queue. Payload-cap `0` means unlimited.
* Queue-count `0` disables that queue. Payload-cap `0` uses the compiled maximum for that payload type.
* `setStream()` writes formatted realtime logs to any Arduino `Print` stream such as `Serial`, `Serial1`, `WiFiClient`, or a custom sink.
* Stream output uses ANSI colors by default. Callback, flush, and query `TraceLog::formatted` values stay plain text.
* After `Trace::init()`, accepted direct C-string log calls do not allocate on the internal enqueue path when all target queues have capacity and no output-boundary conversion is triggered.
* Query APIs, flush batch conversion, `onLog()` conversion, stream implementations, and user-created `std::string` values may still allocate.
* `onLog()` is for realtime observation; `onFlush()` is for persistence.
* Callbacks should avoid long blocking work and should not recursively call Trace logging methods.
* Trace does not own attached `Print` or `Tempo` instances. Keep them alive until `Trace::end()` completes.
* Detaching or replacing `Print` or `Tempo` while Trace is active does not synchronize already snapshotted worker use.
* Stack sizes are FreeRTOS byte sizes on ESP32 and must be at least 1024 bytes.
* `TraceStackType::Auto` prefers PSRAM task stacks when supported and falls back to internal RAM.
* `TraceStorageMemory::PreferPsram` opts recent and pending log buffers into PSRAM with internal fallback. Realtime delivery stays internal.

## Examples

Expand DownExpand Up@@ -149,16 +152,17 @@ For the full API, see [`docs/api.md`](docs/api.md).
| Platform | `espressif32` |
| Language | C++20 |
| Filesystem | none |
| PSRAM | Optional for task stacks when ESP-IDF support is available |
| PSRAM | Optional for task stacks and opt-in recent/pending log storage |
| Dependencies | `bblanchon/ArduinoJson >= 7.0.0` |
| Exceptions | Not used |
| Status | Early-stage `0.1.0` |
| Status | Release `0.2.0` |

## Configuration

```cpp
TraceConfig config;
config.stackSize = 4096;
config.storageMemory = TraceStorageMemory::Internal;
config.maxRecentLogs = 100;
config.maxRealtimeLogs = 100;
config.maxPendingLogs = 50;
Expand Down
4 changes: 3 additions & 1 deletion docs/api.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,7 +12,7 @@

`TraceFlushResult` values are `Ok`, `Failed`, and `Retry`.

`Ok` removes flushed pending logs. `Failed` is terminal for the current `flushAndWait()` call but retains pending logs. `Retry` retains pending logs, schedules another attempt using `TraceConfig::retryIntervalMs`, and keeps `flushAndWait()` waiting until `Ok`, `Failed`, or timeout.
`Ok` removes flushed pending logs. `Failed` is terminal for the current `flushAndWait()` call but retains pending logs. `Retry` retains pending logs, schedules another attempt using `TraceConfig::retryIntervalMs`, and keeps `flushAndWait()` waiting until `Ok`, `Failed`, or timeout. Normal flush requests do not bypass the retry deadline; urgent error and fatal flush requests may bypass it.

## Main methods

Expand DownExpand Up@@ -67,6 +67,8 @@ std::vector<TraceLog> getLastLogs(size_t count);
std::vector<TraceLog> getLogsByTag(const char *tag);
```

`TraceDiag` includes queue counts, drop and flush counters, task stack information, queue allocation byte counts, and PSRAM placement flags for recent, realtime, and pending queues.

## TraceLog

`TraceLog` stores `sequence`, `level`, `tag`, `message`, `formatted`, `timeText`, `uptimeMs`, and `truncated`.
Expand Down
34 changes: 30 additions & 4 deletions docs/configuration.md
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,14 @@
# Configuration

`TraceConfig` controls the internal task, log limits, flush timing, overflow behavior, JSON formatting, level filtering, and stream colors.
`TraceConfig` controls the internal task, log storage memory, log limits, flush timing, overflow behavior, JSON formatting, level filtering, and stream colors.

```cpp
TraceConfig config;
config.stackSize = 4096;
config.priority = 1;
config.coreId = tskNO_AFFINITY;
config.stackType = TraceStackType::Auto;
config.storageMemory = TraceStorageMemory::Internal;
config.maxRecentLogs = 100;
config.maxRealtimeLogs = 100;
config.maxPendingLogs = 50;
Expand DownExpand Up@@ -35,7 +36,21 @@ config.maxFormattedLength = 384;

`maxPendingLogs = 0` disables persistence buffering. Logs are still accepted for recent history and realtime delivery, but they are counted as dropped for persistence.

Queue-count `0` means disabled. Payload-cap `0` means unlimited.
Queue-count `0` means disabled. Payload-cap `0` means use the compiled maximum for that payload type.

## Storage memory

Trace stores recent, realtime, and pending logs in fixed-capacity ring buffers. Each enabled queue is allocated once during `init()`.

After `Trace::init()`, accepted direct C-string log calls do not allocate on the internal enqueue path when all target queues have capacity and no output-boundary conversion is triggered.

This guarantee is intentionally narrow. Query APIs allocate `std::vector<TraceLog>`, flush batch conversion and `onLog()` conversion can allocate because public `TraceLog` contains `std::string`, stream implementations may allocate internally, and user code may allocate before passing `std::string` values to Trace.

`TraceStorageMemory::Internal` is the deterministic default. Recent, realtime, and pending queues use internal-capable memory.

`TraceStorageMemory::PreferPsram` uses PSRAM for recent and pending queues when PSRAM is available, otherwise it falls back to internal-capable memory. The realtime queue remains internal.

`TraceStorageMemory::RequirePsram` requires recent and pending queues to allocate in PSRAM. `init()` returns `TraceStatus::OutOfMemory` if PSRAM is unavailable or allocation fails. The realtime queue remains internal.

## Payload limits

Expand All@@ -45,6 +60,17 @@ Queue-count `0` means disabled. Payload-cap `0` means unlimited.

`maxFormattedLength` applies to `printf`-style and JSON-formatted input before it becomes `TraceLog::message`.

Runtime payload limits are bounded by compile-time caps:

```cpp
TRACE_RECORD_MAX_TAG_LENGTH
TRACE_RECORD_MAX_MESSAGE_LENGTH
TRACE_FORMATTED_BUFFER_LENGTH
TRACE_TIME_TEXT_BUFFER_LENGTH
```

Setting a runtime limit above the compiled cap clamps to the compiled cap. Setting a runtime payload limit to `0` uses the compiled cap.

When Trace truncates a log, `TraceLog::truncated` is set. `TraceDiag::truncatedLogCount` increments once per log record, even if both tag and message were truncated.

## Flush triggers
Expand All@@ -56,7 +82,7 @@ Trace flushes pending logs when:
* `flushIntervalMs` elapses with pending logs.
* `flushOnError` is enabled and an `Error` or `Fatal` log is queued.

`retryIntervalMs` controls the delay after `TraceFlushResult::Retry`. Values smaller than the worker poll interval are clamped so retry cannot spin in a tight loop.
`retryIntervalMs` controls the delay after `TraceFlushResult::Retry`. Values smaller than the worker poll interval are clamped so retry cannot spin in a tight loop. Normal flush requests set `flushRequested` but do not clear or bypass the retry deadline. Urgent error and fatal flush requests may bypass the retry deadline.

## Flush results

Expand All@@ -74,7 +100,7 @@ Trace flushes pending logs when:

`BlockCaller` requests a flush and waits up to `blockCallerTimeoutMs` for pending space.

`FlushImmediately` requests a flush immediately and retries until space appears or `blockCallerTimeoutMs` expires.
`FlushImmediately` requests a flush immediately and queues the new record only if pending space becomes available before `blockCallerTimeoutMs` expires.

## Stack policy

Expand Down
2 changes: 1 addition & 1 deletion library.json
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
{
"name": "Trace",
"version": "0.1.0",
"version": "0.2.0",
"description": "Logging and diagnostics library for ESP32 devices with bounded buffers and task-side flushing.",
"keywords": [
"esp32",
Expand Down
2 changes: 1 addition & 1 deletion library.properties
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
name=Trace
version=0.1.0
version=0.2.0
author=zekageri
maintainer=zekageri
sentence=Logging and diagnostics library for ESP32 devices.
Expand Down
51 changes: 46 additions & 5 deletions src/Trace.h
Original file line numberDiff line numberDiff line change
Expand Up@@ -13,6 +13,22 @@
#include <freertos/FreeRTOS.h>
#include <freertos/task.h>

#ifndef TRACE_RECORD_MAX_TAG_LENGTH
#define TRACE_RECORD_MAX_TAG_LENGTH 32
#endif

#ifndef TRACE_RECORD_MAX_MESSAGE_LENGTH
#define TRACE_RECORD_MAX_MESSAGE_LENGTH 256
#endif

#ifndef TRACE_FORMATTED_BUFFER_LENGTH
#define TRACE_FORMATTED_BUFFER_LENGTH 384
#endif

#ifndef TRACE_TIME_TEXT_BUFFER_LENGTH
#define TRACE_TIME_TEXT_BUFFER_LENGTH 48
#endif

class Tempo;
struct TraceImpl;

Expand DownExpand Up@@ -43,6 +59,12 @@ enum class TraceStackType : uint8_t {
Psram,
};

enum class TraceStorageMemory : uint8_t {
Internal,
PreferPsram,
RequirePsram,
};

enum class TraceOverflowPolicy : uint8_t {
DropOldestPending,
DropNewest,
Expand DownExpand Up@@ -89,6 +111,7 @@ struct TraceConfig {
UBaseType_t priority = 1;
BaseType_t coreId = tskNO_AFFINITY;
TraceStackType stackType = TraceStackType::Auto;
TraceStorageMemory storageMemory = TraceStorageMemory::Internal;
size_t maxRecentLogs = 100;
size_t maxRealtimeLogs = 100;
size_t maxPendingLogs = 50;
Expand DownExpand Up@@ -146,6 +169,12 @@ struct TraceDiag {
size_t stackHighWaterMarkBytes = 0;
TraceStackType requestedStackType = TraceStackType::Auto;
TraceStackType actualStackType = TraceStackType::Internal;
size_t recentAllocatedBytes = 0;
size_t realtimeAllocatedBytes = 0;
size_t pendingAllocatedBytes = 0;
bool recentLogsInPsram = false;
bool realtimeLogsInPsram = false;
bool pendingLogsInPsram = false;
};

using TraceTimeFormatter = bool (*)(const Tempo &tempo, char *buffer, size_t bufferSize);
Expand DownExpand Up@@ -242,6 +271,14 @@ class Trace {

private:
TraceResult log(TraceLevel level, const char *tag, const std::string &message);
TraceResult logRaw(
TraceLevel level,
const char *tag,
size_t tagLen,
const char *message,
size_t messageLen,
bool alreadyTruncated
);
TraceResult logJson(TraceLevel level, const char *tag, const JsonDocument &doc);
TraceResult logVPrintf(TraceLevel level, const char *tag, const char *format, va_list args);

Expand All@@ -260,14 +297,18 @@ class Trace {
return TraceResult::failure(TraceStatus::InvalidArgument, "format failed");
}
const size_t limit = getMaxFormattedLength();
const bool unlimited = limit == 0;
const size_t outputLength = static_cast<size_t>(needed);
const size_t boundedLength = unlimited ? outputLength : std::min(outputLength, limit);
std::vector<char> buffer(boundedLength + 1);
snprintf(buffer.data(), buffer.size(), format, args...);
return log(level, tag, std::string(buffer.data()), !unlimited && outputLength > limit);
const size_t boundedLength = std::min(outputLength, limit);
char buffer[TRACE_FORMATTED_BUFFER_LENGTH + 1] = {};
snprintf(buffer, boundedLength + 1, format, args...);
const size_t tagLimit = getMaxTagLength();
const size_t tagLen = boundedStrLen(tag, tagLimit + 1);
return logRaw(level, tag, tagLen, buffer, boundedLength, outputLength > limit);
}

static size_t boundedStrLen(const char *value, size_t maxLen);
size_t getMaxTagLength() const;
size_t getMaxMessageLength() const;
size_t getMaxFormattedLength() const;
TraceResult log(
TraceLevel level,
Expand Down
30 changes: 18 additions & 12 deletions src/TraceFlush.cpp
Original file line numberDiff line numberDiff line change
Expand Up@@ -17,8 +17,14 @@ void TraceImpl::performFlush() {
return;
}
callback = onFlush;
batch.logs = pendingLogs;
batch.createdAtUptimeMs = millis();
batch.logs.reserve(pendingLogs.size());
for (size_t i = 0; i < pendingLogs.size(); ++i) {
TraceRecord record;
if (pendingLogs.peek(i, record)) {
batch.logs.push_back(toPublicLog(record));
}
}
if (!batch.logs.empty()) {
maxSequence = batch.logs.back().sequence;
}
Expand DownExpand Up@@ -46,16 +52,10 @@ void TraceImpl::performFlush() {
if (flushResult == TraceFlushResult::Ok) {
nextFlushAttemptMs = 0;
if (maxSequence > 0) {
pendingLogs.erase(
std::remove_if(
pendingLogs.begin(),
pendingLogs.end(),
[maxSequence](const TraceLog &log) {
return log.sequence <= maxSequence;
}
),
pendingLogs.end()
);
TraceRecord record;
while (pendingLogs.peek(0, record) && record.sequence <= maxSequence) {
pendingLogs.pop(record);
}
}
flushSuccessCount++;
} else if (flushResult == TraceFlushResult::Retry) {
Expand All@@ -76,13 +76,19 @@ bool TraceImpl::shouldFlushNow() {
if (!lock) {
return false;
}
const uint64_t nowMs = millis();
if (
!pendingLogs.empty() && nextFlushAttemptMs > 0 && nowMs < nextFlushAttemptMs &&
!urgentFlushRequested
) {
return false;
}
if (flushRequested || urgentFlushRequested) {
return true;
}
if (pendingLogs.empty()) {
return false;
}
const uint64_t nowMs = millis();
if (nextFlushAttemptMs > 0) {
return nowMs >= nextFlushAttemptMs;
}
Expand Down
Loading
Loading