Skip to content

feat(#44): skip feature detection while tracking (part B) - #54

Merged
kalwalt merged 1 commit into
webarkit:devfrom
kalwalt:feat/detection-guard-44b
Jun 19, 2026
Merged

feat(#44): skip feature detection while tracking (part B)#54
kalwalt merged 1 commit into
webarkit:devfrom
kalwalt:feat/detection-guard-44b

Conversation

@kalwalt

@kalwalt kalwalt commented Jun 19, 2026

Copy link
Copy Markdown
Member

What

Part B of #44. Guard the per-frame feature detection (extractFeatures + descriptor matching via MatchFeatures) with if (!_isTracking). Once optical flow holds a lock, detection is skipped and the pose is maintained by optical flow + template matching. When the marker is lost (_isTracking cleared in runOpticalFlow / RunTemplateMatching), detection resumes on the next frame to re-acquire.

Why

Detection is the dominant per-frame cost. Running it every frame even while a lock is held is wasteful — optical flow already produces the pose. Part A (#52) cut the cost of each detection; part B removes most detections entirely.

Measured (Teblid webcam example, 640×480): ~10–15 fps detecting every frame → ~45 fps while tracking (≈3×). Re-acquisition latency is one frame after loss.

Why !_isTracking and not the ArtoolkitX counter

ArtoolkitX gates detection on _currentlyTrackedMarkers < _maxNumberOfMarkersToTrack. That doesn't translate directly here: _isDetected is reset to false at the top of every frame, and optical flow is skipped on the very first detection frame (_frameCount == 0), so the marker is detected but not yet tracking. A counter-based guard would then see _currentlyTrackedMarkers == _max on the next frame and skip both detection and optical flow (the OF/pose gate is _isDetected || _isTracking, both false) → permanent freeze. This is guaranteed in the static-image example, which feeds the same frame repeatedly. Gating on _isTracking re-detects until optical flow actually holds a lock, then skips.

State walk-through (single marker)

Frame _isTracking in Detection? Result
0 (acquire) false yes match → _isDetected=true; OF skipped (_frameCount==0)
1 false yes match; OF runs → _isTracking=true
2…N (hold) true no OF + template maintain pose (the win)
loss true→false OF/template fail → _isTracking=false, _valid=false (#46)
N+1 false yes detection resumes

Unchanged

Optical-flow / template / solvePnP paths (gated on _isDetected || _isTracking), #46 tracking-loss, #38 centered origin, and part A's downsampling/rescale.

Testing

  • Static (1920×1440, pyrLevel 1): acquires and holds — no freeze (the case the counter-guard would break).
  • Webcam (640×480): acquire → marker LOST on removal → marker FOUND on re-show; steady-state fps ~10–15 → ~45.

Full design notes: docs/design-detection-guard.md in the webarkit-testing companion PR.

🤖 Generated with Claude Code

Guard the per-frame feature detection (extractFeatures + descriptor
matching) with `if (!_isTracking)`. Once optical flow holds a lock,
detection is skipped and the pose is maintained by optical flow + template
matching; when the marker is lost (_isTracking cleared in runOpticalFlow /
RunTemplateMatching), detection resumes on the next frame to re-acquire.

Detection is the dominant per-frame cost, so skipping it in the steady
tracking state is a large win: at 640x480 the webcam example goes from
~10-15 fps (detecting every frame) to ~45 fps while tracking. This is
part B of webarkit#44; part A (detection-side pyramid downsampling) landed in webarkit#52.

The guard is on _isTracking rather than ArtoolkitX's
`_currentlyTrackedMarkers < _maxNumberOfMarkersToTrack`: _isDetected is
reset to false every frame, and optical flow is skipped on the first
detection frame (_frameCount == 0), so a counter-based guard would skip
both detection and optical flow on the following frame and freeze (this
bites the static-image example, which feeds the same frame repeatedly).
Gating on _isTracking re-detects until optical flow actually holds a lock.

The optical-flow/template/pose paths (gated on _isDetected || _isTracking)
and webarkit#46 tracking-loss are unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@kalwalt kalwalt self-assigned this Jun 19, 2026
@kalwalt kalwalt added enhancement New feature or request C/C++ code concerning the C/C++ code design and improvements Emscripten labels Jun 19, 2026
@kalwalt kalwalt moved this from To do to Review in progress in New markerless image tracking Jun 19, 2026
@kalwalt kalwalt added this to the Markerless tracking milestone Jun 19, 2026
@kalwalt kalwalt moved this from Review in progress to Reviewer approved in New markerless image tracking Jun 19, 2026
@kalwalt
kalwalt merged commit 28ff065 into webarkit:dev Jun 19, 2026
@github-project-automation github-project-automation Bot moved this from Reviewer approved to Done in New markerless image tracking Jun 19, 2026
kalwalt added a commit to webarkit/webarkit-testing that referenced this pull request Jun 19, 2026
Bump WebARKitLib to the part-B detection guard (webarkit/WebARKitLib#54)
and rebuild build/ + dist/. Feature detection is now skipped while optical
flow holds a lock, so the steady tracking state is much cheaper.

Measured on the Teblid webcam example (640x480): ~10-15 fps when detecting
every frame -> ~45 fps while tracking. Static example (1920x1440, pyrLevel 1)
still acquires and holds (no freeze).

- docs: add docs/design-detection-guard.md (understanding, ArtoolkitX
  reference, why the guard is on _isTracking rather than the marker counter,
  state walk-through, testing, risks, non-goals).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kalwalt added a commit to webarkit/webarkit-testing that referenced this pull request Jun 21, 2026
Bump WebARKitLib to the part-B detection guard (webarkit/WebARKitLib#54)
and rebuild build/ + dist/. Feature detection is now skipped while optical
flow holds a lock, so the steady tracking state is much cheaper.

Measured on the Teblid webcam example (640x480): ~10-15 fps when detecting
every frame -> ~45 fps while tracking. Static example (1920x1440, pyrLevel 1)
still acquires and holds (no freeze).

- docs: add docs/design-detection-guard.md (understanding, ArtoolkitX
  reference, why the guard is on _isTracking rather than the marker counter,
  state walk-through, testing, risks, non-goals).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

C/C++ code concerning the C/C++ code design and improvements Emscripten enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Restore downsampled feature detection + single-marker detection guard for performance (reconcile with ArtoolkitX OCVT)

1 participant