feat(#44): skip feature detection while tracking (part B) - #54
Merged
Conversation
Guard the per-frame feature detection (extractFeatures + descriptor matching) with `if (!_isTracking)`. Once optical flow holds a lock, detection is skipped and the pose is maintained by optical flow + template matching; when the marker is lost (_isTracking cleared in runOpticalFlow / RunTemplateMatching), detection resumes on the next frame to re-acquire. Detection is the dominant per-frame cost, so skipping it in the steady tracking state is a large win: at 640x480 the webcam example goes from ~10-15 fps (detecting every frame) to ~45 fps while tracking. This is part B of webarkit#44; part A (detection-side pyramid downsampling) landed in webarkit#52. The guard is on _isTracking rather than ArtoolkitX's `_currentlyTrackedMarkers < _maxNumberOfMarkersToTrack`: _isDetected is reset to false every frame, and optical flow is skipped on the first detection frame (_frameCount == 0), so a counter-based guard would skip both detection and optical flow on the following frame and freeze (this bites the static-image example, which feeds the same frame repeatedly). Gating on _isTracking re-detects until optical flow actually holds a lock. The optical-flow/template/pose paths (gated on _isDetected || _isTracking) and webarkit#46 tracking-loss are unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kalwalt
added a commit
to webarkit/webarkit-testing
that referenced
this pull request
Jun 19, 2026
Bump WebARKitLib to the part-B detection guard (webarkit/WebARKitLib#54) and rebuild build/ + dist/. Feature detection is now skipped while optical flow holds a lock, so the steady tracking state is much cheaper. Measured on the Teblid webcam example (640x480): ~10-15 fps when detecting every frame -> ~45 fps while tracking. Static example (1920x1440, pyrLevel 1) still acquires and holds (no freeze). - docs: add docs/design-detection-guard.md (understanding, ArtoolkitX reference, why the guard is on _isTracking rather than the marker counter, state walk-through, testing, risks, non-goals). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kalwalt
added a commit
to webarkit/webarkit-testing
that referenced
this pull request
Jun 21, 2026
Bump WebARKitLib to the part-B detection guard (webarkit/WebARKitLib#54) and rebuild build/ + dist/. Feature detection is now skipped while optical flow holds a lock, so the steady tracking state is much cheaper. Measured on the Teblid webcam example (640x480): ~10-15 fps when detecting every frame -> ~45 fps while tracking. Static example (1920x1440, pyrLevel 1) still acquires and holds (no freeze). - docs: add docs/design-detection-guard.md (understanding, ArtoolkitX reference, why the guard is on _isTracking rather than the marker counter, state walk-through, testing, risks, non-goals). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Part B of #44. Guard the per-frame feature detection (
extractFeatures+ descriptor matching viaMatchFeatures) withif (!_isTracking). Once optical flow holds a lock, detection is skipped and the pose is maintained by optical flow + template matching. When the marker is lost (_isTrackingcleared inrunOpticalFlow/RunTemplateMatching), detection resumes on the next frame to re-acquire.Why
Detection is the dominant per-frame cost. Running it every frame even while a lock is held is wasteful — optical flow already produces the pose. Part A (#52) cut the cost of each detection; part B removes most detections entirely.
Measured (Teblid webcam example, 640×480): ~10–15 fps detecting every frame → ~45 fps while tracking (≈3×). Re-acquisition latency is one frame after loss.
Why
!_isTrackingand not the ArtoolkitX counterArtoolkitX gates detection on
_currentlyTrackedMarkers < _maxNumberOfMarkersToTrack. That doesn't translate directly here:_isDetectedis reset tofalseat the top of every frame, and optical flow is skipped on the very first detection frame (_frameCount == 0), so the marker is detected but not yet tracking. A counter-based guard would then see_currentlyTrackedMarkers == _maxon the next frame and skip both detection and optical flow (the OF/pose gate is_isDetected || _isTracking, both false) → permanent freeze. This is guaranteed in the static-image example, which feeds the same frame repeatedly. Gating on_isTrackingre-detects until optical flow actually holds a lock, then skips.State walk-through (single marker)
_isTrackingin_isDetected=true; OF skipped (_frameCount==0)_isTracking=true_isTracking=false,_valid=false(#46)Unchanged
Optical-flow / template /
solvePnPpaths (gated on_isDetected || _isTracking), #46 tracking-loss, #38 centered origin, and part A's downsampling/rescale.Testing
marker LOSTon removal →marker FOUNDon re-show; steady-state fps ~10–15 → ~45.Full design notes:
docs/design-detection-guard.mdin the webarkit-testing companion PR.🤖 Generated with Claude Code