Skip to content

Declare tracking lost when the marker leaves the frame (#46) - #47

Merged
kalwalt merged 1 commit into
webarkit:devfrom
kalwalt:fix/tracking-loss-46
Jun 15, 2026
Merged

Declare tracking lost when the marker leaves the frame (#46)#47
kalwalt merged 1 commit into
webarkit:devfrom
kalwalt:fix/tracking-loss-46

Conversation

@kalwalt

@kalwalt kalwalt commented Jun 13, 2026

Copy link
Copy Markdown
Member

Summary

Fixes #46: after a marker is acquired, isValid() / the getMarker event kept emitting found with a frozen pose indefinitely once the marker left the frame. Two coupled changes in WebARKitTracker.cpp make the tracker declare loss, matching the ArtoolkitX OCVT semantics.

Root cause

WebARKit diverged from ArtoolkitX's tracker state machine in two ways:

  1. Track loop gated on _isDetected only. WebARKit resets _isDetected = false every frame and runs detection every frame, so _isDetected means "matched this frame." Once the marker leaves, MatchFeatures fails → _isDetected = false → the optical-flow + template-match path was skipped entirely, so loss was never evaluated. (ArtoolkitX runs that path on a persistent _isDetected with guarded detection, so it re-validates every frame.)
  2. _valid not reset on template-match failure. Template matching is the appearance check. When the marker is replaced by a static background, optical flow keeps the points (they don't move → no failure), but the marker template no longer matches → updateTrackableHomography fails. runOpticalFlow's failure path already cleared _valid; RunTemplateMatching's did not — so isValid() stayed true and getMarker kept firing.

ArtoolkitX has no _valid: IsTrackableVisible/GetTrackablePose are gated purely on _isDetected || _isTracking, and RunTemplateMatching failure sets both false.

Changes

  • resetTracking(): run the optical-flow/template-match block on _isDetected || _isTracking (was _isDetected), so a tracked-but-not-detected frame still re-validates (and fails → clears state) when the marker is gone.
  • RunTemplateMatching() failure path: also set _valid = false (mirrors runOpticalFlow's failure path), so a lost marker propagates to isValid().

Behavior

  • Marker present: unchanged — found, stable pose (template matching passes on the real marker; detection re-acquires every frame).
  • Marker leaves: within ~1–2 frames, template matching fails → _isDetected/_isTracking/_valid cleared → isValid() false → not found. Content can be hidden by consumers.

Testing

Verified against the live-webcam Teblid example (webarkit/webarkit-testing#37): acquiring the marker shows the AR content; removing it now flips to "Searching…" and hides the content within a frame or two; re-introducing re-acquires; no flicker while the marker is held in view.

Notes / follow-ups

Fixes #46

🤖 Generated with Claude Code

Two coupled changes so the tracker stops emitting 'found' with a stale pose after the marker is gone:

1. Run the optical-flow + template-match path on (_isDetected || _isTracking), not only _isDetected. WebARKit resets _isDetected every frame and detects every frame, so once the marker leaves, MatchFeatures fails (_isDetected=false) and the track/validate path was skipped entirely -- loss was never evaluated. (ArtoolkitX gets this for free because its _isDetected persists and detection is guarded.)

2. Reset _valid in RunTemplateMatching's failure path. Template matching is the appearance check; on a static background optical flow keeps the points (no failure) but the marker template no longer matches, so updateTrackableHomography fails. runOpticalFlow's failure path already cleared _valid; RunTemplateMatching's did not, so isValid() stayed true and getMarker kept firing. ArtoolkitX has no _valid and treats !_isDetected && !_isTracking as not-visible (IsTrackableVisible).

Fixes webarkit#46

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@kalwalt

kalwalt commented Jun 13, 2026

Copy link
Copy Markdown
Member Author

Companion integration PR (submodule bump + rebuilt artifacts + webcam-example verification): webarkit/webarkit-testing#39

@kalwalt kalwalt self-assigned this Jun 13, 2026
@kalwalt kalwalt added bug Something isn't working enhancement New feature or request C/C++ code concerning the C/C++ code design and improvements Emscripten labels Jun 13, 2026
@kalwalt kalwalt moved this from In progress to Review in progress in New markerless image tracking Jun 13, 2026
@kalwalt kalwalt added this to the Markerless tracking milestone Jun 13, 2026
@kalwalt kalwalt moved this from Review in progress to Reviewer approved in New markerless image tracking Jun 15, 2026
@kalwalt
kalwalt merged commit 39183f0 into webarkit:dev Jun 15, 2026
@github-project-automation github-project-automation Bot moved this from Reviewer approved to Done in New markerless image tracking Jun 15, 2026
kalwalt added a commit to webarkit/webarkit-testing that referenced this pull request Jun 15, 2026
webarkit/WebARKitLib#47 merged into dev (rebased to 39183f0). Re-point the submodule from the fork-branch commit to the canonical dev tip; source is identical, build/ and dist/ unchanged.

Refs #36, webarkit/WebARKitLib#46, webarkit/WebARKitLib#47

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kalwalt added a commit to webarkit/webarkit-testing that referenced this pull request Jun 15, 2026
Pick up the tracking-loss fix (webarkit/WebARKitLib#47, #46): the tracker now declares 'not found' when the marker leaves the frame instead of staying stuck on a stale pose. Rebuild build/ + dist/ and bump the submodule. Adds the design doc.

Verified with the webcam Teblid example: removing the marker hides the AR content and shows 'Searching...'; no flicker while the marker is held.

Refs #36, webarkit/WebARKitLib#46, webarkit/WebARKitLib#47

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kalwalt added a commit to webarkit/webarkit-testing that referenced this pull request Jun 15, 2026
webarkit/WebARKitLib#47 merged into dev (rebased to 39183f0). Re-point the submodule from the fork-branch commit to the canonical dev tip; source is identical, build/ and dist/ unchanged.

Refs #36, webarkit/WebARKitLib#46, webarkit/WebARKitLib#47

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@kalwalt
kalwalt deleted the fix/tracking-loss-46 branch June 15, 2026 22:07
kalwalt added a commit to webarkit/webarkit-testing that referenced this pull request Jun 21, 2026
Pick up the tracking-loss fix (webarkit/WebARKitLib#47, #46): the tracker now declares 'not found' when the marker leaves the frame instead of staying stuck on a stale pose. Rebuild build/ + dist/ and bump the submodule. Adds the design doc.

Verified with the webcam Teblid example: removing the marker hides the AR content and shows 'Searching...'; no flicker while the marker is held.

Refs #36, webarkit/WebARKitLib#46, webarkit/WebARKitLib#47

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kalwalt added a commit to webarkit/webarkit-testing that referenced this pull request Jun 21, 2026
webarkit/WebARKitLib#47 merged into dev (rebased to 39183f0). Re-point the submodule from the fork-branch commit to the canonical dev tip; source is identical, build/ and dist/ unchanged.

Refs #36, webarkit/WebARKitLib#46, webarkit/WebARKitLib#47

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working C/C++ code concerning the C/C++ code design and improvements Emscripten enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Tracking never declared lost when the marker leaves the frame (_valid/_isTracking stuck)

1 participant