Skip to content

Tracking never declared lost when the marker leaves the frame (_valid/_isTracking stuck) #46

Description

@kalwalt

Summary

Once a marker is detected, isValid() stays true (and the getMarker event keeps firing) indefinitely after the marker has left the camera frame. Tracking is never declared lost: the reported pose freezes at the last good value and never clears.

Surfaced by the new live-webcam Teblid example (webarkit/webarkit-testing#36): the on-screen "Marker tracked" status never returns to "searching" even minutes after the marker is gone.

Root cause (WebARKitTracker.cpp detect→track state machine)

  • _valid and _isTracking are only updated inside runOpticalFlow() (this->_valid = homoInfo.validHomography;, and cleared on failure).
  • runOpticalFlow() is gated behind if (_isDetected) { … } — it runs only when the marker is also freshly matched this frame.
  • When the marker leaves the frame, MatchFeatures fails → _isDetected = false → the if (_isDetected) block is skipped → runOpticalFlow never runs, so _valid / _isTracking keep their last true value.
  • if (_isDetected || _isTracking) is then still true (stale _isTracking), so a pose is recomputed from the stale tracked points and getMarker fires → found, every frame, forever.

So the only code path that could detect loss (optical flow failing / homography going invalid) is unreachable once detection fails. The tracker is stuck "valid" with a frozen pose.

Suggested fixes (any / combination)

  1. Run optical flow on _isDetected || _isTracking (not just _isDetected): when detection drops but we were tracking, optical flow runs, fails on the now-absent marker, and clears _isTracking / _valid.
  2. Gate _valid on a confidence metric: RunTemplateMatching's TM_SQDIFF_NORMED correlation (already computed but not used to drop tracking), the homography inlier count/ratio, and/or homography sanity (non-degenerate scale/rotation) — as ArtoolkitX OCVT does.
  3. Add an explicit "tracking lost" transition that resets _isDetected / _isTracking / _valid and clears output, so consumers receive a clean not found.

Impact

Any continuous (video/webcam) consumer cannot tell when tracking is lost; AR content stays frozen on screen.

Repro

Run the webcam Teblid example (webarkit/webarkit-testing#36), acquire the marker, then remove it from view — "Marker tracked" stays indefinitely (observed >1 min).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

C/C++ codeconcerning the C/C++ code design and improvementsEmscriptenbugSomething isn't workingenhancementNew feature or request

Type

Projects

Relationships

None yet

Development

No branches or pull requests

Issue actions