Skip to content

Restore downsampled feature detection + single-marker detection guard for performance (reconcile with ArtoolkitX OCVT) #44

Description

@kalwalt

Background

WebARKitTracker currently runs feature detection (Teblid/AKAZE/ORB) on the full-resolution frame, every frame. The two ArtoolkitX OCVT performance optimizations are disabled:

  1. Pyramid downsamplingPlanarTracker detects on a pyrDown'd detectionFrame and scales keypoints back up via m_featureDetectScaleFactor. In WebARKit the pyrDown block in resetTracking() is commented out.
  2. Detection guardif (m_currentlyTrackedMarkers < m_maxNumberOfMarkersToTrack) skips detection while already tracking, relying on optical flow. In WebARKit this if is commented out (the original author noted it "doesn't work as expected ... makes tracking unstable").

The leftover scale-up that this created was a correctness bug, fixed in #43 by forcing _featureDetectScaleFactor to identity. The follow-up cleanup (removing the now-vestigial _featureDetectPyrLevel / _featureDetectScaleFactor machinery, keeping full-res detection) is the companion change to this issue.

Goal

Re-introduce the performance optimizations correctly, matching the ArtoolkitX OCVT reference, without the instability the original author hit.

Work

  • Restore the detectionFrame pyramid (pyrDown to _featureDetectPyrLevel).
  • Detect features on detectionFrame (downsampled), not the full-res frame.
  • Restore the keypoint scale-up (pt *= _featureDetectScaleFactor) in MatchFeatures.
  • Fix createFeatureMask: it currently builds the mask on frame.size() (full-res) yet divides the bbox by _featureDetectScaleFactor. With downsampling restored it must build the mask on detectionFrame (downsampled) so the /scaleFactor bbox coords match — otherwise the mask is misaligned. (Latent landmine noted in Feature keypoints double-scaled for frames larger than featureImageMinSize (marker mis-localized at ~2x position) #43.)
  • Investigate and fix the detection guard instability before re-enabling it, or document why detection must run every frame.
  • Benchmark: full-res-every-frame vs downsampled+guard on HD frames.

Notes

  • 640x480 frames have pyrLevel 0 (factor 1.0), so the perf win is only on larger frames (e.g. 1920x1080, 2000x1500).
  • Regression canaries: the static-image Teblid example (2000x1500) and a live 640x480 webcam example.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions