Summary
WebARKitTracker mis-localizes the marker for any frame larger than featureImageMinSize (640x480): the homography and solvePnP pose place tracked content at roughly 2x the marker's true position. Typical 640x480 webcams are unaffected, which is why this has gone unnoticed.
Root cause
WebARKitTrackerImpl::initialize() computes _featureDetectScaleFactor from an image-pyramid downsample level (_featureDetectPyrLevel), intended to scale keypoints detected on a downsampled image back up to full-frame coordinates.
But the downsampling is disabled — the pyrDown block in resetTracking() is commented out and extractFeatures() runs on the full-resolution frame. The matched frame keypoints (already full-frame) are then still multiplied by the factor in MatchFeatures():
finalMatched1[i].pt.x *= _featureDetectScaleFactor[0];
finalMatched1[i].pt.y *= _featureDetectScaleFactor[1];
With featureImageMinSize = 640x480:
| Frame |
pyrLevel |
factor |
effect |
| 640x480 |
0 |
1.0 |
OK (hidden) |
| 2000x1500 |
1 |
2.0 |
every keypoint doubled |
| 1920x1080 |
1 |
2.0 |
broken |
Doubled keypoints -> homography fit to doubled points (degenerate/exploded bbox) -> solvePnP fed doubled image points -> wrong translation -> content rendered at ~2x position.
Evidence
Centroid of matched reference vs frame keypoints on a 2000x1500 frame: ref(1034,908) -> frame(1816,1250), while the marker physically sits at ~(908,625). (1816,1250) = 2 x (908,625).
Fix
While detection runs full-res, the factor must be identity:
_featureDetectScaleFactor = cv::Vec2f(1.0f, 1.0f);
(Keep the pyramid-derived computation commented out; restore it together with the pyrDown detection path if downsampled detection is re-enabled.)
Follow-up
The disabled downsampling block (// This if cond. doesn't works as expected in artoolkitx ...) and the related _featureDetectScaleFactor machinery should be reconciled with the ArtoolkitX OCVT reference: either restore downsampled detection (perf) with a consistent rescale on both the match side and createFeatureMask, or remove the now-vestigial scale path entirely.
Summary
WebARKitTrackermis-localizes the marker for any frame larger thanfeatureImageMinSize(640x480): the homography andsolvePnPpose place tracked content at roughly 2x the marker's true position. Typical 640x480 webcams are unaffected, which is why this has gone unnoticed.Root cause
WebARKitTrackerImpl::initialize()computes_featureDetectScaleFactorfrom an image-pyramid downsample level (_featureDetectPyrLevel), intended to scale keypoints detected on a downsampled image back up to full-frame coordinates.But the downsampling is disabled — the
pyrDownblock inresetTracking()is commented out andextractFeatures()runs on the full-resolution frame. The matched frame keypoints (already full-frame) are then still multiplied by the factor inMatchFeatures():With
featureImageMinSize = 640x480:Doubled keypoints -> homography fit to doubled points (degenerate/exploded bbox) ->
solvePnPfed doubled image points -> wrong translation -> content rendered at ~2x position.Evidence
Centroid of matched reference vs frame keypoints on a 2000x1500 frame:
ref(1034,908) -> frame(1816,1250), while the marker physically sits at ~(908,625). (1816,1250) = 2 x (908,625).Fix
While detection runs full-res, the factor must be identity:
(Keep the pyramid-derived computation commented out; restore it together with the
pyrDowndetection path if downsampled detection is re-enabled.)Follow-up
The disabled downsampling block (
// This if cond. doesn't works as expected in artoolkitx ...) and the related_featureDetectScaleFactormachinery should be reconciled with the ArtoolkitX OCVT reference: either restore downsampled detection (perf) with a consistent rescale on both the match side andcreateFeatureMask, or remove the now-vestigial scale path entirely.