Fix AR pose/localization on >640x480 frames: keypoint scale bug, projection X-sign, marker handedness - #45
Merged
Conversation
… full-res Feature detection runs on the full-resolution frame (the pyrDown downsampling block in resetTracking() is disabled), but MatchFeatures still multiplied the matched frame keypoints by _featureDetectScaleFactor, derived from the image-pyramid level. For frames larger than featureImageMinSize (640x480) that factor is >= 2, so every matched keypoint was doubled -- the marker localized at ~2x its true position, exploding the homography and the solvePnP pose. 640x480 webcams have pyrLevel 0 (factor 1.0), which is why this stayed hidden. Force _featureDetectScaleFactor to identity (1,1) while detection is full-res; keep the pyramid-derived computation commented out so it can be restored together with downsampled detection. Fixes webarkit#43 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…edness cameraProjectionMatrix: use +2*f_x/w (was -2*f_x/w). The modelview already applies the CV->GL Y,Z row negation (arglCameraViewRHf), so the projection must be the plain pinhole form with both focals positive; the negative X focal mirrored screen-X. updateTrackable: negate the Y,Z rotation columns (D*R*D, matching ArtoolkitX ARTrackable2d::updateWithTwoDResults) so the marker frame is right-handed with Z toward the viewer. The translation column is untouched, so position is unchanged. gtests: projection [0] -> +1.7851850084276433. Refs webarkit#35, webarkit#42 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Feature detection has run on the full-resolution frame for some time (the pyrDown block and the single-marker detection guard were commented out), which made _featureDetectScaleFactor an identity no-op after the webarkit#43 fix. Remove the now-dead members (_featureDetectPyrLevel, _featureDetectScaleFactor), their initialize() computation, the commented pyrDown/guard block, the no-op keypoint rescale in MatchFeatures, and the no-op bbox divide in createFeatureMask. Behavior is unchanged (all removed paths were x1/div1 or commented). Restoring downsampled detection + the detection guard for performance is tracked in webarkit#44. Refs webarkit#43, webarkit#44 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
3 tasks
Member
Author
|
Companion example + build PR: webarkit/webarkit-testing#35 |
Closed
3 tasks
kalwalt
added a commit
to webarkit/webarkit-testing
that referenced
this pull request
Jun 10, 2026
webarkit/WebARKitLib#45 merged into dev (rebased to 1021616, f448ef3, c1ae096). Re-point the submodule from the fork branch commit to the canonical dev tip. Source is identical, so build/ and dist/ are unchanged. Refs #30, webarkit/WebARKitLib#45 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kalwalt
added a commit
to webarkit/webarkit-testing
that referenced
this pull request
Jun 10, 2026
webarkit/WebARKitLib#45 merged into dev (rebased to 1021616, f448ef3, c1ae096). Re-point the submodule from the fork branch commit to the canonical dev tip. Source is identical, so build/ and dist/ are unchanged. Refs #30, webarkit/WebARKitLib#45 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
4 tasks
kalwalt
added a commit
to webarkit/webarkit-testing
that referenced
this pull request
Jun 21, 2026
webarkit/WebARKitLib#45 merged into dev (rebased to 1021616, f448ef3, c1ae096). Re-point the submodule from the fork branch commit to the canonical dev tip. Source is identical, so build/ and dist/ are unchanged. Refs #30, webarkit/WebARKitLib#45 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This was referenced Jul 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes AR pose/localization for the OCVT planar tracker on frames larger than
featureImageMinSize(640x480) — the case exercised by the webarkit-testing static-image Teblid example at 2000x1500. What presented as a single "mirror" was three orthogonal bugs, headlined by a feature-detection scale bug that masked the others:Plus a behavior-preserving refactor removing the now-dead scale machinery (#44).
Details
1. Scale-factor double-application (
WebARKitTracker.cpp) — Fixes #43Detection runs on the full-resolution frame (the
pyrDownblock inresetTracking()is disabled), butMatchFeaturesstill multiplied matched keypoints by_featureDetectScaleFactor(derived from the pyramid level):Doubled keypoints → marker localized at ~2x position → exploded homography → wrong
solvePnPtranslation. Evidence: matched centroidref(1034,908) → frame(1816,1250)= exactly 2×(908,625). Fix: force the factor to identity while detection is full-res.2. Projection X-focal sign (
WebARKitGL.cpp) — Refs #35cameraProjectionMatrixused-2*f_x/w, mirroring screen-X. The modelview already applies the CV→GL Y,Z row negation (arglCameraViewRHf), so the projection must be plain pinhole with both focals positive (+2*f_x/w). Verified by projecting the tracked origin to NDC. gtests updated ([0]→+1.7851850084276433).3. Marker handedness — DRD (
WebARKitPattern.cpp) — Refs #42Negate the Y,Z rotation columns of
trans(matching ArtoolkitXARTrackable2d::updateWithTwoDResults). With the downstream row negation this yieldsD*R*D— a right-handed frame (X=right, Y=up, Z=toward viewer). The translation column is untouched, so position is unchanged.4. Refactor (
WebARKitTracker.cpp) — Refs #44Remove the dead
_featureDetectPyrLevel/_featureDetectScaleFactormembers, the commentedpyrDown/detection-guard block, and the no-op rescales. Restoring downsampled detection + the guard for performance is tracked in #44.Risk / behavior
Testing
Review checklist
createFeatureMaskbbox now used directly (see Restore downsampled feature detection + single-marker detection guard for performance (reconcile with ArtoolkitX OCVT) #44 caveat if downsampling is restored)Companion example/build PR: webarkit/webarkit-testing (linked below).
🤖 Generated with Claude Code