Render Teblid static-image example with correct pose (#31) - #33
Merged
Conversation
Static-image example now reads `matrixGL_RH` (the GL right-handed modelview the library computes after the CV->GL handedness flip) instead of `pose` (raw OpenCV camera pose), so the tracked object sits in front of the GL camera and renders on the marker instead of being frustum-clipped behind it. Library side (pulled in via the submodule bump to fix/pose-cv-to-gl): - WebARKitPattern::updateTrackable() no longer applies its partial rotation-column negation; `trans` / getPoseMatrix2() now carry the raw OpenCV pose, and downstream conversions (arglCameraViewRHf in JS for matrixGL_RH; WebARKitManager::getGLViewMatrix in C++) become clean diag(1, -1, -1, 1) * pose CV->GL conversions. Example side: - examples/worker_threejs.js: forward event.data.matrixGL_RH in the `found` message (additive; existing `pose` consumers unaffected). - examples/threejs_static_image_worker_ES6.js: parse msg.matrixGL_RH into the root matrix; wrap AR content in a `markerFrame` Object3D rotated 180 deg around Y so +Z faces the camera (corrects the marker's image-coordinate convention to a natural three.js placement). - examples/threejs_teblid_static_image_ES6_example.html: display the static <img> full-bleed under the canvas overlay (matches the webcam examples' #video layout) so the marker is visible behind the 3D content. Also rebuild dist/WebARKit.js and dist/* / build/* artifacts from the new WASM, and bump the npm tooling to current minor/patch (Babel 7.29.7, webpack 5.107.2, babel-loader 10.1.1, rimraf 6.1.3) plus webpack-cli to 7.0.3 (major) for build cleanliness. A remaining projection X-mirror in the library leaves the AR content's axes visibly tilted relative to the marker edges; this is tracked separately in webarkit/WebARKitLib#35 and cannot be fully corrected from the example side. Refs: #31, webarkit/WebARKitLib#34 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
webarkit/WebARKitLib#36 was squash-merged into upstream/dev as 3429eb3 (same content as the prior dab5073 SHA, different hash due to the squash). Re-pin the submodule pointer to the canonical commit so a fresh `git submodule update --init` against the .gitmodules URL (canonical webarkit/WebARKitLib) resolves successfully. Refs: #31, webarkit/WebARKitLib#34 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Resolves #31. The Teblid static-image example now renders the AR sphere on the marker instead of having it frustum-clipped behind the camera. The fix has two halves:
webarkit/WebARKitLib#36) —WebARKitPatternTrackingInfo::updateTrackable()no longer applies a partial CV→GL conversion;getPoseMatrix2()/transcarry the raw OpenCV pose, and downstream conversions (arglCameraViewRHfin JS formatrixGL_RH;WebARKitManager::getGLViewMatrix()in C++) become cleandiag(1, -1, -1, 1) * posemodelviews.matrixGL_RH(GL right-handed modelview) instead of the rawpose, and the worker forwards that field.Changes
matrixGL_RHand wraps content in amarkerFramerotated 180° around Y so +Z faces the cameraexamples/threejs_static_image_worker_ES6.jsmatrixGL_RHin thefoundmessageexamples/worker_threejs.js<img id="static-image">rendered full-bleed beneath the canvas overlay so the marker is visible behind the 3D contentexamples/threejs_teblid_static_image_ES6_example.htmldist/WebARKit.js,dist/WebARKit.js.LICENSE.txt,dist/SpeedyVisionSinkImageData.js,dist/GrayScale.js,build/webarkit_ES6_wasm.js,build/webarkit_ES6_wasm.simd.jsdab5073onfix/pose-cv-to-gl(will become a canonical SHA onwebarkit/WebARKitLibdev once #36 merges)emscripten/WebARKitLibpackage.json,package-lock.jsondocs/design-pose-cv-to-gl-fix.mdTest plan
npm install&& serveexamples/over HTTPthreejs_teblid_static_image_ES6_example.html— console showsMarker tracked ! Num. matches : N, loader fades, sphere appears anchored on the pinball marker (no longer clipped behind the camera)worker_threejs.jschange is additive)Known follow-up (out of scope)
The 3D axes appear visibly tilted relative to the marker edges — that's an unresolved projection X-mirror in the library, tracked in webarkit/WebARKitLib#35. The pose fix in this PR is independent of that and renders correctly given the current projection.
Refs
🤖 Generated with Claude Code