Skip to content

Add static image example with Teblid tracker (#30) - #32

Merged
kalwalt merged 1 commit into
devfrom
feat/static-image-teblid-example
May 29, 2026
Merged

Add static image example with Teblid tracker (#30)#32
kalwalt merged 1 commit into
devfrom
feat/static-image-teblid-example

Conversation

@kalwalt

@kalwalt kalwalt commented May 29, 2026

Copy link
Copy Markdown
Member

Summary

Implements #30: a static-image AR example that feeds a fixed JPEG into the Teblid tracker instead of a webcam stream, so the tracker can be exercised without camera access (useful for debugging and CI). Structure mirrors threejs_ES6_example.html, reusing worker_threejs.js and the existing data/pinball.jpg marker unchanged.

The marker tracks successfully. The 3D sphere is not yet visible due to a pre-existing pose-convention issue, filed separately as #31 (out of scope here).

What's included

File Purpose
examples/threejs_teblid_static_image_ES6_example.html Page with a hidden <img> input (no initCamera); load event starts tracking
examples/threejs_static_image_worker_ES6.js Client worker: marker decode + full-res RGBA processing + warmup/ping-pong
examples/data/pinball-demo.jpg Test image containing the marker
docs/design-static-image-teblid-example.md Design notes, decision log, and bug analysis

worker_threejs.js, data/pinball.jpg, and the CSS/JS assets are reused without modification.

Bugs found & fixed (to make tracking actually work)

All fixes live in the new client worker — worker_threejs.js is untouched:

  1. Marker was passed as raw JPEG bytes. initTrackerGray() treats its buffer as raw pixels (no JPEG decode), so the ~764 KB compressed file was read as a 13 MB RGBA reference → garbage marker, matching always failed. Now decoded to RGBA pixels via a plain 2D canvas (no GrayScale/WebGL class, per request).
  2. Init/process dimension mismatch. The WASM frame buffer is sized videoWidth × videoHeight, but the original threejs worker sent a downscaled frame. Now processes the image at full resolution to match.
  3. First-frame perspectiveTransform crash. When a marker is detected on frame 0 (_frameCount == 0), the tracker calls GetTrackedFeaturesWarped() before the tracked-feature set is populated → OpenCV assert (scn+1 == m.cols). Worked around with a blank warmup frame; frames are fed via worker ping-pong (throttles heavy full-res processing) and stop once tracked.

Review checklist

  • worker_threejs.js and data/pinball.jpg are genuinely unchanged
  • Marker decode path uses only a 2D canvas (no GrayScale class)
  • Blank warmup frame + ping-pong logic reads cleanly
  • Comfortable including the design doc in docs/ (vs. PR description only)
  • pinball-demo.jpg is the intended test asset

Risk assessment

  • Low risk / additive. New files only; no existing example or shared worker is modified, so no regression surface for current examples.
  • The full-res frame (2000×1500) is heavy, but ping-pong + stop-after-found keeps it to a few frames on a static image.

Test plan

  • Serve examples/ over HTTP and open threejs_teblid_static_image_ES6_example.html
  • Console shows the blank warmup frame ("No features…"), then Marker tracked ! Num. matches : N
  • Loader overlay disappears
  • Confirm existing webcam examples are unaffected

Known follow-up

🤖 Generated with Claude Code

Adds a static-image AR example that feeds a fixed JPEG into the Teblid
tracker instead of a webcam stream, enabling testing without camera access.

- threejs_teblid_static_image_ES6_example.html: hidden <img> input, no initCamera
- threejs_static_image_worker_ES6.js: full-resolution RGBA processing
- data/pinball-demo.jpg: test image containing the marker

Fixes needed to make tracking work (worker_threejs.js left unchanged):
- decode the marker JPEG to RGBA pixels via a 2D canvas (initTrackerGray does
  not decode JPEG; raw file bytes produced a garbage reference marker)
- process the frame at full resolution so it matches the tracker's frame buffer
- send a blank warmup frame before the real image to avoid a first-frame
  perspectiveTransform crash in the tracker

The marker tracks successfully; the 3D sphere is not yet visible due to a
pre-existing pose-convention issue tracked separately in #31.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant