Summary
Add src/freak/freak.ts — the FREAK descriptor (Alahi et al., CVPR 2012),
ported from OpenCV's xfeatures2d, alongside the existing orb binary
descriptor.
Scope narrowed from the original wording: this issue is FREAK and only FREAK.
TEBLID and GMS now have issues of their own.
Motivation
jsfeatNext ships one binary descriptor (orb, ORB/rBRIEF-style). FREAK uses a
retinal sampling pattern with pairwise intensity comparisons and occupies a
niche none of the alternatives cover:
It estimates orientation internally, from 45 long-distance sampling pairs
(NB_ORIENPAIRS), instead of relying on an angle supplied by the detector. That
matters concretely: detectors that produce no orientation (StarDetector/CenSurE
being the obvious candidate for PureCV) work with FREAK out of the box, whereas
pairing them with TEBLID would require a separate orientation estimator such as
the intensity centroid ORB uses.
It also supports cascade matching — the first 16 bytes reject most candidates
before the remaining 48 are compared — which sits naturally on top of a
Hamming brute-force matcher.
Scope
- Retinal sampling pattern, pairwise intensity comparisons → binary descriptor.
- Internal orientation estimation from the 45 long-distance pairs.
- API shaped like
orb.describe(...) for consistency.
- Attached as
jsfeatNext.freak.
The design decision this issue must make explicitly
Pattern generation is not a porting detail. OpenCV's buildPattern()
allocates NB_SCALES × NB_ORIENTATION × NB_POINTS = 64 × 256 × 43 = 704,512
PatternPoint structures of 12 bytes each — roughly 8.5 MB on the heap at
construction. That is heavy for WASM and simply impossible for no_std on
constrained targets, which is what the PureCV port faces downstream.
This is not intrinsic to the algorithm, only to OpenCV's implementation, which
trades memory for computation. Three options:
- Compute on demand. Pattern points are
(x·cosθ − y·sinθ, x·sinθ + y·cosθ) scaled; recomputing them per keypoint
costs a few dozen float operations against 8.5 MB saved. Preferred.
- Reduce
NB_ORIENTATION from 256 to 32 or 64 → a ~1–2 MB table, with
negligible loss of angular precision (256 steps is 1.4°, well beyond the real
accuracy of any orientation estimator).
- Lazy per-scale population of only the scale indices the frame's keypoints
actually use.
Suggested resolution: (1) as the default, (2) available as an option. Whichever
is chosen must be stated in the PR description, because PureCV inherits it — and
if the decision stays implicit, the OpenCV implementation gets ported literally
and the memory profile comes with it.
Acceptance criteria
Non-goals
Dependencies
Related
Summary
Add
src/freak/freak.ts— the FREAK descriptor (Alahi et al., CVPR 2012),ported from OpenCV's
xfeatures2d, alongside the existingorbbinarydescriptor.
Scope narrowed from the original wording: this issue is FREAK and only FREAK.
TEBLID and GMS now have issues of their own.
Motivation
jsfeatNext ships one binary descriptor (
orb, ORB/rBRIEF-style). FREAK uses aretinal sampling pattern with pairwise intensity comparisons and occupies a
niche none of the alternatives cover:
It estimates orientation internally, from 45 long-distance sampling pairs
(
NB_ORIENPAIRS), instead of relying on an angle supplied by the detector. Thatmatters concretely: detectors that produce no orientation (StarDetector/CenSurE
being the obvious candidate for PureCV) work with FREAK out of the box, whereas
pairing them with TEBLID would require a separate orientation estimator such as
the intensity centroid ORB uses.
It also supports cascade matching — the first 16 bytes reject most candidates
before the remaining 48 are compared — which sits naturally on top of a
Hamming brute-force matcher.
Scope
orb.describe(...)for consistency.jsfeatNext.freak.The design decision this issue must make explicitly
Pattern generation is not a porting detail. OpenCV's
buildPattern()allocates
NB_SCALES × NB_ORIENTATION × NB_POINTS = 64 × 256 × 43 = 704,512PatternPointstructures of 12 bytes each — roughly 8.5 MB on the heap atconstruction. That is heavy for WASM and simply impossible for
no_stdonconstrained targets, which is what the PureCV port faces downstream.
This is not intrinsic to the algorithm, only to OpenCV's implementation, which
trades memory for computation. Three options:
(x·cosθ − y·sinθ, x·sinθ + y·cosθ)scaled; recomputing them per keypointcosts a few dozen float operations against 8.5 MB saved. Preferred.
NB_ORIENTATIONfrom 256 to 32 or 64 → a ~1–2 MB table, withnegligible loss of angular precision (256 steps is 1.4°, well beyond the real
accuracy of any orientation estimator).
actually use.
Suggested resolution: (1) as the default, (2) available as an option. Whichever
is chosen must be stated in the PR description, because PureCV inherits it — and
if the decision stays implicit, the OpenCV implementation gets ported literally
and the memory profile comes with it.
Acceptance criteria
keypoint set.
the PR, with its memory cost stated.
descriptor stability without any detector-supplied angle.
contaminated (cf. orb.describe silently produces contaminated descriptors for keypoints near the image edge #110).
any(Replace remaininganyusages with shared types/interfaces (TypedArray, MotionKernel, …) #85).Non-goals
features2d-equivalent suite.haar/bbf(Port haar module (Haar cascade object/face detection) #43/Port bbf module (BBF face detection) #44).Dependencies
image).
Related
docs/features2d-expansion-plan.md