Endpoint for stats about verified occurrences

[mihow]

Summary

Adds GET /api/v2/occurrences/stats/model-agreement/ — verified-occurrence rate + human↔model agreement rates (with Wilson 95% confidence intervals and Cohen's kappa) over the same filter set the /occurrences/ list view accepts. Designed for the project-overview dashboard widget and occurrence-list sidebar panel (consumed by #1308).

Stats viewset convention established in #1296 (see docs/claude/reference/api-stats-pattern.md): scalar response under the entity it's computed over, namespaced under /stats/.

Filter parity

Stats endpoint accepts every query param the /occurrences/ list endpoint accepts, minus ordering/search (don't apply to scalars).

Param	Source	Notes
project_id=<int>	ProjectMixin	required
apply_defaults=true/false	apply_default_filters	Defaults to true. Bypass = ignore project default taxa lists + score thresholds.
taxon=<id> or determination=<id>	CustomOccurrenceDeterminationFilter	Hierarchical via parents_json — matches the taxon and all descendants.
event=<id>	DjangoFilterBackend
deployment=<id>	DjangoFilterBackend
determination__rank=<RANK>	DjangoFilterBackend	e.g. SPECIES, GENUS, FAMILY.
detections__source_image=<id>	DjangoFilterBackend
collection=<id> or collection_id=<id>	OccurrenceCollectionFilter	Capture collection containing the occurrence's detections.
algorithm=<id> (repeatable)	OccurrenceAlgorithmFilter	Inclusive list of detection-algorithm ids.
not_algorithm=<id> (repeatable)	OccurrenceAlgorithmFilter	Exclusive list.
date_start=<YYYY-MM-DD>	OccurrenceDateFilter	ISO date.
date_end=<YYYY-MM-DD>	OccurrenceDateFilter	ISO date.
verified=true/false	OccurrenceVerified	Has-any-ident filter (independent of identifier).
verified_by_me=true/false	OccurrenceVerifiedByMeFilter	Auth-gated — short-circuits to no-op for anon.
taxa_list_id=<id>	OccurrenceTaxaListFilter
not_taxa_list_id=<id>	OccurrenceTaxaListFilter

Backed by the same OCCURRENCE_FILTER_BACKENDS + OCCURRENCE_FILTERSET_FIELDS tuples wired into OccurrenceViewSet, so the two endpoints stay in lock-step.

Endpoint-specific param

Param	Default	Notes
agreement_coarsest_rank=<RANK>	absent (no threshold)	Optional. When supplied, response includes agreed_coarser_rank_* counting only LCAs at or deeper than the given rank. Accepts any TaxonRank name (case-insensitive); UNKNOWN and unknown strings → 400.

Response shape

{
"project_id": 18,
"total_occurrences": 43149,
"verified_count": 45,
"verified_pct": 0.001,
"verified_with_prediction_count": 24,
"no_prediction_count": 21,
"verified_without_taxon_count": 0,
"comparable_count": 24,
"agreed_exact_count": 12,
"agreed_exact_pct": 0.5,
"agreed_exact_ci_low": 0.3146,
"agreed_exact_ci_high": 0.6854,
"agreed_any_rank_count": 17,
"agreed_any_rank_pct": 0.7083,
"agreed_any_rank_ci_low": 0.5023,
"agreed_any_rank_ci_high": 0.8521,
"cohens_kappa": 0.41,
"agreement_coarsest_rank": null,
"agreed_coarser_rank_count": null,
"agreed_coarser_rank_pct": null
}

With ?agreement_coarsest_rank=FAMILY, the bottom three fields populate:

{
"agreement_coarsest_rank": "FAMILY",
"agreed_coarser_rank_count": 14,
"agreed_coarser_rank_pct": 0.5833
}

Field semantics

• verified_* = at least one non-withdrawn Identification.
• verified_with_prediction_count = verified AND has a machine prediction.
• no_prediction_count = verified but no machine prediction (surfaced so consumers can see why the comparable denominator differs from verified_count).
• verified_without_taxon_count = verified AND has a machine prediction but the human identification has no taxon (e.g. a comment-only verification, since Identification.taxon is nullable). Excluded from the agreement denominator since there is no human label to compare.
• comparable_count = verified occurrences with BOTH a machine prediction and a human taxon. This is the denominator for every agreed_*_pct and the Wilson CIs. Equals verified_with_prediction_count − verified_without_taxon_count. Using it (rather than verified_with_prediction_count) keeps the numerator and denominator describing the same set — a verified row that can't agree or disagree is excluded from both.
• agreed_exact_* = user's best identification taxon equals the model's best prediction.
• agreed_any_rank_* = exact matches plus disagreements whose LCA is at any real taxonomic rank (UNKNOWN excluded, since it sorts after SPECIES in TaxonRank.OrderedEnum). The upstream filter (e.g. a Lepidoptera include list) is what bounds the meaningful scope, not a hardcoded threshold in this function.
• agreed_*_ci_low / agreed_*_ci_high = Wilson score 95% confidence interval bounds for the corresponding agreed_*_pct. Wilson stays inside [0, 1] and is honest at the small n typical of verified sets, where the normal approximation breaks down. null when comparable_count is 0. Reads as "94%, somewhere between 87% and 97%" — wide when few comparable, tight when many.
• cohens_kappa = exact-taxon Cohen's kappa: human↔model agreement beyond chance, range [-1, 1] (negative = worse than chance). Plain agreement % rewards luck — in a project dominated by one common species, human and model agree most of the time just by both naming the common one. Kappa subtracts that expected-by-chance agreement. null when there are no comparable pairs or expected agreement is exactly 1.0 (a single taxon category, undefined). Computed from the same comparable pairs already in memory — no extra query.
• agreed_coarser_rank_* = exact matches plus disagreements whose LCA is at the supplied agreement_coarsest_rank or deeper. null when no threshold supplied.
• agreement_coarsest_rank = the threshold rank that was applied (echoed back to the caller). null when the param was absent.

Disagreement counts are not surfaced explicitly — derivable as comparable_count − agreed_*_count.

Usage examples

# Whole project, project defaults applied
curl '.../api/v2/occurrences/stats/model-agreement/?project_id=18'# Bypass project default filters (broader denominator)
curl '.../api/v2/occurrences/stats/model-agreement/?project_id=18&apply_defaults=false'# One deployment
curl '.../api/v2/occurrences/stats/model-agreement/?project_id=18&deployment=42'# Apply a coarser-rank threshold (also count LCAs at FAMILY or deeper)
curl '.../api/v2/occurrences/stats/model-agreement/?project_id=18&agreement_coarsest_rank=FAMILY'# One taxon and its descendants (hierarchical match)
curl '.../api/v2/occurrences/stats/model-agreement/?project_id=18&taxon=567'# Multiple algorithms (repeated param)
curl '.../api/v2/occurrences/stats/model-agreement/?project_id=18&algorithm=3&algorithm=7'# Combine: deployment + collection + bypass project defaults
curl '.../api/v2/occurrences/stats/model-agreement/?project_id=18&deployment=42&collection=99&apply_defaults=false'

The frontend consumer (#1308) wraps this in useModelAgreement(projectId, filters), which accepts an arbitrary filter map (including arrays for repeated params), so the occurrence list page's filter state can be threaded through unchanged.

Discovering the response shape (OPTIONS)

The viewset sets a custom metadata_class so an OPTIONS request returns the response serializer's field schema — including each field's help_text — under actions.GET. Frontends can fetch this once and key tooltips / labels off the field name instead of hardcoding stat descriptions in the UI.

curl -X OPTIONS '.../api/v2/occurrences/stats/model-agreement/?project_id=18'

{
"name": "Model agreement",
"description": "Verified / human↔model agreement rates over the filtered occurrence set. ...",
"actions": {
"GET": {
"verified_pct": {
"type": "float",
"label": "Verified pct",
"help_text": "verified_count / total_occurrences",
"min_value": 0.0,
"max_value": 1.0
},
"comparable_count": {
"type": "integer",
"label": "Comparable count",
"help_text": "Verified occurrences with BOTH a machine prediction and a human taxon — the denominator for all agreed_*_pct and the Wilson CIs. ..."
}
// ... every response field, with type / label / help_text / bounds
}
}
}

DRF's default SimpleMetadata only emits field schema for write methods (POST / PUT), so this is provided by ResponseSchemaMetadata (ami/base/metadata.py). The convention is documented in docs/claude/reference/api-stats-pattern.md for future stats endpoints. Interpretation copy (e.g. "a wide confidence interval means the rate is based on few verifications") stays in the frontend next to the visualization — help_text describes what a field is, not how to read it as a human.

Implementation notes

• Base queryset is deduplicated (queryset.distinct()) before counting so the join chain from apply_default_filters (e.g. verified_by_me → Identification, taxa_list_id → parents_json) can't inflate total_occurrences vs the verified branch.
• Aggregation is scoped to the verified set (occurrences with at least one non-withdrawn identification), which is typically a small fraction of total. The expensive correlated subqueries — best user identification (over Identification) and best machine prediction (over Classification) — evaluate only on verified rows, not on the full filtered queryset.
• LCA on disagreements is deduplicated to distinct (user_taxon, machine_taxon) pairs before computation.
• The agreement denominator is the comparable cohort (both a machine prediction and a human taxon), not all verified-with-prediction rows. A comment-only verification (Identification.taxon is nullable) has a prediction but no human label, so it can neither agree nor disagree and is excluded from both numerator and denominator.
• Reuses apply_default_filters so apply_defaults=false bypasses project default taxa lists + score thresholds.
• agreement_coarsest_rank is validated with a DRF ChoiceField through SingleParamSerializer: blank, unknown, and UNKNOWN values all 400 at the boundary, and the choices propagate to the OpenAPI schema as an enum.
• *_pct fields are bounded to [0.0, 1.0] in the serializer; cohens_kappa to [-1.0, 1.0].
• Wilson CI + Cohen's kappa are pure-Python over the verified_rows list already materialized for the agreement counts — one extra pass, zero extra DB queries.

Bench

Project 18 (43,149 occurrences, 45 verified): 928ms → 350ms cold / 146ms warm after scoping subqueries to the verified set.

Across all production projects with non-zero identifications:

Project	Total	Verified	Cold	Warm
P#85 SEC-SEQ AI Symposium	36,253	13,140	1.18s	343ms
P#20 Barro Colorado Island	40,958	1,351	0.92s	153ms
P#84 Pennsylvania Tebufenozide	18,407	251	0.56s	139ms
P#24 Atlantic Forestry Centre	2,797	274	0.50s	203ms
P#46 AMBER - Panama	10,248	48	0.44s	125ms
P#23 Insectarium de Montréal	20,393	74	0.43s	190ms
P#16 Aarhus Ecoscience	644	8	0.38s	128ms
P#18 Vermont Atlas of Life	43,149	45	0.35s	146ms
P#49 Marc Bélisle's Lab	439	71	0.32s	128ms
P#38 MothBox - Gamboa	21	10	0.31s	122ms

Pre-rework state on project 18 with apply_defaults=false: 159s curl timeout.

Test plan

• ami.main.tests.TestLcaRankBetween — 7 unit tests including UNKNOWN-rank regression.
• ami.main.tests.TestModelAgreementForProject — empty-project + 4-bucket canonical case + coarsest_rank threshold filtering + taxon-less verification excluded from the comparable denominator.
• ami.main.tests.TestOccurrenceStatsViewSet — HTTP coverage for envelope shape, draft-project 404, filter passthrough, apply_defaults=false bypass, exact-match happy path, sister-species any-rank bucket, invalid rank → 400, UNKNOWN rejection, blank rank → 400, threshold echo, CI/kappa field presence + null-on-empty, and OPTIONS returning the response field schema with help_text.
• ami.main.tests.TestWilsonInterval + ami.main.tests.TestCohensKappa — 10 pure-Python unit tests (textbook 8/10 CI value, unit-interval clamping, interval-tightens-with-n, out-of-range successes raises; kappa known 2×2 value, perfect agreement, single-category-undefined, negative kappa, empty).
• Full model-agreement suite green locally: 41/41 (TestLcaRankBetween, TestModelAgreementForProject, TestOccurrenceStatsViewSet, TestWilsonInterval, TestCohensKappa).
• Live smoke against project 18 + 9 other production projects, including the heaviest verified-set (P#85, 13,140 idents).

This PR is backend-only. The frontend consumer — the useModelAgreement hook + the occurrence-list stats panel — lives in #1308.

Follow-ups (out of scope, calling out for next rounds)

apply_default_filters is the dominant cost on hot stats paths

Update:#1317 (taxa verification counts) addresses part of this — its perf rewrite (precompute + CASE aggregation) eliminated a 30s timeout on the per-taxon stats path that has the same root cause (the apply_default_filters + valid() stack being applied per-row). The compound index + GIN index below are still outstanding follow-ups.

For the heaviest project (P#85, 36k post-filter occurrences) the agreement subqueries on the verified set run in <50ms. The rest of the response time is the apply_default_filters + valid() filter stack on Occurrence. EXPLAIN ANALYZE on P#85 reveals:

1. No compound index on (project_id, determination_score) — Postgres does a Parallel Seq Scan on main_occurrence and discards 195,549 rows by filter to find 36,253 matching the project-default score threshold. Hot path: ~60ms.
2. No GIN index on Taxon.parents_json — for projects with default include/exclude taxa lists, the parents_json__contains JSONB containment check is a row-by-row evaluation. P#85 has no taxa lists so this didn't show up here, but it would dominate for projects that do (e.g. via OccurrenceFilter's recursive taxa filter).
3. valid()'s anti-join to main_detection is fine (index scan, 36k loops on hot cache, <60ms).

This affects every endpoint that calls apply_default_filters() or Occurrence.objects.valid() — /occurrences/, /captures/, /events/, and the other stats actions on this viewset. Anywhere a project default threshold is non-zero, the same seq-scan is happening.

Note: the entire bench table above was measured without these indexes (they don't exist yet), so those numbers are the no-index baseline — worst case 1.18s cold / 343ms warm. That's acceptable for a cached dashboard widget, so the indexes are a follow-up, not a blocker for this PR.

Likely cheap wins:

• CREATE INDEX CONCURRENTLY main_occurrence_project_score_idx ON main_occurrence (project_id, determination_score) — index range scan instead of seq-scan-then-filter.
• CREATE INDEX CONCURRENTLY main_taxon_parents_json_gin_idx ON main_taxon USING gin (parents_json jsonb_path_ops) — index lookup for parents_json__contains instead of full-row JSONB eval.

Filter-driven occurrence exports

This PR's filter parity wiring (OCCURRENCE_FILTER_BACKENDS + OCCURRENCE_FILTERSET_FIELDS) sets up a natural follow-up: let users click "Export" on /occurrences/ with the current filters applied and get a job whose output matches that filtered set, without first materializing a SourceImageCollection. The export infra already has a "filters JSON → re-run backends in worker" pattern (ami/exports/utils.py:13-72generate_fake_request() + apply_filters()) but is hardwired to OccurrenceCollectionFilter. Wiring the same shared backend tuple into the exporter would close the gap.

Explicit auth gate on stats viewset

OccurrenceStatsViewSet uses IsActiveStaffOrReadOnly. verified_by_me=true from an anon caller is safe today only because OccurrenceVerifiedByMeFilter.filter_queryset short-circuits on is_authenticated. Worth an explicit gate at the viewset level rather than relying on the filter's internal short-circuit.

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

• New Features
  • Added a new API endpoint to retrieve agreement metrics between human and machine predictions, including agreement rates, confidence intervals, and statistical measures like Cohen's kappa. Optional filtering by coarser taxonomic ranks is now supported.

[netlify]

✅ Deploy Preview for antenna-preview canceled.

Name	Link
🔨 Latest commit	b65100f
🔍 Latest deploy log	https://app.netlify.com/projects/antenna-preview/deploys/6a18c5355986f30008ecf677

[netlify]

✅ Deploy Preview for antenna-ssec canceled.

Name	Link
🔨 Latest commit	b65100f
🔍 Latest deploy log	https://app.netlify.com/projects/antenna-ssec/deploys/6a18c535070aa90008719159

[coderabbitai]

Caution

Review failed

Failed to post review comments

📝 Walkthrough

Walkthrough

This PR adds a stats endpoint /occurrences/stats/model-agreement/ that computes verified-occurrence counts and human–model agreement metrics (exact, any-rank, optional coarser-rank) over filtered occurrences, plus Wilson 95% CIs and Cohen's kappa, with supporting utilities, serializer, tests, and docs update.

Changes

Model Agreement Stats Endpoint

Layer / File(s)	Summary
Statistical utility functions ami/utils/stats.py	New module adds WILSON_Z_95, wilson_interval() for 95% Wilson confidence bounds on proportions, and cohens_kappa() for inter-rater agreement; both return rounded, bounded results or None when undefined.
Taxonomic LCA and agreement computation ami/main/models_future/occurrence.py	Adds TaxonTuple type, lca_rank_between() to find deepest shared non-UNKNOWN rank, and model_agreement_for_project() to compute deduplicated totals, verified subsets, exact/any/coarser-rank agreement counts/percents, Wilson CIs, and Cohen's kappa.
API response serializer ami/main/api/serializers.py	Adds ModelAgreementSerializer describing project-scoped totals, verified counts/pcts, exact and any-rank agreement counts/pcts with nullable Wilson CI fields, nullable Cohen's kappa, and optional coarser-rank agreement fields keyed to agreement_coarsest_rank.
API endpoint wiring and exposure ami/main/api/views.py	Centralizes occurrence filter settings, adds filter plumbing to OccurrenceStatsViewSet, and implements a non-paginated model_agreement action that validates project visibility and agreement_coarsest_rank, applies default and DRF filters, computes agreement via model_agreement_for_project, and returns serialized results.
OPTIONS response metadata ami/base/metadata.py	Adds ResponseSchemaMetadata to include the response serializer's field schema under actions["GET"] in OPTIONS when GET is allowed and a serializer can be instantiated.
Tests ami/main/tests.py	Adds unit tests for LCA, Wilson interval, Cohen's kappa, model_agreement_for_project, and comprehensive endpoint tests for /api/v2/occurrences/stats/model-agreement/ (validation, filtering, coarsest-rank handling, CI/kappa presence, and OPTIONS schema).
Docs docs/claude/reference/api-stats-pattern.md	Documents the response-field-via-OPTIONS pattern and updates the example endpoint path to GET /occurrences/stats/model-agreement/.

Sequence Diagram

sequenceDiagram
participant Client
participant ViewSet as OccurrenceStatsViewSet
participant DRFFilter as DRF Filters
participant Logic as model_agreement_for_project
participant Serializer as ModelAgreementSerializer
Client->>ViewSet: GET /stats/model-agreement?project_id=X
ViewSet->>ViewSet: Validate project visibility
ViewSet->>ViewSet: Parse & validate agreement_coarsest_rank
ViewSet->>ViewSet: Build base queryset with defaults
ViewSet->>DRFFilter: filter_queryset()
DRFFilter->>ViewSet: Filtered QuerySet
ViewSet->>Logic: Compute agreement metrics
Logic->>Logic: Deduplicate, count verified/unverified
Logic->>Logic: Compute exact & any-rank agreement
Logic->>Logic: Calculate rates, Wilson CIs, Cohen's kappa
Logic->>ViewSet: Agreement dict
ViewSet->>Serializer: Serialize result
Serializer->>Client: JSON response (counts, rates, CIs, kappa)

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related issues

• Add guidance and stats about which taxa need verification #1316: Related — proposes per-taxon agreement/verification metrics and could reuse model_agreement_for_project() and lca_rank_between() introduced here.

Poem

🐰 I nudge through taxon trees with nimble feet,

Counting matches where human and model meet.
Wilson bounds hush the noisy debate,
Kappa whispers chance we should negate —
A hop, a stat, agreement looks sweet!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'Endpoint for stats about verified occurrences' is clear and directly describes the main change—a new stats endpoint for agreement metrics over verified occurrences.
Description check	✅ Passed	The PR description comprehensively covers all required sections: summary, list of changes, detailed implementation notes, comprehensive test plan, benchmarks, deployment notes, usage examples, and documented follow-ups.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/human-model-agreement-endpoint

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Pull request overview

Adds a new scalar stats endpoint GET /occurrences/stats/human-model-agreement/ that reports verified-occurrence and human-vs-model agreement rates (exact and "under-order") for an occurrence queryset, reusing the existing /occurrences/ filter stack and apply_default_filters.

Changes:

• New aggregation helper human_model_agreement_for_project plus pure-Python lca_rank_between over Taxon.parents_json in ami/main/models_future/occurrence.py.
• New action on OccurrenceStatsViewSet plus HumanModelAgreementSerializer; extracts OccurrenceViewSet filter backends/fields into module-level tuples (OCCURRENCE_FILTER_BACKENDS, OCCURRENCE_FILTERSET_FIELDS) shared by both viewsets.
• React Query hook useHumanModelAgreement and supporting planning/scoping docs.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
ami/main/models_future/occurrence.py	New LCA helper + Python-side aggregation function over a pre-filtered Occurrence queryset.
ami/main/api/views.py	Extracts shared occurrence filter config and adds human_model_agreement action on OccurrenceStatsViewSet.
ami/main/api/serializers.py	New HumanModelAgreementSerializer describing the response shape.
ami/main/tests.py	Unit tests for lca_rank_between, aggregation tests, and HTTP-level tests for the new action.
ui/src/data-services/hooks/occurrences/stats/useHumanModelAgreement.ts	New typed React Query hook for the endpoint.
docs/claude/planning/2026-05-14-human-model-agreement-endpoint.md	Implementation plan document for the feature.
docs/claude/planning/occurrence-filter-driven-exports.md	Side-research scoping stub for filter-driven exports (out of scope of this PR).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[coderabbitai]

Actionable comments posted: 2

🧹 Nitpick comments (3)

ami/main/api/serializers.py (1)

1765-1769: ⚡ Quick win

Constrain percentage fields to the documented 0.0..1.0 range

These fields are contractually bounded; adding serializer bounds gives fast failure on accidental regressions and keeps response validation self-documenting.

Proposed diff

- verified_pct = serializers.FloatField(help_text="verified_count / total_occurrences")+ verified_pct = serializers.FloatField(+ min_value=0.0,+ max_value=1.0,+ help_text="verified_count / total_occurrences",+ )
@@
- agreed_exact_pct = serializers.FloatField(help_text="agreed_exact_count / verified_count")+ agreed_exact_pct = serializers.FloatField(+ min_value=0.0,+ max_value=1.0,+ help_text="agreed_exact_count / verified_count",+ )
@@
- agreed_under_order_pct = serializers.FloatField(help_text="agreed_under_order_count / verified_count")+ agreed_under_order_pct = serializers.FloatField(+ min_value=0.0,+ max_value=1.0,+ help_text="agreed_under_order_count / verified_count",+ )

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ami/main/api/serializers.py` around lines 1765 - 1769, The percentage fields
verified_pct, agreed_exact_pct, and agreed_under_order_pct are currently
unbounded; update their declarations to add validation bounds (min_value=0.0,
max_value=1.0) on the serializers.FloatField instances so the serializer
enforces the documented 0.0..1.0 contract and fails fast on invalid values.

ui/src/data-services/hooks/occurrences/stats/useHumanModelAgreement.ts (2)

4-13: 💤 Low value

Consider renaming Response to avoid shadowing the global DOM type.

Response is the name of the global fetch response type. Shadowing it at module scope is harmless today but creates a foot-gun if anyone later references the DOM Response in this file. A domain-prefixed name (e.g., HumanModelAgreementResponse) is clearer at call sites too.

♻️ Proposed rename

-interface Response {+interface HumanModelAgreementResponse {
project_id: number
total_occurrences: number
verified_count: number
verified_pct: number
agreed_exact_count: number
agreed_exact_pct: number
agreed_under_order_count: number
agreed_under_order_pct: number
}
@@
- const { data, isLoading, isFetching, error } = useAuthorizedQuery<Response>({+ const { data, isLoading, isFetching, error } = useAuthorizedQuery<HumanModelAgreementResponse>({

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ui/src/data-services/hooks/occurrences/stats/useHumanModelAgreement.ts`
around lines 4 - 13, Rename the module-scoped interface named Response to a
domain-specific name (e.g., HumanModelAgreementResponse) to avoid shadowing the
global DOM Response type; update the interface declaration and all references to
it within useHumanModelAgreement.ts (and any exported types/imports) so code
that needs the DOM Response can still reference it unambiguously and call sites
use the new HumanModelAgreementResponse identifier.

---

20-32: ⚡ Quick win

Single-value filter map drops multi-value query params (e.g., algorithm).

OccurrenceAlgorithmFilter reads algorithm and not_algorithm via request.query_params.getlist(...) on the backend, so callers can legitimately pass multiple algorithm IDs. The current Record<string, string | number | boolean | undefined> plus params.set(...) collapses any such filter to a single value, so this hook can't fully reproduce the /occurrences/ filter set the PR objectives describe.

Consider widening the value type and switching to append per item:

♻️ Proposed change

 export const useHumanModelAgreement = (
projectId?: string,
- filters?: Record<string, string | number | boolean | undefined>+ filters?: Record<+ string,+ string | number | boolean | Array<string | number> | undefined+ >
) => {
const url = `${API_URL}/${API_ROUTES.OCCURRENCES}/stats/human-model-agreement/`
const params = new URLSearchParams()
if (projectId) params.set('project_id', projectId)
if (filters) {
Object.entries(filters).forEach(([key, value]) => {
- if (value !== undefined && value !== '' && value !== null) {- params.set(key, String(value))- }+ if (value === undefined || value === null || value === '') return+ if (Array.isArray(value)) {+ value.forEach((v) => {+ if (v !== undefined && v !== null && v !== '') {+ params.append(key, String(v))+ }+ })+ } else {+ params.set(key, String(value))+ }
})
}

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ui/src/data-services/hooks/occurrences/stats/useHumanModelAgreement.ts`
around lines 20 - 32, The hook useHumanModelAgreement currently types filters as
Record<string, string | number | boolean | undefined> and calls params.set(...),
which collapses multi-value query params; update the filters param type to allow
string[] (e.g., Record<string, string | number | boolean | string[] |
undefined>) and when iterating Object.entries(filters) detect arrays and call
params.append(key, String(item)) for each element (fall back to params.set for
single values), ensuring multi-value keys like "algorithm" and "not_algorithm"
are preserved in the generated URL.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@ami/main/models_future/occurrence.py`:
- Line 187: The code eagerly materializes the entire QuerySet into memory by
doing occurrences = list(qs); change this to a memory-safe iteration or paging
approach: replace the full list() with chunked processing using
qs.iterator(chunk_size=1000) or loop over qs in paginated batches (e.g.,
Paginator or manual offset/limit) and aggregate/write results per-chunk, and
avoid prefetching everything at once (adjust or remove the prefetch_related on
identifications or use values()/only()/defer() to limit fetched fields) so
memory usage stays bounded; update any downstream logic that expects a full list
to work with incremental processing or collect results into a streaming response
instead.
In `@docs/claude/planning/2026-05-14-human-model-agreement-endpoint.md`:
- Around line 26-43: The fenced code block listing project files lacks a
language tag (triggering markdownlint MD040); update the opening fence for the
block that contains entries like "ami/ ... occurrence.py # ADD:
human_model_agreement_for_project(), _lca_rank_of() helper", "serializers.py #
ADD: HumanModelAgreementSerializer", and "useHumanModelAgreement.ts # ADD: typed
React Query hook" to include a language identifier (e.g., ```text) so the block
is explicitly labeled; keep the same block content and closing fence unchanged.
---
Nitpick comments:
In `@ami/main/api/serializers.py`:
- Around line 1765-1769: The percentage fields verified_pct, agreed_exact_pct,
and agreed_under_order_pct are currently unbounded; update their declarations to
add validation bounds (min_value=0.0, max_value=1.0) on the
serializers.FloatField instances so the serializer enforces the documented
0.0..1.0 contract and fails fast on invalid values.
In `@ui/src/data-services/hooks/occurrences/stats/useHumanModelAgreement.ts`:
- Around line 4-13: Rename the module-scoped interface named Response to a
domain-specific name (e.g., HumanModelAgreementResponse) to avoid shadowing the
global DOM Response type; update the interface declaration and all references to
it within useHumanModelAgreement.ts (and any exported types/imports) so code
that needs the DOM Response can still reference it unambiguously and call sites
use the new HumanModelAgreementResponse identifier.
- Around line 20-32: The hook useHumanModelAgreement currently types filters as
Record<string, string | number | boolean | undefined> and calls params.set(...),
which collapses multi-value query params; update the filters param type to allow
string[] (e.g., Record<string, string | number | boolean | string[] |
undefined>) and when iterating Object.entries(filters) detect arrays and call
params.append(key, String(item)) for each element (fall back to params.set for
single values), ensuring multi-value keys like "algorithm" and "not_algorithm"
are preserved in the generated URL.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: a97a61f4-518e-4bf7-b6b5-fb325dc4e97d

📥 Commits

Reviewing files that changed from the base of the PR and between aeb57c1 and b81a987.

📒 Files selected for processing (7)

• ami/main/api/serializers.py
• ami/main/api/views.py
• ami/main/models_future/occurrence.py
• ami/main/tests.py
• docs/claude/planning/2026-05-14-human-model-agreement-endpoint.md
• docs/claude/planning/occurrence-filter-driven-exports.md
• ui/src/data-services/hooks/occurrences/stats/useHumanModelAgreement.ts

[annavik]

Oh yes!!

[coderabbitai]

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

ui/src/data-services/hooks/occurrences/stats/useModelAgreement.ts (1)

22-33: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Support repeated query params for multi-select filters.

Record<string, primitive> + params.set(...) drops repeated keys, so multi-value filters (e.g., repeated algorithm / not_algorithm) can’t be forwarded faithfully from occurrence filters.

💡 Proposed fix

-export const useModelAgreement = (- projectId?: string,- filters?: Record<string, string | number | boolean | undefined>-) => {+type FilterPrimitive = string | number | boolean+type FilterValue = FilterPrimitive | FilterPrimitive[] | null | undefined++export const useModelAgreement = (+ projectId?: string,+ filters?: Record<string, FilterValue>+) => {
const url = `${API_URL}/${API_ROUTES.OCCURRENCES}/stats/model-agreement/`
const params = new URLSearchParams()
if (projectId) params.set('project_id', projectId)
if (filters) {
Object.entries(filters).forEach(([key, value]) => {
- if (value !== undefined && value !== '' && value !== null) {- params.set(key, String(value))- }+ if (Array.isArray(value)) {+ value.forEach((item) => {+ if (item !== undefined && item !== '' && item !== null) {+ params.append(key, String(item))+ }+ })+ return+ }+ if (value !== undefined && value !== '' && value !== null) {+ params.set(key, String(value))+ }
})
}
+ const queryString = params.toString()
const { data, isLoading, isFetching, error } =
useAuthorizedQuery<ModelAgreementResponse>({
queryKey: [
API_ROUTES.OCCURRENCES,
'stats',
'model-agreement',
projectId,
- filters,+ queryString,
],
- url: `${url}?${params.toString()}`,+ url: `${url}?${queryString}`,
})

Also applies to: 38-46

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ui/src/data-services/hooks/occurrences/stats/useModelAgreement.ts` around
lines 22 - 33, The current implementation converts multi-value filters into a
Record<string, primitive> and uses params.set(...), which overwrites duplicate
query keys and loses multi-select filters; update the filters type to allow
arrays (e.g., Record<string, string | number | boolean | string[] | undefined>),
and when building URLSearchParams switch to using params.append(...) for
repeated values: if the filter value is an array loop and params.append(key,
String(v)) for each entry, otherwise call params.append(key, String(value));
ensure the same change is applied in the other occurrence mentioned (the block
around the second params handling at lines ~38-46). Use the existing params and
filters identifiers so the change is localized.

🧹 Nitpick comments (1)

docs/claude/prompts/NEXT_SESSION_PROMPT.md (1)

1-86: ⚡ Quick win

Planning document appears stale and may confuse future readers.

This file is titled "Next session" and describes tasks "for this session" (lines 7-68), but according to the PR objectives summary, the work described here has already been completed:

• Renaming from "human-model-agreement" to "model-agreement" ✓
• SQL aggregation push ✓
• UNKNOWN rank bug fix ✓
• Denominator fix (verified_with_prediction_count) ✓

Including a "NEXT_SESSION_PROMPT" document that describes already-completed work as if it's pending creates confusion for future developers who might try to execute these tasks again or wonder what state the codebase is in.

Additionally, line 5 references the old endpoint URL that will 404 after the renaming.

Consider one of:

1. Archive/rename this to docs/claude/planning/2026-05-14-session-notes-pr-1307.md (historical record, past tense)
2. Remove it if the other planning doc at line 18 already serves as the historical record
3. Add a header clearly stating "Historical planning document - work completed in commits X-Y"

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/claude/prompts/NEXT_SESSION_PROMPT.md` around lines 1 - 86, The
"NEXT_SESSION_PROMPT.md" planning doc is stale and misleading; update it by
either (a) renaming/archiving it (e.g., to
docs/claude/planning/2026-05-14-session-notes-pr-1307.md) and leaving as
historical record, (b) deleting it if redundant, or (c) editing the top of
NEXT_SESSION_PROMPT.md to a clear "Historical planning document — work completed
in commits <sha-range>" header and update/remove the old endpoint URL reference;
ensure you touch the file named NEXT_SESSION_PROMPT.md and fix the line that
references the old endpoint URL
(http://localhost:8000/api/v2/occurrences/stats/human-model-agreement/?) so it
no longer points to a non-existent route and include the completed-commits SHAs
or a pointer to the merged PR in the header.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@ami/main/models_future/occurrence.py`:
- Around line 201-213: The count is taken from the raw queryset (total =
queryset.count()) but the verified branch uses a deduped queryset (.distinct()),
so duplicates in the incoming queryset can inflate total; change to operate on a
deduplicated base queryset (e.g., replace/count using queryset.distinct() or
assign deduped = queryset.distinct() and use deduped for total and downstream
operations like the block that builds verified_rows and any other aggregations)
so that total, verified_rows and agreement numerators use the same deduplicated
set (refer to total, verified_rows and the use of .distinct() in this
file/function).
In `@docs/claude/prompts/NEXT_SESSION_PROMPT.md`:
- Line 86: The TODO about updating MEMORY.md is incomplete—either perform the
update or remove/clarify the note: add a new entry named
project_pr_1307_human_model_agreement.md into MEMORY.md summarizing the current
PR state (references to PR `#1307`, the plan doc at
docs/claude/planning/occurrence-filter-driven-exports.md, and the exported
stub), or delete the parenthetical “(TODO this session start)” and replace it
with a clear status line (e.g., “updated” or “needs follow-up”) so the commit
message and NEXT_SESSION_PROMPT.md reflect an accurate, actionable state.
---
Outside diff comments:
In `@ui/src/data-services/hooks/occurrences/stats/useModelAgreement.ts`:
- Around line 22-33: The current implementation converts multi-value filters
into a Record<string, primitive> and uses params.set(...), which overwrites
duplicate query keys and loses multi-select filters; update the filters type to
allow arrays (e.g., Record<string, string | number | boolean | string[] |
undefined>), and when building URLSearchParams switch to using
params.append(...) for repeated values: if the filter value is an array loop and
params.append(key, String(v)) for each entry, otherwise call params.append(key,
String(value)); ensure the same change is applied in the other occurrence
mentioned (the block around the second params handling at lines ~38-46). Use the
existing params and filters identifiers so the change is localized.
---
Nitpick comments:
In `@docs/claude/prompts/NEXT_SESSION_PROMPT.md`:
- Around line 1-86: The "NEXT_SESSION_PROMPT.md" planning doc is stale and
misleading; update it by either (a) renaming/archiving it (e.g., to
docs/claude/planning/2026-05-14-session-notes-pr-1307.md) and leaving as
historical record, (b) deleting it if redundant, or (c) editing the top of
NEXT_SESSION_PROMPT.md to a clear "Historical planning document — work completed
in commits <sha-range>" header and update/remove the old endpoint URL reference;
ensure you touch the file named NEXT_SESSION_PROMPT.md and fix the line that
references the old endpoint URL
(http://localhost:8000/api/v2/occurrences/stats/human-model-agreement/?) so it
no longer points to a non-existent route and include the completed-commits SHAs
or a pointer to the merged PR in the header.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c229eb53-6f25-4d87-bd0b-622992ee75eb

📥 Commits

Reviewing files that changed from the base of the PR and between b81a987 and 0924027.

📒 Files selected for processing (8)

• ami/main/api/serializers.py
• ami/main/api/views.py
• ami/main/models_future/occurrence.py
• ami/main/tests.py
• docs/claude/planning/2026-05-14-human-model-agreement-endpoint.md
• docs/claude/prompts/NEXT_SESSION_PROMPT.md
• docs/claude/reference/api-stats-pattern.md
• ui/src/data-services/hooks/occurrences/stats/useModelAgreement.ts

✅ Files skipped from review due to trivial changes (1)

• docs/claude/planning/2026-05-14-human-model-agreement-endpoint.md

[mihow]

Claude says: Pushed two commits addressing this round of CodeRabbit feedback + a follow-on design extension the user signed off on.

Code changes (0565f06) — feat(occurrence-stats): drop ORDER threshold; add coarsest_rank query param

• Drops the hardcoded lca >= TaxonRank.ORDER agreement gate. Replaces agreed_under_order_* with agreed_any_rank_* (exact + any non-null LCA at a real rank; UNKNOWN still excluded). Idea: the upstream filter (e.g. a Lepidoptera include list) already bounds the meaningful scope, so a fixed threshold in this function is unnecessary noise.
• Adds optional ?agreement_coarsest_rank=<RANK> for callers who do want an explicit floor. When supplied, response also includes agreed_coarser_rank_* (exact + LCAs at-or-deeper-than the threshold). Always echoes the applied rank in agreement_coarsest_rank so consumers can disambiguate. Invalid rank or UNKNOWN → 400.
• Addresses CodeRabbit:
  • Dedupes base queryset (queryset.distinct()) so default-filter joins (verified_by_me, taxa_list_id) can't inflate total_occurrences vs the verified branch.
  • Bounds the *_pct FloatField serializers to [0.0, 1.0].
• Tests: existing buckets renamed; new cases for the coarsest-rank threshold filtering shallow LCAs, invalid rank → 400, UNKNOWN rejection, and the threshold echo.

FE follow-up (4a92c0b) — feat(ui): align model-agreement hook with BE rename + multi-value query params

• Renames agreed_under_order_* → agreed_any_rank_* in the hook's response type; adds the optional coarser-rank fields.
• Widens filters to accept arrays and uses params.append(...) so multi-value filters (algorithm, not_algorithm — backend reads via request.query_params.getlist(...)) survive instead of getting collapsed by params.set.

Tests: 19/19 in TestModelAgreementForProject + TestOccurrenceStatsViewSet pass (--keepdb). Live render on localhost:4000 still shows VERIFIED 0% / AGREEMENT 71% on P18.

Stale NEXT_SESSION_PROMPT.md flagged in the latest review is deferred — useful as narrative for this rename and not on the production path.

🤖 Generated with Claude Code

[mihow]

Claude says: Follow-up from #1316 / #1317, which builds the per-taxon version of this. Two findings here that are relevant to this PR:

1. The parents_json GIN index follow-up — shipped, but it isn't the fix.

#1316 carried over the suggestion to add a GIN index on Taxon.parents_json (jsonb_path_ops) as the cure for slow higher-rank (Family/Order) rollups. We shipped it (migration 0085). Measured outcome: with the index in place, the per-taxon rollup still timed out (30s statement limit) on a production-scale project (~1k observed taxa, ~17k occurrences).

Reason: the rollup ran as a correlated subquery with parents_json @> [{"id": OuterRef(...)}], and a GIN index can't serve a containment whose right-hand side is an OuterRef — it's a per-row parameter, not a constant. So it degraded to a per-row scan, run once per page row plus once per taxon in the pagination COUNT. The index is still worth keeping (it serves the literal-RHS containment filters: occurrence-list taxon=<id>, the default-taxa filter), but it does not fix correlated rollups.

2. The pattern that did fix it — relevant to this endpoint's in-memory concern.

The reviews here flagged model_agreement_for_project materializing the full filtered queryset into Python on large projects. The approach we landed in #1317 sidesteps both problems at once: the counts only concern verified occurrences (those with a non-withdrawn Identification), which are sparse — bounded by human review effort, not total occurrences. Compute the rollup in a single pass over just that small set → constant-time CASE annotations (and id__in for the membership filter). Same project went from 30s timeout to ~0.3s across the default list / verified=true|false / ordering=verified_count. Cost is O(verified occurrences × ancestor depth), not O(all occurrences) — might be a cheaper path for this endpoint's large-project case too.

Writeup with the anti-pattern vs pattern and the gotchas (pagination COUNT annotation stripping, cachalot polluting repeated benchmarks, django-pydantic-field.values() returning pydantic objects rather than dicts): docs/claude/reference/hierarchical-rollup-query-performance.md (added in #1317).

[mihow]

Claude says: Nice-to-have follow-up from the closed CodeRabbit PR #1325: extract an EnumChoiceField helper into ami/base/serializers.py — a reusable enum-backed ChoiceField that accepts member names case-insensitively and supports excluding sentinel values like UNKNOWN.

Right now the agreement_coarsest_rank param is validated inline in model_agreement (ami/main/api/views.py, ~line 1463). That works and gives strict 400s for blank/unknown/UNKNOWN, but EnumChoiceField would DRY it up and become worthwhile once a second enum-backed query param shows up. Good first issue — small scope, easy to test (case-insensitive match, exclude rejection, blank-value 400) and review. Not blocking anything; parking it here so it doesn't get lost.