docs(social): the scrape contract — per-platform posts, what one scrape stores, post engagement, follower history (app#2018)

[sweetmantech]

Contract PR for recoupable/app#2018 (scrape persistence keystone). Absorbs the docs rows of app#2017 (posts per platform) and app#2026 (history on /socials).

What changes

POST /api/socials/{id}/scrape (social.json)

• Credits formula (5 + posts), a per-platform table for posts (honored / actor parameter / default when omitted), a "What gets stored" list (profile, daily follower snapshot, posts with engagement, Instagram comments + commenter enrichment) and the one-hop rule for fan discovery.
• Corrects the old text: startInstagramProfileScraping never forwards posts, so Instagram ignores it and always returns the profile's latest posts (up to 12); TikTok honors it (resultsPerPage). The previous "other platforms ignore this parameter" was wrong for TikTok and silent about Instagram.
• posts parameter description shortened to point at the table.

GET /api/artists/{id}/posts (social.json)

• ArtistPost gains nullable views, likes, comments, reposts, each with the per-platform source field (e.g. TikTok playCount, X viewCount); description states they are the values at the last scrape and null where the platform does not report them.

GET /api/artists/{id}/socials (releases.json)

• New history=<days> query (1–90) returning history: SocialSnapshot[] per profile, newest first, one point per social per day (latest scrape that day wins). New SocialSnapshot schema (captured_at, follower_count, following_count, post_count).
• Documents page / limit (the validator already accepts them) and the real 401 / 403 / 404 the route returns; previously only 400 was listed.

POST /api/artist/socials/scrape (releases.json) — posts description aligned with the single-social page, credits per profile stated.

GET /api/artists/{id}/fans (releases.json) — states that fans are commenters on scraped posts (Instagram today) and that the commenter profile run fills avatar/bio/follower counts. (The per-platform coverage table stays with app#2024.)

Merge order

docs (this) → database (social_snapshots, posts engagement columns, apify_scraper_runs lineage) → api keystone → api budget. This PR is the contract the api PR is tested against.

Edits were applied as anchored text replacements (the files do not round-trip byte-identically through json.dumps); both files re-parse.

🤖 Generated with Claude Code

https://claude.ai/code/session_012PS8hmiwR1rGD6c41n6gD8

---

Summary by cubic

Defines the scrape persistence contract in the OpenAPI specs for recoupable/app#2018: per-platform post depth, what one scrape stores, post engagement fields, and follower history.

What changed

• POST /api/socials/{id}/scrape now documents the credits formula (5 + posts) and a per-platform table; Instagram ignores posts and always returns the latest 12 posts, while TikTok honors it — correcting previously wrong text.
• ArtistPost gains nullable views, likes, comments, and reposts, each with the per-platform source field.
• GET /api/artists/{id}/socials adds history=<days> (1–90) returning SocialSnapshot[] per profile and documents the real 401/403/404 responses; following_count and post_count are null where the platform does not report them.
• GET /api/artists/{id}/fans now states fans are commenters on scraped posts (Instagram today) and that the enrichment pass fills avatar, bio, and follower and following counts.
• The batch scrape posts description is aligned with the single-social endpoint.

^{Written for commit b147e63. Summary will update on new commits.}

[coderabbitai]

Warning

Review limit reached

Next included review available in 28 minutes.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 6f410e62-2e66-4462-9034-63fbe63b458b

📥 Commits

Reviewing files that changed from the base of the PR and between 1bc3f0a and b147e63.

📒 Files selected for processing (2)

• api-reference/openapi/releases.json
• api-reference/openapi/social.json

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[cubic-dev-ai]

3 issues found across 2 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="api-reference/openapi/releases.json">
<violation number="1" location="api-reference/openapi/releases.json:1187">
P3: The new fans description claims the commenter profile run fills `following_count`, but the Scrape page's "What gets stored" bullet in the same PR says it fills only avatar, bio and follower count. These two statements conflict; a consumer reading either could be misled. Reconcile them so both docs state the same set of filled fan fields.</violation>
<violation number="2" location="api-reference/openapi/releases.json:4483">
P3: SocialSnapshot.following_count and post_count are nullable but not in `required`, so consumers can't rely on them being present, while the `history` parameter description promises every snapshot stores follower, following and post counts. Since the descriptions say these are null (not absent) where a platform doesn't report them, add both to `required` so the schema matches the contract text.</violation>
</file>
<file name="api-reference/openapi/social.json">
<violation number="1" location="api-reference/openapi/social.json:1467">
P3: The field is named `updated_at`, but this description now says the value is when the post was *published* on the platform. That clashes with the other `updated_at` fields in these specs (`SocialProfile.updated_at` = last updated, `SocialPost.updated_at` = last updated). Either keep the field semantic consistent with its name, or rename the property to reflect that it carries the publish timestamp, so consumers aren't misled about what refresh time it represents.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P3: The new fans description claims the commenter profile run fills following_count, but the Scrape page's "What gets stored" bullet in the same PR says it fills only avatar, bio and follower count. These two statements conflict; a consumer reading either could be misled. Reconcile them so both docs state the same set of filled fan fields.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At api-reference/openapi/releases.json, line 1187:
<comment>The new fans description claims the commenter profile run fills `following_count`, but the Scrape page's "What gets stored" bullet in the same PR says it fills only avatar, bio and follower count. These two statements conflict; a consumer reading either could be misled. Reconcile them so both docs state the same set of filled fan fields.</comment>
<file context>
@@ -1121,7 +1184,7 @@
"/api/artists/{id}/fans": {
"get": {
- "description": "Retrieve all social profiles from fans of an artist across all platforms. This endpoint aggregates fan data from all connected social media platforms. Supports pagination for large fan lists.",
+ "description": "Retrieve the social profiles of an artist's fans: the accounts that commented on the artist's scraped posts (Instagram today). Every Instagram scrape is followed by one batched profile run over the new commenters, which fills `avatar`, `bio`, `follower_count` and `following_count` on each fan; `region` is null unless the platform reports one. Supports pagination for large fan lists.",
"parameters": [
{
</file context>

[cubic-dev-ai]

P3: SocialSnapshot.following_count and post_count are nullable but not in required, so consumers can't rely on them being present, while the history parameter description promises every snapshot stores follower, following and post counts. Since the descriptions say these are null (not absent) where a platform doesn't report them, add both to required so the schema matches the contract text.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At api-reference/openapi/releases.json, line 4483:
<comment>SocialSnapshot.following_count and post_count are nullable but not in `required`, so consumers can't rely on them being present, while the `history` parameter description promises every snapshot stores follower, following and post counts. Since the descriptions say these are null (not absent) where a platform doesn't report them, add both to `required` so the schema matches the contract text.</comment>
<file context>
@@ -4391,6 +4454,41 @@
+ "type": "integer",
+ "description": "Followers (subscribers on YouTube) at capture time"
+ },
+ "following_count": {
+ "type": "integer",
+ "nullable": true,
</file context>

[cubic-dev-ai]

P3: The field is named updated_at, but this description now says the value is when the post was published on the platform. That clashes with the other updated_at fields in these specs (SocialProfile.updated_at = last updated, SocialPost.updated_at = last updated). Either keep the field semantic consistent with its name, or rename the property to reflect that it carries the publish timestamp, so consumers aren't misled about what refresh time it represents.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At api-reference/openapi/social.json, line 1467:
<comment>The field is named `updated_at`, but this description now says the value is when the post was *published* on the platform. That clashes with the other `updated_at` fields in these specs (`SocialProfile.updated_at` = last updated, `SocialPost.updated_at` = last updated). Either keep the field semantic consistent with its name, or rename the property to reflect that it carries the publish timestamp, so consumers aren't misled about what refresh time it represents.</comment>
<file context>
@@ -1464,7 +1464,27 @@
"type": "string",
"format": "date-time",
- "description": "ISO timestamp of when the post was last updated"
+ "description": "ISO timestamp of when the post was published on the platform"
+ },
+ "views": {
</file context>

[sweetmantech]

Local Mintlify verification (2026-08-27, npx mintlify@latest dev on this branch @ 248b0b9)

Page	Documented	Rendered
POST /api/socials/{id}/scrape	credits line, per-platform posts table (Instagram ignored, TikTok honored), "What gets stored" list, one-hop rule	✅ all render; the markdown table inside the OpenAPI description renders as a table. Mintlify wraps the narrow first-column header ("Platform"): column sizing, not markup
GET /api/artists/{id}/socials	page, limit, history (1–90) params; history[] of SocialSnapshot in the 200 example; 400/401/403/404 tabs	✅ params + ranges render; response example shows history[{captured_at, follower_count, following_count, post_count}]; five status tabs
GET /api/artists/{id}/posts	views/likes/comments/reposts on ArtistPost; description states last-scrape semantics	✅ example carries the four fields
GET /api/artists/{id}/fans	commenter-enrichment sentence, Instagram-only today	✅

Both OpenAPI files re-parse (json.load); docs.json nav unchanged (no new pages, so llms.txt picks the text up from the existing entries).

Social Scrape

Get Artist Socials

Get Artist Posts

Get Artist Fans

[sweetmantech]

Review triage (b147e63): fixed the two wording conflicts — the scrape page's commenter bullet now says avatar, bio, follower and following counts (the profile scraper returns followsCount), and the snapshot bullet qualifies following/post counts as "where the platform reports them", matching SocialSnapshot's nullable fields. Declined the updated_at rename: the handlers have always written the platform publish timestamp into posts.updated_at and GET /posts orders by it; the description now says what the value is rather than hiding it behind the column name.

[sweetmantech]

Preview testing, round 2 (2026-08-27, branch @ b147e63, npx mintlify@latest dev on :3333, driven with Chrome DevTools MCP)

Covers the review-fix commit (b147e63) and exercises the interactive pieces, not just the rendered text.

Check	Expected	Observed
Social Scrape description	credits line, 3-column posts table, "What gets stored" list, one-hop rule	✅ 8 table rows (header + 7 platforms), header Platform / posts / When posts is omitted; commenter bullet now says "avatar, bio, follower and following counts" (matches /fans)
Social Scrape cross-links	every /api-reference/… link on the page resolves	✅ fetch() from the page: artists/socials 200, apify/scraper 200, posts/get 200, fans/get 200, artist/socials-scrape 200
Get Artist Socials params	page (default 1), limit (default 20, ≤100), history (1–90)	✅ all three render with defaults/ranges
Get Artist Socials response tabs	200 / 400 / 401 / 403 / 404	✅ five tabs; clicking 404 shows { "status": "error", "message": "<string>" } (ArtistSocialsErrorResponse)
Get Artist Socials history schema	SocialSnapshot child attributes	✅ captured_at (required), follower_count (required), following_count (nullable), post_count (nullable) with descriptions
Get Artist Socials playground	"Try it" → set history=14 → cURL includes it	✅ generated URL: …/api/artists/{id}/socials?page=1&limit=20&history=14
Get Artist Posts	views/likes/comments/reposts in the 200 example and as child attributes with per-platform source fields	✅ example shows all four; expanded attributes carry the platform mapping (e.g. views: YouTube viewCount, TikTok playCount, X viewCount)
Get Artist Fans	Instagram-only commenter coverage + enrichment sentence	✅

Not changed by this PR, confirmed unaffected: docs.json nav (no new pages); both OpenAPI files still parse.

Capture note: Mintlify's sticky navbar and side panel repeat on stitched full-page captures, so #navbar / #content-side-layout were set to position: relative in the page before each shot; content is otherwise untouched.

Social Scrape

Get Artist Socials (children expanded)

Get Artist Socials: 404 tab, and the playground with history=14

Get Artist Posts (children expanded)

Get Artist Fans

[sweetmantech]

hyperlink "Social Posts" and "Social Scrape" to the respective endpoints.