fix(brightdata): align integration with live API docs, add markdown/mode/error-report support

[waleedlatif1]

Summary

• fixed Discover numResults docs (real max is 20, not 1000) and contentFormat value (API only accepts "md", never "markdown" — the old value always 400'd)
• added Discover mode param (standard/deep/fast/zeroRanking) for search depth/ranking control
• added markdown output support to Web Unlocker scrape_url via data_format
• added include_errors support to scrape_dataset, matching existing sync_scrape behavior
• added .trim() on datasetId in scrape_dataset/sync_scrape URLs to guard against copy-paste whitespace
• added wandConfig on complex fields, tightened block output descriptions

Full re-validation against Bright Data's live OpenAPI specs (Web Unlocker, SERP, Discover, Datasets v3) confirms every tool is aligned field-for-field. No backward-incompatible changes — the two corrected values were never valid API inputs, so no working workflow could have depended on them.

Type of Change

• Bug fix

Testing

Tested manually — validated every tool against Bright Data's live docs/OpenAPI specs and re-verified with 5 independent verification passes (web-unlocker/serp, discover, datasets-v3, block wiring/registry, lint/typecheck/diff review). bunx biome check clean, bunx tsc --noEmit clean (exit 0).

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs	Skipped		Jul 7, 2026 3:36pm

[cursor]

PR Summary

Low Risk
Third-party API parameter and UI wiring fixes with no auth or core execution changes; corrected values were previously invalid API inputs.

Overview
Aligns the Bright Data block and tools with the live API: Discover docs now cap results at 20 (not 1000), Response Format sends md instead of the invalid markdown value, and a new Search Mode (deep / fast / zeroRanking) is wired through to the discover request body.

Scrape URL gains an advanced Convert To option that maps to Web Unlocker data_format for markdown output. Scrape dataset and sync scrape expose Include errors (matching sync scrape’s existing API flag on dataset trigger), and dataset_id is trimmed in request URLs to avoid whitespace failures.

The workflow UI adds wandConfig on discover intent and URL JSON fields, plus clearer block output descriptions; types and tool param wiring are updated for the new fields.

^{Reviewed by Cursor Bugbot for commit aef2da2. Configure here.}

[greptile-apps]

Greptile Summary

This PR aligns the Bright Data integration with the live API specs, fixing two values that caused 400 errors and adding several new optional parameters. All changes are backwards-compatible since the corrected values were never valid API inputs.

• Bug fixes: Discover's contentFormat dropdown corrected from "markdown" to "md" (the only value the Discover API accepts), and numResults documentation updated from 1,000 to 20.
• New params:mode for Discover search depth, data_format: "markdown" for Web Unlocker scrape-to-markdown, include_errors for both dataset tools, and .trim() on datasetId to guard against whitespace.
• Block wiring:mode correctly defaults to '' (not forwarded), matching the pattern used by dataFormat; syncIncludeErrors/datasetIncludeErrors switches have no value initializer so they don't default to a truthy state.

Confidence Score: 5/5

Safe to merge — all changes are verified bug fixes and additive optional parameters with no breaking changes.

The corrected API values ("md" for Discover format, data_format: "markdown" for Web Unlocker) are confirmed against Bright Data's live OpenAPI spec. The new optional params (mode, dataFormat, includeErrors) are all properly guarded and only forwarded when explicitly set. The previous mode: "standard" default issue flagged in the earlier review thread was resolved in the head commit.

No files require special attention.

Important Files Changed

Filename	Overview
apps/sim/blocks/blocks/brightdata.ts	Adds block inputs/dropdowns for dataFormat, mode, syncIncludeErrors, and datasetIncludeErrors. After the fix in aef2da2, mode defaults to '' and is only forwarded when explicitly set, matching dataFormat's opt-in pattern.
apps/sim/tools/brightdata/discover.ts	Fixes numResults max (1000→20) and adds mode forwarding. Tool correctly passes mode only when truthy, so the API's default "standard" applies by omission.
apps/sim/tools/brightdata/scrape_url.ts	Adds optional dataFormat param that forwards data_format: "markdown" to the Web Unlocker API — confirmed valid against Bright Data's live OpenAPI spec.
apps/sim/tools/brightdata/scrape_dataset.ts	Adds includeErrors param and .trim() on datasetId. include_errors query param is appended only when truthy, consistent with the sync_scrape tool's existing pattern.
apps/sim/tools/brightdata/sync_scrape.ts	Applies .trim() on datasetId to match scrape_dataset; includeErrors support was pre-existing — no logic changes, just whitespace guard.
apps/sim/tools/brightdata/types.ts	Type interfaces updated consistently with new optional fields: dataFormat?: string, includeErrors?: boolean, and mode?: string on the relevant param types.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant User
participant Block as BrightData Block
participant Tool as Tool Layer
participant API as Bright Data API
User->>Block: Set operation + params
Note over Block: transform() maps block params<br/>to tool params
alt scrape_url
Block->>Tool: "{zone, url, format, country, dataFormat?}"
Tool->>API: POST /request with data_format: "markdown" (if set)
end
alt discover
Block->>Tool: "{query, numResults?, mode?, intent?, format?}"
Note over Tool: mode only forwarded when non-empty
Tool->>API: POST /discover with format: "md"
API-->>Tool: "{task_id}"
loop Poll until done/failed/timeout
Tool->>API: "GET /discover?task_id=..."
API-->>Tool: "{status, results}"
end
end
alt sync_scrape / scrape_dataset
Block->>Tool: "{datasetId.trim(), urls, format, includeErrors?}"
Tool->>API: POST /datasets/v3/scrape or /trigger
end
API-->>Tool: Response
Tool-->>Block: Transformed output
Block-->>User: results / content / data

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant User
participant Block as BrightData Block
participant Tool as Tool Layer
participant API as Bright Data API
User->>Block: Set operation + params
Note over Block: transform() maps block params<br/>to tool params
alt scrape_url
Block->>Tool: "{zone, url, format, country, dataFormat?}"
Tool->>API: POST /request with data_format: "markdown" (if set)
end
alt discover
Block->>Tool: "{query, numResults?, mode?, intent?, format?}"
Note over Tool: mode only forwarded when non-empty
Tool->>API: POST /discover with format: "md"
API-->>Tool: "{task_id}"
loop Poll until done/failed/timeout
Tool->>API: "GET /discover?task_id=..."
API-->>Tool: "{status, results}"
end
end
alt sync_scrape / scrape_dataset
Block->>Tool: "{datasetId.trim(), urls, format, includeErrors?}"
Tool->>API: POST /datasets/v3/scrape or /trigger
end
API-->>Tool: Response
Tool-->>Block: Transformed output
Block-->>User: results / content / data

Loading

_{Reviews (2): Last reviewed commit: "fix(brightdata): default mode field to e..." | Re-trigger Greptile}

[waleedlatif1]

@greptile review

[waleedlatif1]

@cursor review

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit aef2da2. Configure here.}