The official Python SDK for Bright Data APIs. Scrape any website, get SERP results, bypass bot detection and CAPTCHAs, and access 100+ ready-made datasets.
pip install brightdata-sdkGet your API Token from the Bright Data Control Panel:
export BRIGHTDATA_API_TOKEN="your_api_token_here"Already logged in with the CLI? The SDK works with no configuration — it automatically
falls back to the credentials stored by brightdata login.
This SDK is async-native. A sync client is also available (see Sync Client).
importasynciofrombrightdataimportBrightDataClientasyncdefmain():
asyncwithBrightDataClient() asclient:
result=awaitclient.scrape_url("https://example.com")
print(result.data)
asyncio.run(main())asyncwithBrightDataClient() asclient:
result=awaitclient.scrape_url("https://example.com")
print(result.data)For non-blocking web scraping, use mode="async". This triggers a request and returns a response_id, which the SDK automatically polls until results are ready:
asyncwithBrightDataClient() asclient:
# Triggers request → gets response_id → polls until readyresult=awaitclient.scrape_url(
url="https://example.com",
mode="async",
poll_interval=5, # Check every 5 secondspoll_timeout=180# Web Unlocker async can take ~2 minutes
)
print(result.data)
# Batch scraping multiple URLs concurrentlyurls= ["https://example.com", "https://example.org", "https://example.net"]
results=awaitclient.scrape_url(url=urls, mode="async", poll_timeout=180)How it works:
- Sends request to
/unblocker/req→ returnsresponse_idimmediately - Polls
/unblocker/get_result?response_id=...until ready or timeout - Returns the scraped data
When to use async mode:
- Batch scraping with many URLs
- Background processing while continuing other work
Performance note: Web Unlocker async mode typically takes ~2 minutes to complete. For faster results on single URLs, use the default sync mode (no mode parameter).
Search across Google, Bing, and Yandex. Google and Bing return parsed organic results (title/url/description/position); Yandex has no upstream parser and returns raw HTML.
asyncwithBrightDataClient() asclient:
# Google — parsed resultsresult=awaitclient.search.google(query="python scraping", num_results=10)
foriteminresult.data:
print(item["title"], item["url"])
# Bing — parsed results (same shape as Google)result=awaitclient.search.bing(query="python scraping", num_results=10)
foriteminresult.data:
print(item["title"], item["url"])
# Yandex — raw HTML only (parse it yourself with BeautifulSoup etc.)result=awaitclient.search.yandex(query="python scraping", num_results=10)
print(f"{len(result.raw_html)} chars of HTML")Batch queries: pass a list of strings instead of a single string — each query runs concurrently and you get back a List[SearchResult].
For non-blocking SERP requests, use mode="async":
asyncwithBrightDataClient() asclient:
# Non-blocking - polls for resultsresult=awaitclient.search.google(
query="python programming",
mode="async",
poll_interval=2, # Check every 2 secondspoll_timeout=30# Give up after 30 seconds
)
foriteminresult.data:
print(item['title'], item['link'])When to use async mode:
- Batch operations with many queries
- Background processing while continuing other work
- When scraping may take longer than usual
Note: Async mode uses the same zones and returns the same data structure as sync mode - no extra configuration needed!
The SDK includes ready-to-use scrapers for popular websites: Amazon, LinkedIn, Instagram, Facebook, and more.
Pattern:client.scrape.<platform>.<method>(url)
Example: Amazon
asyncwithBrightDataClient() asclient:
# Product detailsresult=awaitclient.scrape.amazon.products(url="https://amazon.com/dp/B0CRMZHDG8")
# Reviewsresult=awaitclient.scrape.amazon.reviews(url="https://amazon.com/dp/B0CRMZHDG8")
# Sellersresult=awaitclient.scrape.amazon.sellers(url="https://amazon.com/dp/B0CRMZHDG8")Available scrapers:
client.scrape.amazon— products, reviews, sellersclient.scrape.linkedin— profiles, companies, jobs, postsclient.scrape.instagram— profiles, posts, comments, reelsclient.scrape.facebook— posts, comments, reels, pages, marketplace, eventsclient.scrape.tiktok— profiles, posts by keyword/profile/urlclient.scrape.youtube— videos, channels, commentsclient.scrape.reddit— posts, comments, posts by keywordclient.scrape.pinterest— posts, profiles, posts by keyword/profileclient.scrape.chatgpt— send prompts, retrieve responsesclient.scrape.perplexity— queries with AI-ranked resultsclient.scrape.digikey— electronic component products
Cloud-hosted Chrome instances accessible via the Chrome DevTools Protocol (CDP). The SDK builds the connection URL — you drive the browser with Playwright, Puppeteer, or Selenium.
frombrightdataimportBrightDataClientfromplaywright.async_apiimportasync_playwrightclient=BrightDataClient(
browser_username="brd-customer-<id>-zone-<zone>",
browser_password="<password>",
)
url=client.browser.get_connect_url(country="us") # country is optionalasyncwithasync_playwright() aspw:
browser=awaitpw.chromium.connect_over_cdp(url)
page=awaitbrowser.new_page()
awaitpage.goto("https://example.com")
html=awaitpage.content()
awaitbrowser.close()When to use: sites that require full browser automation — JS rendering, login flows, interactive clicks. For plain HTML fetches, prefer client.scrape_url().
AI-ranked web search. Unlike SERP (which returns engine-ordered results), Discover takes a query plus an intent phrase and re-ranks by relevance. Optionally extracts full page content as markdown.
asyncwithBrightDataClient() asclient:
result=awaitclient.discover(
query="artificial intelligence trends 2026",
intent="latest AI technology developments",
country="us",
num_results=10,
)
foriteminresult.data:
print(f"[{item['relevance_score']:.2f}] {item['title']}{item['link']}")For long-running discoveries, trigger and poll separately:
job=awaitclient.discover_trigger(query="...", intent="...")
result=awaitjob.wait_and_fetch(timeout=60)When to use Discover vs SERP: Discover when you want entity-level relevance ranking driven by a natural-language intent (e.g. "find sustainability-focused AI companies"). SERP when you want raw search engine results.
Run custom collectors built in Bright Data's Scraper Studio. One call triggers the job, polls until ready, and returns the records:
asyncwithBrightDataClient() asclient:
data=awaitclient.scraper_studio.run(
collector="c_abc123", # your collector IDinput={"url": "https://example.com"}, # collector-specific inputtimeout=180,
)
print(f"Got {len(data)} records")For manual control over the lifecycle, use client.scraper_studio.trigger() → .status() → .fetch().
Access 100+ ready-made datasets from Bright Data — pre-collected, structured data from popular platforms.
asyncwithBrightDataClient() asclient:
# Filter a dataset — returns a snapshot_idsnapshot_id=awaitclient.datasets.imdb_movies(
filter={"name": "title", "operator": "includes", "value": "black"},
records_limit=5
)
# Download when ready (polls until snapshot is complete)data=awaitclient.datasets.imdb_movies.download(snapshot_id)
print(f"Got {len(data)} records")
# Quick sample: .sample() auto-discovers fields, no filter needed# Works on any datasetsnapshot_id=awaitclient.datasets.imdb_movies.sample(records_limit=5)Export results to file:
frombrightdata.datasetsimportexportexport(data, "results.json") # JSONexport(data, "results.csv") # CSVexport(data, "results.jsonl") # JSONLAvailable dataset categories:
- E-commerce: Amazon, Walmart, Shopee, Lazada, Zalando, Zara, H&M, Shein, IKEA, Sephora, and more
- Business intelligence: ZoomInfo, PitchBook, Owler, Slintel, VentureRadar, Manta
- Jobs & HR: Glassdoor (companies, reviews, jobs), Indeed (companies, jobs), Xing
- Reviews: Google Maps, Yelp, G2, Trustpilot, TrustRadius
- Social media: Pinterest (posts, profiles), Facebook Pages
- Real estate: Zillow, Airbnb, and 8+ regional platforms
- Luxury brands: Chanel, Dior, Prada, Balenciaga, Hermes, YSL, and more
- Entertainment: IMDB, NBA, Goodreads
Discover available fields:
metadata=awaitclient.datasets.imdb_movies.get_metadata()
forname, fieldinmetadata.fields.items():
print(f"{name}: {field.type}")Run multiple requests concurrently:
importasynciofrombrightdataimportBrightDataClientasyncdefmain():
asyncwithBrightDataClient() asclient:
urls= ["https://example.com/page1", "https://example.com/page2", "https://example.com/page3"]
tasks= [client.scrape_url(url) forurlinurls]
results=awaitasyncio.gather(*tasks)
asyncio.run(main())For long-running scrapes:
asyncwithBrightDataClient() asclient:
# Triggerjob=awaitclient.scrape.amazon.products_trigger(url="https://amazon.com/dp/B123")
# Wait for completionawaitjob.wait(timeout=180)
# Fetch resultsdata=awaitjob.fetch()For simpler use cases, use SyncBrightDataClient:
frombrightdataimportSyncBrightDataClientwithSyncBrightDataClient() asclient:
result=client.scrape_url("https://example.com")
print(result.data)
# All methods work the sameresult=client.scrape.amazon.products(url="https://amazon.com/dp/B123")
result=client.search.google(query="python")See docs/sync_client.md for details.
Inspect the account, list zones, and clean up unused ones:
asyncwithBrightDataClient() asclient:
# Verify token + connectivity (never raises — returns False on any error)ifawaitclient.test_connection():
print("Connected")
# Account status, usage stats, credit balanceinfo=awaitclient.get_account_info()
print(info)
# All zones on this account, with type and usagezones=awaitclient.list_zones()
forzinzones:
print(z["name"], z["type"])
# Delete a zone — destructive, confirm before callingawaitclient.delete_zone("zone_to_remove")RuntimeError: SyncBrightDataClient cannot be used inside async context
# Wrong - using sync client in async functionasyncdefmain():
withSyncBrightDataClient() asclient: # Error!
...
# Correct - use async clientasyncdefmain():
asyncwithBrightDataClient() asclient:
result=awaitclient.scrape_url("https://example.com")RuntimeError: BrightDataClient not initialized
# Wrong - forgot context managerclient=BrightDataClient()
result=awaitclient.scrape_url("...") # Error!# Correct - use context managerasyncwithBrightDataClient() asclient:
result=awaitclient.scrape_url("...")MIT License