Skip to content

Tier unsupported sites by scraping difficulty - #357

Closed
guergabo wants to merge 6 commits into
mainfrom
hypeship/scraperly-link
Closed

Tier unsupported sites by scraping difficulty#357
guergabo wants to merge 6 commits into
mainfrom
hypeship/scraperly-link

Conversation

@guergabo

@guergaboguergabo commented May 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Restructures the Unsupported Websites section of the browsers FAQ into a tiered list (Very Hard / Hard) of sites that are difficult or infeasible to automate today, with a short description of what to expect at each tier.

Inspired by per-site difficulty resources elsewhere in the scraping community, but using our own observations rather than linking out.

Drops the earlier <Info> callout in bot-detection/overview.mdx.

Preview

https://tbd-6fc993ce-hypeship-scraperly-link.mintlify.app/browsers/faq#unsupported-websites

Test plan

  • Mintlify preview renders the tiered list cleanly
  • #unsupported-websites anchor still resolves

Note

Low Risk
Documentation-only FAQ copy and structure; no product or runtime behavior changes in this diff.

Overview
The Unsupported Websites FAQ section is rewritten from a flat bullet list into Very Hard and Hard tiers, each with guidance on what automation friction to expect (login walls/CAPTCHAs vs. stealth/proxy mitigations).

Very Hard groups LinkedIn, Facebook, and Instagram; Hard covers X, Amazon, and Reddit. A new intro clarifies the list is incomplete and based on internal experience, including when stealth, residential proxies, and profiles still aren’t enough.

Reviewed by Cursor Bugbot for commit 23689d7. Bugbot is set up for automated code reviews on this repo. Configure here.

@mintlify

mintlifyBot commented May 11, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

ProjectStatusPreviewUpdated (UTC)
Kernel🟢 ReadyView PreviewMay 11, 2026, 5:22 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

cursor[bot]
cursorBot approved these changes May 11, 2026
cursor[bot]
cursorBot approved these changes May 11, 2026

@cursorcursorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk assessment: Very Low.

The actual diff only adds a short informational <Info> callout with an external link in browsers/bot-detection/overview.mdx. There are no codepath, config, navigation, prompt, infrastructure, auth, permissions, or data changes, and I didn’t find a CODEOWNERS file in the repo.

The PR is already approved on the current head commit, so I’m leaving this assessment as a comment only and not re-approving.

Open in WebView Automation

Sent by Cursor Automation: Assign PR reviewers

@guergaboguergabo changed the title Link to Scraperly in bot anti-detection overviewTier unsupported sites by scraping difficultyMay 11, 2026

@cursorcursorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk assessment: Very Low.

The actual diff only adds a short <Info> callout with an external Scraperly link in browsers/bot-detection/overview.mdx. There are no codepath, config, navigation, prompt/model-instruction, infrastructure, auth, permissions, or data changes, and I didn’t find a CODEOWNERS file in the repo.

The PR already has an approval recorded, so I’m leaving this assessment as a comment only and not re-approving.

Open in WebView Automation

Sent by Cursor Automation: Assign PR reviewers

@cursorcursorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Risk assessment: Very Low.

The current diff only updates copy in browsers/faq.mdx to categorize unsupported websites by scraping difficulty. There are no codepath, config, navigation, infrastructure, auth, permissions, data, or model-instruction changes, and I didn’t find a CODEOWNERS file in the repo.

The PR is already approved, so I’m leaving this assessment as a comment only and not re-approving.

Open in WebView Automation

Sent by Cursor Automation: Assign PR reviewers

@masnwilliams
masnwilliams marked this pull request as ready for review July 22, 2026 06:16

@cursorcursorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Risk assessment: Very Low.

The actual diff changes only user-facing documentation in browsers/faq.mdx: it reorganizes six unsupported websites into two scraping-difficulty tiers and expands the explanatory copy. It doesn't modify executable code, prompts/model instructions, site configuration, navigation, infrastructure, authentication, permissions, or data, so the blast radius is limited to the rendered FAQ content. No CODEOWNERS file is present in the repository.

The PR already has an approval recorded, so I haven't approved it again.

Open in WebView Automation

Sent by Cursor Automation: Assign PR reviewers

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@guergabo@masnwilliams