docs(ruby-gem): sync NativeFeed and entry resolution for gem 0.28.0

src/content/docs/ruby-gem/guides/ai-agent-workflows.mdx

@@ -84,16 +84,16 @@ Add `html2rss` to your Claude Desktop configuration (`~/Library/Application Supp
 
 When an agent needs articles immediately without saving a feed configuration:
 
-1. Call `scrape_url` with `strategy: "auto"`. Auto runs Faraday then Botasaurus when `BOTASAURUS_SCRAPER_URL` is set on the MCP process.
+1. Call `scrape_url` with `strategy: "auto"`. Auto runs Faraday then Botasaurus when `BOTASAURUS_SCRAPER_URL` is set on the MCP process, prefers native RSS/Atom when present, and may rewrite a weak homepage URL via entry resolution.
 2. An empty item list is still success for articles-now. Read `payload.items`. Follow `next_step` (including `read_runtime` / `html2rss://runtime` when Botasaurus is unset).
-3. Call `inspect_url` when the result is weak or you need recon (`final_url`, `status`, `scheme_downgrade`, native RSS/Atom).
+3. Call `inspect_url` when the result is weak or you need recon (`final_url`, `status`, `scheme_downgrade`, `alternate_feeds`, `surface_category`). Follow `next_step` — often `scrape_url` when `alternate_feeds` is present.
 
 ### Pattern B: Iterative Feed Config Authoring
 
 When an agent is tasked with creating a durable YAML feed configuration:
 
 1. **Capture:** Call `capture_config`. YAML is `payload.yaml` (`items` + `enhance: true`). Check `payload.articles_count` and `has_selectors`.
-2. **Recon (optional):** Call `inspect_url` for `final_url`, `status`, `scheme_downgrade`, and `alternate_feeds` if the draft is weak.
+2. **Recon (optional):** Call `inspect_url` for `final_url`, `status`, `scheme_downgrade`, `alternate_feeds`, and `surface_category` if the draft is weak. Follow `next_step` (often `scrape_url` when alternates are present).
 3. **Rewrite:** If the destination is html2rss-configs, add `directory.topics` and explicit channel `title`/`url`. Strive to keep `enhance: true` (set `false` only when chrome leaks into items).
 4. **Validate:** Pass `yaml` (or `config`) to `validate_config`. Exactly one of those arguments.
 5. **Apply:** Call `apply_config`. `isError` plus `payload.item_count: 0` means the config is not shippable — channel title in the XML is not success.

src/content/docs/ruby-gem/guides/managing-feed-configs.mdx

@@ -76,7 +76,12 @@ Prefer `Html2rss.feed_result` when one scrape must render as both RSS and JSON F
 - `to_rss` / `to_json_feed(feed_url:)` — render formats
 - `status` — scrape telemetry (`Html2rss::Status`)
 
-`status.to_h` is the stable observability payload. Always includes `version` and `dedup_dropped`. When present, it may also include `scraper_tallies`, `selected_strategy`, `attempt_count`, `strategy_attempts` (auto-fallback attempts; empty outside `strategy: auto`), and `admission_drops` (Cleanup reason → count).
+`status.to_h` is the stable observability payload. Always includes `version` and `dedup_dropped`. When present, it may also include:
+
+- `scraper_tallies`, `selected_strategy`, `attempt_count`, `strategy_attempts` (auto-fallback attempts; empty outside `strategy: auto`)
+- `admission_drops` (Cleanup reason → count)
+- `entry_url` / `scrape_url` (pasted vs effective URL after entry resolution)
+- `entry_resolution` (`applied`, `probe_count`, `reason`, optional `winner_score`)
 
 For URL-only auto discovery, `Html2rss.auto_feed_result(url)` returns the same `FeedResult` (CLI `html2rss auto --explain` prints `status.to_h` on stderr).
 

src/content/docs/ruby-gem/reference/auto-source.mdx

@@ -22,13 +22,15 @@ To enable it, add `auto_source: {}` to your configuration:
 
 One fetch, then scrapers run in **tiers**. Within a tier, results merge. The pipeline stops when Cleanup has admitted enough articles (`limit`, default `25`).
 
-1. **In-page structured:** `schema`, `microdata`, `microformats2`, `json_state`, `xhr_articles`
-2. **Follow-up IO:** `wordpress_api`, `sitemap`, `meta_oembed`
-3. **Heuristic:** `semantic_html`
-4. **Heuristic:** `html` — skipped when an earlier tier already admitted at least one clean article
+1. **Native syndication:** `native_feed` — discovers same-origin RSS/Atom (`rel=alternate` plus common feed paths) and parses them before HTML heuristics. Direct feed entry URLs (Content-Type or body sniff) skip HTML AutoSource and parse immediately.
+2. **In-page structured:** `schema`, `microdata`, `microformats2`, `json_state`, `xhr_articles`
+3. **Follow-up IO:** `wordpress_api`, `sitemap`, `meta_oembed`
+4. **Heuristic:** `semantic_html`
+5. **Heuristic:** `html` — skipped when an earlier tier already admitted at least one clean article
 
 What each scraper does:
 
+- **`native_feed`:** Prefer native RSS/Atom when present. Tallies appear as `AutoSource::NativeFeed` under `--explain`.
 - **`schema`:** Parses `<script type="json/ld">` tags containing structured data (e.g., [Schema.org](https://schema.org/)), extracting authors and publishers.
 - **`microdata`:** Extracts HTML Microdata annotations (`itemscope itemtype`).
 - **`microformats2`:** Parses Microformats2 `h-entry` markup, extracting `p-name`, `e-content`, `u-url`, `dt-published`, `p-author`, `p-category`, and `u-photo` / `u-featured` media.
@@ -48,25 +50,44 @@ Drop reasons (counts) land on `Html2rss::Status#admission_drops`. Print them wit
 
 Ruby `limit:` and CLI `--limit` cap kept articles (default `25`).
 
-**`json_state` / `xhr_articles` Limitations:** both scrapers require discoverable arrays of hashes containing clear `title` and `url` fields. Minified or
-obfuscated payloads, heavily encoded values, or blobs that require executing embedded functions are ignored. `xhr_articles` only sees JSON bodies captured during the current Botasaurus browser scrape (capped by the scrape API).
+**`json_state` / `xhr_articles`:** both need discoverable arrays of hashes with clear `title` and `url` fields. Minified or obfuscated payloads, heavily encoded values, or blobs that require executing embedded functions are ignored. `xhr_articles` only sees JSON bodies captured during the current Botasaurus browser scrape (capped by the scrape API).
 
-**`wordpress_api` Limitations:** this scraper depends on the page exposing a public WordPress REST API root. The current implementation fetches post records directly, but it does not yet resolve category names or featured media metadata.
+**`wordpress_api`:** depends on a public WordPress REST API root. The scraper fetches post records directly; it does not resolve category names or featured media metadata.
 
-## Fine-Tuning
+## Entry URL resolution
 
-You can customize `auto_source` to improve its accuracy.
+Under `strategy: auto` with `auto_source` and no custom `selectors`, a weak homepage or hub extract can trigger **entry URL resolution** before Botasaurus escalation. The pipeline probes same-origin listing and feed candidates, then sticky-rewrites the effective scrape URL only when a retry extract yields items.
 
-### Scraper Options
+Resolution runs when enabled, the page is not a blocked surface, and any of:
+
+- fewer than 3 admitted articles
+- a weak surface (`high_entropy_surface`, `app_shell`, `unsupported_surface`)
+- at least half of the articles came from `native_feed`
+
+Inspect results with `html2rss auto URL --explain` or `Html2rss::Status#to_h`:
+
+| Field | Meaning |
+| ------------------ | --------------------------------------------------------------- |
+| `entry_url` | Pasted / channel URL |
+| `scrape_url` | Effective fetch URL after resolution |
+| `entry_resolution` | `{ applied, probe_count, reason }` plus optional `winner_score` |
+
+`entry_resolution.applied: true` means a winner was selected. The sticky rewrite still requires a successful retry extract.
 
-Enable or disable specific scrapers and adjust their settings:
+Defaults: `enabled: true`, `max_probes: 5`. Set `enabled: false` to keep the pasted URL only.
+
+## Fine-Tuning
+
+### Scraper Options
 
 <Code
  code={`
  channel:
  url: https://example.com
  auto_source:
  scraper:
+ native_feed:
+ enabled: true # default: true
  wordpress_api:
  enabled: false # default: true
  sitemap:
@@ -97,9 +118,21 @@ Enable or disable specific scrapers and adjust their settings:
  lang="yaml"
 />
 
-### Cleanup Options
+### Entry Resolution Options
 
-Remove unwanted items from the results:
+<Code
+ code={`
+ channel:
+ url: https://example.com
+ auto_source:
+ entry_resolution:
+ enabled: true # default: true
+ max_probes: 5 # default: 5; must be > 0
+ `}
+ lang="yaml"
+/>
+
+### Cleanup Options
 
 <Code
  code={`

src/content/docs/ruby-gem/reference/cli-reference.mdx

@@ -44,7 +44,7 @@ Available options:
 - `--max-redirects`: Maximum redirects to follow per request.
 - `--max-requests`: Maximum requests to allow for this feed build (defaults to `4` for auto discovery).
 - `--input`: Local HTML file path to read input from without making network requests. When provided without a `URL`, `html2rss` attempts to detect the base URL from `<link rel="canonical">` or OpenGraph metadata in the HTML.
-- `--explain`: Print `Html2rss::Status` JSON to stderr (`version`, `dedup_dropped`, and when present `scraper_tallies`, `selected_strategy`, `attempt_count`, `strategy_attempts`, `admission_drops`). Stdout stays the RSS or JSON Feed.
+- `--explain`: Print `Html2rss::Status` JSON to stderr (`version`, `dedup_dropped`, and when present `scraper_tallies`, `selected_strategy`, `attempt_count`, `strategy_attempts`, `admission_drops`, `entry_url`, `scrape_url`, `entry_resolution`). Stdout stays the RSS or JSON Feed.
 
 #### URL Surface Guidance For `auto`
 

src/content/docs/ruby-gem/reference/mcp-server.mdx

@@ -38,13 +38,13 @@ The MCP server supports two transport mechanisms:
 
 ## Strategy Resolution in MCP
 
-`scrape_url` and `capture_config` with `strategy: "auto"` (the default) run Faraday → Botasaurus AutoFallback in one call.
+`scrape_url` and `capture_config` with `strategy: "auto"` (the default) run Faraday → Botasaurus AutoFallback in one call. That path prefers native RSS/Atom when present and may run entry URL resolution on weak homepage/hub extracts — see [Auto Source](/ruby-gem/reference/auto-source/) and [Strategy](/ruby-gem/reference/strategy/#auto-default).
 
-`inspect_url` with `auto` uses Faraday for a cheap diagnostic. Pin `strategy: "botasaurus"` when you need browser rendering for inspect.
+`inspect_url` with `auto` uses Faraday for a cheap diagnostic. Pin `strategy: "botasaurus"` when you need browser rendering for inspect. Inspect reports head `rel=alternate` feeds only (no `/feed` path guessing).
 
 Read `html2rss://runtime` for `botasaurus_configured` (boolean only). Set `BOTASAURUS_SCRAPER_URL` on the **MCP process** env (see [AI agent workflows](/ruby-gem/guides/ai-agent-workflows/)).
 
-**Tool-call budget:** `scrape_url` = 1 call. Durable config = `capture_config` → `validate_config` → `apply_config`. Call `inspect_url` only when scrape/capture is weak or you need recon. Follow envelope `next_step` / `guidance` rather than guessing the next tool.
+**Tool-call budget:** `scrape_url` = 1 call. Durable config = `capture_config` → `validate_config` → `apply_config`. Call `inspect_url` only when scrape/capture is weak or you need recon. Follow envelope `next_step` / `guidance`.
 
 ## Result envelope
 
@@ -94,11 +94,13 @@ Diagnostic page analysis plus recon from the same fetch. `auto` uses Faraday; pi
  - `status`: HTTP status when the strategy reports one.
  - `scheme_downgrade`: `true` when the fetch went https → http.
  - `alternate_feeds`: `rel=alternate` RSS/Atom links from the document head (no `/feed` path guessing).
- - `content_type` / `html_response` / `scraper_eligibility` / `sst_stats` / `sst.segment_stats`.
+ - `surface_category`: page surface class (listing, weak, blocked, and related).
+ - `content_type` / `html_response` / `scraper_eligibility`.
+ - `sst`: `{ node_count, degraded, segment_stats }` when SST ran; `segment_stats` is also emitted at the top level when present.
  - `articles_count` and `admission_drops` when the response is HTML (Cleanup drop-reason counts).
  - `blocked_surface` when an interstitial signature is detected.
  - `xhr_capture` when the inspect strategy is `botasaurus` (`count`, redacted `sample_endpoints`, `candidate_articles`).
-- **next_step:** `done` when native alternate feeds exist, `capture_config` when articles were found, otherwise `scrape_url`.
+- **next_step:** Follow the envelope. Typical values: `scrape_url` when `alternate_feeds` is present or when no articles were found; `capture_config` when articles were found and there are no alternates.
 
 ### `capture_config`
 

src/content/docs/ruby-gem/reference/strategy.mdx

@@ -22,9 +22,16 @@ The default strategy chain is:
 
 `faraday` -> `botasaurus`
 
-Auto fallback shares one request budget across all strategy attempts. For pagination-heavy or dynamic pages, increase `request.max_requests` (or `--max-requests`) when retries exhaust the budget.
+When the config uses `auto_source` (no custom `selectors`), each strategy attempt:
 
-Under `auto`, `Html2rss.feed_result(...).status` exposes scrape telemetry: `selected_strategy`, `attempt_count`, and `strategy_attempts` (see [Managing Feed Configs](/ruby-gem/guides/managing-feed-configs/#ruby-api-feedresult)). Auto fallback decisions are also visible at `LOG_LEVEL=info` (hidden at the default `LOG_LEVEL=warn`).
+1. Runs AutoSource scrapers, starting with `native_feed` for same-origin RSS/Atom (direct feed URLs parse without HTML AutoSource).
+2. May run **entry URL resolution** on a weak homepage/hub extract — probe same-origin listing/feed candidates and sticky-rewrite the scrape URL only when the retry yields items — **before** escalating to the next strategy.
+
+Gates, defaults, and Status fields: [Auto Source — Entry URL resolution](/ruby-gem/reference/auto-source/#entry-url-resolution).
+
+Auto fallback shares one request budget across all strategy attempts. The baseline reserves a NativeFeed follow-up slot and, when entry resolution is enabled, `max_probes` probe slots plus one retry GET. Raise `request.max_requests` (or `--max-requests`) when builds exhaust the budget.
+
+Under `auto`, `Html2rss.feed_result(...).status` exposes scrape telemetry: `selected_strategy`, `attempt_count`, `strategy_attempts`, and when resolution ran `entry_url`, `scrape_url`, and `entry_resolution` (see [Managing Feed Configs](/ruby-gem/guides/managing-feed-configs/#ruby-api-feedresult)). Auto fallback decisions are also visible at `LOG_LEVEL=info` (hidden at the default `LOG_LEVEL=warn`).
 
 ## `faraday`
 
@@ -43,6 +50,8 @@ Requirements:
 
 html2rss enforces local request policy preflight and a feed-build timeout budget. The Botasaurus scrape API splits **total** scrape time (queue, boot, navigate, wait) from **work** time (navigate, selector wait, and scroll after the browser is ready). Feed YAML `wait_timeout_seconds` is validated against the work cap (`1..30` by default); the gem forwards `request.botasaurus` options to the scrape API without shrinking retries or waits to fit a remaining feed budget. Faraday transport timeout for POST `/scrape` is the lesser of the remaining feed-build budget and `BOTASAURUS_SCRAPE_TIMEOUT_SECONDS` plus a small buffer (default **45s** scrape total, **47s** transport cap).
 
+When the scrape API reports a timed-out stage, `Html2rss::RequestService::RequestTimedOut` may include `timeout_phase` (`queue`, `boot`, or `work`) and the same value in the exception message (`timeout_phase=<stage>`). Transport-hop timeouts leave `timeout_phase` unset.
+
 During the **browser** execution tier, the Botasaurus scrape API always captures JSON XHR/fetch response bodies (`xhr_responses`). html2rss forwards them as `Response#captured_responses` so AutoSource `xhr_articles` can extract articles without extra requests. The HTTP-request tier returns an empty capture list. Caps and filtering live in the scrape API (see its README); article-likeness filtering is a client concern.
 
 The client speaks OpenAPI 2.0 `ScrapeSuccess` / `ScrapeError` envelopes. Success requires an `html` string. Extra `request.botasaurus` keys fail YAML validation.