fix(gh_trending): scrape github.com/trending for real trending data

[datj9]

Problem

The `gh_trending` Lambda never read github.com/trending. It queried the GitHub Search API for repos created in the last 7/30 days, sorted by all-time stars — a fundamentally different list from github.com/trending (which ranks by stars gained in a window). This is why the trending data on the live site looked wrong.

Two latent bugs rode along:

• `_fetch_readme` referenced an undefined `HEADERS` → `NameError`, silently swallowed, so README context was never used.
• The handler emitted a nested schema (`{generated_at, weekly:{period,repos}}`) that did not match what the Astro site reads. The site expects a flat `{updated_at, weekly[], monthly[]}` with a `stars_this_period` field — so the "+N this week" badges were always broken.
• `email_digest` read the old nested shape (`data.get("weekly", {}).get("repos")`), which throws `AttributeError` against the real (flat) data — the Friday email's trending section was already broken on master.

Change

• Scrape `https://github.com/trending?since=weekly|monthly\` (public HTML, no token required) via BeautifulSoup. Parse name / url / description / language / stars / stars_this_period per `article.Box-row`.
• Emit the flat schema the site expects: `{updated_at, weekly[], monthly[]}` — fixes the broken period badges.
• README enrichment is now best-effort: works unauthenticated, returns `""` on failure so summaries still build from the trending page's own name/description/language.
• Fix `email_digest` to read the flat `weekly[]`.
• Add `beautifulsoup4` dependency + parser unit tests with a fixture.

Cron schedule (weekly Mon 07:00 UTC) and all infra are unchanged. CDK bundling installs the new dependency automatically from `requirements.txt`.

Test plan

• 7 parser unit tests pass (`pytest tests/test_gh_trending.py`)
• ruff clean on changed files
• Live smoke test: parsed 20 weekly + 19 monthly repos from the real page, correct stars/period/language, no token used
• `email_digest._extract_top_repos` verified against flat data
• infra TypeScript compiles (`tsc --noEmit`); other 3 Lambdas unaffected (no cross-imports)
• Post-merge: CI `deploy.yml` rebuilds + deploys; verify `gh_trending` next run writes real trending data

Out of scope

The site has been stale since 2026-05-12 because SSM `/tech-bytes/github-token` is still `PLACEHOLDER`, so the auto-rebuild trigger no-ops. That's a separate ops fix (set a real PAT) and does not block this correctness fix.

[datj9]

Problem

The `gh_trending` Lambda never read github.com/trending. It queried the GitHub Search API for repos created in the last 7/30 days, sorted by all-time stars — a fundamentally different list from github.com/trending (which ranks by stars gained in a window). This is why the trending data on the live site looked wrong.

Two latent bugs rode along:

• `_fetch_readme` referenced an undefined `HEADERS` → `NameError`, silently swallowed, so README context was never used.
• The handler emitted a nested schema (`{generated_at, weekly:{period,repos}}`) that did not match what the Astro site reads. The site expects a flat `{updated_at, weekly[], monthly[]}` with a `stars_this_period` field — so the "+N this week" badges were always broken.
• `email_digest` read the old nested shape (`data.get("weekly", {}).get("repos")`), which throws `AttributeError` against the real (flat) data — the Friday email's trending section was already broken on master.

Change

• Scrape `https://github.com/trending?since=weekly|monthly\` (public HTML, no token required) via BeautifulSoup. Parse name / url / description / language / stars / stars_this_period per `article.Box-row`.
• Emit the flat schema the site expects: `{updated_at, weekly[], monthly[]}` — fixes the broken period badges.
• README enrichment is now best-effort: works unauthenticated, returns `""` on failure so summaries still build from the trending page's own name/description/language.
• Fix `email_digest` to read the flat `weekly[]`.
• Add `beautifulsoup4` dependency + parser unit tests with a fixture.

Cron schedule (weekly Mon 07:00 UTC) and all infra are unchanged. CDK bundling installs the new dependency automatically from `requirements.txt`.

Test plan

• 7 parser unit tests pass (`pytest tests/test_gh_trending.py`)
• ruff clean on changed files
• Live smoke test: parsed 20 weekly + 19 monthly repos from the real page, correct stars/period/language, no token used
• `email_digest._extract_top_repos` verified against flat data
• infra TypeScript compiles (`tsc --noEmit`); other 3 Lambdas unaffected (no cross-imports)
• Post-merge: CI `deploy.yml` rebuilds + deploys; verify `gh_trending` next run writes real trending data

Out of scope

The site has been stale since 2026-05-12 because SSM `/tech-bytes/github-token` is still `PLACEHOLDER`, so the auto-rebuild trigger no-ops. That's a separate ops fix (set a real PAT) and does not block this correctness fix.

[datj9]

Problem

The `gh_trending` Lambda never read github.com/trending. It queried the GitHub Search API for repos created in the last 7/30 days, sorted by all-time stars — a fundamentally different list from github.com/trending (which ranks by stars gained in a window). This is why the trending data on the live site looked wrong.

Two latent bugs rode along:

• `_fetch_readme` referenced an undefined `HEADERS` → `NameError`, silently swallowed, so README context was never used.
• The handler emitted a nested schema (`{generated_at, weekly:{period,repos}}`) that did not match what the Astro site reads. The site expects a flat `{updated_at, weekly[], monthly[]}` with a `stars_this_period` field — so the "+N this week" badges were always broken.
• `email_digest` read the old nested shape (`data.get("weekly", {}).get("repos")`), which throws `AttributeError` against the real (flat) data — the Friday email's trending section was already broken on master.

Change

• Scrape `https://github.com/trending?since=weekly|monthly\` (public HTML, no token required) via BeautifulSoup. Parse name / url / description / language / stars / stars_this_period per `article.Box-row`.
• Emit the flat schema the site expects: `{updated_at, weekly[], monthly[]}` — fixes the broken period badges.
• README enrichment is now best-effort: works unauthenticated, returns `""` on failure so summaries still build from the trending page's own name/description/language.
• Fix `email_digest` to read the flat `weekly[]`.
• Add `beautifulsoup4` dependency + parser unit tests with a fixture.

Cron schedule (weekly Mon 07:00 UTC) and all infra are unchanged. CDK bundling installs the new dependency automatically from `requirements.txt`.

Test plan

• 7 parser unit tests pass (`pytest tests/test_gh_trending.py`)
• ruff clean on changed files
• Live smoke test: parsed 20 weekly + 19 monthly repos from the real page, correct stars/period/language, no token used
• `email_digest._extract_top_repos` verified against flat data
• infra TypeScript compiles (`tsc --noEmit`); other 3 Lambdas unaffected (no cross-imports)
• Post-merge: CI `deploy.yml` rebuilds + deploys; verify `gh_trending` next run writes real trending data

Out of scope

The site has been stale since 2026-05-12 because SSM `/tech-bytes/github-token` is still `PLACEHOLDER`, so the auto-rebuild trigger no-ops. That's a separate ops fix (set a real PAT) and does not block this correctness fix.

[datj9]

Problem

The `gh_trending` Lambda never read github.com/trending. It queried the GitHub Search API for repos created in the last 7/30 days, sorted by all-time stars — a fundamentally different list from github.com/trending (which ranks by stars gained in a window). This is why the trending data on the live site looked wrong.

Two latent bugs rode along:

• `_fetch_readme` referenced an undefined `HEADERS` → `NameError`, silently swallowed, so README context was never used.
• The handler emitted a nested schema (`{generated_at, weekly:{period,repos}}`) that did not match what the Astro site reads. The site expects a flat `{updated_at, weekly[], monthly[]}` with a `stars_this_period` field — so the "+N this week" badges were always broken.
• `email_digest` read the old nested shape (`data.get("weekly", {}).get("repos")`), which throws `AttributeError` against the real (flat) data — the Friday email's trending section was already broken on master.

Change

• Scrape `https://github.com/trending?since=weekly|monthly\` (public HTML, no token required) via BeautifulSoup. Parse name / url / description / language / stars / stars_this_period per `article.Box-row`.
• Emit the flat schema the site expects: `{updated_at, weekly[], monthly[]}` — fixes the broken period badges.
• README enrichment is now best-effort: works unauthenticated, returns `""` on failure so summaries still build from the trending page's own name/description/language.
• Fix `email_digest` to read the flat `weekly[]`.
• Add `beautifulsoup4` dependency + parser unit tests with a fixture.

Cron schedule (weekly Mon 07:00 UTC) and all infra are unchanged. CDK bundling installs the new dependency automatically from `requirements.txt`.

Test plan

• 7 parser unit tests pass (`pytest tests/test_gh_trending.py`)
• ruff clean on changed files
• Live smoke test: parsed 20 weekly + 19 monthly repos from the real page, correct stars/period/language, no token used
• `email_digest._extract_top_repos` verified against flat data
• infra TypeScript compiles (`tsc --noEmit`); other 3 Lambdas unaffected (no cross-imports)
• Post-merge: CI `deploy.yml` rebuilds + deploys; verify `gh_trending` next run writes real trending data

Out of scope

The site has been stale since 2026-05-12 because SSM `/tech-bytes/github-token` is still `PLACEHOLDER`, so the auto-rebuild trigger no-ops. That's a separate ops fix (set a real PAT) and does not block this correctness fix.

[datj9]

Problem

The `gh_trending` Lambda never read github.com/trending. It queried the GitHub Search API for repos created in the last 7/30 days, sorted by all-time stars — a fundamentally different list from github.com/trending (which ranks by stars gained in a window). This is why the trending data on the live site looked wrong.

Two latent bugs rode along:

• `_fetch_readme` referenced an undefined `HEADERS` → `NameError`, silently swallowed, so README context was never used.
• The handler emitted a nested schema (`{generated_at, weekly:{period,repos}}`) that did not match what the Astro site reads. The site expects a flat `{updated_at, weekly[], monthly[]}` with a `stars_this_period` field — so the "+N this week" badges were always broken.
• `email_digest` read the old nested shape (`data.get("weekly", {}).get("repos")`), which throws `AttributeError` against the real (flat) data — the Friday email's trending section was already broken on master.

Change

• Scrape `https://github.com/trending?since=weekly|monthly\` (public HTML, no token required) via BeautifulSoup. Parse name / url / description / language / stars / stars_this_period per `article.Box-row`.
• Emit the flat schema the site expects: `{updated_at, weekly[], monthly[]}` — fixes the broken period badges.
• README enrichment is now best-effort: works unauthenticated, returns `""` on failure so summaries still build from the trending page's own name/description/language.
• Fix `email_digest` to read the flat `weekly[]`.
• Add `beautifulsoup4` dependency + parser unit tests with a fixture.

Cron schedule (weekly Mon 07:00 UTC) and all infra are unchanged. CDK bundling installs the new dependency automatically from `requirements.txt`.

Test plan

• 7 parser unit tests pass (`pytest tests/test_gh_trending.py`)
• ruff clean on changed files
• Live smoke test: parsed 20 weekly + 19 monthly repos from the real page, correct stars/period/language, no token used
• `email_digest._extract_top_repos` verified against flat data
• infra TypeScript compiles (`tsc --noEmit`); other 3 Lambdas unaffected (no cross-imports)
• Post-merge: CI `deploy.yml` rebuilds + deploys; verify `gh_trending` next run writes real trending data

Out of scope

The site has been stale since 2026-05-12 because SSM `/tech-bytes/github-token` is still `PLACEHOLDER`, so the auto-rebuild trigger no-ops. That's a separate ops fix (set a real PAT) and does not block this correctness fix.

[datj9]

Problem

The `gh_trending` Lambda never read github.com/trending. It queried the GitHub Search API for repos created in the last 7/30 days, sorted by all-time stars — a fundamentally different list from github.com/trending (which ranks by stars gained in a window). This is why the trending data on the live site looked wrong.

Two latent bugs rode along:

• `_fetch_readme` referenced an undefined `HEADERS` → `NameError`, silently swallowed, so README context was never used.
• The handler emitted a nested schema (`{generated_at, weekly:{period,repos}}`) that did not match what the Astro site reads. The site expects a flat `{updated_at, weekly[], monthly[]}` with a `stars_this_period` field — so the "+N this week" badges were always broken.
• `email_digest` read the old nested shape (`data.get("weekly", {}).get("repos")`), which throws `AttributeError` against the real (flat) data — the Friday email's trending section was already broken on master.

Change

• Scrape `https://github.com/trending?since=weekly|monthly\` (public HTML, no token required) via BeautifulSoup. Parse name / url / description / language / stars / stars_this_period per `article.Box-row`.
• Emit the flat schema the site expects: `{updated_at, weekly[], monthly[]}` — fixes the broken period badges.
• README enrichment is now best-effort: works unauthenticated, returns `""` on failure so summaries still build from the trending page's own name/description/language.
• Fix `email_digest` to read the flat `weekly[]`.
• Add `beautifulsoup4` dependency + parser unit tests with a fixture.

Cron schedule (weekly Mon 07:00 UTC) and all infra are unchanged. CDK bundling installs the new dependency automatically from `requirements.txt`.

Test plan

• 7 parser unit tests pass (`pytest tests/test_gh_trending.py`)
• ruff clean on changed files
• Live smoke test: parsed 20 weekly + 19 monthly repos from the real page, correct stars/period/language, no token used
• `email_digest._extract_top_repos` verified against flat data
• infra TypeScript compiles (`tsc --noEmit`); other 3 Lambdas unaffected (no cross-imports)
• Post-merge: CI `deploy.yml` rebuilds + deploys; verify `gh_trending` next run writes real trending data

Out of scope

The site has been stale since 2026-05-12 because SSM `/tech-bytes/github-token` is still `PLACEHOLDER`, so the auto-rebuild trigger no-ops. That's a separate ops fix (set a real PAT) and does not block this correctness fix.

[datj9]

Problem

The `gh_trending` Lambda never read github.com/trending. It queried the GitHub Search API for repos created in the last 7/30 days, sorted by all-time stars — a fundamentally different list from github.com/trending (which ranks by stars gained in a window). This is why the trending data on the live site looked wrong.

Two latent bugs rode along:

• `_fetch_readme` referenced an undefined `HEADERS` → `NameError`, silently swallowed, so README context was never used.
• The handler emitted a nested schema (`{generated_at, weekly:{period,repos}}`) that did not match what the Astro site reads. The site expects a flat `{updated_at, weekly[], monthly[]}` with a `stars_this_period` field — so the "+N this week" badges were always broken.
• `email_digest` read the old nested shape (`data.get("weekly", {}).get("repos")`), which throws `AttributeError` against the real (flat) data — the Friday email's trending section was already broken on master.

Change

• Scrape `https://github.com/trending?since=weekly|monthly\` (public HTML, no token required) via BeautifulSoup. Parse name / url / description / language / stars / stars_this_period per `article.Box-row`.
• Emit the flat schema the site expects: `{updated_at, weekly[], monthly[]}` — fixes the broken period badges.
• README enrichment is now best-effort: works unauthenticated, returns `""` on failure so summaries still build from the trending page's own name/description/language.
• Fix `email_digest` to read the flat `weekly[]`.
• Add `beautifulsoup4` dependency + parser unit tests with a fixture.

Cron schedule (weekly Mon 07:00 UTC) and all infra are unchanged. CDK bundling installs the new dependency automatically from `requirements.txt`.

Test plan

• 7 parser unit tests pass (`pytest tests/test_gh_trending.py`)
• ruff clean on changed files
• Live smoke test: parsed 20 weekly + 19 monthly repos from the real page, correct stars/period/language, no token used
• `email_digest._extract_top_repos` verified against flat data
• infra TypeScript compiles (`tsc --noEmit`); other 3 Lambdas unaffected (no cross-imports)
• Post-merge: CI `deploy.yml` rebuilds + deploys; verify `gh_trending` next run writes real trending data

Out of scope

The site has been stale since 2026-05-12 because SSM `/tech-bytes/github-token` is still `PLACEHOLDER`, so the auto-rebuild trigger no-ops. That's a separate ops fix (set a real PAT) and does not block this correctness fix.

[datj9]

Problem

The `gh_trending` Lambda never read github.com/trending. It queried the GitHub Search API for repos created in the last 7/30 days, sorted by all-time stars — a fundamentally different list from github.com/trending (which ranks by stars gained in a window). This is why the trending data on the live site looked wrong.

Two latent bugs rode along:

• `_fetch_readme` referenced an undefined `HEADERS` → `NameError`, silently swallowed, so README context was never used.
• The handler emitted a nested schema (`{generated_at, weekly:{period,repos}}`) that did not match what the Astro site reads. The site expects a flat `{updated_at, weekly[], monthly[]}` with a `stars_this_period` field — so the "+N this week" badges were always broken.
• `email_digest` read the old nested shape (`data.get("weekly", {}).get("repos")`), which throws `AttributeError` against the real (flat) data — the Friday email's trending section was already broken on master.

Change

• Scrape `https://github.com/trending?since=weekly|monthly\` (public HTML, no token required) via BeautifulSoup. Parse name / url / description / language / stars / stars_this_period per `article.Box-row`.
• Emit the flat schema the site expects: `{updated_at, weekly[], monthly[]}` — fixes the broken period badges.
• README enrichment is now best-effort: works unauthenticated, returns `""` on failure so summaries still build from the trending page's own name/description/language.
• Fix `email_digest` to read the flat `weekly[]`.
• Add `beautifulsoup4` dependency + parser unit tests with a fixture.

Cron schedule (weekly Mon 07:00 UTC) and all infra are unchanged. CDK bundling installs the new dependency automatically from `requirements.txt`.

Test plan

• 7 parser unit tests pass (`pytest tests/test_gh_trending.py`)
• ruff clean on changed files
• Live smoke test: parsed 20 weekly + 19 monthly repos from the real page, correct stars/period/language, no token used
• `email_digest._extract_top_repos` verified against flat data
• infra TypeScript compiles (`tsc --noEmit`); other 3 Lambdas unaffected (no cross-imports)
• Post-merge: CI `deploy.yml` rebuilds + deploys; verify `gh_trending` next run writes real trending data

Out of scope

The site has been stale since 2026-05-12 because SSM `/tech-bytes/github-token` is still `PLACEHOLDER`, so the auto-rebuild trigger no-ops. That's a separate ops fix (set a real PAT) and does not block this correctness fix.