feat: genericize into self-hostable OSS project - #6

Open
datj9 wants to merge 1 commit into
masterfrom
feat/genericize-oss
Open

feat: genericize into self-hostable OSS project#6
datj9 wants to merge 1 commit into
masterfrom
feat/genericize-oss

Conversation

@datj9

Copy link
Copy Markdown
Owner

Summary

Turns tech-bytes into a real OSS project: anyone can fork, edit one config file, plug in their LLM key (OpenAI / Anthropic / local), and launch with zero AWS. GitHub Actions builds the digest on a cron and publishes to GitHub Pages. AWS/CDK stays as an optional advanced path.

Plan: ~/.claude/plans/purring-kindling-cray.md

What changed

Pipeline (lambdas/pipeline/)

  • Pluggable LLM (pipeline/providers/): openai, anthropic, local (OpenAI-compatible: Ollama, LM Studio, Groq, OpenRouter) behind one Protocol. Selected by llm.provider in config; keys from env, SSM fallback on AWS.
  • Pluggable storage (pipeline/storage/): local_fs (default, writes data/) and s3 (AWS path) behind one Protocol. boto3 fully lazy — free path imports no AWS libs.
  • Single CLI: python -m pipeline.run [all|release_radar|hn_digest|gh_trending|email_digest].
  • Single config: config/techbytes.config.yml owns sources + branding + LLM choice. Icons are emoji literals in config (the site reads them directly; the old hardcoded categoryIcons map is gone).
  • CloudWatch metrics no-op outside AWS. trigger_rebuild no-op on the free path.

Site (Astro)

  • Branding read from config at build time; PUBLIC_* env overrides for CI.
  • Pages base path auto-detected from GITHUB_REPOSITORY for project Pages sites.
  • Data dir resolved from $DATA_DIR (default ../data).
  • contribute.astro repo URL resolved from env — no hardcoded datj9/tech-bytes.

Workflows

  • digest.yml (NEW) — free path: cron → run pipeline → commit data/ → build → deploy Pages. No AWS.
  • deploy-aws.yml (renamed from deploy.yml) — optional AWS path; reads domain/zone/repo from vars/context.
  • preview.yml — ruff + pytest + site build + best-effort cdk synth on every PR.

Infra (CDK, optional)

  • Fully parameterized: domain.name, hosted.zone.name, github.repo, ssm.prefix from context. No hardcoded values; clear error if missing. See infra/README.md.

OSS scaffolding

  • LICENSE (MIT), README.md (60-second quickstart + config reference + provider setup), CONTRIBUTING.md, regenerated .env.example.
  • All personal identifiers stripped (domain, AWS profile, repo slug, SSM paths, rebuild URL). Grep gate clean across source and build output.

Decisions (locked at execution)

  • License: MIT
  • Brand: keeps "Tech Bytes" as the demo default (forkers override via config)
  • Dir rename: lambdas/pipeline/ (git mv preserves history)
  • Anthropic default model: claude-sonnet-4-6

Verification

All green locally:

CheckResult
ruff check pipeline/clean
pytest pipeline/tests/34/34 pass
pnpm run build (site)success
tsc --noEmit (infra)clean
python -m pipeline.run release_radar (smoke, local storage, fake key)writes data/release-radar.json, no boto3 import
Grep gate finaldivision|datj9|super-personal in site/dist/clean
Grep gate in source (excluding intentional default /tech-bytes/ SSM namespace)clean

preview.yml runs ruff + pytest + site build + cdk synth on this PR.

Test plan for reviewer

  1. Free path (local):cp config/techbytes.config.example.yml config/techbytes.config.yml → set OPENAI_API_KEY (or LLM_PROVIDER=anthropic + ANTHROPIC_API_KEY) → python -m pipeline.run all → confirm data/*.json written, no AWS deps required.
  2. Site:cd site && npm i && npm run build → confirm generic branding, no finaldivision / hardcoded icons in dist/.
  3. Provider swap: toggle llm.provider between openai / anthropic / local in config; confirm the factory picks the right provider and default model.
  4. Fork simulation: set GITHUB_REPOSITORY=myuser/myrepo before pnpm run build; confirm Pages base path and contribute page use the fork's identity.

Breaking changes

  • config/technologies.yml removed → migrate to config/techbytes.config.yml (see example).
  • data/release-radar.json icon field now an emoji literal (was a categoryIcons key string). Seed data migrated; first pipeline run regenerates everything.
  • Lambda handler paths changed from release_radar.handler.handler to pipeline.release_radar.handler.handler — CDK bundling updated to match. Existing AWS deployments need cdk deploy to pick up the new handler paths.

Turn tech-bytes into a real OSS project anyone can fork, configure, and run
with zero AWS. The default deploy path is now GitHub Pages via a scheduled
GitHub Actions workflow; AWS/CDK is demoted to an optional advanced path.
Architecture
- pipeline/ (renamed from lambdas/): framework-agnostic Python package with a
single CLI entrypoint `python -m pipeline.run [all|source]`.
- pipeline/providers/: pluggable LLM (openai, anthropic, local/OAI-compatible)
behind one Protocol. Selected by config; keys from env (SSM fallback on AWS).
- pipeline/storage/: pluggable storage (local_fs default, s3 optional) behind
one Protocol. Selected by $STORAGE. boto3 fully lazy — free path needs no AWS.
- config/techbytes.config.yml: single user-facing config (sources + branding +
LLM choice). Icons live here as emoji literals, not in site code.
- pipeline/shared/{config,rebuild,utils,secrets}.py: cross-cutting helpers.
CloudWatch metrics no-op outside AWS; rebuild trigger no-op on free path.
Site (Astro)
- Branding (title/tagline/url) read from config at build, PUBLIC_* env overrides.
- Hardcoded `categoryIcons` map deleted; icons come from data (which comes from
config) as emoji literals.
- Data dir resolved via $DATA_DIR; Pages base path auto-detected from
GITHUB_REPOSITORY.
- Contribute page repo URL resolved from $GITHUB_REPOSITORY / PUBLIC_GITHUB_REPO.
Workflows
- digest.yml (NEW): cron -> run pipeline -> commit data/ -> build -> Pages.
- deploy-aws.yml (renamed): optional advanced path; pulls domain/zone/repo from
vars/context, no hardcoded values.
- preview.yml: ruff + pytest + site build + best-effort cdk synth.
Infra (CDK, optional)
- Fully parameterized via context: domain.name, hosted.zone.name, github.repo,
ssm.prefix. No hardcoded domain/account/repo.
- Lambda handler paths updated for the pipeline/ rename.
OSS scaffolding
- LICENSE (MIT), README (quickstart + config + provider setup), CONTRIBUTING,
regenerated .env.example.
- All personal identifiers removed (domain, AWS profile, repo slug, SSM paths,
model id, rebuild URL). Grep gate clean across source + build output.
Tests
- 34 pass: existing release_radar/gh_trending schema tests updated for the new
config-driven icon; new test_pipeline.py covers provider factory, storage
factory, and local_fs behavior.
Decisions
- License: MIT. Brand: keeps "Tech Bytes" as the demo default (forkers override
via config). Dir renamed lambdas/ -> pipeline/. Anthropic default model:
claude-sonnet-4-6.
@datj9
datj9force-pushed the feat/genericize-oss branch from d7a1562 to fdd6384CompareAugust 16, 2026 23:17
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@datj9
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

feat: genericize into self-hostable OSS project - #6

Open
datj9 wants to merge 1 commit into
masterfrom
feat/genericize-oss
Open

feat: genericize into self-hostable OSS project#6
datj9 wants to merge 1 commit into
masterfrom
feat/genericize-oss

Conversation

@datj9

Copy link
Copy Markdown
Owner

Summary

Turns tech-bytes into a real OSS project: anyone can fork, edit one config file, plug in their LLM key (OpenAI / Anthropic / local), and launch with zero AWS. GitHub Actions builds the digest on a cron and publishes to GitHub Pages. AWS/CDK stays as an optional advanced path.

Plan: ~/.claude/plans/purring-kindling-cray.md

What changed

Pipeline (lambdas/pipeline/)

  • Pluggable LLM (pipeline/providers/): openai, anthropic, local (OpenAI-compatible: Ollama, LM Studio, Groq, OpenRouter) behind one Protocol. Selected by llm.provider in config; keys from env, SSM fallback on AWS.
  • Pluggable storage (pipeline/storage/): local_fs (default, writes data/) and s3 (AWS path) behind one Protocol. boto3 fully lazy — free path imports no AWS libs.
  • Single CLI: python -m pipeline.run [all|release_radar|hn_digest|gh_trending|email_digest].
  • Single config: config/techbytes.config.yml owns sources + branding + LLM choice. Icons are emoji literals in config (the site reads them directly; the old hardcoded categoryIcons map is gone).
  • CloudWatch metrics no-op outside AWS. trigger_rebuild no-op on the free path.

Site (Astro)

  • Branding read from config at build time; PUBLIC_* env overrides for CI.
  • Pages base path auto-detected from GITHUB_REPOSITORY for project Pages sites.
  • Data dir resolved from $DATA_DIR (default ../data).
  • contribute.astro repo URL resolved from env — no hardcoded datj9/tech-bytes.

Workflows

  • digest.yml (NEW) — free path: cron → run pipeline → commit data/ → build → deploy Pages. No AWS.
  • deploy-aws.yml (renamed from deploy.yml) — optional AWS path; reads domain/zone/repo from vars/context.
  • preview.yml — ruff + pytest + site build + best-effort cdk synth on every PR.

Infra (CDK, optional)

  • Fully parameterized: domain.name, hosted.zone.name, github.repo, ssm.prefix from context. No hardcoded values; clear error if missing. See infra/README.md.

OSS scaffolding

  • LICENSE (MIT), README.md (60-second quickstart + config reference + provider setup), CONTRIBUTING.md, regenerated .env.example.
  • All personal identifiers stripped (domain, AWS profile, repo slug, SSM paths, rebuild URL). Grep gate clean across source and build output.

Decisions (locked at execution)

  • License: MIT
  • Brand: keeps "Tech Bytes" as the demo default (forkers override via config)
  • Dir rename: lambdas/pipeline/ (git mv preserves history)
  • Anthropic default model: claude-sonnet-4-6

Verification

All green locally:

CheckResult
ruff check pipeline/clean
pytest pipeline/tests/34/34 pass
pnpm run build (site)success
tsc --noEmit (infra)clean
python -m pipeline.run release_radar (smoke, local storage, fake key)writes data/release-radar.json, no boto3 import
Grep gate finaldivision|datj9|super-personal in site/dist/clean
Grep gate in source (excluding intentional default /tech-bytes/ SSM namespace)clean

preview.yml runs ruff + pytest + site build + cdk synth on this PR.

Test plan for reviewer

  1. Free path (local):cp config/techbytes.config.example.yml config/techbytes.config.yml → set OPENAI_API_KEY (or LLM_PROVIDER=anthropic + ANTHROPIC_API_KEY) → python -m pipeline.run all → confirm data/*.json written, no AWS deps required.
  2. Site:cd site && npm i && npm run build → confirm generic branding, no finaldivision / hardcoded icons in dist/.
  3. Provider swap: toggle llm.provider between openai / anthropic / local in config; confirm the factory picks the right provider and default model.
  4. Fork simulation: set GITHUB_REPOSITORY=myuser/myrepo before pnpm run build; confirm Pages base path and contribute page use the fork's identity.

Breaking changes

  • config/technologies.yml removed → migrate to config/techbytes.config.yml (see example).
  • data/release-radar.json icon field now an emoji literal (was a categoryIcons key string). Seed data migrated; first pipeline run regenerates everything.
  • Lambda handler paths changed from release_radar.handler.handler to pipeline.release_radar.handler.handler — CDK bundling updated to match. Existing AWS deployments need cdk deploy to pick up the new handler paths.

Turn tech-bytes into a real OSS project anyone can fork, configure, and run
with zero AWS. The default deploy path is now GitHub Pages via a scheduled
GitHub Actions workflow; AWS/CDK is demoted to an optional advanced path.
Architecture
- pipeline/ (renamed from lambdas/): framework-agnostic Python package with a
single CLI entrypoint `python -m pipeline.run [all|source]`.
- pipeline/providers/: pluggable LLM (openai, anthropic, local/OAI-compatible)
behind one Protocol. Selected by config; keys from env (SSM fallback on AWS).
- pipeline/storage/: pluggable storage (local_fs default, s3 optional) behind
one Protocol. Selected by $STORAGE. boto3 fully lazy — free path needs no AWS.
- config/techbytes.config.yml: single user-facing config (sources + branding +
LLM choice). Icons live here as emoji literals, not in site code.
- pipeline/shared/{config,rebuild,utils,secrets}.py: cross-cutting helpers.
CloudWatch metrics no-op outside AWS; rebuild trigger no-op on free path.
Site (Astro)
- Branding (title/tagline/url) read from config at build, PUBLIC_* env overrides.
- Hardcoded `categoryIcons` map deleted; icons come from data (which comes from
config) as emoji literals.
- Data dir resolved via $DATA_DIR; Pages base path auto-detected from
GITHUB_REPOSITORY.
- Contribute page repo URL resolved from $GITHUB_REPOSITORY / PUBLIC_GITHUB_REPO.
Workflows
- digest.yml (NEW): cron -> run pipeline -> commit data/ -> build -> Pages.
- deploy-aws.yml (renamed): optional advanced path; pulls domain/zone/repo from
vars/context, no hardcoded values.
- preview.yml: ruff + pytest + site build + best-effort cdk synth.
Infra (CDK, optional)
- Fully parameterized via context: domain.name, hosted.zone.name, github.repo,
ssm.prefix. No hardcoded domain/account/repo.
- Lambda handler paths updated for the pipeline/ rename.
OSS scaffolding
- LICENSE (MIT), README (quickstart + config + provider setup), CONTRIBUTING,
regenerated .env.example.
- All personal identifiers removed (domain, AWS profile, repo slug, SSM paths,
model id, rebuild URL). Grep gate clean across source + build output.
Tests
- 34 pass: existing release_radar/gh_trending schema tests updated for the new
config-driven icon; new test_pipeline.py covers provider factory, storage
factory, and local_fs behavior.
Decisions
- License: MIT. Brand: keeps "Tech Bytes" as the demo default (forkers override
via config). Dir renamed lambdas/ -> pipeline/. Anthropic default model:
claude-sonnet-4-6.
@datj9
datj9force-pushed the feat/genericize-oss branch from d7a1562 to fdd6384CompareAugust 16, 2026 23:17
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@datj9
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat: genericize into self-hostable OSS project - #6

Open
datj9 wants to merge 1 commit into
masterfrom
feat/genericize-oss
Open

feat: genericize into self-hostable OSS project#6
datj9 wants to merge 1 commit into
masterfrom
feat/genericize-oss

Conversation

@datj9

Copy link
Copy Markdown
Owner

Summary

Turns tech-bytes into a real OSS project: anyone can fork, edit one config file, plug in their LLM key (OpenAI / Anthropic / local), and launch with zero AWS. GitHub Actions builds the digest on a cron and publishes to GitHub Pages. AWS/CDK stays as an optional advanced path.

Plan: ~/.claude/plans/purring-kindling-cray.md

What changed

Pipeline (lambdas/pipeline/)

  • Pluggable LLM (pipeline/providers/): openai, anthropic, local (OpenAI-compatible: Ollama, LM Studio, Groq, OpenRouter) behind one Protocol. Selected by llm.provider in config; keys from env, SSM fallback on AWS.
  • Pluggable storage (pipeline/storage/): local_fs (default, writes data/) and s3 (AWS path) behind one Protocol. boto3 fully lazy — free path imports no AWS libs.
  • Single CLI: python -m pipeline.run [all|release_radar|hn_digest|gh_trending|email_digest].
  • Single config: config/techbytes.config.yml owns sources + branding + LLM choice. Icons are emoji literals in config (the site reads them directly; the old hardcoded categoryIcons map is gone).
  • CloudWatch metrics no-op outside AWS. trigger_rebuild no-op on the free path.

Site (Astro)

  • Branding read from config at build time; PUBLIC_* env overrides for CI.
  • Pages base path auto-detected from GITHUB_REPOSITORY for project Pages sites.
  • Data dir resolved from $DATA_DIR (default ../data).
  • contribute.astro repo URL resolved from env — no hardcoded datj9/tech-bytes.

Workflows

  • digest.yml (NEW) — free path: cron → run pipeline → commit data/ → build → deploy Pages. No AWS.
  • deploy-aws.yml (renamed from deploy.yml) — optional AWS path; reads domain/zone/repo from vars/context.
  • preview.yml — ruff + pytest + site build + best-effort cdk synth on every PR.

Infra (CDK, optional)

  • Fully parameterized: domain.name, hosted.zone.name, github.repo, ssm.prefix from context. No hardcoded values; clear error if missing. See infra/README.md.

OSS scaffolding

  • LICENSE (MIT), README.md (60-second quickstart + config reference + provider setup), CONTRIBUTING.md, regenerated .env.example.
  • All personal identifiers stripped (domain, AWS profile, repo slug, SSM paths, rebuild URL). Grep gate clean across source and build output.

Decisions (locked at execution)

  • License: MIT
  • Brand: keeps "Tech Bytes" as the demo default (forkers override via config)
  • Dir rename: lambdas/pipeline/ (git mv preserves history)
  • Anthropic default model: claude-sonnet-4-6

Verification

All green locally:

CheckResult
ruff check pipeline/clean
pytest pipeline/tests/34/34 pass
pnpm run build (site)success
tsc --noEmit (infra)clean
python -m pipeline.run release_radar (smoke, local storage, fake key)writes data/release-radar.json, no boto3 import
Grep gate finaldivision|datj9|super-personal in site/dist/clean
Grep gate in source (excluding intentional default /tech-bytes/ SSM namespace)clean

preview.yml runs ruff + pytest + site build + cdk synth on this PR.

Test plan for reviewer

  1. Free path (local):cp config/techbytes.config.example.yml config/techbytes.config.yml → set OPENAI_API_KEY (or LLM_PROVIDER=anthropic + ANTHROPIC_API_KEY) → python -m pipeline.run all → confirm data/*.json written, no AWS deps required.
  2. Site:cd site && npm i && npm run build → confirm generic branding, no finaldivision / hardcoded icons in dist/.
  3. Provider swap: toggle llm.provider between openai / anthropic / local in config; confirm the factory picks the right provider and default model.
  4. Fork simulation: set GITHUB_REPOSITORY=myuser/myrepo before pnpm run build; confirm Pages base path and contribute page use the fork's identity.

Breaking changes

  • config/technologies.yml removed → migrate to config/techbytes.config.yml (see example).
  • data/release-radar.json icon field now an emoji literal (was a categoryIcons key string). Seed data migrated; first pipeline run regenerates everything.
  • Lambda handler paths changed from release_radar.handler.handler to pipeline.release_radar.handler.handler — CDK bundling updated to match. Existing AWS deployments need cdk deploy to pick up the new handler paths.

Turn tech-bytes into a real OSS project anyone can fork, configure, and run
with zero AWS. The default deploy path is now GitHub Pages via a scheduled
GitHub Actions workflow; AWS/CDK is demoted to an optional advanced path.
Architecture
- pipeline/ (renamed from lambdas/): framework-agnostic Python package with a
single CLI entrypoint `python -m pipeline.run [all|source]`.
- pipeline/providers/: pluggable LLM (openai, anthropic, local/OAI-compatible)
behind one Protocol. Selected by config; keys from env (SSM fallback on AWS).
- pipeline/storage/: pluggable storage (local_fs default, s3 optional) behind
one Protocol. Selected by $STORAGE. boto3 fully lazy — free path needs no AWS.
- config/techbytes.config.yml: single user-facing config (sources + branding +
LLM choice). Icons live here as emoji literals, not in site code.
- pipeline/shared/{config,rebuild,utils,secrets}.py: cross-cutting helpers.
CloudWatch metrics no-op outside AWS; rebuild trigger no-op on free path.
Site (Astro)
- Branding (title/tagline/url) read from config at build, PUBLIC_* env overrides.
- Hardcoded `categoryIcons` map deleted; icons come from data (which comes from
config) as emoji literals.
- Data dir resolved via $DATA_DIR; Pages base path auto-detected from
GITHUB_REPOSITORY.
- Contribute page repo URL resolved from $GITHUB_REPOSITORY / PUBLIC_GITHUB_REPO.
Workflows
- digest.yml (NEW): cron -> run pipeline -> commit data/ -> build -> Pages.
- deploy-aws.yml (renamed): optional advanced path; pulls domain/zone/repo from
vars/context, no hardcoded values.
- preview.yml: ruff + pytest + site build + best-effort cdk synth.
Infra (CDK, optional)
- Fully parameterized via context: domain.name, hosted.zone.name, github.repo,
ssm.prefix. No hardcoded domain/account/repo.
- Lambda handler paths updated for the pipeline/ rename.
OSS scaffolding
- LICENSE (MIT), README (quickstart + config + provider setup), CONTRIBUTING,
regenerated .env.example.
- All personal identifiers removed (domain, AWS profile, repo slug, SSM paths,
model id, rebuild URL). Grep gate clean across source + build output.
Tests
- 34 pass: existing release_radar/gh_trending schema tests updated for the new
config-driven icon; new test_pipeline.py covers provider factory, storage
factory, and local_fs behavior.
Decisions
- License: MIT. Brand: keeps "Tech Bytes" as the demo default (forkers override
via config). Dir renamed lambdas/ -> pipeline/. Anthropic default model:
claude-sonnet-4-6.
@datj9
datj9force-pushed the feat/genericize-oss branch from d7a1562 to fdd6384CompareAugust 16, 2026 23:17
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@datj9
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat: genericize into self-hostable OSS project - #6

Open
datj9 wants to merge 1 commit into
masterfrom
feat/genericize-oss
Open

feat: genericize into self-hostable OSS project#6
datj9 wants to merge 1 commit into
masterfrom
feat/genericize-oss

Conversation

@datj9

Copy link
Copy Markdown
Owner

Summary

Turns tech-bytes into a real OSS project: anyone can fork, edit one config file, plug in their LLM key (OpenAI / Anthropic / local), and launch with zero AWS. GitHub Actions builds the digest on a cron and publishes to GitHub Pages. AWS/CDK stays as an optional advanced path.

Plan: ~/.claude/plans/purring-kindling-cray.md

What changed

Pipeline (lambdas/pipeline/)

  • Pluggable LLM (pipeline/providers/): openai, anthropic, local (OpenAI-compatible: Ollama, LM Studio, Groq, OpenRouter) behind one Protocol. Selected by llm.provider in config; keys from env, SSM fallback on AWS.
  • Pluggable storage (pipeline/storage/): local_fs (default, writes data/) and s3 (AWS path) behind one Protocol. boto3 fully lazy — free path imports no AWS libs.
  • Single CLI: python -m pipeline.run [all|release_radar|hn_digest|gh_trending|email_digest].
  • Single config: config/techbytes.config.yml owns sources + branding + LLM choice. Icons are emoji literals in config (the site reads them directly; the old hardcoded categoryIcons map is gone).
  • CloudWatch metrics no-op outside AWS. trigger_rebuild no-op on the free path.

Site (Astro)

  • Branding read from config at build time; PUBLIC_* env overrides for CI.
  • Pages base path auto-detected from GITHUB_REPOSITORY for project Pages sites.
  • Data dir resolved from $DATA_DIR (default ../data).
  • contribute.astro repo URL resolved from env — no hardcoded datj9/tech-bytes.

Workflows

  • digest.yml (NEW) — free path: cron → run pipeline → commit data/ → build → deploy Pages. No AWS.
  • deploy-aws.yml (renamed from deploy.yml) — optional AWS path; reads domain/zone/repo from vars/context.
  • preview.yml — ruff + pytest + site build + best-effort cdk synth on every PR.

Infra (CDK, optional)

  • Fully parameterized: domain.name, hosted.zone.name, github.repo, ssm.prefix from context. No hardcoded values; clear error if missing. See infra/README.md.

OSS scaffolding

  • LICENSE (MIT), README.md (60-second quickstart + config reference + provider setup), CONTRIBUTING.md, regenerated .env.example.
  • All personal identifiers stripped (domain, AWS profile, repo slug, SSM paths, rebuild URL). Grep gate clean across source and build output.

Decisions (locked at execution)

  • License: MIT
  • Brand: keeps "Tech Bytes" as the demo default (forkers override via config)
  • Dir rename: lambdas/pipeline/ (git mv preserves history)
  • Anthropic default model: claude-sonnet-4-6

Verification

All green locally:

CheckResult
ruff check pipeline/clean
pytest pipeline/tests/34/34 pass
pnpm run build (site)success
tsc --noEmit (infra)clean
python -m pipeline.run release_radar (smoke, local storage, fake key)writes data/release-radar.json, no boto3 import
Grep gate finaldivision|datj9|super-personal in site/dist/clean
Grep gate in source (excluding intentional default /tech-bytes/ SSM namespace)clean

preview.yml runs ruff + pytest + site build + cdk synth on this PR.

Test plan for reviewer

  1. Free path (local):cp config/techbytes.config.example.yml config/techbytes.config.yml → set OPENAI_API_KEY (or LLM_PROVIDER=anthropic + ANTHROPIC_API_KEY) → python -m pipeline.run all → confirm data/*.json written, no AWS deps required.
  2. Site:cd site && npm i && npm run build → confirm generic branding, no finaldivision / hardcoded icons in dist/.
  3. Provider swap: toggle llm.provider between openai / anthropic / local in config; confirm the factory picks the right provider and default model.
  4. Fork simulation: set GITHUB_REPOSITORY=myuser/myrepo before pnpm run build; confirm Pages base path and contribute page use the fork's identity.

Breaking changes

  • config/technologies.yml removed → migrate to config/techbytes.config.yml (see example).
  • data/release-radar.json icon field now an emoji literal (was a categoryIcons key string). Seed data migrated; first pipeline run regenerates everything.
  • Lambda handler paths changed from release_radar.handler.handler to pipeline.release_radar.handler.handler — CDK bundling updated to match. Existing AWS deployments need cdk deploy to pick up the new handler paths.

Turn tech-bytes into a real OSS project anyone can fork, configure, and run
with zero AWS. The default deploy path is now GitHub Pages via a scheduled
GitHub Actions workflow; AWS/CDK is demoted to an optional advanced path.
Architecture
- pipeline/ (renamed from lambdas/): framework-agnostic Python package with a
single CLI entrypoint `python -m pipeline.run [all|source]`.
- pipeline/providers/: pluggable LLM (openai, anthropic, local/OAI-compatible)
behind one Protocol. Selected by config; keys from env (SSM fallback on AWS).
- pipeline/storage/: pluggable storage (local_fs default, s3 optional) behind
one Protocol. Selected by $STORAGE. boto3 fully lazy — free path needs no AWS.
- config/techbytes.config.yml: single user-facing config (sources + branding +
LLM choice). Icons live here as emoji literals, not in site code.
- pipeline/shared/{config,rebuild,utils,secrets}.py: cross-cutting helpers.
CloudWatch metrics no-op outside AWS; rebuild trigger no-op on free path.
Site (Astro)
- Branding (title/tagline/url) read from config at build, PUBLIC_* env overrides.
- Hardcoded `categoryIcons` map deleted; icons come from data (which comes from
config) as emoji literals.
- Data dir resolved via $DATA_DIR; Pages base path auto-detected from
GITHUB_REPOSITORY.
- Contribute page repo URL resolved from $GITHUB_REPOSITORY / PUBLIC_GITHUB_REPO.
Workflows
- digest.yml (NEW): cron -> run pipeline -> commit data/ -> build -> Pages.
- deploy-aws.yml (renamed): optional advanced path; pulls domain/zone/repo from
vars/context, no hardcoded values.
- preview.yml: ruff + pytest + site build + best-effort cdk synth.
Infra (CDK, optional)
- Fully parameterized via context: domain.name, hosted.zone.name, github.repo,
ssm.prefix. No hardcoded domain/account/repo.
- Lambda handler paths updated for the pipeline/ rename.
OSS scaffolding
- LICENSE (MIT), README (quickstart + config + provider setup), CONTRIBUTING,
regenerated .env.example.
- All personal identifiers removed (domain, AWS profile, repo slug, SSM paths,
model id, rebuild URL). Grep gate clean across source + build output.
Tests
- 34 pass: existing release_radar/gh_trending schema tests updated for the new
config-driven icon; new test_pipeline.py covers provider factory, storage
factory, and local_fs behavior.
Decisions
- License: MIT. Brand: keeps "Tech Bytes" as the demo default (forkers override
via config). Dir renamed lambdas/ -> pipeline/. Anthropic default model:
claude-sonnet-4-6.
@datj9
datj9force-pushed the feat/genericize-oss branch from d7a1562 to fdd6384CompareAugust 16, 2026 23:17
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@datj9
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

feat: genericize into self-hostable OSS project - #6

Open
datj9 wants to merge 1 commit into
masterfrom
feat/genericize-oss
Open

feat: genericize into self-hostable OSS project#6
datj9 wants to merge 1 commit into
masterfrom
feat/genericize-oss

Conversation

@datj9

Copy link
Copy Markdown
Owner

Summary

Turns tech-bytes into a real OSS project: anyone can fork, edit one config file, plug in their LLM key (OpenAI / Anthropic / local), and launch with zero AWS. GitHub Actions builds the digest on a cron and publishes to GitHub Pages. AWS/CDK stays as an optional advanced path.

Plan: ~/.claude/plans/purring-kindling-cray.md

What changed

Pipeline (lambdas/pipeline/)

  • Pluggable LLM (pipeline/providers/): openai, anthropic, local (OpenAI-compatible: Ollama, LM Studio, Groq, OpenRouter) behind one Protocol. Selected by llm.provider in config; keys from env, SSM fallback on AWS.
  • Pluggable storage (pipeline/storage/): local_fs (default, writes data/) and s3 (AWS path) behind one Protocol. boto3 fully lazy — free path imports no AWS libs.
  • Single CLI: python -m pipeline.run [all|release_radar|hn_digest|gh_trending|email_digest].
  • Single config: config/techbytes.config.yml owns sources + branding + LLM choice. Icons are emoji literals in config (the site reads them directly; the old hardcoded categoryIcons map is gone).
  • CloudWatch metrics no-op outside AWS. trigger_rebuild no-op on the free path.

Site (Astro)

  • Branding read from config at build time; PUBLIC_* env overrides for CI.
  • Pages base path auto-detected from GITHUB_REPOSITORY for project Pages sites.
  • Data dir resolved from $DATA_DIR (default ../data).
  • contribute.astro repo URL resolved from env — no hardcoded datj9/tech-bytes.

Workflows

  • digest.yml (NEW) — free path: cron → run pipeline → commit data/ → build → deploy Pages. No AWS.
  • deploy-aws.yml (renamed from deploy.yml) — optional AWS path; reads domain/zone/repo from vars/context.
  • preview.yml — ruff + pytest + site build + best-effort cdk synth on every PR.

Infra (CDK, optional)

  • Fully parameterized: domain.name, hosted.zone.name, github.repo, ssm.prefix from context. No hardcoded values; clear error if missing. See infra/README.md.

OSS scaffolding

  • LICENSE (MIT), README.md (60-second quickstart + config reference + provider setup), CONTRIBUTING.md, regenerated .env.example.
  • All personal identifiers stripped (domain, AWS profile, repo slug, SSM paths, rebuild URL). Grep gate clean across source and build output.

Decisions (locked at execution)

  • License: MIT
  • Brand: keeps "Tech Bytes" as the demo default (forkers override via config)
  • Dir rename: lambdas/pipeline/ (git mv preserves history)
  • Anthropic default model: claude-sonnet-4-6

Verification

All green locally:

CheckResult
ruff check pipeline/clean
pytest pipeline/tests/34/34 pass
pnpm run build (site)success
tsc --noEmit (infra)clean
python -m pipeline.run release_radar (smoke, local storage, fake key)writes data/release-radar.json, no boto3 import
Grep gate finaldivision|datj9|super-personal in site/dist/clean
Grep gate in source (excluding intentional default /tech-bytes/ SSM namespace)clean

preview.yml runs ruff + pytest + site build + cdk synth on this PR.

Test plan for reviewer

  1. Free path (local):cp config/techbytes.config.example.yml config/techbytes.config.yml → set OPENAI_API_KEY (or LLM_PROVIDER=anthropic + ANTHROPIC_API_KEY) → python -m pipeline.run all → confirm data/*.json written, no AWS deps required.
  2. Site:cd site && npm i && npm run build → confirm generic branding, no finaldivision / hardcoded icons in dist/.
  3. Provider swap: toggle llm.provider between openai / anthropic / local in config; confirm the factory picks the right provider and default model.
  4. Fork simulation: set GITHUB_REPOSITORY=myuser/myrepo before pnpm run build; confirm Pages base path and contribute page use the fork's identity.

Breaking changes

  • config/technologies.yml removed → migrate to config/techbytes.config.yml (see example).
  • data/release-radar.json icon field now an emoji literal (was a categoryIcons key string). Seed data migrated; first pipeline run regenerates everything.
  • Lambda handler paths changed from release_radar.handler.handler to pipeline.release_radar.handler.handler — CDK bundling updated to match. Existing AWS deployments need cdk deploy to pick up the new handler paths.

Turn tech-bytes into a real OSS project anyone can fork, configure, and run
with zero AWS. The default deploy path is now GitHub Pages via a scheduled
GitHub Actions workflow; AWS/CDK is demoted to an optional advanced path.
Architecture
- pipeline/ (renamed from lambdas/): framework-agnostic Python package with a
single CLI entrypoint `python -m pipeline.run [all|source]`.
- pipeline/providers/: pluggable LLM (openai, anthropic, local/OAI-compatible)
behind one Protocol. Selected by config; keys from env (SSM fallback on AWS).
- pipeline/storage/: pluggable storage (local_fs default, s3 optional) behind
one Protocol. Selected by $STORAGE. boto3 fully lazy — free path needs no AWS.
- config/techbytes.config.yml: single user-facing config (sources + branding +
LLM choice). Icons live here as emoji literals, not in site code.
- pipeline/shared/{config,rebuild,utils,secrets}.py: cross-cutting helpers.
CloudWatch metrics no-op outside AWS; rebuild trigger no-op on free path.
Site (Astro)
- Branding (title/tagline/url) read from config at build, PUBLIC_* env overrides.
- Hardcoded `categoryIcons` map deleted; icons come from data (which comes from
config) as emoji literals.
- Data dir resolved via $DATA_DIR; Pages base path auto-detected from
GITHUB_REPOSITORY.
- Contribute page repo URL resolved from $GITHUB_REPOSITORY / PUBLIC_GITHUB_REPO.
Workflows
- digest.yml (NEW): cron -> run pipeline -> commit data/ -> build -> Pages.
- deploy-aws.yml (renamed): optional advanced path; pulls domain/zone/repo from
vars/context, no hardcoded values.
- preview.yml: ruff + pytest + site build + best-effort cdk synth.
Infra (CDK, optional)
- Fully parameterized via context: domain.name, hosted.zone.name, github.repo,
ssm.prefix. No hardcoded domain/account/repo.
- Lambda handler paths updated for the pipeline/ rename.
OSS scaffolding
- LICENSE (MIT), README (quickstart + config + provider setup), CONTRIBUTING,
regenerated .env.example.
- All personal identifiers removed (domain, AWS profile, repo slug, SSM paths,
model id, rebuild URL). Grep gate clean across source + build output.
Tests
- 34 pass: existing release_radar/gh_trending schema tests updated for the new
config-driven icon; new test_pipeline.py covers provider factory, storage
factory, and local_fs behavior.
Decisions
- License: MIT. Brand: keeps "Tech Bytes" as the demo default (forkers override
via config). Dir renamed lambdas/ -> pipeline/. Anthropic default model:
claude-sonnet-4-6.
@datj9
datj9force-pushed the feat/genericize-oss branch from d7a1562 to fdd6384CompareAugust 16, 2026 23:17
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@datj9
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat: genericize into self-hostable OSS project - #6

Open
datj9 wants to merge 1 commit into
masterfrom
feat/genericize-oss
Open

feat: genericize into self-hostable OSS project#6
datj9 wants to merge 1 commit into
masterfrom
feat/genericize-oss

Conversation

@datj9

Copy link
Copy Markdown
Owner

Summary

Turns tech-bytes into a real OSS project: anyone can fork, edit one config file, plug in their LLM key (OpenAI / Anthropic / local), and launch with zero AWS. GitHub Actions builds the digest on a cron and publishes to GitHub Pages. AWS/CDK stays as an optional advanced path.

Plan: ~/.claude/plans/purring-kindling-cray.md

What changed

Pipeline (lambdas/pipeline/)

  • Pluggable LLM (pipeline/providers/): openai, anthropic, local (OpenAI-compatible: Ollama, LM Studio, Groq, OpenRouter) behind one Protocol. Selected by llm.provider in config; keys from env, SSM fallback on AWS.
  • Pluggable storage (pipeline/storage/): local_fs (default, writes data/) and s3 (AWS path) behind one Protocol. boto3 fully lazy — free path imports no AWS libs.
  • Single CLI: python -m pipeline.run [all|release_radar|hn_digest|gh_trending|email_digest].
  • Single config: config/techbytes.config.yml owns sources + branding + LLM choice. Icons are emoji literals in config (the site reads them directly; the old hardcoded categoryIcons map is gone).
  • CloudWatch metrics no-op outside AWS. trigger_rebuild no-op on the free path.

Site (Astro)

  • Branding read from config at build time; PUBLIC_* env overrides for CI.
  • Pages base path auto-detected from GITHUB_REPOSITORY for project Pages sites.
  • Data dir resolved from $DATA_DIR (default ../data).
  • contribute.astro repo URL resolved from env — no hardcoded datj9/tech-bytes.

Workflows

  • digest.yml (NEW) — free path: cron → run pipeline → commit data/ → build → deploy Pages. No AWS.
  • deploy-aws.yml (renamed from deploy.yml) — optional AWS path; reads domain/zone/repo from vars/context.
  • preview.yml — ruff + pytest + site build + best-effort cdk synth on every PR.

Infra (CDK, optional)

  • Fully parameterized: domain.name, hosted.zone.name, github.repo, ssm.prefix from context. No hardcoded values; clear error if missing. See infra/README.md.

OSS scaffolding

  • LICENSE (MIT), README.md (60-second quickstart + config reference + provider setup), CONTRIBUTING.md, regenerated .env.example.
  • All personal identifiers stripped (domain, AWS profile, repo slug, SSM paths, rebuild URL). Grep gate clean across source and build output.

Decisions (locked at execution)

  • License: MIT
  • Brand: keeps "Tech Bytes" as the demo default (forkers override via config)
  • Dir rename: lambdas/pipeline/ (git mv preserves history)
  • Anthropic default model: claude-sonnet-4-6

Verification

All green locally:

CheckResult
ruff check pipeline/clean
pytest pipeline/tests/34/34 pass
pnpm run build (site)success
tsc --noEmit (infra)clean
python -m pipeline.run release_radar (smoke, local storage, fake key)writes data/release-radar.json, no boto3 import
Grep gate finaldivision|datj9|super-personal in site/dist/clean
Grep gate in source (excluding intentional default /tech-bytes/ SSM namespace)clean

preview.yml runs ruff + pytest + site build + cdk synth on this PR.

Test plan for reviewer

  1. Free path (local):cp config/techbytes.config.example.yml config/techbytes.config.yml → set OPENAI_API_KEY (or LLM_PROVIDER=anthropic + ANTHROPIC_API_KEY) → python -m pipeline.run all → confirm data/*.json written, no AWS deps required.
  2. Site:cd site && npm i && npm run build → confirm generic branding, no finaldivision / hardcoded icons in dist/.
  3. Provider swap: toggle llm.provider between openai / anthropic / local in config; confirm the factory picks the right provider and default model.
  4. Fork simulation: set GITHUB_REPOSITORY=myuser/myrepo before pnpm run build; confirm Pages base path and contribute page use the fork's identity.

Breaking changes

  • config/technologies.yml removed → migrate to config/techbytes.config.yml (see example).
  • data/release-radar.json icon field now an emoji literal (was a categoryIcons key string). Seed data migrated; first pipeline run regenerates everything.
  • Lambda handler paths changed from release_radar.handler.handler to pipeline.release_radar.handler.handler — CDK bundling updated to match. Existing AWS deployments need cdk deploy to pick up the new handler paths.

Turn tech-bytes into a real OSS project anyone can fork, configure, and run
with zero AWS. The default deploy path is now GitHub Pages via a scheduled
GitHub Actions workflow; AWS/CDK is demoted to an optional advanced path.
Architecture
- pipeline/ (renamed from lambdas/): framework-agnostic Python package with a
single CLI entrypoint `python -m pipeline.run [all|source]`.
- pipeline/providers/: pluggable LLM (openai, anthropic, local/OAI-compatible)
behind one Protocol. Selected by config; keys from env (SSM fallback on AWS).
- pipeline/storage/: pluggable storage (local_fs default, s3 optional) behind
one Protocol. Selected by $STORAGE. boto3 fully lazy — free path needs no AWS.
- config/techbytes.config.yml: single user-facing config (sources + branding +
LLM choice). Icons live here as emoji literals, not in site code.
- pipeline/shared/{config,rebuild,utils,secrets}.py: cross-cutting helpers.
CloudWatch metrics no-op outside AWS; rebuild trigger no-op on free path.
Site (Astro)
- Branding (title/tagline/url) read from config at build, PUBLIC_* env overrides.
- Hardcoded `categoryIcons` map deleted; icons come from data (which comes from
config) as emoji literals.
- Data dir resolved via $DATA_DIR; Pages base path auto-detected from
GITHUB_REPOSITORY.
- Contribute page repo URL resolved from $GITHUB_REPOSITORY / PUBLIC_GITHUB_REPO.
Workflows
- digest.yml (NEW): cron -> run pipeline -> commit data/ -> build -> Pages.
- deploy-aws.yml (renamed): optional advanced path; pulls domain/zone/repo from
vars/context, no hardcoded values.
- preview.yml: ruff + pytest + site build + best-effort cdk synth.
Infra (CDK, optional)
- Fully parameterized via context: domain.name, hosted.zone.name, github.repo,
ssm.prefix. No hardcoded domain/account/repo.
- Lambda handler paths updated for the pipeline/ rename.
OSS scaffolding
- LICENSE (MIT), README (quickstart + config + provider setup), CONTRIBUTING,
regenerated .env.example.
- All personal identifiers removed (domain, AWS profile, repo slug, SSM paths,
model id, rebuild URL). Grep gate clean across source + build output.
Tests
- 34 pass: existing release_radar/gh_trending schema tests updated for the new
config-driven icon; new test_pipeline.py covers provider factory, storage
factory, and local_fs behavior.
Decisions
- License: MIT. Brand: keeps "Tech Bytes" as the demo default (forkers override
via config). Dir renamed lambdas/ -> pipeline/. Anthropic default model:
claude-sonnet-4-6.
@datj9
datj9force-pushed the feat/genericize-oss branch from d7a1562 to fdd6384CompareAugust 16, 2026 23:17
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@datj9
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat: genericize into self-hostable OSS project - #6

Open
datj9 wants to merge 1 commit into
masterfrom
feat/genericize-oss
Open

feat: genericize into self-hostable OSS project#6
datj9 wants to merge 1 commit into
masterfrom
feat/genericize-oss

Conversation

@datj9

Copy link
Copy Markdown
Owner

Summary

Turns tech-bytes into a real OSS project: anyone can fork, edit one config file, plug in their LLM key (OpenAI / Anthropic / local), and launch with zero AWS. GitHub Actions builds the digest on a cron and publishes to GitHub Pages. AWS/CDK stays as an optional advanced path.

Plan: ~/.claude/plans/purring-kindling-cray.md

What changed

Pipeline (lambdas/pipeline/)

  • Pluggable LLM (pipeline/providers/): openai, anthropic, local (OpenAI-compatible: Ollama, LM Studio, Groq, OpenRouter) behind one Protocol. Selected by llm.provider in config; keys from env, SSM fallback on AWS.
  • Pluggable storage (pipeline/storage/): local_fs (default, writes data/) and s3 (AWS path) behind one Protocol. boto3 fully lazy — free path imports no AWS libs.
  • Single CLI: python -m pipeline.run [all|release_radar|hn_digest|gh_trending|email_digest].
  • Single config: config/techbytes.config.yml owns sources + branding + LLM choice. Icons are emoji literals in config (the site reads them directly; the old hardcoded categoryIcons map is gone).
  • CloudWatch metrics no-op outside AWS. trigger_rebuild no-op on the free path.

Site (Astro)

  • Branding read from config at build time; PUBLIC_* env overrides for CI.
  • Pages base path auto-detected from GITHUB_REPOSITORY for project Pages sites.
  • Data dir resolved from $DATA_DIR (default ../data).
  • contribute.astro repo URL resolved from env — no hardcoded datj9/tech-bytes.

Workflows

  • digest.yml (NEW) — free path: cron → run pipeline → commit data/ → build → deploy Pages. No AWS.
  • deploy-aws.yml (renamed from deploy.yml) — optional AWS path; reads domain/zone/repo from vars/context.
  • preview.yml — ruff + pytest + site build + best-effort cdk synth on every PR.

Infra (CDK, optional)

  • Fully parameterized: domain.name, hosted.zone.name, github.repo, ssm.prefix from context. No hardcoded values; clear error if missing. See infra/README.md.

OSS scaffolding

  • LICENSE (MIT), README.md (60-second quickstart + config reference + provider setup), CONTRIBUTING.md, regenerated .env.example.
  • All personal identifiers stripped (domain, AWS profile, repo slug, SSM paths, rebuild URL). Grep gate clean across source and build output.

Decisions (locked at execution)

  • License: MIT
  • Brand: keeps "Tech Bytes" as the demo default (forkers override via config)
  • Dir rename: lambdas/pipeline/ (git mv preserves history)
  • Anthropic default model: claude-sonnet-4-6

Verification

All green locally:

CheckResult
ruff check pipeline/clean
pytest pipeline/tests/34/34 pass
pnpm run build (site)success
tsc --noEmit (infra)clean
python -m pipeline.run release_radar (smoke, local storage, fake key)writes data/release-radar.json, no boto3 import
Grep gate finaldivision|datj9|super-personal in site/dist/clean
Grep gate in source (excluding intentional default /tech-bytes/ SSM namespace)clean

preview.yml runs ruff + pytest + site build + cdk synth on this PR.

Test plan for reviewer

  1. Free path (local):cp config/techbytes.config.example.yml config/techbytes.config.yml → set OPENAI_API_KEY (or LLM_PROVIDER=anthropic + ANTHROPIC_API_KEY) → python -m pipeline.run all → confirm data/*.json written, no AWS deps required.
  2. Site:cd site && npm i && npm run build → confirm generic branding, no finaldivision / hardcoded icons in dist/.
  3. Provider swap: toggle llm.provider between openai / anthropic / local in config; confirm the factory picks the right provider and default model.
  4. Fork simulation: set GITHUB_REPOSITORY=myuser/myrepo before pnpm run build; confirm Pages base path and contribute page use the fork's identity.

Breaking changes

  • config/technologies.yml removed → migrate to config/techbytes.config.yml (see example).
  • data/release-radar.json icon field now an emoji literal (was a categoryIcons key string). Seed data migrated; first pipeline run regenerates everything.
  • Lambda handler paths changed from release_radar.handler.handler to pipeline.release_radar.handler.handler — CDK bundling updated to match. Existing AWS deployments need cdk deploy to pick up the new handler paths.

Turn tech-bytes into a real OSS project anyone can fork, configure, and run
with zero AWS. The default deploy path is now GitHub Pages via a scheduled
GitHub Actions workflow; AWS/CDK is demoted to an optional advanced path.
Architecture
- pipeline/ (renamed from lambdas/): framework-agnostic Python package with a
single CLI entrypoint `python -m pipeline.run [all|source]`.
- pipeline/providers/: pluggable LLM (openai, anthropic, local/OAI-compatible)
behind one Protocol. Selected by config; keys from env (SSM fallback on AWS).
- pipeline/storage/: pluggable storage (local_fs default, s3 optional) behind
one Protocol. Selected by $STORAGE. boto3 fully lazy — free path needs no AWS.
- config/techbytes.config.yml: single user-facing config (sources + branding +
LLM choice). Icons live here as emoji literals, not in site code.
- pipeline/shared/{config,rebuild,utils,secrets}.py: cross-cutting helpers.
CloudWatch metrics no-op outside AWS; rebuild trigger no-op on free path.
Site (Astro)
- Branding (title/tagline/url) read from config at build, PUBLIC_* env overrides.
- Hardcoded `categoryIcons` map deleted; icons come from data (which comes from
config) as emoji literals.
- Data dir resolved via $DATA_DIR; Pages base path auto-detected from
GITHUB_REPOSITORY.
- Contribute page repo URL resolved from $GITHUB_REPOSITORY / PUBLIC_GITHUB_REPO.
Workflows
- digest.yml (NEW): cron -> run pipeline -> commit data/ -> build -> Pages.
- deploy-aws.yml (renamed): optional advanced path; pulls domain/zone/repo from
vars/context, no hardcoded values.
- preview.yml: ruff + pytest + site build + best-effort cdk synth.
Infra (CDK, optional)
- Fully parameterized via context: domain.name, hosted.zone.name, github.repo,
ssm.prefix. No hardcoded domain/account/repo.
- Lambda handler paths updated for the pipeline/ rename.
OSS scaffolding
- LICENSE (MIT), README (quickstart + config + provider setup), CONTRIBUTING,
regenerated .env.example.
- All personal identifiers removed (domain, AWS profile, repo slug, SSM paths,
model id, rebuild URL). Grep gate clean across source + build output.
Tests
- 34 pass: existing release_radar/gh_trending schema tests updated for the new
config-driven icon; new test_pipeline.py covers provider factory, storage
factory, and local_fs behavior.
Decisions
- License: MIT. Brand: keeps "Tech Bytes" as the demo default (forkers override
via config). Dir renamed lambdas/ -> pipeline/. Anthropic default model:
claude-sonnet-4-6.
@datj9
datj9force-pushed the feat/genericize-oss branch from d7a1562 to fdd6384CompareAugust 16, 2026 23:17
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@datj9
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

feat: genericize into self-hostable OSS project - #6

Open
datj9 wants to merge 1 commit into
masterfrom
feat/genericize-oss
Open

feat: genericize into self-hostable OSS project#6
datj9 wants to merge 1 commit into
masterfrom
feat/genericize-oss

Conversation

@datj9

Copy link
Copy Markdown
Owner

Summary

Turns tech-bytes into a real OSS project: anyone can fork, edit one config file, plug in their LLM key (OpenAI / Anthropic / local), and launch with zero AWS. GitHub Actions builds the digest on a cron and publishes to GitHub Pages. AWS/CDK stays as an optional advanced path.

Plan: ~/.claude/plans/purring-kindling-cray.md

What changed

Pipeline (lambdas/pipeline/)

  • Pluggable LLM (pipeline/providers/): openai, anthropic, local (OpenAI-compatible: Ollama, LM Studio, Groq, OpenRouter) behind one Protocol. Selected by llm.provider in config; keys from env, SSM fallback on AWS.
  • Pluggable storage (pipeline/storage/): local_fs (default, writes data/) and s3 (AWS path) behind one Protocol. boto3 fully lazy — free path imports no AWS libs.
  • Single CLI: python -m pipeline.run [all|release_radar|hn_digest|gh_trending|email_digest].
  • Single config: config/techbytes.config.yml owns sources + branding + LLM choice. Icons are emoji literals in config (the site reads them directly; the old hardcoded categoryIcons map is gone).
  • CloudWatch metrics no-op outside AWS. trigger_rebuild no-op on the free path.

Site (Astro)

  • Branding read from config at build time; PUBLIC_* env overrides for CI.
  • Pages base path auto-detected from GITHUB_REPOSITORY for project Pages sites.
  • Data dir resolved from $DATA_DIR (default ../data).
  • contribute.astro repo URL resolved from env — no hardcoded datj9/tech-bytes.

Workflows

  • digest.yml (NEW) — free path: cron → run pipeline → commit data/ → build → deploy Pages. No AWS.
  • deploy-aws.yml (renamed from deploy.yml) — optional AWS path; reads domain/zone/repo from vars/context.
  • preview.yml — ruff + pytest + site build + best-effort cdk synth on every PR.

Infra (CDK, optional)

  • Fully parameterized: domain.name, hosted.zone.name, github.repo, ssm.prefix from context. No hardcoded values; clear error if missing. See infra/README.md.

OSS scaffolding

  • LICENSE (MIT), README.md (60-second quickstart + config reference + provider setup), CONTRIBUTING.md, regenerated .env.example.
  • All personal identifiers stripped (domain, AWS profile, repo slug, SSM paths, rebuild URL). Grep gate clean across source and build output.

Decisions (locked at execution)

  • License: MIT
  • Brand: keeps "Tech Bytes" as the demo default (forkers override via config)
  • Dir rename: lambdas/pipeline/ (git mv preserves history)
  • Anthropic default model: claude-sonnet-4-6

Verification

All green locally:

CheckResult
ruff check pipeline/clean
pytest pipeline/tests/34/34 pass
pnpm run build (site)success
tsc --noEmit (infra)clean
python -m pipeline.run release_radar (smoke, local storage, fake key)writes data/release-radar.json, no boto3 import
Grep gate finaldivision|datj9|super-personal in site/dist/clean
Grep gate in source (excluding intentional default /tech-bytes/ SSM namespace)clean

preview.yml runs ruff + pytest + site build + cdk synth on this PR.

Test plan for reviewer

  1. Free path (local):cp config/techbytes.config.example.yml config/techbytes.config.yml → set OPENAI_API_KEY (or LLM_PROVIDER=anthropic + ANTHROPIC_API_KEY) → python -m pipeline.run all → confirm data/*.json written, no AWS deps required.
  2. Site:cd site && npm i && npm run build → confirm generic branding, no finaldivision / hardcoded icons in dist/.
  3. Provider swap: toggle llm.provider between openai / anthropic / local in config; confirm the factory picks the right provider and default model.
  4. Fork simulation: set GITHUB_REPOSITORY=myuser/myrepo before pnpm run build; confirm Pages base path and contribute page use the fork's identity.

Breaking changes

  • config/technologies.yml removed → migrate to config/techbytes.config.yml (see example).
  • data/release-radar.json icon field now an emoji literal (was a categoryIcons key string). Seed data migrated; first pipeline run regenerates everything.
  • Lambda handler paths changed from release_radar.handler.handler to pipeline.release_radar.handler.handler — CDK bundling updated to match. Existing AWS deployments need cdk deploy to pick up the new handler paths.

Turn tech-bytes into a real OSS project anyone can fork, configure, and run
with zero AWS. The default deploy path is now GitHub Pages via a scheduled
GitHub Actions workflow; AWS/CDK is demoted to an optional advanced path.
Architecture
- pipeline/ (renamed from lambdas/): framework-agnostic Python package with a
single CLI entrypoint `python -m pipeline.run [all|source]`.
- pipeline/providers/: pluggable LLM (openai, anthropic, local/OAI-compatible)
behind one Protocol. Selected by config; keys from env (SSM fallback on AWS).
- pipeline/storage/: pluggable storage (local_fs default, s3 optional) behind
one Protocol. Selected by $STORAGE. boto3 fully lazy — free path needs no AWS.
- config/techbytes.config.yml: single user-facing config (sources + branding +
LLM choice). Icons live here as emoji literals, not in site code.
- pipeline/shared/{config,rebuild,utils,secrets}.py: cross-cutting helpers.
CloudWatch metrics no-op outside AWS; rebuild trigger no-op on free path.
Site (Astro)
- Branding (title/tagline/url) read from config at build, PUBLIC_* env overrides.
- Hardcoded `categoryIcons` map deleted; icons come from data (which comes from
config) as emoji literals.
- Data dir resolved via $DATA_DIR; Pages base path auto-detected from
GITHUB_REPOSITORY.
- Contribute page repo URL resolved from $GITHUB_REPOSITORY / PUBLIC_GITHUB_REPO.
Workflows
- digest.yml (NEW): cron -> run pipeline -> commit data/ -> build -> Pages.
- deploy-aws.yml (renamed): optional advanced path; pulls domain/zone/repo from
vars/context, no hardcoded values.
- preview.yml: ruff + pytest + site build + best-effort cdk synth.
Infra (CDK, optional)
- Fully parameterized via context: domain.name, hosted.zone.name, github.repo,
ssm.prefix. No hardcoded domain/account/repo.
- Lambda handler paths updated for the pipeline/ rename.
OSS scaffolding
- LICENSE (MIT), README (quickstart + config + provider setup), CONTRIBUTING,
regenerated .env.example.
- All personal identifiers removed (domain, AWS profile, repo slug, SSM paths,
model id, rebuild URL). Grep gate clean across source + build output.
Tests
- 34 pass: existing release_radar/gh_trending schema tests updated for the new
config-driven icon; new test_pipeline.py covers provider factory, storage
factory, and local_fs behavior.
Decisions
- License: MIT. Brand: keeps "Tech Bytes" as the demo default (forkers override
via config). Dir renamed lambdas/ -> pipeline/. Anthropic default model:
claude-sonnet-4-6.
@datj9
datj9force-pushed the feat/genericize-oss branch from d7a1562 to fdd6384CompareAugust 16, 2026 23:17
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@datj9