Uh oh!
There was an error while loading. Please reload this page.
feat: daily arXiv researcher — scan papers for gh-aw improvement opportunities - #49609
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
| @@ -0,0 +1,94 @@ | |||
| """ | |||
There was a problem hiding this comment.
@copilot use JavaScript/github-script inlined into the AW, then remove this file
There was a problem hiding this comment.
Done in 85e3b9b. The Python script has been removed and its logic is now inlined as an actions/github-script@v9.0.0 step in the workflow. The JavaScript uses Node.js built-in https and fs to fetch and parse the arXiv Atom XML, with the same dedup logic against cache-memory. The export.arxiv.org entry was also dropped from network: since custom steps run outside the AWF firewall sandbox.
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Pull request overview
Adds a daily Claude workflow to discover arXiv research relevant to gh-aw improvements.
Changes:
- Fetches, parses, and deduplicates recent arXiv papers.
- Uses sub-agents to screen papers and extract opportunities.
- Persists a research ledger and creates discussions for findings.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/daily-arxiv-researcher.md | Defines fetching, analysis, memory, and reporting behavior. |
.github/workflows/daily-arxiv-researcher.lock.yml | Compiled GitHub Actions workflow. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 1/2 changed files
- Comments generated: 5
- Review effort level: Balanced
| safe-outputs: | ||
| create-discussion: | ||
| category: "research" |
| while ((em = entryRe.exec(xml)) !== null) { | ||
| const entry = em[1]; | ||
| const idUrl = getText('id', entry); | ||
| const arxivId = idUrl.replace(/.*abs\//, '').trim(); |
| try { | ||
| xml = await new Promise((resolve, reject) => { | ||
| const req = https.get(ARXIV_URL, { timeout: 30000 }, res => { | ||
| const chunks = []; | ||
| res.on('data', c => chunks.push(c)); | ||
| res.on('end', () => resolve(Buffer.concat(chunks).toString('utf8'))); | ||
| }); | ||
| req.on('error', reject); | ||
| req.on('timeout', () => { req.destroy(); reject(new Error('request timed out')); }); | ||
| }); | ||
| } catch (e) { | ||
| core.warning(`arXiv fetch failed: ${e.message}`); | ||
| } |
| Write the updated ledger to `/tmp/gh-aw/repo-memory/default/paper-ledger.md`. | ||
| Write the updated index to `/tmp/gh-aw/repo-memory/default/paper-index.json`. |
| - name: Fetch and parse arXiv papers | ||
| uses: actions/github-script@v9.0.0 |
🎉 This pull request is included in a new release. Release: |
Daily Claude workflow that skims latest arXiv papers (cs.AI/SE/LG), extracts actionable gh-aw improvement opportunities, creates a
researchdiscussion when findings exist, and maintains a permanent per-paper ledger inrepo-memory.New files
.github/workflows/daily-arxiv-researcher.md— daily scheduled workflow (Claude engine,max-ai-credits: 250).github/scripts/arxiv-fetch-and-filter.py— DataOps helper: parses arXiv Atom XML, deduplicates againstcache-memorybefore agent starts; extracted to a separate file to avoid YAML/Python indentation conflicts in heredoc blocks.github/workflows/daily-arxiv-researcher.lock.yml— compiled outputOptimization design
curl+ Python parsecache-memorydedup (90-day key)paper-screenersub-agent (model: small)opportunity-extractorsub-agent (model: large, max 8)repo-memoryonmemory/arxiv-paper-ledgerFlow
Security
ANTHROPIC_API_KEY: standard Claude engine secret, used by other Claude workflows in this repoexport.arxiv.org: public read-only API, no credentials sent