Skip to content

test: add deterministic scout validation - #2

Draft
gkmraju wants to merge 1 commit into
mainfrom
test/deterministic-ranking-validation
Draft

gkmraju wants to merge 1 commit into
mainfrom
test/deterministic-ranking-validation

Conversation

@gkmraju

@gkmraju gkmraju commented Aug 17, 2026

Copy link
Copy Markdown
Owner

Summary

Adds focused pull-request validation for the standalone Top Quant Gits scout without changing its default runtime behavior.

Changes

  • adds deterministic tests for repository ranking, digest rendering, default category integrity, and seen-repository persistence
  • allows tests to supply an explicit clock to the pure ranking and digest functions while preserving the existing real-time default
  • adds read-only pull-request CI on Python 3.11 and 3.13
  • documents the local validation commands

Validation

  • py -3.13 -m unittest discover -s tests -v — 9 tests passed
  • py -3.13 -m compileall -q top_quant_gits tests — passed
  • git diff --check — passed

Scope

  • no GitHub API search was performed during validation
  • no Telegram message was sent
  • the scheduled daily workflow and committed output/latest_digest.md were not changed
  • existing draft PR chore: add fallback contribution workspace #1 is unrelated and remains untouched
  • this remains a separate project from GitHub-Contribution-Agent

Summary by Sourcery

Add deterministic validation for repository ranking, digest generation, categories, and seen-repository storage.

Enhancements:

  • Make repository ranking and digest generation deterministic when supplied with an explicit reference time while retaining real-time defaults.

CI:

  • Add read-only pull-request CI across Python 3.11 and 3.13 for unit tests and source compilation.

Documentation:

  • Document local validation commands that avoid external GitHub and Telegram calls.

Tests:

  • Add deterministic coverage for ranking behavior, digest output, default category integrity, and seen-repository persistence.

@sourcery-ai

sourcery-ai Bot commented Aug 17, 2026

Copy link
Copy Markdown

Reviewer's Guide

Adds deterministic, clock-controllable ranking and digest generation, introduces focused tests for ranking, digest, store, and categories, and wires a read-only pull-request CI workflow that runs those tests and compile checks on Python 3.11 and 3.13 while documenting the validation commands in the README.

Sequence diagram for deterministic ranking and digest generation

sequenceDiagram
    actor Developer
    participant Tests
    participant Ranker as score_repositories
    participant DigestMD as build_markdown_digest
    participant DigestTG as build_telegram_link_digest

    Developer->>Tests: run unittest
    Tests->>Ranker: score_repositories(category, repos, now=fixed_datetime)
    Ranker-->>Tests: ranked_repos

    Tests->>DigestMD: build_markdown_digest(categories, ranked_repos, top_n, generated_at=fixed_datetime)
    DigestMD-->>Tests: markdown_digest

    Tests->>DigestTG: build_telegram_link_digest(categories, ranked_repos, top_n, generated_at=fixed_datetime)
    DigestTG-->>Tests: telegram_digest
Loading

File-Level Changes

Change Details Files
Make repository scoring deterministic by allowing an explicit reference time without changing default runtime behavior.
  • Extend the scoring function signature with an optional now parameter used as a reference clock.
  • Compute age and idle days using the provided reference time while retaining the previous behavior when no value is passed.
  • Add unit tests that exercise ranking order, keyword matching, tie-breaking, and future-dated repositories with a fixed clock.
  • Introduce a small test helper for constructing RepoCandidate instances with sensible defaults.
top_quant_gits/ranker.py
tests/test_ranker.py
tests/helpers.py
tests/__init__.py
Make digest rendering reproducible by allowing callers to supply a fixed generation timestamp.
  • Add an optional generated_at parameter to markdown and Telegram digest builders.
  • Derive formatted text from the supplied timestamp when present, falling back to the current UTC time by default.
  • Add tests that assert stable markdown and Telegram output, including headers, scores, links, and empty-category messaging.
top_quant_gits/digest.py
tests/test_digest.py
Verify store and category integrity with deterministic filesystem and configuration tests.
  • Add tests for SeenRepoStore covering behavior for missing files and round-trip save/load with sorted, deduplicated repos.
  • Add tests that assert default categories have unique slugs and non-empty search terms and keywords.
tests/test_store_and_categories.py
Introduce a scoped CI workflow that runs deterministic tests and compile checks on pull requests only.
  • Add a GitHub Actions workflow that triggers on pull_request and manual dispatch with read-only repository permissions.
  • Set up a Python version matrix for 3.11 and 3.13 and install the project in editable mode.
  • Run the unittest test suite and compileall against the package and tests as part of CI.
.github/workflows/ci.yml
Document the local validation commands alongside the project description.
  • Extend README with a validation section describing the deterministic test suite and compile checks.
  • Show equivalent local commands for running unit tests and compile checks without hitting external services.
README.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant