Skip to content

Harden the Python data baseline - #1

Merged
DataTideHH merged 21 commits into
mainfrom
feat/harden-python-baseline
Jul 28, 2026
Merged

Harden the Python data baseline#1
DataTideHH merged 21 commits into
mainfrom
feat/harden-python-baseline

Conversation

@DataTideHH

@DataTideHHDataTideHH commented Jul 28, 2026

Copy link
Copy Markdown
Owner

Summary

This PR turns python-data-basics from a locally documented learning setup into a small, testable and cross-platform Python Data/BI baseline.

Dependency and project model

  • adds pyproject.toml as the dependency and tooling source of truth
  • separates runtime, notebook, optional ML and development dependencies
  • replaces the local transitive package freeze with small requirement wrappers
  • removes the redundant requirements-core.txt

Baseline and examples

  • replaces the primary logistic-regression demo in main.py with a deterministic environment and pandas baseline check
  • moves logistic regression into examples/optional/ with an explicit limitation and optional dependency group
  • keeps CSV, JSON, public API and localhost Ollama examples as bounded learning modules
  • normalizes the older example modules for the current Ruff quality gate

Notebook hygiene

  • clears committed outputs and execution counts
  • removes local interpreter paths and IDE execution timestamps
  • corrects kernel and language metadata to Python 3.12
  • adds tests that prevent these local traces from returning

Quality assurance

  • adds pytest tests for the baseline transformation, invalid inputs, notebook hygiene and optional ML example
  • adds Ruff lint and format configuration
  • adds Python source compilation checks
  • adds a GitHub Actions matrix for Python 3.12 on Ubuntu 24.04 and Windows 2025
  • executes the baseline script, optional ML example and a temporary notebook copy in CI
  • preserves short-lived Ruff diagnostics as workflow artifacts when quality checks fail

Documentation

  • rewrites the README around the verified baseline story
  • documents Windows PowerShell, Windows Command Prompt and macOS/Linux setup
  • avoids unsafe machine-wide PowerShell policy changes
  • documents dependency groups, local checks, CI behaviour, safety boundaries and repository scope

Verified GitHub-hosted execution

The final workflow run completed successfully on both matrix targets:

  • Ubuntu 24.04 with Python 3.12
  • Windows 2025 with Python 3.12

Both jobs successfully completed:

  • project and optional dependency installation
  • Python bytecode compilation
  • Ruff lint checks
  • Ruff formatting verification
  • pytest suite
  • deterministic baseline entry point
  • optional logistic-regression example
  • execution of a clean temporary notebook copy
  • final Ruff quality gate

Safety and scope

  • repository access is read-only in CI
  • checkout credentials are not persisted
  • examples use synthetic data, a public unauthenticated endpoint or localhost only
  • no token, secret, credential file or executed notebook is committed
  • this remains a learning and quality baseline, not a production package or predictive-model claim

@chatgpt-codex-connectorchatgpt-codex-connectorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:ced33cf3a2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread.github/workflows/python-quality.yml Outdated
run: python -m compileall -q main.py examples tests

- name: Run Ruff lint checks
run: python -m ruff check main.py examples tests

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Format the existing examples before enforcing Ruff

In the added Python quality workflow, this command scans all four pre-existing top-level example scripts in both matrix jobs. Running the exact command with Ruff 0.15.12 exits with status 1 because examples/01_csv_pandas_basics.py, 02_json_basics.py, 03_api_request_basics.py, and 04_ollama_local_api_basics.py each trigger I001 for an extra blank line after their imports, so every Ubuntu and Windows run stops before pytest. Format those files or adjust the lint configuration before making this check mandatory.

Useful? React with 👍 / 👎.

@DataTideHH
DataTideHH merged commit 3eacea7 into mainJul 28, 2026
2 checks passed
@DataTideHH
DataTideHH deleted the feat/harden-python-baseline branch July 28, 2026 15:49
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@DataTideHH