Skip to content

Introduce cruft to keep the repository up to date with template changes#144

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Kircheneer wants to merge 15 commits into from
Closed

Introduce cruft to keep the repository up to date with template changes #144

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
fcb0829
Introduce cruft.
Kircheneer Jul 27, 2022
452023a
Comply to cruft.
Kircheneer Jul 27, 2022
ecbab5c
Add invoke tasks for cruft.
Kircheneer Jul 27, 2022
d6ed906
Refactor CI to have local differences from template live in separate …
Kircheneer Jul 27, 2022
239b512
Update ci.yml to match template.
Kircheneer Jul 27, 2022
9686221
Add newline at the end of CI yml file.
Kircheneer Jul 27, 2022
b073f92
Restrict CI to only run once for branches with open PRs.
Kircheneer Jul 27, 2022
5dc6220
Satisfy yamllint.
Kircheneer Jul 27, 2022
9973ebc
Remove .idea folder again.
Kircheneer Jul 27, 2022
a095c53
Try to fix build.
Kircheneer Jul 27, 2022
4e2f70a
Remove pytest-redislite.
Kircheneer Jul 27, 2022
78e6f3a
Fix cruft_update invoke task.
Kircheneer Jul 27, 2022
c05f392
Update to latest cookiecutter changes.
Kircheneer Jul 27, 2022
1e1c636
Update .cruft.json.
Kircheneer Jul 27, 2022
2e66012
Fix ci step name for mypy.
Kircheneer Jul 27, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .bandit.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -3,3 +3,4 @@ skips: []
# No need to check for security issues in the test scripts!
exclude_dirs:
- "./tests/"
- "./.venv/"
20 changes: 20 additions & 0 deletions .cruft.json
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
{
"template": "https://github.com/networktocode-llc/cookiecutter-ntc",
"commit": null,
"context": {
"cookiecutter": {
"codeowner_github_usernames": "@glennmatthews @Kircheneer @chadell",
"description": "Library to easily sync/diff/update 2 different data sources",
"project_name": "DiffSync",
"project_slug": "diffsync",
"project_python_name": "diffsync",
"project_python_base_version": "3.7",
"project_with_config_settings": "yes",
"generate_docs": "no",
"version": "1.6.0",
"_template": "https://github.com/networktocode-llc/cookiecutter-ntc"
}
},
"directory": "python/",
"checkout": "main"
}
28 changes: 23 additions & 5 deletions .dockerignore
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,27 @@
# Docker related
development/Dockerfile
development/docker-compose*.yml
development/*.env
*.env
environments/

# Python
**/*.pyc
**/*.pyo
**/*.log
**/__pycache__/
**/.pytest_cache/
**/.venv/


# Other
docs/_build
FAQ.md
.git/
.gitignore
Dockerfile
docker-compose.yml
.env
docs/_build
.github
tasks.py
LICENSE
**/*.log
**/.vscode/
invoke*.yml
tasks.py
2 changes: 2 additions & 0 deletions .flake8
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -2,3 +2,5 @@
# E501: Line length is enforced by Black, so flake8 doesn't need to check it
# W503: Black disagrees with this rule, as does PEP 8; Black wins
ignore = E501, W503
exclude =
.venv
2 changes: 1 addition & 1 deletion .github/CODEOWNERS
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
# Default owners for all files in this repository
# Default owner(s) of all files in this repository
* @glennmatthews @Kircheneer @chadell
15 changes: 8 additions & 7 deletions .github/ISSUE_TEMPLATE/bug_report.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,24 @@
---
name: 🐛 Bug Report
about: Report a reproducible bug in the current release of DiffSync
about: Report a reproducible bug in the current release of diffsync

@glennmatthewsglennmatthewsJul 27, 2022

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DiffSync is the project name, diffsync is the Python package name. I'd prefer to keep it as DiffSync when we're referring to the project as a whole.

@KircheneerKircheneerJul 27, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. We need to address this at the source, because currently this is cruft working as intended.

---

### Environment
* DiffSync version: <!-- Example: 1.0.0 -->
* Python version <!-- Example: 3.7.7 -->

<!-- What happened instead? -->
### Observed Behavior
* Python version: <!-- Example: 3.7.7 -->
* diffsync version: <!-- Example: 1.0.0 -->

<!-- What did you expect to happen? -->
### Expected Behavior


<!-- What happened instead? -->
### Observed Behavior

<!--
Describe in detail the exact steps that someone else can take to reproduce
this bug using the current release.
-->
### Steps to Reproduce
1.
2.
3.
3.
6 changes: 3 additions & 3 deletions .github/ISSUE_TEMPLATE/feature_request.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,21 @@
---
name: ✨ Feature Request
about: Propose a new feature or enhancement

---

### Environment
* DiffSync version: <!-- Example: 1.0.0 -->
* diffsync version: <!-- Example: 1.0.0 -->

<!--
Describe in detail the new functionality you are proposing.
-->
### Proposed Functionality


<!--
Convey an example use case for your proposed feature. Write from the
perspective of a user who would benefit from the proposed
functionality and describe how.
-->
--->
### Use Case

133 changes: 89 additions & 44 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,16 @@
---
name: "CI"
on: # yamllint disable-line rule:truthy
- "push"
- "pull_request"
on: # yamllint disable-line rule:truthy rule:comments
push:
branches:
- "main"
- "develop"
pull_request: ~

env:
IMAGE_NAME: "diffsync"

jobs:
validate_renovate:
runs-on: "ubuntu-20.04"
steps:
- name: "Check out repository code"
uses: "actions/checkout@v2"
- name: "Validate renovate configuration"
uses: "rinchsan/renovate-config-validator@v0.0.11"
with:
pattern: "renovate.json"
black:
runs-on: "ubuntu-20.04"
env:
Expand All@@ -21,7 +19,7 @@ jobs:
- name: "Check out repository code"
uses: "actions/checkout@v2"
- name: "Setup environment"
uses: "networktocode/gh-action-setup-poetry-environment@v1"
uses: "networktocode/gh-action-setup-poetry-environment@v2"
- name: "Linting: black"
run: "poetry run invoke black"
bandit:
Expand All@@ -32,7 +30,7 @@ jobs:
- name: "Check out repository code"
uses: "actions/checkout@v2"
- name: "Setup environment"
uses: "networktocode/gh-action-setup-poetry-environment@v1"
uses: "networktocode/gh-action-setup-poetry-environment@v2"
- name: "Linting: bandit"
run: "poetry run invoke bandit"
needs:
Expand All@@ -45,7 +43,7 @@ jobs:
- name: "Check out repository code"
uses: "actions/checkout@v2"
- name: "Setup environment"
uses: "networktocode/gh-action-setup-poetry-environment@v1"
uses: "networktocode/gh-action-setup-poetry-environment@v2"
- name: "Linting: pydocstyle"
run: "poetry run invoke pydocstyle"
needs:
Expand All@@ -58,24 +56,11 @@ jobs:
- name: "Check out repository code"
uses: "actions/checkout@v2"
- name: "Setup environment"
uses: "networktocode/gh-action-setup-poetry-environment@v1"
uses: "networktocode/gh-action-setup-poetry-environment@v2"
- name: "Linting: flake8"
run: "poetry run invoke flake8"
needs:
- "black"
mypy:
runs-on: "ubuntu-20.04"
env:
INVOKE_LOCAL: "True"
steps:
- name: "Check out repository code"
uses: "actions/checkout@v2"
- name: "Setup environment"
uses: "networktocode/gh-action-setup-poetry-environment@v1"
- name: "Linting: flake8"
run: "poetry run invoke mypy"
needs:
- "black"
yamllint:
runs-on: "ubuntu-20.04"
env:
Expand All@@ -84,40 +69,84 @@ jobs:
- name: "Check out repository code"
uses: "actions/checkout@v2"
- name: "Setup environment"
uses: "networktocode/gh-action-setup-poetry-environment@v1"
uses: "networktocode/gh-action-setup-poetry-environment@v2"
- name: "Linting: yamllint"
run: "poetry run invoke yamllint"
needs:
- "black"
build:
strategy:
fail-fast: true
matrix:
python-version: ["3.7", "3.8", "3.9", "3.10"]
runs-on: "ubuntu-20.04"
env:
PYTHON_VER: "${{ matrix.python-version }}"
steps:
- name: "Check out repository code"
uses: "actions/checkout@v2"
- name: "Setup environment"
uses: "networktocode/gh-action-setup-poetry-environment@v1"
- name: "Build Container"
run: "poetry run invoke build"
uses: "networktocode/gh-action-setup-poetry-environment@v2"
- name: "Get image version"
run: "echo IMAGE_VER=`poetry version -s`-py${{ matrix.python-version }} >> $GITHUB_ENV"
- name: "Set up Docker Buildx"
id: "buildx"
uses: "docker/setup-buildx-action@v1"
- name: "Build"
uses: "docker/build-push-action@v2"
with:
builder: "${{ steps.buildx.outputs.name }}"
context: "./"
push: false
tags: "${{ env.IMAGE_NAME }}:${{ env.IMAGE_VER }}"
file: "./Dockerfile"
cache-from: "type=gha,scope=${{ env.IMAGE_NAME }}-${{ env.IMAGE_VER }}-py${{ matrix.python-version }}"
cache-to: "type=gha,scope=${{ env.IMAGE_NAME }}-${{ env.IMAGE_VER }}-py${{ matrix.python-version }}"
build-args: |
PYTHON_VER=${{ env.PYTHON_VER }}
needs:
- "bandit"
- "pydocstyle"
- "flake8"
- "yamllint"
- "mypy"
pylint:
runs-on: "ubuntu-20.04"
strategy:
fail-fast: true
matrix:
python-version: ["3.7"]
env:
PYTHON_VER: "${{ matrix.python-version }}"
steps:
- name: "Check out repository code"
uses: "actions/checkout@v2"
- name: "Setup environment"
uses: "networktocode/gh-action-setup-poetry-environment@v1"
- name: "Build Container"
run: "poetry run invoke build"
uses: "networktocode/gh-action-setup-poetry-environment@v2"
- name: "Get image version"
run: "echo IMAGE_VER=`poetry version -s`-py${{ matrix.python-version }} >> $GITHUB_ENV"
- name: "Set up Docker Buildx"
id: "buildx"
uses: "docker/setup-buildx-action@v1"
- name: "Load the image from cache"
uses: "docker/build-push-action@v2"
with:
builder: "${{ steps.buildx.outputs.name }}"
context: "./"
push: false
load: true
tags: "${{ env.IMAGE_NAME }}:${{ env.IMAGE_VER }}"
file: "./Dockerfile"
cache-from: "type=gha,scope=${{ env.IMAGE_NAME }}-${{ env.IMAGE_VER }}-py${{ matrix.python-version }}"
cache-to: "type=gha,scope=${{ env.IMAGE_NAME }}-${{ env.IMAGE_VER }}-py${{ matrix.python-version }}"
build-args: |
PYTHON_VER=${{ env.PYTHON_VER }}
- name: "Debug: Show docker images"
run: "docker image ls"
- name: "Linting: Pylint"
run: "poetry run invoke pylint"
needs:
- "build"
unittest:
pytest:
strategy:
fail-fast: true
matrix:
Expand All@@ -130,12 +159,28 @@ jobs:
uses: "actions/checkout@v2"
- name: "Setup environment"
uses: "networktocode/gh-action-setup-poetry-environment@v2"
- name: "Get image version"
run: "echo IMAGE_VER=`poetry version -s`-py${{ matrix.python-version }} >> $GITHUB_ENV"
- name: "Set up Docker Buildx"
id: "buildx"
uses: "docker/setup-buildx-action@v1"
- name: "Load the image from cache"
uses: "docker/build-push-action@v2"
with:
python-version: "${{ matrix.python-version }}"
- name: "Run poetry Install"
run: "poetry install"
builder: "${{ steps.buildx.outputs.name }}"
context: "./"
push: false
load: true
tags: "${{ env.IMAGE_NAME }}:${{ env.IMAGE_VER }}"
file: "./Dockerfile"
cache-from: "type=gha,scope=${{ env.IMAGE_NAME }}-${{ env.IMAGE_VER }}-py${{ matrix.python-version }}"
cache-to: "type=gha,scope=${{ env.IMAGE_NAME }}-${{ env.IMAGE_VER }}-py${{ matrix.python-version }}"
build-args: |
PYTHON_VER=${{ env.PYTHON_VER }}
- name: "Debug: Show docker images"
run: "docker image ls"
- name: "Run Tests"
run: "poetry run invoke pytest --local"
run: "poetry run invoke pytest"
needs:
- "pylint"
publish_gh:
Expand DownExpand Up@@ -166,7 +211,7 @@ jobs:
overwrite: true
file_glob: true
needs:
- "unittest"
- "pytest"
publish_pypi:
name: "Push Package to PyPI"
runs-on: "ubuntu-20.04"
Expand All@@ -192,4 +237,4 @@ jobs:
user: "__token__"
password: "${{ secrets.PYPI_API_TOKEN }}"
needs:
- "unittest"
- "pytest"
29 changes: 29 additions & 0 deletions .github/workflows/local.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
---

@glennmatthewsglennmatthewsJul 27, 2022

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a separate file just so as to keep ci.yml identical with the cookiecutter? I would have hoped that it'd be possible/straightforward with cruft to have a locally modified version of the file (i.e. some consistent diff against the baseline). This feels like a bit of a step backwards to me.

@KircheneerKircheneerJul 27, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately this doesn't seem to be possible right now in cruft. I agree that this is worse, because now we can't cross-reference between the two sets of workflows. Possibly this is something that we can address by using another paradigm in GitHub actions.

name: "CI (non-template jobs)"
on: # yamllint disable-line rule:truthy
push:
branches:
- "main"
- "develop"
pull_request: ~
jobs:
validate_renovate:
runs-on: "ubuntu-20.04"
steps:
- name: "Check out repository code"
uses: "actions/checkout@v2"
- name: "Validate renovate configuration"
uses: "rinchsan/renovate-config-validator@v0.0.11"
with:
pattern: "renovate.json"
mypy:
runs-on: "ubuntu-20.04"
env:
INVOKE_LOCAL: "True"
steps:
- name: "Check out repository code"
uses: "actions/checkout@v2"
- name: "Setup environment"
uses: "networktocode/gh-action-setup-poetry-environment@v1"
- name: "Linting: mypy"
run: "poetry run invoke mypy"
Loading