Skip to content

Add a workflow that builds the docs and deploys them at staged or production#104

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kou merged 12 commits into from
Dec 7, 2022
Merged

Add a workflow that builds the docs and deploys them at staged or production #104

Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
aea783e
Add a workflow that builds the docs and deploys them at staged or pro…
martin-g Dec 5, 2022
91d97e6
Add a branch for unsupported branch/tag
martin-g Dec 6, 2022
289f36e
Improve the glob to ignored RCx tags
martin-g Dec 6, 2022
e0a6910
"git commit" only if there are changed files
martin-g Dec 6, 2022
617ead0
Test building tags
martin-g Dec 6, 2022
832cc6d
Delete everything but .asf.yaml and .git/ when copying the new docs
martin-g Dec 6, 2022
8f46ef9
Remove the filtering by paths
martin-g Dec 6, 2022
822153e
Do not try to delete . and ..
martin-g Dec 6, 2022
d5e32a2
Use .asf.yaml from master/tag
martin-g Dec 6, 2022
2a65b5d
Remove empty line
martin-g Dec 7, 2022
8b92e71
Fix wording
martin-g Dec 7, 2022
e0692fa
Change the tested branch back to master
martin-g Dec 7, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 75 additions & 0 deletions .github/workflows/docs.yaml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
on:
push:
branches:
- master
tags-ignore:
- "**-rc**"

name: Deploy DataFusion Python site

jobs:
build-docs:
name: Build docs
runs-on: ubuntu-latest
steps:
- name: Set target branch
id: target-branch
run: |
set -x
if test '${{ github.ref }}' = 'refs/heads/master'; then
echo "value=asf-staging" >> $GITHUB_OUTPUT
elif test '${{ github.ref_type }}' = 'tag'; then
echo "value=asf-site" >> $GITHUB_OUTPUT
else
echo "Unsupported input: ${{ github.ref }} / ${{ github.ref_type }}"
exit 1
fi
Comment thread
martin-g marked this conversation as resolved.
- name: Checkout docs sources
uses: actions/checkout@v3
- name: Checkout docs target branch
uses: actions/checkout@v3
with:
fetch-depth: 0
ref: ${{ steps.target-branch.outputs.value }}
path: docs-target
Comment on lines +29 to +34

@koukouDec 7, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that we need to prepare asf-site and asf-staging branches for this: https://github.com/martin-g/arrow-datafusion-python/actions/runs/3633415170/jobs/6130403408

How about using git worktree instead?

diff --git a/.github/workflows/docs.yaml b/.github/workflows/docs.yaml
index 806202a..bfafafa 100644
--- a/.github/workflows/docs.yaml+++ b/.github/workflows/docs.yaml@@ -26,12 +26,6 @@ jobs:
fi
- name: Checkout docs sources
uses: actions/checkout@v3
- - name: Checkout docs target branch- uses: actions/checkout@v3- with:- fetch-depth: 0- ref: ${{ steps.target-branch.outputs.value }}- path: docs-target
- name: Setup Python
uses: actions/setup-python@v4
with:
@@ -61,7 +55,12 @@ jobs:
- name: Copy & push the generated HTML
run: |
set -x
+ git worktree add docs-target
cd docs-target
+ if ! git checkout ${{ steps.target-branch.outputs.value }}; then+ git checkout --orphan ${{ steps.target-branch.outputs.value }}+ git rm -rf .+ fi
# delete anything but: 1) '.'; 2) '..'; 3) .git/
find ./ | grep -vE "^./$|^../$|^./.git" | xargs rm -rf
cp ../.asf.yaml .

@martin-gmartin-gDec 7, 2022

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO this is more complex/confusing than the current solution.

@martin-gmartin-gDec 7, 2022

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A task before merging this PR is to create asf-staging branch.

@koukouDec 7, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created: https://github.com/apache/arrow-datafusion-python/tree/asf-staging

- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: "3.10"

- name: Install dependencies
run: |
set -x
python3 -m venv venv
source venv/bin/activate
pip install -r requirements-310.txt
pip install -r docs/requirements.txt
- name: Build Datafusion
run: |
set -x
source venv/bin/activate
maturin develop

- name: Build docs
run: |
set -x
source venv/bin/activate
cd docs
make html

- name: Copy & push the generated HTML
run: |
set -x
cd docs-target
# delete anything but: 1) '.'; 2) '..'; 3) .git/
find ./ | grep -vE "^./$|^../$|^./.git" | xargs rm -rf
cp ../.asf.yaml .
cp -r ../docs/build/html/* .
git status --porcelain
if [ "$(git status --porcelain)" != "" ]; then
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add --all
git commit -m 'Publish built docs triggered by ${{ github.sha }}'
git push || git push --force
fi