ARROW-14505: [CI][Docs] Exercise documentation builds more frequently by kszucs · Pull Request #11567 · apache/arrow

kszucs · 2021-10-28T11:53:31Z

No description provided.

github-actions · 2021-10-28T12:22:44Z

https://issues.apache.org/jira/browse/ARROW-14505

github-actions · 2021-10-28T12:22:46Z

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

jorisvandenbossche · 2021-10-28T13:29:27Z

Does this mean the workflow only runs on PRs if any of those scripts is edited?
(this is not meant to catch build errors due to changes in the actual docs?)

This build will test the complete documentation build only on master and other implementation/binding dependent builds are going to be responsible to test the relevant sections triggered on all PR updates:
conda-cpp: exercise doxygen
conda-python: exercise sphinx
debian-js: exercise js docs
ubuntu-r: exercise R docs

It sounds less fragile to just add the ci directory here.

The ubuntu-docs build is too expensive so it's better to restrict for PRs.
It will be executed for all commits on master though.

Do we get any notification when CI runs fail on master?

Good question, even if we would get GHA notifications I assume most of us have filtered those emails out.
We can always inspect the build statuses from https://github.com/apache/arrow/commits/master which helps to identify the breaking commit.

kou

+1

pitrou · 2021-11-03T12:37:46Z

Is there a point in building docs outside of the dedicated ubuntu-docs build?

Yes. The ubuntu-docs build can take more than 90 minutes so we shouldn't run that on all PRs. Instead we can build language specific docs for PRs and execute the full docs built on the master branch.

kszucs

+1, merging on green

kou · 2021-11-30T01:30:45Z

https://github.com/apache/arrow/runs/4355980806?check_suite_focus=true was filed as ARROW-14896.

@jonkeane @ianmcook Could you take a look at this?

jonkeane · 2021-11-30T23:08:27Z

I've got some cleanups / skips related to DuckDB in #11795. I'm ok merging this + #11795 separately, and if the docs builds still fail after that, I'll skip those examples until we can stabilize that integration (there are a few other tickets, and DuckDB is a bit of a moving target cause we sometimes need to wait for their upstream changes before we can test them reliably)

kou · 2021-11-30T23:50:12Z

OK. I closed ARROW-14896.

…pendency

kszucs

+1, merging on green

ursabot · 2021-12-02T12:35:06Z

Benchmark runs are scheduled for baseline = 55ebeb3 and contender = be9a22b. be9a22b is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed ⬇️1.35% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.27% ⬆️0.18%] ursa-thinkcentre-m75q
Supported benchmarks:
ursa-i9-9960x: langs = Python, R, JavaScript
ursa-thinkcentre-m75q: langs = C++, Java
ec2-t3-xlarge-us-east-2: cloud = True

A follow-up of apache#11567. https://github.com/ursacomputing/crossbow/runs/4415960154?check_suite_focus=true + mkdir -p /build/docs/c_glib + rsync -a /usr/local/share/gtk-doc/html/ /build/docs/c_glib /arrow/ci/scripts/c_glib_build.sh: line 52: rsync: command not found 127

A follow-up of #11567. https://github.com/ursacomputing/crossbow/runs/4415960154?check_suite_focus=true + mkdir -p /build/docs/c_glib + rsync -a /usr/local/share/gtk-doc/html/ /build/docs/c_glib /arrow/ci/scripts/c_glib_build.sh: line 52: rsync: command not found 127 Closes#11859 from kou/ci-debian-rsync Authored-by: Sutou Kouhei <kou@clear-code.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com>

kszucs changed the title ~~[CI][Docs] Exercise documentation builds on the main branch~~ARROW-14505: [CI][Docs] Exercise documentation builds on the main branchOct 28, 2021

jorisvandenbossche reviewed Oct 28, 2021
View reviewed changes

kszucs changed the title ~~ARROW-14505: [CI][Docs] Exercise documentation builds on the main branch~~ARROW-14505: [CI][Docs] Exercise documentation builds more frequentlyOct 28, 2021

apache deleted a comment from github-actionsBotOct 28, 2021

kszucs commented Oct 28, 2021
View reviewed changes

Comment threaddocker-compose.yml Outdated

jonkeane reviewed Oct 28, 2021
View reviewed changes

Comment threaddocker-compose.yml Outdated
Comment threaddocker-compose.yml Outdated
Comment threaddocker-compose.yml Outdated
Comment threaddocker-compose.yml Outdated

kszucs requested a review from kou October 29, 2021 13:52

kou approved these changes Oct 29, 2021
View reviewed changes

Comment threadci/scripts/js_build.sh Outdated
Comment threadci/scripts/python_build.sh Outdated
Comment threadci/scripts/r_build.sh Outdated
Comment threadci/scripts/c_glib_build.sh Outdated

pitrou reviewed Nov 3, 2021
View reviewed changes

kszucs force-pushed the build-docs-on-main-branch branch from bf1f042 to 1ffdf41Compare November 29, 2021 13:20

kszucs commented Nov 29, 2021
View reviewed changes

kszucs force-pushed the build-docs-on-main-branch branch from 1848bc1 to 1ffdf41Compare November 29, 2021 16:01

kou reviewed Nov 30, 2021
View reviewed changes

Comment thread.github/workflows/docs.yml Outdated

jorisvandenbossche mentioned this pull request Nov 30, 2021
MINOR: [Docs][Python] Update the Grouped Aggregations API section now functions are hidden #11803
Closed

kszucs force-pushed the build-docs-on-main-branch branch from 2941ff4 to 7d70b79Compare December 1, 2021 18:16

kszucs added 10 commits December 1, 2021 22:33

Exercise documentation builds on the main branch
975a391

Trigger on pull request
91d0958

Build docs individually for each impls
dc4dd10

Only rsync the documentations from docs_build.sh
77d4671

Trigger python doc builds
d1fc0e3

Install sphinx to the python docker image
b6578e1

Add sphinx conda env file
c195666

ncpus
1e29cd1

Remove docs_build.sh
e265f11

Add rsync to the ubuntu images
d3b3808

kszucs added 10 commits December 1, 2021 22:34

Build docs in pandas build
95c668e

Turn off javadoc in debian-java
9fda07c

Increase timeouts
0b827d2

Ensure directory existst for c_glib docs
6413276

Address review comments
6067e1c

Fix JS build
92ef80f

Ensure JS doc dir exists
68116eb

Use environment variables to enable apidoc builds
87db05a

Reuse conda_env_sphinx.txt in the ubuntu-docs image
3e80c17

Use pip flavoured dependency constraint
896e856

kszucs force-pushed the build-docs-on-main-branch branch from 7d70b79 to 896e856Compare December 1, 2021 21:34

kszucs added 2 commits December 1, 2021 22:51

Install sphinx-tabs via pip
216b854

Cannot use conda env file with pip because doxygen is not a python de…
bdbf926
…pendency

kszucs commented Dec 1, 2021
View reviewed changes

kou reviewed Dec 1, 2021
View reviewed changes

Comment thread.github/workflows/docs.yml Outdated

kouand others added 2 commits December 2, 2021 08:25

Fix path
e1f2120

Increase build timeout
a4a6389

kszucs closed this in be9a22bDec 2, 2021

kou mentioned this pull request Dec 4, 2021
ARROW-14984: [CI][Debian] rsync is missing #11859
Closed

asfimport mentioned this pull request Dec 4, 2021
[CI][Docs] Exercise documentation builds on the main branch #18915
Closed

Uh oh!

Conversation

kszucs commented Oct 28, 2021

Uh oh!

github-actionsBot commented Oct 28, 2021

Uh oh!

github-actionsBot commented Oct 28, 2021

Uh oh!

jorisvandenbosscheOct 28, 2021

Choose a reason for hiding this comment

Uh oh!

kszucsOct 28, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pitrouNov 3, 2021

Choose a reason for hiding this comment

Uh oh!

kszucsNov 3, 2021

Choose a reason for hiding this comment

Uh oh!

pitrouNov 3, 2021

Choose a reason for hiding this comment

Uh oh!

kszucsNov 3, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kou left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pitrouNov 3, 2021

Choose a reason for hiding this comment

Uh oh!

kszucsNov 3, 2021

Choose a reason for hiding this comment

Uh oh!

kszucs left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kou commented Nov 30, 2021

Uh oh!

jonkeane commented Nov 30, 2021

Uh oh!

kou commented Nov 30, 2021

Uh oh!

kszucs left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ursabot commented Dec 2, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

kszucsOct 28, 2021•
edited
Loading

ursabot commented Dec 2, 2021•
edited
Loading