Skip to content

ARROW-3470: [C++] Fix row-wise example - #3078

Closed
fsaintjacques wants to merge 2 commits into
apache:masterfrom
fsaintjacques:ARROW-3470-out-of-date-example
Closed

ARROW-3470: [C++] Fix row-wise example#3078
fsaintjacques wants to merge 2 commits into
apache:masterfrom
fsaintjacques:ARROW-3470-out-of-date-example

Conversation

@fsaintjacques

Copy link
Copy Markdown
Contributor
  • Implement the ADD_EXAMPLE cmake function with new ctest label
    example, also covered by the runexample target. This can be
    toggled via the ARROW_BUILD_EXAMPLES option which is ON by default.
  • Implement fully working row-wise-conversion-example.cc and add it to
    the default build.
  • Update documentation to embed (manually) the newly created example.

@wesm

wesm commented Dec 3, 2018

Copy link
Copy Markdown
Member

This might be conflicting with #2856. Can we include the C++ file in the generated docs without copying it manually?

@xhochy

Copy link
Copy Markdown
Member

We should be able to include a whole file into the Sphinx docs via https://breathe.readthedocs.io/en/latest/directives.html#doxygenfile

I'll merge #2856 as soon as the build passes, so that this PR can rebase on top. We should get rid of the doxygen tutorials completely soon. We may still build the doxygen HTML but only as API docs. All prose documentation should be in Sphinx.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These builders are only used in the local scope. We should make them simple stack variables without wrapping them into std::shared_ptr.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wrapped them out of uniformity. The cost_components_builder has to be in a shared_ptr to be passed to the components_builder. The previous version was wrongly using a std::move then referencing again the (moved out) builder.

Our choice is between:

  • 3 local and 1 shared_ptr builder
  • 4 shared_ptr builders

I can revert this choice.

@pitrou

Copy link
Copy Markdown
Member

If you want to include the source for a file in a Sphinx document, you can use the literalinclude directive: https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#directive-literalinclude

@fsaintjacques

Copy link
Copy Markdown
ContributorAuthor

Yeah, pre #2856 I was looking at Doxygen documentation to achieve this.

@fsaintjacques

Copy link
Copy Markdown
ContributorAuthor

I tend to prefer prose as a single file with prose comments as it allows to copy-paste in a single pass for users. Anyone objects that I go this route? It'll also give CI testing of C++ examples documentation.

@wesm

wesm commented Dec 7, 2018

Copy link
Copy Markdown
Member

No objections here

@fsaintjacques
fsaintjacquesforce-pushed the ARROW-3470-out-of-date-example branch 5 times, most recently from 0f26ba2 to 602b417CompareDecember 10, 2018 19:50
@fsaintjacques

Copy link
Copy Markdown
ContributorAuthor

@xhochy that would be ready to review

@xhochyxhochy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, this is looking good but sadly needs a rebase now.

Anyone: Feel free to merge once build passes.

@wesmwesm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a few other comments

Comment threadcpp/CMakeLists.txt Outdated
Comment threadcpp/cmake_modules/BuildUtils.cmake Outdated
Comment threadcpp/cmake_modules/BuildUtils.cmake Outdated
Comment threadcpp/examples/arrow/CMakeLists.txt Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cruft

Comment threadcpp/examples/arrow/row-wise-conversion-example.cc Outdated
@fsaintjacques
fsaintjacquesforce-pushed the ARROW-3470-out-of-date-example branch from 602b417 to 11e4336CompareDecember 12, 2018 16:41
@fsaintjacques

Copy link
Copy Markdown
ContributorAuthor

I addressed the comments and replicated what was done to CI scripts with the new tests default to off.

@wesmwesm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, thanks @fsaintjacques! Will merge once the build runs

Comment threaddocs/source/cpp/index.rst Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The examples should come before the API IMHO.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll let the CI finish, and if it passes I'll push with this tiny change.

- Implement the `ADD_EXAMPLE` cmake function with new ctest label
`example`, also covered by the `runexample` target. This can be
toggled via the `ARROW_BUILD_EXAMPLES` option which is ON by default.
- Implement fully working `row-wise-conversion-example.cc` and add it to
the default build.
- Update documentation to embed (manually) the newly created example.
When ARROW_EXTRA_ERROR_CONTEXT is enabled and the ARROW_RETURN_NOT_OK
macro is used in a non-arrow namespace, the compiler would throw an
error.
@fsaintjacques
fsaintjacquesforce-pushed the ARROW-3470-out-of-date-example branch from 11e4336 to fab63f6CompareDecember 12, 2018 17:52
@fsaintjacques

Copy link
Copy Markdown
ContributorAuthor

I don't think the failures were related to this change (gandiva micro benchmark), the second one is odd (https://travis-ci.org/apache/arrow/jobs/467085379)

Error message: Invalid: /home/travis/build/apache/arrow/cpp/src/arrow/ipc/json-integration-test.cc:151 code: RecordBatchFileReader::Open(arrow_file.get(), &arrow_reader)
/home/travis/build/apache/arrow/cpp/src/arrow/ipc/reader.cc:624 code: ReadFooter()
File is too small: 0

Concurrency issue?

@wesm

wesm commented Dec 12, 2018

Copy link
Copy Markdown
Member

I've been seeing this pretty consistently https://travis-ci.org/apache/arrow/jobs/467092615#L2567. I opened https://issues.apache.org/jira/browse/ARROW-4008. It's not related to this patch

@fsaintjacques

Copy link
Copy Markdown
ContributorAuthor

I think this is now safe to merge.

@fsaintjacques
fsaintjacques deleted the ARROW-3470-out-of-date-example branch December 18, 2018 12:55
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@fsaintjacques@wesm@xhochy@pitrou