Skip to content

GH-3289: Introduce Parquet E2E test harness to support CLI testing - #3296

Closed
ArnavBalyan wants to merge 6 commits into
apache:masterfrom
ArnavBalyan:arnavb/support-e2e-cli-tests
Closed

GH-3289: Introduce Parquet E2E test harness to support CLI testing#3296
ArnavBalyan wants to merge 6 commits into
apache:masterfrom
ArnavBalyan:arnavb/support-e2e-cli-tests

Conversation

@ArnavBalyan

@ArnavBalyanArnavBalyan commented Aug 30, 2025

Copy link
Copy Markdown
Member
  • Currently parquet does not have a good mechanism to have e2e testing for CLI
  • Developers have to manually call methods on the concrete class instead of the actual CLI route, this makes testing hard and prone to errors. Further the method return is tested instead of the actual console output.

  • This change introduces a test harness which is a wrapper over main CLI class and provides utilities to implement CLI tests easily.
  • Users can simply test with single line of code such as: cli("command").ok().matchOutputFromFile("txt"). The harness provides some other utilities like linecount, partial match etc.
  • The CLI output can be added to a txt to ensure readability and test the actual console output

@ArnavBalyan

ArnavBalyan commented Aug 30, 2025

Copy link
Copy Markdown
MemberAuthor

cc @gszadovszky as followup thanks!

@ArnavBalyanArnavBalyan changed the title GH-3289: Introduce e2e test harness to support cli testsGH-3289: Introduce e2e test harness to support cliAug 30, 2025
@ArnavBalyanArnavBalyan changed the title GH-3289: Introduce e2e test harness to support cliGH-3289: Introduce Parquet e2e test harness to support cliAug 30, 2025
@ArnavBalyan
ArnavBalyanforce-pushed the arnavb/support-e2e-cli-tests branch from 5d453a4 to 497858cCompareAugust 30, 2025 14:06
@ArnavBalyanArnavBalyan changed the title GH-3289: Introduce Parquet e2e test harness to support cliGH-3289: Introduce Parquet E2E test harness to support CLI testingAug 30, 2025

@gszadovszkygszadovszky left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @ArnavBalyan for picking this up. It is a great improvement to improve the test coverage of the cli!

Comment threadparquet-cli/src/test/resources/cli-outputs/size-stats.txt Outdated
Comment threadparquet-cli/src/test/java/org/apache/parquet/cli/commands/FileTest.java Outdated
Comment threadparquet-cli/src/main/java/org/apache/parquet/cli/Main.java Outdated
@ArnavBalyan

Copy link
Copy Markdown
MemberAuthor

cc @gszadovszky this is ready thanks!

@gszadovszky

Copy link
Copy Markdown
Contributor

@ArnavBalyan, could you check the failures?

@gszadovszkygszadovszky left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @ArnavBalyan.

@gszadovszky

Copy link
Copy Markdown
Contributor

Oh, the tests are still failing...

@ArnavBalyan

Copy link
Copy Markdown
MemberAuthor

Strangely I am not seeing it locally let me fix it

@ArnavBalyan
ArnavBalyanforce-pushed the arnavb/support-e2e-cli-tests branch from 3f7caa8 to 57011aeCompareSeptember 5, 2025 07:36
@Fokko

Fokko commented Dec 9, 2025

Copy link
Copy Markdown
Contributor

@ArnavBalyan Gentle ping! Did you find some time to fix the test?

@ArnavBalyan

Copy link
Copy Markdown
MemberAuthor

Hi @Fokko thanks for the ping, I'll revive it in the coming week, looks like it was a formatting issue. Will try to fix soon!

@github-actions

Copy link
Copy Markdown

This pull request has been automatically marked as stale because it has had no activity for at least 2 months. If you are still working on this change or plan to move it forward, please leave a comment or push a new commit so we know to keep it open. Otherwise, this PR will be closed automatically in about one month. Thank you for your contribution to Apache Parquet!

@github-actions

Copy link
Copy Markdown

Closing this pull request due to at least 3 months of inactivity. If you would like to continue the work, please feel free to reopen this pull request or open a new one. Thank you for your contribution to Apache Parquet!

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@ArnavBalyan@gszadovszky@Fokko