Skip to content

[Feature] Support for storage layer benchmark - #6506

Merged
yangzhg merged 6 commits into
apache:masterfrom
BiteTheDDDDt:dev_benchmark
Sep 2, 2021
Merged

[Feature] Support for storage layer benchmark#6506
yangzhg merged 6 commits into
apache:masterfrom
BiteTheDDDDt:dev_benchmark

Conversation

@BiteTheDDDDt

Copy link
Copy Markdown
Contributor

Proposed changes

like here #6505

Types of changes

What types of changes does your code introduce to Doris?
Put an x in the boxes that apply

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation Update (if none of the other choices apply)
  • Code refactor (Modify the code structure, format the code, etc...)
  • Optimization. Including functional usability improvements and performance improvements.
  • Dependency. Such as changes related to third-party components.
  • Other.

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

  • I have created an issue on (Fix #ISSUE) and described the bug/feature there in detail
  • Compiling and unit tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • If these changes need document changes, I have updated the document
  • Any dependent changes have been merged

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

trans benchmark_tool to unit test
fix eof
remove unused code and file
add SegmentWrite&& support multi benchmark test
add PageEncodeBenchmark
remove time count at first turn run && move some util fuction to test_util&& add benchmark name display
add MultiBenchmark to manage benchmarks
add SegmentWriteByFileBenchmark
add SegmentScanByFileBenchmark
add ignore case compare
add benchmark include build
promote usage

auto* dest_slice = (Slice*)target;
dest_slice->size = src_len;
dest_slice->data = (char*)pool->allocate(src_len);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add String type

Comment threadbe/test/olap/tablet_schema_helper.h Outdated
void set_column_value_by_type(FieldType fieldType, std::string src, char* target, MemPool* pool,
size_t _length = 0) {
if (fieldType == OLAP_FIELD_TYPE_CHAR) {
char* src_value = &src[0];

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use src.data() or use src.copy() at line 135

Comment threadbe/test/olap/tablet_schema_helper.h Outdated
}
}

void set_column_value_by_type(FieldType fieldType, std::string src, char* target, MemPool* pool,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
voidset_column_value_by_type(FieldTypefieldType, std::stringsrc, char*target, MemPool*pool,
voidset_column_value_by_type(FieldTypefieldType, conststd::string&src, char*target, MemPool*pool,

Comment threadbe/test/test_util/test_util.cpp Outdated
}
}

std::vector<std::string> split_str(const std::string& str, char separation) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use strings::Split

Comment threadthirdparty/build-thirdparty.sh Outdated
sed -i '160 i \ \ add_cxx_compiler_flag(-lrt)' ./CMakeLists.txt

cmake -E make_directory "build"
cmake -E chdir "build" cmake -DBENCHMARK_DOWNLOAD_DEPENDENCIES=on -DCMAKE_BUILD_TYPE=Release ../

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
cmake -E chdir "build" cmake -DBENCHMARK_DOWNLOAD_DEPENDENCIES=on -DCMAKE_BUILD_TYPE=Release ../
CXXFLAGS="-lresolv -pthread -lrt"cmake -E chdir "build" cmake -DBENCHMARK_DOWNLOAD_DEPENDENCIES=on -DCMAKE_BUILD_TYPE=Release ../

Comment threadthirdparty/build-thirdparty.sh Outdated

sed -i '160 i \ \ add_cxx_compiler_flag(-lresolv)' ./CMakeLists.txt
sed -i '160 i \ \ add_cxx_compiler_flag(-pthread)' ./CMakeLists.txt
sed -i '160 i \ \ add_cxx_compiler_flag(-lrt)' ./CMakeLists.txt

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove those sed

@BiteTheDDDDt

Copy link
Copy Markdown
ContributorAuthor

@yangzhg I do some change to fix those problems

@yangzhgyangzhg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Sep 1, 2021
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@yangzhg
yangzhg merged commit 4dd610c into apache:masterSep 2, 2021
@morningmanmorningman mentioned this pull request Oct 10, 2021
@BiteTheDDDDt
BiteTheDDDDt deleted the dev_benchmark branch January 20, 2025 06:45
Hastyshell added a commit to Hastyshell/doris that referenced this pull request Jan 11, 2026
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approvedIndicates a PR has been approved by one committer.area/toolsreviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@BiteTheDDDDt@yangzhg@morningman