Uh oh!
There was an error while loading. Please reload this page.
[Fix](jdk) Fix jdk17 crash on some envs. - #32865
Closed
kaka11chen wants to merge 2 commits into
Closed
Conversation
This reverts commit 5c7964d.
doris-robot
commented
Mar 26, 2024
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
Contributor
clang-tidy review says "All clean, LGTM! 👍" |
kaka11chenforce-pushed
the
fix_jdk17_crash
branch
from
March 26, 2024 15:21
92666da to
2444954CompareContributor
|
Contributor
clang-tidy review says "All clean, LGTM! 👍" |
xinyiZzz
commented
Mar 27, 2024
Contributor
good fix!, jemalloc is used in JVM, use dlopen to dynamically load JVM has a problem, see: |
xinyiZzz added a commit
that referenced
this pull request
May 10, 2024
…#34578) Now in order to support malloc hook, compiling jemalloc will add tow compilation options --with-jemalloc-prefix=je and --disable-cxx, then overwrite malloc/free as follows to allow BE to use jemalloc: #define ALIAS(doris_fn) __attribute__((alias(#doris_fn), used)) void* malloc(size_t size) __THROW ALIAS(doris_malloc); void free(void* p) __THROW ALIAS(doris_free); void* realloc(void* p, size_t size) __THROW ALIAS(doris_realloc); void* calloc(size_t n, size_t size) __THROW ALIAS(doris_calloc); void cfree(void* ptr) __THROW ALIAS(doris_cfree); but after such overwrite like this, Doris BE on jdk17 cannot be started in some environments. there are three solutions: Modify overwrite malloc/free, fixed in Use jemalloc's experimental features to hook jemalloc #33897 Modify BE dynamically loads JVM, fixed in [Fix](jdk) Fix jdk17 crash on some envs. #32865 Commit of this PR, modify jemalloc compilation options, after remove --with-jemalloc-prefix=je, there is not need to overwrite malloc/free, BE will use Jemalloc by default. So, if need to use memory hook, this PR still cannot solve this problem. Looking forward to modify BE dynamically loads JVM, which will not only solve this problem, also fix other crashes caused by BE use JVM.
xinyiZzz added a commit
to xinyiZzz/incubator-doris
that referenced
this pull request
May 11, 2024
…apache#34578) Now in order to support malloc hook, compiling jemalloc will add tow compilation options --with-jemalloc-prefix=je and --disable-cxx, then overwrite malloc/free as follows to allow BE to use jemalloc: void* malloc(size_t size) __THROW ALIAS(doris_malloc); void free(void* p) __THROW ALIAS(doris_free); void* realloc(void* p, size_t size) __THROW ALIAS(doris_realloc); void* calloc(size_t n, size_t size) __THROW ALIAS(doris_calloc); void cfree(void* ptr) __THROW ALIAS(doris_cfree); but after such overwrite like this, Doris BE on jdk17 cannot be started in some environments. there are three solutions: Modify overwrite malloc/free, fixed in Use jemalloc's experimental features to hook jemalloc apache#33897 Modify BE dynamically loads JVM, fixed in [Fix](jdk) Fix jdk17 crash on some envs. apache#32865 Commit of this PR, modify jemalloc compilation options, after remove --with-jemalloc-prefix=je, there is not need to overwrite malloc/free, BE will use Jemalloc by default. So, if need to use memory hook, this PR still cannot solve this problem. Looking forward to modify BE dynamically loads JVM, which will not only solve this problem, also fix other crashes caused by BE use JVM.
ByteYue pushed a commit
to ByteYue/doris
that referenced
this pull request
May 15, 2024
…apache#34578) Now in order to support malloc hook, compiling jemalloc will add tow compilation options --with-jemalloc-prefix=je and --disable-cxx, then overwrite malloc/free as follows to allow BE to use jemalloc: #define ALIAS(doris_fn) __attribute__((alias(#doris_fn), used)) void* malloc(size_t size) __THROW ALIAS(doris_malloc); void free(void* p) __THROW ALIAS(doris_free); void* realloc(void* p, size_t size) __THROW ALIAS(doris_realloc); void* calloc(size_t n, size_t size) __THROW ALIAS(doris_calloc); void cfree(void* ptr) __THROW ALIAS(doris_cfree); but after such overwrite like this, Doris BE on jdk17 cannot be started in some environments. there are three solutions: Modify overwrite malloc/free, fixed in Use jemalloc's experimental features to hook jemalloc apache#33897 Modify BE dynamically loads JVM, fixed in [Fix](jdk) Fix jdk17 crash on some envs. apache#32865 Commit of this PR, modify jemalloc compilation options, after remove --with-jemalloc-prefix=je, there is not need to overwrite malloc/free, BE will use Jemalloc by default. So, if need to use memory hook, this PR still cannot solve this problem. Looking forward to modify BE dynamically loads JVM, which will not only solve this problem, also fix other crashes caused by BE use JVM.
HappenLee pushed a commit
to HappenLee/incubator-doris
that referenced
this pull request
Apr 24, 2026
…apache#34578) Now in order to support malloc hook, compiling jemalloc will add tow compilation options --with-jemalloc-prefix=je and --disable-cxx, then overwrite malloc/free as follows to allow BE to use jemalloc: #define ALIAS(doris_fn) __attribute__((alias(#doris_fn), used)) void* malloc(size_t size) __THROW ALIAS(doris_malloc); void free(void* p) __THROW ALIAS(doris_free); void* realloc(void* p, size_t size) __THROW ALIAS(doris_realloc); void* calloc(size_t n, size_t size) __THROW ALIAS(doris_calloc); void cfree(void* ptr) __THROW ALIAS(doris_cfree); but after such overwrite like this, Doris BE on jdk17 cannot be started in some environments. there are three solutions: Modify overwrite malloc/free, fixed in Use jemalloc's experimental features to hook jemalloc apache#33897 Modify BE dynamically loads JVM, fixed in [Fix](jdk) Fix jdk17 crash on some envs. apache#32865 Commit of this PR, modify jemalloc compilation options, after remove --with-jemalloc-prefix=je, there is not need to overwrite malloc/free, BE will use Jemalloc by default. So, if need to use memory hook, this PR still cannot solve this problem. Looking forward to modify BE dynamically loads JVM, which will not only solve this problem, also fix other crashes caused by BE use JVM.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
Issue Number: close #xxx
Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...