Skip to content

HBASE-26723 Switch to use log4j2.properties file to configure log4j2#4096

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Apache9 merged 2 commits into from
Mar 2, 2022
Merged

HBASE-26723 Switch to use log4j2.properties file to configure log4j2 #4096

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2afa94c
HBASE-26723 Switch to use log4j2.properties file to configure log4j2
Apache9 Feb 27, 2022
b28c2c2
fix concurrency issue
Apache9 Feb 27, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 5 additions & 12 deletions bin/hbase
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -751,8 +751,8 @@ elif [ "$COMMAND" = "hbtop" ] ; then
done
fi

if [ -f "${HBASE_HOME}/conf/log4j-hbtop.properties" ] ; then
HBASE_HBTOP_OPTS="${HBASE_HBTOP_OPTS} -Dlog4j.configuration=file:${HBASE_HOME}/conf/log4j-hbtop.properties"
if [ -f "${HBASE_HOME}/conf/log4j2-hbtop.properties" ] ; then
HBASE_HBTOP_OPTS="${HBASE_HBTOP_OPTS} -Dlog4j2.configurationFile=file:${HBASE_HOME}/conf/log4j2-hbtop.properties"

@apurtellapurtellFeb 7, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this work will resolve the issue I see starting hbtop with latest branch-2 HEAD

Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/bridge/SLF4JBridgeHandler
at org.apache.hadoop.hbase.logging.JulToSlf4jInitializer.<clinit>(JulToSlf4jInitializer.java:36)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

This can be "fixed" by removing the JUL to SLF4J bridge stuff from default HBASE_OPTS. Seems not appropriate as a generic option for all processes.

I wonder if hbtop should have logging system properties set at all. It is a curses based command line application, not a daemon. If it has anything to log/print, it should go to the console.

@Apache9Apache9Feb 27, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redirecting the log message all the slf4j should be default I think, and then we could decide whether to actually output it, but changing the log level, appenders, etc.

And I do not think the error here can be fixed by this change, it seems when starting hbtop we do not add some slf4j related jars. Should be a sperated issue.

fi
HBASE_OPTS="${HBASE_OPTS} ${HBASE_HBTOP_OPTS}"
else
Expand DownExpand Up@@ -826,25 +826,18 @@ HBASE_OPTS="$HBASE_OPTS -Dhbase.log.dir=$HBASE_LOG_DIR"
HBASE_OPTS="$HBASE_OPTS -Dhbase.log.file=$HBASE_LOGFILE"
HBASE_OPTS="$HBASE_OPTS -Dhbase.home.dir=$HBASE_HOME"
HBASE_OPTS="$HBASE_OPTS -Dhbase.id.str=$HBASE_IDENT_STRING"
# log4j2 does not support setting log level and appender at once, so we need to split HBASE_ROOT_LOGGER
HBASE_ROOT_LOGGER=${HBASE_ROOT_LOGGER:-INFO,console}
array=(${HBASE_ROOT_LOGGER//,/ })
HBASE_OPTS="$HBASE_OPTS -Dhbase.root.logger.level=${array[0]}"
HBASE_OPTS="$HBASE_OPTS -Dhbase.root.logger.appender=${array[1]}"
HBASE_OPTS="$HBASE_OPTS -Dhbase.root.logger=${HBASE_ROOT_LOGGER:-INFO,console}"
if [ "x$JAVA_LIBRARY_PATH" != "x" ]; then
HBASE_OPTS="$HBASE_OPTS -Djava.library.path=$JAVA_LIBRARY_PATH"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$JAVA_LIBRARY_PATH"
fi

# Enable security logging on the master and regionserver only
if [ "$COMMAND" = "master" ] || [ "$COMMAND" = "regionserver" ]; then
HBASE_SECURITY_LOGGER=${HBASE_SECURITY_LOGGER:-INFO,RFAS}
HBASE_OPTS="$HBASE_OPTS -Dhbase.security.logger=${HBASE_SECURITY_LOGGER:-INFO,RFAS}"
else
HBASE_SECURITY_LOGGER=${HBASE_SECURITY_LOGGER:-INFO,NullAppender}
HBASE_OPTS="$HBASE_OPTS -Dhbase.security.logger=${HBASE_SECURITY_LOGGER:-INFO,NullAppender}"
fi
array=(${HBASE_SECURITY_LOGGER//,/ })
HBASE_OPTS="$HBASE_OPTS -Dhbase.security.logger.level=${array[0]}"
HBASE_OPTS="$HBASE_OPTS -Dhbase.security.logger.appender=${array[1]}"

HEAP_SETTINGS="$JAVA_HEAP_MAX $JAVA_OFFHEAP_MAX"
# by now if we're running a command it means we need logging
Expand Down
16 changes: 2 additions & 14 deletions bin/hbase.cmd
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -332,13 +332,7 @@ set HBASE_OPTS=%HBASE_OPTS% -Djava.util.logging.config.class="org.apache.hadoop.
if not defined HBASE_ROOT_LOGGER (
set HBASE_ROOT_LOGGER=INFO,console
)

for /F "tokens=1,2 delims=," %%a in ("%HBASE_ROOT_LOGGER%") do (
set HBASE_ROOT_LOGGER_LEVEL=%%a
set HBASE_ROOT_LOGGER_APPENDER=%%b
)

set HBASE_OPTS=%HBASE_OPTS% -Dhbase.root.logger.level="%HBASE_ROOT_LOGGER_LEVEL% -Dhbase.root.logger.appender="%HBASE_ROOT_LOGGER_APPENDER% "
set HBASE_OPTS=%HBASE_OPTS% -Dhbase.root.logger="%HBASE_ROOT_LOGGER%"

if defined JAVA_LIBRARY_PATH (
set HBASE_OPTS=%HBASE_OPTS% -Djava.library.path="%JAVA_LIBRARY_PATH%"
Expand All@@ -354,13 +348,7 @@ if not defined HBASE_SECURITY_LOGGER (
set HBASE_SECURITY_LOGGER=INFO,DRFAS
)
)

for /F "tokens=1,2 delims=," %%a in ("%HBASE_SECURITY_LOGGER%") do (
set HBASE_SECURITY_LOGGER_LEVEL=%%a
set HBASE_SECURITY_LOGGER_APPENDER=%%b
)

set HBASE_OPTS=%HBASE_OPTS% -Dhbase.security.logger.level="%HBASE_SECURITY_LOGGER_LEVEL% -Dhbase.security.logger.appender="%HBASE_SECURITY_LOGGER_APPENDER%"
set HBASE_OPTS=%HBASE_OPTS% -Dhbase.security.logger="%HBASE_SECURITY_LOGGER%"

set HEAP_SETTINGS=%JAVA_HEAP_MAX% %JAVA_OFFHEAP_MAX%
set java_arguments=%HEAP_SETTINGS% %HBASE_OPTS% -classpath "%CLASSPATH%" %CLASS% %hbase-command-arguments%
Expand Down
5 changes: 4 additions & 1 deletion conf/hbase-env.cmd
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -32,7 +32,7 @@
@rem set HBASE_OFFHEAPSIZE=1000

@rem For example, to allocate 8G of offheap, to 8G:
@rem etHBASE_OFFHEAPSIZE=8G
@rem set HBASE_OFFHEAPSIZE=8G

@rem Extra Java runtime options.
@rem Below are what we set by default. May only work with SUN JVM.
Expand DownExpand Up@@ -82,6 +82,9 @@ set HBASE_OPTS=%HBASE_OPTS% "-XX:+UseConcMarkSweepGC" "-Djava.net.preferIPv4Stac
@rem Tell HBase whether it should manage it's own instance of ZooKeeper or not.
@rem set HBASE_MANAGES_ZK=true

@rem Tell HBase the logger level and appenders
@rem set HBASE_ROOT_LOGGER=INFO,DRFA

@rem Uncomment to enable trace, you can change the options to use other exporters such as jaeger or
@rem zipkin. See https://github.com/open-telemetry/opentelemetry-java-instrumentation on how to
@rem configure exporters and other components through system properties.
Expand Down
4 changes: 2 additions & 2 deletions conf/hbase-env.sh
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -127,11 +127,11 @@
# export HBASE_MANAGES_ZK=true

# The default log rolling policy is RFA, where the log file is rolled as per the size defined for the
# RFA appender. Please refer to the log4j.properties file to see more details on this appender.
# RFA appender. Please refer to the log4j2.properties file to see more details on this appender.
# In case one needs to do log rolling on a date change, one should set the environment property
# HBASE_ROOT_LOGGER to "<DESIRED_LOG LEVEL>,DRFA".
# For example:
# HBASE_ROOT_LOGGER=INFO,DRFA
# export HBASE_ROOT_LOGGER=INFO,DRFA
# The reason for changing default to RFA is to avoid the boundary case of filling out disk space as
# DRFA doesn't put any cap on the log size. Please refer to HBase-5655 for more context.

Expand Down
35 changes: 35 additions & 0 deletions conf/log4j2-hbtop.properties
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
#/**
# * Licensed to the Apache Software Foundation (ASF) under one
# * or more contributor license agreements. See the NOTICE file
# * distributed with this work for additional information
# * regarding copyright ownership. The ASF licenses this file
# * to you under the Apache License, Version 2.0 (the
# * "License"); you may not use this file except in compliance
# * with the License. You may obtain a copy of the License at
# *
# * http://www.apache.org/licenses/LICENSE-2.0
# *
# * Unless required by applicable law or agreed to in writing, software
# * distributed under the License is distributed on an "AS IS" BASIS,
# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# * See the License for the specific language governing permissions and
# * limitations under the License.
# */

status = warn
dest = err
name = PropertiesConfig

# console
appender.console.type = Console
appender.console.target = SYSTEM_ERR
appender.console.name = console
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = %d{ISO8601} %-5p [%t] %c{2}: %m%n

rootLogger = WARN,console

# ZooKeeper will still put stuff at WARN
logger.zookeeper.name = org.apache.zookeeper
logger.zookeeper.level = ERROR

35 changes: 0 additions & 35 deletions conf/log4j2-hbtop.xml
View file
Open in desktop

This file was deleted.

137 changes: 137 additions & 0 deletions conf/log4j2.properties
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
#/**

@apurtellapurtellFeb 7, 2022•
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can't we continue to use the existing properties file that ships in HEAD of branch-2? It is because even with the Log4J 1 compatibility stuff we still can't make it compatible with the log4j1 properties files?

For operational compatibility in minor releases, after users upgrade, their existing logging configuration should still work. It's fine to recommend that they make changes, like a switch to the XML format, to gain access to improvements, but it would appear log4j has decided to make arbitrary changes that break existing configurations still, so logging wouldn't work if the user doesn't port.

Here are the first few lines from log4j.properties in branch-2.

# Define some default values that can be overridden by system properties
hbase.root.logger=INFO,console
hbase.security.logger=INFO,console
hbase.log.dir=.
hbase.log.file=hbase.log
hbase.log.level=INFO
# Define the root logger to the system property "hbase.root.logger".
log4j.rootLogger=${hbase.root.logger}
# Logging Threshold
log4j.threshold=ALL
#
# Daily Rolling File Appender
#
log4j.appender.DRFA=org.apache.log4j.DailyRollingFileAppender
log4j.appender.DRFA.File=${hbase.log.dir}/${hbase.log.file}

It looks like the new properties file format drops the "log4j" prefix? Can we keep it?

Or, we could implement logic in the launch script that looks at the user's existing log4j properties file and rewrites it, and uses the rewritten version, if it detects this?

Or we could intercept the load of the properties file and do some simple string frobbing to handle this case?

@Apache9Apache9Feb 27, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is log4j2.properties, not log4j.properties. Users could keep their log4j.properties and it will be handled by log4j-1.2-api. Of course if you have some customized appenders then there will be problems.

The log4j2 uses a different syntax, so if users want to fully upgrade to log4j2 then they need to change their configuration ways. But if you want to have the ability to config level and appenders at once, properties file is the only possible way. For other configuration format, such as xml or yaml, there is no way to configuration them at once.

# * Licensed to the Apache Software Foundation (ASF) under one
# * or more contributor license agreements. See the NOTICE file
# * distributed with this work for additional information
# * regarding copyright ownership. The ASF licenses this file
# * to you under the Apache License, Version 2.0 (the
# * "License"); you may not use this file except in compliance
# * with the License. You may obtain a copy of the License at
# *
# * http://www.apache.org/licenses/LICENSE-2.0
# *
# * Unless required by applicable law or agreed to in writing, software
# * distributed under the License is distributed on an "AS IS" BASIS,
# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# * See the License for the specific language governing permissions and
# * limitations under the License.
# */

status = warn
dest = err
name = PropertiesConfig

# Console appender
appender.console.type = Console
appender.console.target = SYSTEM_ERR
appender.console.name = console
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = %d{ISO8601} %-5p [%t] %c{2}: %.1000m%n

# Daily Rolling File Appender
appender.DRFA.type = RollingFile
appender.DRFA.name = DRFA
appender.DRFA.fileName = ${sys:hbase.log.dir:-.}/${sys:hbase.log.file:-hbase.log}
appender.DRFA.filePattern = ${sys:hbase.log.dir:-.}/${sys:hbase.log.file:-hbase.log}.%d{yyyy-MM-dd}
appender.DRFA.createOnDemand = true
appender.DRFA.layout.type = PatternLayout
appender.DRFA.layout.pattern = %d{ISO8601} %-5p [%t] %c{2}: %.1000m%n
appender.DRFA.policies.type = Policies
appender.DRFA.policies.time.type = TimeBasedTriggeringPolicy
appender.DRFA.policies.time.interval = 1
appender.DRFA.policies.time.modulate = true
appender.DRFA.policies.size.type = SizeBasedTriggeringPolicy
appender.DRFA.policies.size.size = ${sys:hbase.log.maxfilesize:-256MB}
appender.DRFA.strategy.type = DefaultRolloverStrategy
appender.DRFA.strategy.max = ${sys:hbase.log.maxbackupindex:-20}

# Rolling File Appender
appender.RFA.type = RollingFile
appender.RFA.name = RFA
appender.RFA.fileName = ${sys:hbase.log.dir:-.}/${sys:hbase.log.file:-hbase.log}
appender.RFA.filePattern = ${sys:hbase.log.dir:-.}/${sys:hbase.log.file:-hbase.log}.%i
appender.RFA.createOnDemand = true
appender.RFA.layout.type = PatternLayout
appender.RFA.layout.pattern = %d{ISO8601} %-5p [%t] %c{2}: %.1000m%n
appender.RFA.policies.type = Policies
appender.RFA.policies.size.type = SizeBasedTriggeringPolicy
appender.RFA.policies.size.size = ${sys:hbase.log.maxfilesize:-256MB}
appender.RFA.strategy.type = DefaultRolloverStrategy
appender.RFA.strategy.max = ${sys:hbase.log.maxbackupindex:-20}

# Security Audit Appender
appender.RFAS.type = RollingFile
appender.RFAS.name = RFAS
appender.RFAS.fileName = ${sys:hbase.log.dir:-.}/${sys:hbase.security.log.file:-SecurityAuth.audit}
appender.RFAS.filePattern = ${sys:hbase.log.dir:-.}/${sys:hbase.security.log.file:-SecurityAuth.audit}.%i
appender.RFAS.createOnDemand = true
appender.RFAS.layout.type = PatternLayout
appender.RFAS.layout.pattern = %d{ISO8601} %-5p [%t] %c{2}: %.1000m%n
appender.RFAS.policies.type = Policies
appender.RFAS.policies.size.type = SizeBasedTriggeringPolicy
appender.RFAS.policies.size.size = ${sys:hbase.security.log.maxfilesize:-256MB}
appender.RFAS.strategy.type = DefaultRolloverStrategy
appender.RFAS.strategy.max = ${sys:hbase.security.log.maxbackupindex:-20}

# Http Access Log RFA, uncomment this if you want an http access.log
# appender.AccessRFA.type = RollingFile
# appender.AccessRFA.name = AccessRFA
# appender.AccessRFA.fileName = /var/log/hbase/access.log
# appender.AccessRFA.filePattern = /var/log/hbase/access.log.%i
# appender.AccessRFA.createOnDemand = true
# appender.AccessRFA.layout.type = PatternLayout
# appender.AccessRFA.layout.pattern = %m%n
# appender.AccessRFA.policies.type = Policies
# appender.AccessRFA.policies.size.type = SizeBasedTriggeringPolicy
# appender.AccessRFA.policies.size.size = 200MB
# appender.AccessRFA.strategy.type = DefaultRolloverStrategy
# appender.AccessRFA.strategy.max = 10

# Null Appender
appender.NullAppender.type = Null
appender.NullAppender.name = NullAppender

rootLogger = ${sys:hbase.root.logger:-INFO,console}

logger.SecurityLogger.name = SecurityLogger
logger.SecurityLogger = ${sys:hbase.security.logger:-INFO,console}
logger.SecurityLogger.additivity = false

# Custom Logging levels
# logger.zookeeper.name = org.apache.zookeeper
# logger.zookeeper.level = ERROR

# logger.FSNamesystem.name = org.apache.hadoop.fs.FSNamesystem
# logger.FSNamesystem.level = DEBUG

# logger.hbase.name = org.apache.hadoop.hbase
# logger.hbase.level = DEBUG

# logger.META.name = org.apache.hadoop.hbase.META
# logger.META.level = DEBUG

# Make these two classes below DEBUG to see more zk debug.
# logger.ZKUtil.name = org.apache.hadoop.hbase.zookeeper.ZKUtil
# logger.ZKUtil.level = DEBUG

# logger.ZKWatcher.name = org.apache.hadoop.hbase.zookeeper.ZKWatcher
# logger.ZKWatcher.level = DEBUG

# logger.dfs.name = org.apache.hadoop.dfs
# logger.dfs.level = DEBUG

# Prevent metrics subsystem start/stop messages (HBASE-17722)
logger.MetricsConfig.name = org.apache.hadoop.metrics2.impl.MetricsConfig
logger.MetricsConfig.level = WARN

logger.MetricsSinkAdapte.name = org.apache.hadoop.metrics2.impl.MetricsSinkAdapter
logger.MetricsSinkAdapte.level = WARN

logger.MetricsSystemImpl.name = org.apache.hadoop.metrics2.impl.MetricsSystemImpl
logger.MetricsSystemImpl.level = WARN

# Disable request log by default, you can enable this by changing the appender
logger.http.name = http.requests
logger.http.additivity = false
logger.http = INFO,NullAppender
# Replace the above with this configuration if you want an http access.log
# logger.http = INFO,AccessRFA
Loading