HADOOP-15019. Honor HADOOP_USER_CLASSPATH_FIRST for deduped entries.

[deepujain]

Description of PR

Summary

When HADOOP_USER_CLASSPATH_FIRST=true, a user-supplied classpath entry that
is already present in Hadoop's computed classpath can stay in its older
position instead of moving to the front. This happens because duplicate
classpath entries were treated as a no-op even when they were being re-added
with before.

This patch keeps the existing de-duplication behavior, but when a duplicate
entry is re-added with before, it is moved to the front of CLASSPATH.
That preserves the expected override behavior for cases such as
HADOOP_CONF_DIR also being present in HADOOP_CLASSPATH.

Change

• hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh:
  update hadoop_add_classpath() so duplicate entries are reordered to the
  front when added with before.
• hadoop-common-project/hadoop-common/src/test/scripts/hadoop_add_classpath.bats:
  add a regression that proves a duplicate entry can be moved to the front.
• hadoop-common-project/hadoop-common/src/test/scripts/hadoop_finalize_classpath.bats:
  add a higher-level regression that matches the JIRA scenario where
  HADOOP_CONF_DIR is duplicated in HADOOP_CLASSPATH while
  HADOOP_USER_CLASSPATH_FIRST=true.

Evidence it works

• bash -n hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
• git diff --check
• Direct shell validation of duplicate reordering:
  /bin/bash -lc 'set -eo pipefail; export HADOOP_SHELL_SCRIPT_DEBUG=true; export QATESTMODE=true; export HADOOP_LIBEXEC_DIR=$(cd -P hadoop-common-project/hadoop-common/src/main/bin >/dev/null && pwd -P); source hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh; tmpdir=$(mktemp -d); mkdir -p "$tmpdir/first" "$tmpdir/second"; CLASSPATH=""; hadoop_add_classpath "$tmpdir/first" after; hadoop_add_classpath "$tmpdir/second" after; hadoop_add_classpath "$tmpdir/second" before; test "$CLASSPATH" = "$tmpdir/second:$tmpdir/first"; rm -rf "$tmpdir"'
• Direct shell validation of the JIRA-style finalize flow:
  /bin/bash -lc 'set -eo pipefail; export HADOOP_SHELL_SCRIPT_DEBUG=true; export QATESTMODE=true; export HADOOP_LIBEXEC_DIR=$(cd -P hadoop-common-project/hadoop-common/src/main/bin >/dev/null && pwd -P); source hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh; tmpdir=$(mktemp -d); mkdir -p "$tmpdir/new" "$tmpdir/old"; CLASSPATH=""; HADOOP_CONF_DIR="$tmpdir"; HADOOP_CLASSPATH="$tmpdir:$tmpdir/new:$tmpdir/old"; HADOOP_USER_CLASSPATH_FIRST=true; HADOOP_USE_CLIENT_CLASSLOADER=""; hadoop_translate_cygwin_path() { true; }; hadoop_finalize_classpath; test "$CLASSPATH" = "$tmpdir:$tmpdir/new:$tmpdir/old"; rm -rf "$tmpdir"'
• bats is not installed in this local environment, so the new .bats
  regressions were validated through the equivalent sourced-function shell
  invocations above.

JIRA

Fixes HADOOP-15019

For code changes:

• Does the title of this PR start with the corresponding JIRA issue id
  (e.g. 'HADOOP-17799. Your PR title ...')?
• Object storage: Have the integration tests been executed and the endpoint
  declared according to the connector-specific documentation? Not
  applicable to this shell-script change.
• If adding new dependencies to the code, are these dependencies licensed
  in a way that is compatible for inclusion under
  ASF 2.0?
  No new dependencies are added.
• If applicable, have you updated the LICENSE, LICENSE-binary,
  NOTICE-binary files? No license or notice changes are required.

AI Tooling

Contains content generated by Codex.

If an AI tool was used:

• The PR includes the phrase "Contains content generated by "
  where is the name of the AI tool used.
• My use of AI contributions follows the ASF legal policy
  https://www.apache.org/legal/generative-tooling.html

[hadoop-yetus]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 35s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	44m 43s		trunk passed
+1 💚	mvnsite	1m 57s		trunk passed
+1 💚	shadedclient	27m 43s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 12s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	1m 25s		the patch passed
+1 💚	shellcheck	0m 4s		No new issues.
+1 💚	shadedclient	27m 0s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 53s		hadoop-common in the patch passed.
+1 💚	asflicense	0m 35s		The patch does not generate ASF License warnings.
		109m 52s

Subsystem	Report/Notes
Docker	ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8670/1/artifact/out/Dockerfile
GITHUB PR	#8670
Optional Tests	dupname asflicense mvnsite unit codespell detsecrets shellcheck shelldocs
uname	Linux 6ca4040a72f8 5.15.0-186-generic #196-Ubuntu SMP Sat Jun 20 16:09:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / fb10cf8
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8670/1/testReport/
Max. process+thread count	615 (vs. ulimit of 10000)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8670/1/console
versions	git=2.43.0 maven=3.9.15 shellcheck=0.9.0
Powered by	Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.