GH-49351: [C++] Check TZDIR environment variable in vendored date library

[canassa]

Rationale for This Change

The vendored Howard Hinnant date library hardcodes /usr/share/zoneinfo as the timezone database path in discover_tz_dir(). It does not check the TZDIR environment variable, which is the POSIX standard mechanism for overriding this path.

This causes timezone operations to fail on non FHS Linux distributions such as NixOS, where zoneinfo resides under a non standard path like /nix/store/.../share/zoneinfo.

The upstream library also lacks TZDIR support.

---

What Changes Are Included in This PR?

• cpp/src/arrow/vendored/datetime/tz.cpp: Check the TZDIR environment variable in discover_tz_dir() before falling back to platform specific hardcoded paths. Uses stat() and S_ISDIR() for validation, matching the existing pattern in the function.
• cpp/src/arrow/vendored/datetime/README.md: Document the patch.
• cpp/src/arrow/public_api_test.cc: Add a non Windows test that sets TZDIR and verifies timezone resolution succeeds through Arrow's compute API.
• python/pyarrow/conftest.py: Respect TZDIR in the emscripten timezone_data test marker.

---

Are These Changes Tested?

Yes. A new Misc.TZDIREnvironmentVariable test sets TZDIR to a valid zoneinfo directory and casts a UTC timestamp to America/New_York, verifying the code path works end to end.

---

Are There Any User Facing Changes?

Arrow now respects the TZDIR environment variable on non Windows platforms, enabling timezone operations on systems without /usr/share/zoneinfo.

Fixes#49351

• GitHub Issue: [C++] Vendored date library does not respect TZDIR environment variable #49351

[github-actions]

⚠️ GitHub issue #49351has been automatically assigned in GitHub to PR creator.

[rok]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

[canassa]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

Yes! I noticed this when working on a Python project that uses PyArrow on my NixOS desktop.

I could work around it by creating a symlink, but I figured it would be nice if Arrow supported the environment variable.

[rok]

Great to hear!
I don't have time to review this right now. I'll take a look next week. One thing right now - we try to avoid changing things in cpp/src/arrow/vendored/ if possible. Could you check if we can solve this without the change in cpp/src/arrow/vendored/datetime/tz.cpp?

[canassa]

iew this right now. I'll take a look next week. One thing right now - we try to avoid changing things in cpp/src/arrow/vendored/ if possible. Could you check if we can solve this without the change in cpp/src/arrow/vendored/datetime/tz.cpp?

Sure, I will take a look!

[rok]

The linting issue can be fixed with (source):

pre-commit run --show-diff-on-failure --color=always --all-files cpp

The cpp paginator issues are unrelated (#49347).

[canassa]

Thanks, I pushed the linting fixes.

I also investigated other approaches, but the vendored library change seem to be unavoidable. The get_tz_dir() is a static singleton, once initialized, the path is fixed for the lifetime of the process, and there's no setter.

[FreaxMATE]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

Yes! I noticed this when working on a Python project that uses PyArrow on my NixOS desktop.

I could work around it by creating a symlink, but I figured it would be nice if Arrow supported the environment variable.

I have the same issue. For anyone trying this quick workaround for nixos:

sudo mkdir-p /usr/share
sudo ln-s /etc/zoneinfo /usr/share/zoneinfo

Thanks @canassa for the fix!

[rok]

Sorry for slow reply @canassa. Thanks for checking if we can avoid changing the vendored lib!

We had recently moved to use std::chrono for accessing timezone data on Windows #48601 circumventing the vendored library and need to deal with timezone files. If you're on Windows, could you test current main branch? (current release doesn't have this yet)

If you're still seeing this issue, it would be more productive to disable vendored library further and switch to std::chrono than to keep hardcoding TZDIR.

If it'd still make sense to keep with currently proposed approach, then - the JNI CI failures could be related to your change. Can you please rebase this branch so we can check with CI?

[canassa]

Rationale for This Change

The vendored Howard Hinnant date library hardcodes /usr/share/zoneinfo as the timezone database path in discover_tz_dir(). It does not check the TZDIR environment variable, which is the POSIX standard mechanism for overriding this path.

This causes timezone operations to fail on non FHS Linux distributions such as NixOS, where zoneinfo resides under a non standard path like /nix/store/.../share/zoneinfo.

The upstream library also lacks TZDIR support.

---

What Changes Are Included in This PR?

• cpp/src/arrow/vendored/datetime/tz.cpp: Check the TZDIR environment variable in discover_tz_dir() before falling back to platform specific hardcoded paths. Uses stat() and S_ISDIR() for validation, matching the existing pattern in the function.
• cpp/src/arrow/vendored/datetime/README.md: Document the patch.
• cpp/src/arrow/public_api_test.cc: Add a non Windows test that sets TZDIR and verifies timezone resolution succeeds through Arrow's compute API.
• python/pyarrow/conftest.py: Respect TZDIR in the emscripten timezone_data test marker.

---

Are These Changes Tested?

Yes. A new Misc.TZDIREnvironmentVariable test sets TZDIR to a valid zoneinfo directory and casts a UTC timestamp to America/New_York, verifying the code path works end to end.

---

Are There Any User Facing Changes?

Arrow now respects the TZDIR environment variable on non Windows platforms, enabling timezone operations on systems without /usr/share/zoneinfo.

Fixes#49351

• GitHub Issue: [C++] Vendored date library does not respect TZDIR environment variable #49351

[github-actions]

⚠️ GitHub issue #49351has been automatically assigned in GitHub to PR creator.

[rok]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

[canassa]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

Yes! I noticed this when working on a Python project that uses PyArrow on my NixOS desktop.

I could work around it by creating a symlink, but I figured it would be nice if Arrow supported the environment variable.

[rok]

Great to hear!
I don't have time to review this right now. I'll take a look next week. One thing right now - we try to avoid changing things in cpp/src/arrow/vendored/ if possible. Could you check if we can solve this without the change in cpp/src/arrow/vendored/datetime/tz.cpp?

[canassa]

iew this right now. I'll take a look next week. One thing right now - we try to avoid changing things in cpp/src/arrow/vendored/ if possible. Could you check if we can solve this without the change in cpp/src/arrow/vendored/datetime/tz.cpp?

Sure, I will take a look!

[rok]

The linting issue can be fixed with (source):

pre-commit run --show-diff-on-failure --color=always --all-files cpp

The cpp paginator issues are unrelated (#49347).

[canassa]

Thanks, I pushed the linting fixes.

I also investigated other approaches, but the vendored library change seem to be unavoidable. The get_tz_dir() is a static singleton, once initialized, the path is fixed for the lifetime of the process, and there's no setter.

[FreaxMATE]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

Yes! I noticed this when working on a Python project that uses PyArrow on my NixOS desktop.

I could work around it by creating a symlink, but I figured it would be nice if Arrow supported the environment variable.

I have the same issue. For anyone trying this quick workaround for nixos:

sudo mkdir-p /usr/share
sudo ln-s /etc/zoneinfo /usr/share/zoneinfo

Thanks @canassa for the fix!

[rok]

Sorry for slow reply @canassa. Thanks for checking if we can avoid changing the vendored lib!

We had recently moved to use std::chrono for accessing timezone data on Windows #48601 circumventing the vendored library and need to deal with timezone files. If you're on Windows, could you test current main branch? (current release doesn't have this yet)

If you're still seeing this issue, it would be more productive to disable vendored library further and switch to std::chrono than to keep hardcoding TZDIR.

If it'd still make sense to keep with currently proposed approach, then - the JNI CI failures could be related to your change. Can you please rebase this branch so we can check with CI?

[canassa]

Rationale for This Change

The vendored Howard Hinnant date library hardcodes /usr/share/zoneinfo as the timezone database path in discover_tz_dir(). It does not check the TZDIR environment variable, which is the POSIX standard mechanism for overriding this path.

This causes timezone operations to fail on non FHS Linux distributions such as NixOS, where zoneinfo resides under a non standard path like /nix/store/.../share/zoneinfo.

The upstream library also lacks TZDIR support.

---

What Changes Are Included in This PR?

• cpp/src/arrow/vendored/datetime/tz.cpp: Check the TZDIR environment variable in discover_tz_dir() before falling back to platform specific hardcoded paths. Uses stat() and S_ISDIR() for validation, matching the existing pattern in the function.
• cpp/src/arrow/vendored/datetime/README.md: Document the patch.
• cpp/src/arrow/public_api_test.cc: Add a non Windows test that sets TZDIR and verifies timezone resolution succeeds through Arrow's compute API.
• python/pyarrow/conftest.py: Respect TZDIR in the emscripten timezone_data test marker.

---

Are These Changes Tested?

Yes. A new Misc.TZDIREnvironmentVariable test sets TZDIR to a valid zoneinfo directory and casts a UTC timestamp to America/New_York, verifying the code path works end to end.

---

Are There Any User Facing Changes?

Arrow now respects the TZDIR environment variable on non Windows platforms, enabling timezone operations on systems without /usr/share/zoneinfo.

Fixes#49351

• GitHub Issue: [C++] Vendored date library does not respect TZDIR environment variable #49351

[github-actions]

⚠️ GitHub issue #49351has been automatically assigned in GitHub to PR creator.

[rok]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

[canassa]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

Yes! I noticed this when working on a Python project that uses PyArrow on my NixOS desktop.

I could work around it by creating a symlink, but I figured it would be nice if Arrow supported the environment variable.

[rok]

Great to hear!
I don't have time to review this right now. I'll take a look next week. One thing right now - we try to avoid changing things in cpp/src/arrow/vendored/ if possible. Could you check if we can solve this without the change in cpp/src/arrow/vendored/datetime/tz.cpp?

[canassa]

iew this right now. I'll take a look next week. One thing right now - we try to avoid changing things in cpp/src/arrow/vendored/ if possible. Could you check if we can solve this without the change in cpp/src/arrow/vendored/datetime/tz.cpp?

Sure, I will take a look!

[rok]

The linting issue can be fixed with (source):

pre-commit run --show-diff-on-failure --color=always --all-files cpp

The cpp paginator issues are unrelated (#49347).

[canassa]

Thanks, I pushed the linting fixes.

I also investigated other approaches, but the vendored library change seem to be unavoidable. The get_tz_dir() is a static singleton, once initialized, the path is fixed for the lifetime of the process, and there's no setter.

[FreaxMATE]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

Yes! I noticed this when working on a Python project that uses PyArrow on my NixOS desktop.

I could work around it by creating a symlink, but I figured it would be nice if Arrow supported the environment variable.

I have the same issue. For anyone trying this quick workaround for nixos:

sudo mkdir-p /usr/share
sudo ln-s /etc/zoneinfo /usr/share/zoneinfo

Thanks @canassa for the fix!

[rok]

Sorry for slow reply @canassa. Thanks for checking if we can avoid changing the vendored lib!

We had recently moved to use std::chrono for accessing timezone data on Windows #48601 circumventing the vendored library and need to deal with timezone files. If you're on Windows, could you test current main branch? (current release doesn't have this yet)

If you're still seeing this issue, it would be more productive to disable vendored library further and switch to std::chrono than to keep hardcoding TZDIR.

If it'd still make sense to keep with currently proposed approach, then - the JNI CI failures could be related to your change. Can you please rebase this branch so we can check with CI?

[canassa]

Rationale for This Change

The vendored Howard Hinnant date library hardcodes /usr/share/zoneinfo as the timezone database path in discover_tz_dir(). It does not check the TZDIR environment variable, which is the POSIX standard mechanism for overriding this path.

This causes timezone operations to fail on non FHS Linux distributions such as NixOS, where zoneinfo resides under a non standard path like /nix/store/.../share/zoneinfo.

The upstream library also lacks TZDIR support.

---

What Changes Are Included in This PR?

• cpp/src/arrow/vendored/datetime/tz.cpp: Check the TZDIR environment variable in discover_tz_dir() before falling back to platform specific hardcoded paths. Uses stat() and S_ISDIR() for validation, matching the existing pattern in the function.
• cpp/src/arrow/vendored/datetime/README.md: Document the patch.
• cpp/src/arrow/public_api_test.cc: Add a non Windows test that sets TZDIR and verifies timezone resolution succeeds through Arrow's compute API.
• python/pyarrow/conftest.py: Respect TZDIR in the emscripten timezone_data test marker.

---

Are These Changes Tested?

Yes. A new Misc.TZDIREnvironmentVariable test sets TZDIR to a valid zoneinfo directory and casts a UTC timestamp to America/New_York, verifying the code path works end to end.

---

Are There Any User Facing Changes?

Arrow now respects the TZDIR environment variable on non Windows platforms, enabling timezone operations on systems without /usr/share/zoneinfo.

Fixes#49351

• GitHub Issue: [C++] Vendored date library does not respect TZDIR environment variable #49351

[github-actions]

⚠️ GitHub issue #49351has been automatically assigned in GitHub to PR creator.

[rok]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

[canassa]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

Yes! I noticed this when working on a Python project that uses PyArrow on my NixOS desktop.

I could work around it by creating a symlink, but I figured it would be nice if Arrow supported the environment variable.

[rok]

Great to hear!
I don't have time to review this right now. I'll take a look next week. One thing right now - we try to avoid changing things in cpp/src/arrow/vendored/ if possible. Could you check if we can solve this without the change in cpp/src/arrow/vendored/datetime/tz.cpp?

[canassa]

iew this right now. I'll take a look next week. One thing right now - we try to avoid changing things in cpp/src/arrow/vendored/ if possible. Could you check if we can solve this without the change in cpp/src/arrow/vendored/datetime/tz.cpp?

Sure, I will take a look!

[rok]

The linting issue can be fixed with (source):

pre-commit run --show-diff-on-failure --color=always --all-files cpp

The cpp paginator issues are unrelated (#49347).

[canassa]

Thanks, I pushed the linting fixes.

I also investigated other approaches, but the vendored library change seem to be unavoidable. The get_tz_dir() is a static singleton, once initialized, the path is fixed for the lifetime of the process, and there's no setter.

[FreaxMATE]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

Yes! I noticed this when working on a Python project that uses PyArrow on my NixOS desktop.

I could work around it by creating a symlink, but I figured it would be nice if Arrow supported the environment variable.

I have the same issue. For anyone trying this quick workaround for nixos:

sudo mkdir-p /usr/share
sudo ln-s /etc/zoneinfo /usr/share/zoneinfo

Thanks @canassa for the fix!

[rok]

Sorry for slow reply @canassa. Thanks for checking if we can avoid changing the vendored lib!

We had recently moved to use std::chrono for accessing timezone data on Windows #48601 circumventing the vendored library and need to deal with timezone files. If you're on Windows, could you test current main branch? (current release doesn't have this yet)

If you're still seeing this issue, it would be more productive to disable vendored library further and switch to std::chrono than to keep hardcoding TZDIR.

If it'd still make sense to keep with currently proposed approach, then - the JNI CI failures could be related to your change. Can you please rebase this branch so we can check with CI?

[canassa]

Rationale for This Change

The vendored Howard Hinnant date library hardcodes /usr/share/zoneinfo as the timezone database path in discover_tz_dir(). It does not check the TZDIR environment variable, which is the POSIX standard mechanism for overriding this path.

This causes timezone operations to fail on non FHS Linux distributions such as NixOS, where zoneinfo resides under a non standard path like /nix/store/.../share/zoneinfo.

The upstream library also lacks TZDIR support.

---

What Changes Are Included in This PR?

• cpp/src/arrow/vendored/datetime/tz.cpp: Check the TZDIR environment variable in discover_tz_dir() before falling back to platform specific hardcoded paths. Uses stat() and S_ISDIR() for validation, matching the existing pattern in the function.
• cpp/src/arrow/vendored/datetime/README.md: Document the patch.
• cpp/src/arrow/public_api_test.cc: Add a non Windows test that sets TZDIR and verifies timezone resolution succeeds through Arrow's compute API.
• python/pyarrow/conftest.py: Respect TZDIR in the emscripten timezone_data test marker.

---

Are These Changes Tested?

Yes. A new Misc.TZDIREnvironmentVariable test sets TZDIR to a valid zoneinfo directory and casts a UTC timestamp to America/New_York, verifying the code path works end to end.

---

Are There Any User Facing Changes?

Arrow now respects the TZDIR environment variable on non Windows platforms, enabling timezone operations on systems without /usr/share/zoneinfo.

Fixes#49351

• GitHub Issue: [C++] Vendored date library does not respect TZDIR environment variable #49351

[github-actions]

⚠️ GitHub issue #49351has been automatically assigned in GitHub to PR creator.

[rok]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

[canassa]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

Yes! I noticed this when working on a Python project that uses PyArrow on my NixOS desktop.

I could work around it by creating a symlink, but I figured it would be nice if Arrow supported the environment variable.

[rok]

Great to hear!
I don't have time to review this right now. I'll take a look next week. One thing right now - we try to avoid changing things in cpp/src/arrow/vendored/ if possible. Could you check if we can solve this without the change in cpp/src/arrow/vendored/datetime/tz.cpp?

[canassa]

iew this right now. I'll take a look next week. One thing right now - we try to avoid changing things in cpp/src/arrow/vendored/ if possible. Could you check if we can solve this without the change in cpp/src/arrow/vendored/datetime/tz.cpp?

Sure, I will take a look!

[rok]

The linting issue can be fixed with (source):

pre-commit run --show-diff-on-failure --color=always --all-files cpp

The cpp paginator issues are unrelated (#49347).

[canassa]

Thanks, I pushed the linting fixes.

I also investigated other approaches, but the vendored library change seem to be unavoidable. The get_tz_dir() is a static singleton, once initialized, the path is fixed for the lifetime of the process, and there's no setter.

[FreaxMATE]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

Yes! I noticed this when working on a Python project that uses PyArrow on my NixOS desktop.

I could work around it by creating a symlink, but I figured it would be nice if Arrow supported the environment variable.

I have the same issue. For anyone trying this quick workaround for nixos:

sudo mkdir-p /usr/share
sudo ln-s /etc/zoneinfo /usr/share/zoneinfo

Thanks @canassa for the fix!

[rok]

Sorry for slow reply @canassa. Thanks for checking if we can avoid changing the vendored lib!

We had recently moved to use std::chrono for accessing timezone data on Windows #48601 circumventing the vendored library and need to deal with timezone files. If you're on Windows, could you test current main branch? (current release doesn't have this yet)

If you're still seeing this issue, it would be more productive to disable vendored library further and switch to std::chrono than to keep hardcoding TZDIR.

If it'd still make sense to keep with currently proposed approach, then - the JNI CI failures could be related to your change. Can you please rebase this branch so we can check with CI?

[canassa]

Rationale for This Change

The vendored Howard Hinnant date library hardcodes /usr/share/zoneinfo as the timezone database path in discover_tz_dir(). It does not check the TZDIR environment variable, which is the POSIX standard mechanism for overriding this path.

This causes timezone operations to fail on non FHS Linux distributions such as NixOS, where zoneinfo resides under a non standard path like /nix/store/.../share/zoneinfo.

The upstream library also lacks TZDIR support.

---

What Changes Are Included in This PR?

• cpp/src/arrow/vendored/datetime/tz.cpp: Check the TZDIR environment variable in discover_tz_dir() before falling back to platform specific hardcoded paths. Uses stat() and S_ISDIR() for validation, matching the existing pattern in the function.
• cpp/src/arrow/vendored/datetime/README.md: Document the patch.
• cpp/src/arrow/public_api_test.cc: Add a non Windows test that sets TZDIR and verifies timezone resolution succeeds through Arrow's compute API.
• python/pyarrow/conftest.py: Respect TZDIR in the emscripten timezone_data test marker.

---

Are These Changes Tested?

Yes. A new Misc.TZDIREnvironmentVariable test sets TZDIR to a valid zoneinfo directory and casts a UTC timestamp to America/New_York, verifying the code path works end to end.

---

Are There Any User Facing Changes?

Arrow now respects the TZDIR environment variable on non Windows platforms, enabling timezone operations on systems without /usr/share/zoneinfo.

Fixes#49351

• GitHub Issue: [C++] Vendored date library does not respect TZDIR environment variable #49351

[github-actions]

⚠️ GitHub issue #49351has been automatically assigned in GitHub to PR creator.

[rok]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

[canassa]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

Yes! I noticed this when working on a Python project that uses PyArrow on my NixOS desktop.

I could work around it by creating a symlink, but I figured it would be nice if Arrow supported the environment variable.

[rok]

Great to hear!
I don't have time to review this right now. I'll take a look next week. One thing right now - we try to avoid changing things in cpp/src/arrow/vendored/ if possible. Could you check if we can solve this without the change in cpp/src/arrow/vendored/datetime/tz.cpp?

[canassa]

iew this right now. I'll take a look next week. One thing right now - we try to avoid changing things in cpp/src/arrow/vendored/ if possible. Could you check if we can solve this without the change in cpp/src/arrow/vendored/datetime/tz.cpp?

Sure, I will take a look!

[rok]

The linting issue can be fixed with (source):

pre-commit run --show-diff-on-failure --color=always --all-files cpp

The cpp paginator issues are unrelated (#49347).

[canassa]

Thanks, I pushed the linting fixes.

I also investigated other approaches, but the vendored library change seem to be unavoidable. The get_tz_dir() is a static singleton, once initialized, the path is fixed for the lifetime of the process, and there's no setter.

[FreaxMATE]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

Yes! I noticed this when working on a Python project that uses PyArrow on my NixOS desktop.

I could work around it by creating a symlink, but I figured it would be nice if Arrow supported the environment variable.

I have the same issue. For anyone trying this quick workaround for nixos:

sudo mkdir-p /usr/share
sudo ln-s /etc/zoneinfo /usr/share/zoneinfo

Thanks @canassa for the fix!

[rok]

Sorry for slow reply @canassa. Thanks for checking if we can avoid changing the vendored lib!

We had recently moved to use std::chrono for accessing timezone data on Windows #48601 circumventing the vendored library and need to deal with timezone files. If you're on Windows, could you test current main branch? (current release doesn't have this yet)

If you're still seeing this issue, it would be more productive to disable vendored library further and switch to std::chrono than to keep hardcoding TZDIR.

If it'd still make sense to keep with currently proposed approach, then - the JNI CI failures could be related to your change. Can you please rebase this branch so we can check with CI?

[canassa]

Rationale for This Change

The vendored Howard Hinnant date library hardcodes /usr/share/zoneinfo as the timezone database path in discover_tz_dir(). It does not check the TZDIR environment variable, which is the POSIX standard mechanism for overriding this path.

This causes timezone operations to fail on non FHS Linux distributions such as NixOS, where zoneinfo resides under a non standard path like /nix/store/.../share/zoneinfo.

The upstream library also lacks TZDIR support.

---

What Changes Are Included in This PR?

• cpp/src/arrow/vendored/datetime/tz.cpp: Check the TZDIR environment variable in discover_tz_dir() before falling back to platform specific hardcoded paths. Uses stat() and S_ISDIR() for validation, matching the existing pattern in the function.
• cpp/src/arrow/vendored/datetime/README.md: Document the patch.
• cpp/src/arrow/public_api_test.cc: Add a non Windows test that sets TZDIR and verifies timezone resolution succeeds through Arrow's compute API.
• python/pyarrow/conftest.py: Respect TZDIR in the emscripten timezone_data test marker.

---

Are These Changes Tested?

Yes. A new Misc.TZDIREnvironmentVariable test sets TZDIR to a valid zoneinfo directory and casts a UTC timestamp to America/New_York, verifying the code path works end to end.

---

Are There Any User Facing Changes?

Arrow now respects the TZDIR environment variable on non Windows platforms, enabling timezone operations on systems without /usr/share/zoneinfo.

Fixes#49351

• GitHub Issue: [C++] Vendored date library does not respect TZDIR environment variable #49351

[github-actions]

⚠️ GitHub issue #49351has been automatically assigned in GitHub to PR creator.

[rok]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

[canassa]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

Yes! I noticed this when working on a Python project that uses PyArrow on my NixOS desktop.

I could work around it by creating a symlink, but I figured it would be nice if Arrow supported the environment variable.

[rok]

Great to hear!
I don't have time to review this right now. I'll take a look next week. One thing right now - we try to avoid changing things in cpp/src/arrow/vendored/ if possible. Could you check if we can solve this without the change in cpp/src/arrow/vendored/datetime/tz.cpp?

[canassa]

iew this right now. I'll take a look next week. One thing right now - we try to avoid changing things in cpp/src/arrow/vendored/ if possible. Could you check if we can solve this without the change in cpp/src/arrow/vendored/datetime/tz.cpp?

Sure, I will take a look!

[rok]

The linting issue can be fixed with (source):

pre-commit run --show-diff-on-failure --color=always --all-files cpp

The cpp paginator issues are unrelated (#49347).

[canassa]

Thanks, I pushed the linting fixes.

I also investigated other approaches, but the vendored library change seem to be unavoidable. The get_tz_dir() is a static singleton, once initialized, the path is fixed for the lifetime of the process, and there's no setter.

[FreaxMATE]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

Yes! I noticed this when working on a Python project that uses PyArrow on my NixOS desktop.

I could work around it by creating a symlink, but I figured it would be nice if Arrow supported the environment variable.

I have the same issue. For anyone trying this quick workaround for nixos:

sudo mkdir-p /usr/share
sudo ln-s /etc/zoneinfo /usr/share/zoneinfo

Thanks @canassa for the fix!

[rok]

Sorry for slow reply @canassa. Thanks for checking if we can avoid changing the vendored lib!

We had recently moved to use std::chrono for accessing timezone data on Windows #48601 circumventing the vendored library and need to deal with timezone files. If you're on Windows, could you test current main branch? (current release doesn't have this yet)

If you're still seeing this issue, it would be more productive to disable vendored library further and switch to std::chrono than to keep hardcoding TZDIR.

If it'd still make sense to keep with currently proposed approach, then - the JNI CI failures could be related to your change. Can you please rebase this branch so we can check with CI?

[canassa]

Rationale for This Change

The vendored Howard Hinnant date library hardcodes /usr/share/zoneinfo as the timezone database path in discover_tz_dir(). It does not check the TZDIR environment variable, which is the POSIX standard mechanism for overriding this path.

This causes timezone operations to fail on non FHS Linux distributions such as NixOS, where zoneinfo resides under a non standard path like /nix/store/.../share/zoneinfo.

The upstream library also lacks TZDIR support.

---

What Changes Are Included in This PR?

• cpp/src/arrow/vendored/datetime/tz.cpp: Check the TZDIR environment variable in discover_tz_dir() before falling back to platform specific hardcoded paths. Uses stat() and S_ISDIR() for validation, matching the existing pattern in the function.
• cpp/src/arrow/vendored/datetime/README.md: Document the patch.
• cpp/src/arrow/public_api_test.cc: Add a non Windows test that sets TZDIR and verifies timezone resolution succeeds through Arrow's compute API.
• python/pyarrow/conftest.py: Respect TZDIR in the emscripten timezone_data test marker.

---

Are These Changes Tested?

Yes. A new Misc.TZDIREnvironmentVariable test sets TZDIR to a valid zoneinfo directory and casts a UTC timestamp to America/New_York, verifying the code path works end to end.

---

Are There Any User Facing Changes?

Arrow now respects the TZDIR environment variable on non Windows platforms, enabling timezone operations on systems without /usr/share/zoneinfo.

Fixes#49351

• GitHub Issue: [C++] Vendored date library does not respect TZDIR environment variable #49351

[github-actions]

⚠️ GitHub issue #49351has been automatically assigned in GitHub to PR creator.

[rok]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

[canassa]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

Yes! I noticed this when working on a Python project that uses PyArrow on my NixOS desktop.

I could work around it by creating a symlink, but I figured it would be nice if Arrow supported the environment variable.

[rok]

Great to hear!
I don't have time to review this right now. I'll take a look next week. One thing right now - we try to avoid changing things in cpp/src/arrow/vendored/ if possible. Could you check if we can solve this without the change in cpp/src/arrow/vendored/datetime/tz.cpp?

[canassa]

iew this right now. I'll take a look next week. One thing right now - we try to avoid changing things in cpp/src/arrow/vendored/ if possible. Could you check if we can solve this without the change in cpp/src/arrow/vendored/datetime/tz.cpp?

Sure, I will take a look!

[rok]

The linting issue can be fixed with (source):

pre-commit run --show-diff-on-failure --color=always --all-files cpp

The cpp paginator issues are unrelated (#49347).

[canassa]

Thanks, I pushed the linting fixes.

I also investigated other approaches, but the vendored library change seem to be unavoidable. The get_tz_dir() is a static singleton, once initialized, the path is fixed for the lifetime of the process, and there's no setter.

[FreaxMATE]

Thanks for opening this @canassa! Is this an issue you're personally seeing on NixOS?

Yes! I noticed this when working on a Python project that uses PyArrow on my NixOS desktop.

I could work around it by creating a symlink, but I figured it would be nice if Arrow supported the environment variable.

I have the same issue. For anyone trying this quick workaround for nixos:

sudo mkdir-p /usr/share
sudo ln-s /etc/zoneinfo /usr/share/zoneinfo

Thanks @canassa for the fix!

[rok]

Sorry for slow reply @canassa. Thanks for checking if we can avoid changing the vendored lib!

We had recently moved to use std::chrono for accessing timezone data on Windows #48601 circumventing the vendored library and need to deal with timezone files. If you're on Windows, could you test current main branch? (current release doesn't have this yet)

If you're still seeing this issue, it would be more productive to disable vendored library further and switch to std::chrono than to keep hardcoding TZDIR.

If it'd still make sense to keep with currently proposed approach, then - the JNI CI failures could be related to your change. Can you please rebase this branch so we can check with CI?