tests: only refresh the minimum sysinfo in mem limit tests.

[ashdnazg]

Rationale for this change

Currently the memory limit tests hang due to lock contention and the overhead from frequently calling sysinfo::System::refresh_all() in the memory monitoring task.

In fact, we don't really need to refresh all info, it's enough to just get the memory data about the current process, which significantly reduces the refresh time and the tests finish quickly.

What changes are included in this PR?

In the memory limit utils, the memory monitoring spawned task now refreshes system info only for what's actually necessary.

Are these changes tested?

It's part of the tests, and they seem to not hang anymore :)

Are there any user-facing changes?

No

[ashdnazg]

Can we run the extended tests on this PR?

[jayzhan211]

Run extended tests

[ashdnazg]

@jayzhan211https://github.com/apache/datafusion/actions/runs/14439917392/job/40487622886 seems to have failed?

[jayzhan211]

it is still running

[ashdnazg]

I don't think it does, the spawning crashed, there's nothing running here: https://github.com/apache/datafusion/actions?query=workflow%3A%22Datafusion+extended+tests%22

[ashdnazg]

The CI issue is hopefully fixed here: #15704

[2010YOUY01]

Thank you for the fix. To test this PR, I think you can push the change to your cloned repository of datafusion's main branch, and the extended test CI will be triggered there. (Maybe you have to also enable GitHub action first in your cloned repo, I can't remember exactly how)

[ashdnazg]

Tests seem to finish nicely: https://github.com/ashdnazg/datafusion/actions/runs/14442804912/job/40496805764

[jayzhan211]

👍🏻

[alamb]

Thank you @ashdnazg and @jayzhan211