Add support for AIX (closes #241)

[HorlogeSkynet]

No description provided.

[HorlogeSkynet]

Up @nir0s@jokurz 🙇

[HorlogeSkynet]

Up @nir0s@jokurz 🙏

[nir0s]

LGTM

[HorlogeSkynet]

Can we have a quick thought from you on this @sethmlarson if you don't mind ? 🙇

[hartwork]

I'd like to raise concerns that AIX is (Unix but) not Linux and that hence putting AIX bits into class LinuxDistribution seems like mis-leading modelling. Also I see a mix of host and target system here, e.g. host sys.platform is inspected no matter if .root_dir is /or not, same for the call to oslevel. Just my two cents

[HorlogeSkynet]

I'd like to raise concerns that AIX is (Unix but) not Linux and that hence putting AIX bits into class LinuxDistribution seems like mis-leading modelling.

You're right, this seems related to the same questions raised in #177 about this module architecture. At the moment, despite renaming LinuxDistribution to UnixDistribution (in a backward-compatible way until v2.0.0 of course), I don't have any other idea about how we could add AIX support here (LinuxDistribution() being a distro singleton and so on...).

Also I see a mix of host and target system here, e.g. host sys.platform is inspected no matter if .root_dir is /or not, same for the call to oslevel. Just my two cents

You were absolutely right, please take a look to the updated (and rebased) patch, it aims at improving host and target system information separation (surprisingly it simplified testing and I think this would improve detection rate OOTB).

Bye, thanks for your time Sebastian 🙇

[hartwork]

@HorlogeSkynet I see some good ideas here, but I'm unsure if this approach will work without use of a chroot-like approach, which I don't see here: adjusting PATH alone will not keep the commands we are calling from reading files outside/upwards .root_dir. Am I missing something?

[HorlogeSkynet]

I see some good ideas here, but I'm unsure if this approach will work without use of a chroot-like approach, which I don't see here: adjusting PATH alone will not keep the commands we are calling from reading files outside/upwards .root_dir. Am I missing something?

No you don't, I actually did think about that before adapting this patch, but then forgot 🤔
I'll replace this mess by a proper os.chroot soon. It will also simplify all os.path.join operations performed around root_dir attribute. Furthermore, replacing ":" literal by os.pathsep won't be required anymore.
I'll keep you posted, bye 👋

---

Note : how could we then support --root-dir option with os.chroot availability limited to UNIX platforms ?

EDIT : I think I got an idea to keep this "BC" for Windows, stay put 😌

[HorlogeSkynet]

So I'm back Sebastian, latest thoughts on this topic :

• os.chroot needs at least CAP_SYS_CHROOT, and I don't think we can want distro to require any privilege in the future
• Actually, even chroot-ing wouldn't solve our issue here. The --root-dir option cannot be extended to third binaries execution :
  • What if targeted rootfs architecture is different from the host one ?
  • What if the binary output is based on running kernel information (see UNAME(2) for instance, I don't know how oslevel works) ?

As AIX detection would be based on information gathered from uname, --root-dir usage from a Linux host (for instance) targeting an AIX rootfs would be incompatible.

Where do you think we should go from here ?
I see two options from my PoV :

1. Reverting my previous changes and accounting for --root-dir limitations, as long as preventing any sub-process execution when this options is used (a.k.a. not trusting include_uname nor include_lsb constructor values)
2. Opting for a proper os.chroot implementation and warning users if they are not on UNIX or running with insufficient privileges.

Bye 👋

[HorlogeSkynet]

Dear Sebastian @hartwork, dear maintainers,

You will find here an updated branch, with 4 (rebased) commits :

1. The first one "naively" brings support for AIX (as it was originally proposed here) ;
2. The second one is actually "breaking", as it removes support for binaries execution when -r/--root-dir (see Add support for detecting a distro from a rootfs even on another OS #161 and Detect distro from arbitrary rootfs root_dir #161 #247) is specified (rationale below) ;
3. The third one updates documentation about LinuxDistribution exceptions that can be raised (a catch and ignore subprocess.CalledProcessError when running lsb_relsase #261 omission) ;
4. The last one adds a proper consideration for root_dir public attribute.

---

Rationale about 2 : According to the above discussion, concerns have been raised about -r/--root-dir argument incompatibility with target rootfs binaries execution.
Bringing support for AIX emphasized this issue, as calling uname might return totally wrong information (running kernel would be probed, not the one from the targeted rootfs).
Moreover, about oslevel (and maybe in the future any interesting binary), we cannot expect it/them from being even executable (architecture, dynamic libraries, specific target system requirement, ...).
The chroot-based approach mentioned above is a dead-end in our case, as, in addition not to fix all of these issues, it requires system privileges that we cannot expect regular users to have/give to.

A new parameter has been introduced (include_third_bins) to control execution of each one of these "third-party" binaries that we might happen to call in the future. I'm not sure about its name though, so any comment would be appreciated.

To conclude, a very quick GitHub search shows(?) that actually no one uses this parameter from API (at least on this forge).

Bye, thanks for your time ! 👋

[hartwork]

Hi @HorlogeSkynet , good summary, sorry for not getting to a review and reply earlier: