instrument pycloudlib failures in lxc delete /mnt/path --force

[holmanb]

Our integration tests are currently seeing lxd vm failures like the following:

failed on teardown with "RuntimeError: Failure (rc=1): Error: Stopping the instance failed: Failed unmounting instance: Failed to unmount "/var/snap/lxd/common/lxd/storage-pools/default/virtual-machines/cloudinit-0111-2315175ouu6z5k": device or resource busy"

Another example

Since lxd doesn't include diagnostic information required to understand why this is happening, I propose temporarily parsing the error message for the mount point and checking the mount for open files.

Update: almost 2 years later and the underlying issue hasn't been resolved and now we are seeing similar issues on lxd VM restarts as well. I just revived this PR because I'd like to get to the bottom of these failures, but for that we need more information. Triaging constant integration test failures is a waste of time and I don't see any better proposals at this currently.

No tests because this is just some simple instrumentation that I plan to rip out as soon as the underlying issue is resolved. It should only run in the failure path anyways so risk of breaking things is low.

This is a pretty ugly hackParsing lxd's error messages isn't ideal, so I didn't go further than open file checking before getting eyes on this. We may want/need to add checks for the various other causes of EBUSY. This is a common cause for EBUSY, so this could be enough, however if a reviewer is okay with the general approach I can also move forward with something a little more comprehensive than just checking for open files.

[TheRealFalcon]

@holmanb is this PR still relevant?

[holmanb]

@holmanb is this PR still relevant?

I think @blackboxsw was had preference for a different approach. I'll close it.