Core: Extend ResolvingFileIO to support prefix-based operations

[nastra]

No description provided.

[bryanck]

We can just call the delegate directly now, I don't believe there is a need for this logic anymore

[nastra]

do you think we should still batch here or would we want to essentially use the same implementation you had?

// peek at the first element to determine the type of FileIO
Iterator<String> originalIterator = pathsToDelete.iterator();
if (!originalIterator.hasNext()) {
return;
}
PeekingIterator<String> iterator = Iterators.peekingIterator(originalIterator);
DelegateFileIO fileIO = io(iterator.peek());
fileIO.deleteFiles(() -> iterator);

To me it seems like batching should probably be handled by the underlying FileIO implementation, but just wanted to see what others think?

[bryanck]

Actually, it is probably better to keep what you have as there may be multiple FileIO types involved, and the peek method wasn’t ideal.

[RussellSpitzer]

Thanks @nastra for finishing this up and @bryanck for the foundational work and review.

[leoluan2009]

@nastra@RussellSpitzer Should ADLSFileIO implement DelgateFileIO as GCSFileIO to make code clean and consistently?

[nastra]

@leoluan2009 yes it should, good catch. I've opened #8563 to address that