[REEF-1357] Allow different caching levels for caching in IInputParti…

[afchung]

…tion

This addressed the issue by

• Adding the interface IDataMover, which abstracts the movement of data between different memory layers.
• Adding the DataCache class, which abstracts the caching of data at different storage layers.
• Changing the IInputPartitionn interface to work with the new ISerializer.

JIRA:
REEF-1357

[dkm2110]

Do we plan to modify or remove this function altogether in near future. Or is it related to the fact that we plan to give range of options from user min,max than simply one cache level.

[afchung]

This is just related to the fact that the API is not stabilized yet, and we don't want to lock ourselves into any commitments with regards to backward compatibility.

[dkm2110]

@afchung left comments.

[afchung]

@dkm2110 I've been trying to add test for DataMover and DataCache, but it seems we can't reliably do so without faking a wrapper interface on top of DirectoryInfo and FileInfo, at the least. Given that adding the wrapper interfaces will take up a lot of lines and obfuscate the CR, I'll split this into two tasks - creating the interfaces and then this PR (REEF-1357). I'll close this PR first. Thanks for the review!

[dkm2110]

@afchung Makes sense. Thanks!!

[afchung]

@dkm2110 Please take a look at #997 when you have time. Thanks!