Skip to content

Allow overriding provision of FileSystem instances to HadoopTableOperations #13

Description

@mccheah

Util.getFS forces the FileSystem objects to be cached at the JVM level using FileSystem.get. In specific situations, sometimes one would prefer finer-grained control over the caching of these objects.

We propose the following:

  • Add protected FileSystem getFS(Path path, Configuration configuration) in HadoopTableOperations. Default implementation calls Util#getFS.
  • Make HadoopTableOperationspublic with a protected constructor so that users can use some of the behavior but override specific components, particularly File I/O Submodule for TableOperations #12 and the FileSystem provision change above.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions