RFC Allow for io.StringIO inputs to certain modules

[weiji14]

Description of proposed changes

Make a tempfile_from_buffer backend function for io.StringIO inputs. Will be useful for modules where:

• an in-memory stream of 'string-like' data makes sense, e.g. Figure.legend's 'spec' option (see also Allow Figure.legend to read from StringIO #571)
• a 'filename' is the only way to pass in data as using GMT virtualfiles don't work, e.g. x2sys_cross (see also Wrap x2sys_init and x2sys_cross #546 (comment))

This is implemented as the 'hacky' Option 1 at #571 (comment).

Happy to discuss about the actual implementation details, such as:

• whether 'tempfile_from_buffer' is a good name
• should this live inside 'tempfile.py', or is it better placed with the other 'virtualfile_from_*' functions (which is what I modeled 'tempfile_from_buffer' after).
• etc

See also:

• https://stackoverflow.com/questions/11892623/stringio-and-compatibility-with-with-statement-context-manager
• https://stackoverflow.com/questions/3253258/what-is-the-best-way-to-write-the-contents-of-a-stringio-to-a-file#3253819
• https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html?highlight=stringio

Fixes#571

Reminders

• Run make format and make check to make sure the code follows the style guide.
• Add tests for new features or tests that would have caught the bug that you're fixing.
• Add new public functions/methods/classes to doc/api/index.rst.
• Write detailed docstrings for all functions/methods.
• If adding new functionality, add an example to docstrings or tutorials.

[weiji14]

@dr-glenn, do you want to try this branch using pip install https://github.com/GenericMappingTools/pygmt/archive/stringio_input.zip and see if it works for your StringIO legend? Documentation (beta) is at https://pygmt-git-stringioinput.gmt.vercel.app/api/generated/pygmt.Figure.legend.html#pygmt.Figure.legend.

Happy to receive any feedback before we merge this. If it's good, we might get this in for the upcoming PyGMT v0.2.0 release 😁

[seisman]

io.StringIO is something new to me. Some googling gives the impression that io.StringIO is one of the file-like objects. Perhaps we should generalize the function to support all file-like objects?

[weiji14]

io.StringIO is something new to me. Some googling gives the impression that io.StringIO is one of the file-like objects. Perhaps we should generalize the function to support all file-like objects?

I'm a pretty new user of io.StringIO too. I'm aware of io.BytesIO which is a binary file-like object, but I'm not quite sure if it is commonly used and/or whether it's worth supporting. Maybe @dr-glenn knows more.

To be fair, we'll probably need to refactor this hacky implementation (writing to a tempfile and having GMT read that file) to use GMT_Read_Data or something else at some point. Not to say that this PR isn't useful, just that we can add these 'enhancements' in one step at a time.

[seisman]

To be fair, we'll probably need to refactor this hacky implementation (writing to a tempfile and having GMT read that file) to use GMT_Read_Data or something else at some point.

Yes, that's another point. Without this PR, users just need to add a few lines of codes (maybe two or three) to write strings into a temporary file, and pass the file name to legend(), text() or any other modules. But here you add several lines of codes to support it, and these codes are most likely to be removed in the future. Not sure if it's really worth the complexity.

[weiji14]

Yes, that's another point. Without this PR, users just need to add a few lines of codes (maybe two or three) to write strings into a temporary file, and pass the file name to legend(), text() or any other modules. But here you add several lines of codes to support it, and these codes are most likely to be removed in the future. Not sure if it's really worth the complexity.

Fair point, the code here is really just a convenience function to simplify maybe 4-5 lines of code:

pygmt/pygmt/helpers/tempfile.py

Lines 150 to 155 in 06ee891

	withGMTTempFile() astmpfile:
	buf.seek(0) # Change stream position back to start
	withopen(file=tmpfile.name, mode="w") asfdst:
	shutil.copyfileobj(fsrc=buf, fdst=fdst)
	yieldtmpfile.name

As for whether it's worth it, there are some GMT Supplemental Modules that don't quite support virtualfiles yet (e.g. x2sys, as mentioned in #546 (comment)), and having this tempfile_from_buffer helper function in the backend would be useful in the interim months to pass in pandas.DataFrame inputs to these modules.

[seisman]

@weiji14 I think it's time to refactor this PR, so that all functions can accept io.StringIO input.

[weiji14]

@weiji14 I think it's time to refactor this PR, so that all functions can accept io.StringIO input.

Ok, but we really should make the input types tutorial at #1268 a priority first. I feel that new users don't know all the I/O options PyGMT supports, like passing in a shapefile...

[seisman]

This PR will be superseded by PR #3326 and #3438. Closing.