Histogram doesn't work for pandas.Series

[seisman]

Description of the problem

Full code that generated the error

import pygmt
import pandas as pd
import numpy as np

table = pd.read_csv("https://raw.githubusercontent.com/GenericMappingTools/pygmt/master/pygmt/tests/data/points.txt", sep=" ", header=None)

fig = pygmt.Figure()
fig.histogram(table[0], pen="1p", series=[0, 90, 10], frame=True)             # Fail
#fig.histogram(np.array(table[0]), pen="1p", series=[0, 90, 10], frame=True)  # Pass
fig.show()

I have to convert table[0] to a numpy array to make it work.

Full error message

Traceback (most recent call last):
  File "test.py", line 8, in <module>
    fig.histogram(table[0], pen="1p", series=[0, 90, 10], frame=True)
  File "pygmt/pygmt/helpers/decorators.py", line 297, in new_module
    return module_func(*args, **kwargs)
  File "pygmt/pygmt/helpers/decorators.py", line 438, in new_module
    return module_func(*args, **kwargs)
  File "pygmt/pygmt/src/histogram.py", line 57, in histogram
    with file_context as infile:
  File "lib/python3.8/contextlib.py", line 113, in __enter__
    return next(self.gen)
  File "pygmt/pygmt/clib/session.py", line 1159, in virtualfile_from_vectors
    rows = len(arrays[0])
TypeError: len() of unsized object

System information

Please paste the output of python -c "import pygmt; pygmt.show_versions()":

PyGMT information:
  version: v0.3.2.dev82+g82ebbbc4.d20210412
System information:
  python: 3.8.5 (default, Sep  4 2020, 02:22:02)  [Clang 10.0.0 ]
  executable: /Users/XXX/.miniconda/bin/python
  machine: macOS-10.16-x86_64-i386-64bit
Dependency information:
  numpy: 1.20.1
  pandas: 1.2.3
  xarray: 0.17.0
  netCDF4: 1.5.6
  packaging: 20.9
  ghostscript: 9.53.3
  gmt: 6.2.0_522a350_2021.04.29
GMT library information:
  binary dir: /Users/XXX/.miniconda/bin
  cores: 8
  grid layout: rows
  library path: /Users/XXX/opt/GMT-master/lib/libgmt.dylib
  padding: 2
  plugin dir: /Users/XXX/opt/GMT-master/lib/gmt/plugins
  share dir: /Users/XXX/opt/GMT-master/share
  version: 6.2.0

[weiji14]

Good catch, this isn't a problem restricted to just histogram though. I've stepped through the logic in pygmt/clib/session.py and believe it's a problem with the vectors_to_arrays function not handling a 1D pandas.Series properly, specifically this line:

pygmt/pygmt/clib/conversion.py

Line 161 in d2cb8b9

arrays = [as_c_contiguous(np.asarray(i)) for i in vectors]

The list comprehension would loop through all the rows in the pandas.Series, when it should just treat the entire pandas.Series as a single array. I.e. arrays = vectors_to_arrays(vectors) produces this (wrong):

[array(43.4847),
 array(22.331),
 array(40.8023),
 array(49.876),
 array(60.6897),
 array(27.779),
 array(40.3582),
 array(18.9422),
 array(20.7545),
 array(19.8565),
 array(19.9855),
 array(46.4844),
 array(43.2328),
 array(59.3526),
 array(16.7597),
 array(32.3278),
 array(11.5309),
 array(61.7074),
 array(28.1125),
 array(47.8333)]

when it should actually produce this (correct):

[array([43.4847, 22.331 , 40.8023, 49.876 , 60.6897, 27.779 , 40.3582,
        18.9422, 20.7545, 19.8565, 19.9855, 46.4844, 43.2328, 59.3526,
        16.7597, 32.3278, 11.5309, 61.7074, 28.1125, 47.8333])]

and this is why rows = len(arrays[0]) produces the TypeError: len() of unsized object, because it is only picking up array(43.4847) instead of array(([43.4847, 22.331 ,...,])

This may or may not be a duplicate of #1132 actually, since a pandas.Series is basically a 1D array, whereas we expect a 2D pandas.DataFrame array input into the table parameter (1D numpy.array objects are treated differently). But it's helpful to have the example test case above to debug the situation 😄

Edit: Actually there's probably several solutions to fix this bug, another way is to refactor the virtualfile_from_data function to handle pandas.Series objects properly since we're thinking of it already at #990 (comment). Will do a bit more testing.