Bigtable and float values

[dmitry-saritasa]

tried to store 0.73 as numeric value in bigtable cell and got this error:

TypeError: 0.67 could not be converted to bytes

Trace

/home/dmitry/Projects/demo/storm/libs/db/models.py in save(self, record, instance_id)
 205 for col in columns:
 206 row.set_cell(
--> 207 column_family, col, getattr(record, col, None)
 208 )
 209 row.commit()
/home/dmitry/Projects/experiments/storm/petrel/chapter3/example1/petrel/lib/python2.7/site-packages/gcloud/bigtable/row.pyc in set_cell(self, column_family_id, column, value, timestamp)
 300 """
 301 self._set_cell(column_family_id, column, value, timestamp=timestamp,
--> 302 state=None)
 303 
 304 def delete(self):

/home/dmitry/Projects/experiments/storm/petrel/chapter3/example1/petrel/lib/python2.7/site-packages/gcloud/bigtable/row.pyc in _set_cell(self, column_family_id, column, value, timestamp, state)
 126 if isinstance(value, six.integer_types):
 127 value = _PACK_I64(value)
--> 128 value = _to_bytes(value)
 129 if timestamp is None:
 130 # Use -1 for current Bigtable server time.

/home/dmitry/Projects/experiments/storm/petrel/chapter3/example1/petrel/lib/python2.7/site-packages/gcloud/_helpers.pyc in _to_bytes(value, encoding)
 457 return result
 458 else:
--> 459 raise TypeError('%r could not be converted to bytes' % (value,))
 460 
 461 

Checked the source code and see that only int or bytes are allowed to be stored. Therefore I have to convert float value into INT *i.e. 0.63 *100 = 63 and store that as a value

Am I correct and why only ints/bytes are allowed?

thanks,
Dmitry

[dhermes]

@dmitry-saritasa Thanks for filing.

1. Can you provide the code you used to trigger this error?
2. The backend allows only bytes but has special methods for counters, which are stored as >q (i.e. a long long / 64-bit signed integer in big-endian byte order).
3. If you'd like to store a Python float (double), you'd need to convert to bytes e.g. struct.pack('>d', 0.75)

---

@jonparrott WDYT of supporting types other than bytes or integers? It'd be easy enough to do, but may go against the spirit of the API.

[dmitry-saritasa]

clear - thanks @dhermes

[theacodes]

@dhermes I think it affords discussion - Python's representation of other types may not work if the data is read in another language.

[lukesneeringer]

I am going to declare bankruptcy on this issue.