Infer data types from CSV columns.
This package provides a single interface for generating the datatype for a given row-column formatted dataset. We support the following datatypes:
- DATE
- TIME
- DATETIME
- NUMBER
- INT
- FLOAT
- CURRENCY
- PERCENT
- STRING
- ARRAY
- OBJECT
- ZIPCODE
- BOOLEAN
- GEOMETRY
- GEOMETRY_FROM_STRING
- PAIR_GEOMETRY_FROM_STRING
- NONE
npm install type-analyzer
Parameters
dataArrayrequired An array of row objectrulesArrayoptional An array of custom regex rulesoptionsObjectoptional Option objectoptions.ignoreDataTypesArrayoptional Data types to ignore
varAnalyzer=require('type-analyzer').Analyzer;vardata=[{"ST_AsText": "MULTIPOLYGON (((30 20, 45 40, 10 40, 30 20)), ((15 5, 40 10, 10 20, 5 10, 15 5)))","name": "san_francisco","lat": "37.7749295","lng": "-122.4194155","launch_date": "2010-06-05","added_at": "2010-06-05 12:00"},{"ST_AsText": "MULTIPOLYGON (((30 20, 45 40, 10 40, 30 20)), ((15 5, 40 10, 10 20, 5 10, 15 5)))","name": "paris","lat": "48.856666","lng": "2.3509871","launch_date": "2011-12-04","added_at": "2010-06-05 12:00"},]varcolMeta=Analyzer.computeColMeta(data);rules
You can pass in an array of custom rules. For example. if you want to ensure that a column full of ids represented as numbers is identified as a column of strings. Rules can be matched with either exact name of the column, or regex used to match names. Note: Analyzer prefers rules using name over regex since better performance.
varAnalyzer=require('type-analyzer').Analyzer;varcolMeta=Analyzer.computeColMeta(data,[{name: 'id',dataType: 'STRING'}]);// orvarcolMeta=Analyzer.computeColMeta(data,[{regex: /id/,dataType: 'STRING'}]);options.ignoreDataTypes
You can also pass in ignoreDataTypes to ignore certain types. This will improve your type checking performance.
varDATA_TYPES=require('type-analyzer').DATA_TYPES;varcolMeta=Analyzer.computeColMeta(arr,[],{ignoredDataTypes: DATA_TYPES.CURRENCY})[0].type,And it will short cut around the usual analysis system and give you back the column formatted as you'd expect.
You can import all availale types as a constant.
Breaking changes with v1.0.0: Regex has moved into src, but can more easily be accessed from the module.exports from the root. As part of a larger clean up many extraneous util files were removed.