Uh oh!
There was an error while loading. Please reload this page.
Support load json-data into Doris by RoutineLoad or StreamLoad - #3553
Conversation
morningman
commented
May 11, 2020
UT compile failed |
| 7. `jsonpaths` | ||
| Set jsonpath for precise imports, otherwise simple imports. |
There was a problem hiding this comment.
what does simple imports mean? Should specify that jsonpaths is only valid when format is json
There was a problem hiding this comment.
i had add more comment
| Set jsonpath for precise imports, otherwise simple imports. | ||
| 8. `strip_outer_array` | ||
| Boolean type, true to indicate that json data starts with an array object and flattens objects in the array object. |
| `format` | ||
| Specifies the format of the imported data. Support csv and json, the default is csv. | ||
| `jsonpaths` | ||
| Set jsonpath for precise imports, otherwise simple imports. | ||
| `strip_outer_array` | ||
| Boolean type, true to indicate that json data starts with an array object and flattens objects in the array object. |
| 7. jsonpaths | ||
| jsonpaths: 导入json方式分为：简单模式和精准模式。如果设置了jsonpath则为精准模式导入，否则为简单模式导入，具体可参考示例。 |
There was a problem hiding this comment.
我觉得这里还是要解释一下什么是简单模式，什么是精准模式，这样能够更好的让用户理解这两种模式的区别。
| 8. strip_outer_array | ||
| 布尔类型，为true表示json数据以数组对象开始且将数组对象中进行展平，默认值是false。 |
There was a problem hiding this comment.
给个例子就能够让用户更好理解了，比如下面的数据，如果这个值设置为true，那么就会导入两条数据。如果设置为false，就会按照给定的jsonpaths来抽取字段进行导入。
[
{"k1" : 1, "v1" : 2},
{"k1" : 3, "v1" : 4},
]
| if (!_json_doc.IsArray() && _strip_outer_array) { | ||
| delete[] json_str; | ||
| return Status::InternalError("JSON ROOT node is array-object, `json_unnest` must be TRUE."); |
There was a problem hiding this comment.
| return Status::InternalError("JSON ROOT node is array-object, `json_unnest` must be TRUE."); | |
| return Status::InternalError("JSON ROOT node is array-object, `strip_outer_array ` must be TRUE."); |
| } | ||
| } | ||
| _next_line++; | ||
| if (!valid || nullcount == _jsonpaths_doc.Size()) { |
There was a problem hiding this comment.
if nullcount == _jsonpaths_doc.Size(), the _counter->num_rows_filtered++ should be called, and the error line should be recorded in runtime state.
| {"RECORDS":[ | ||
| {"category":"C++","author":"avc","title":"C++ primer","price":89.5}, | ||
| {"category":"Java","author":"avc","title":"Effective Java","price":95}, | ||
| {"category":"Linux","author":"avc","title":"Linux kernel","price":195} | ||
| ]} |
There was a problem hiding this comment.
我理解不应该是这样的格式，而应该是下面的格式:
[
{"category":"C++","author":"avc","title":"C++ primer","price":89.5},
{"category":"Java","author":"avc","title":"Effective Java","price":95},
{"category":"Linux","author":"avc","title":"Linux kernel","price":195}
]
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Doris support load json-data by RoutineLoad or StreamLoad. Ref: [https://github.com//pull/3230]