Avro Table Provider - #910

Merged

alamb merged 56 commits into

apache:masterfrom

Igosuki:avro2_m

Sep 15, 2021

Commits on Sep 14, 2021

Add avro as a datasource, file and table provider
Igosuki
committed
7e14f91
wip
Igosuki
committed
0f940ab
Added support composite identifiers for struct type.
jorgecarleitao
authored and
Igosuki
committed
2ca7ccb
Fixed build.
jorgecarleitao
authored and
Igosuki
committed
fcbf43b
cheat and add unions to valid composite column types
Igosuki
committed
bae2f00
Implement the AvroArrayReader
Igosuki
committed
a352dad
Add binary types
Igosuki
committed
d42cdd1
Enable Avro as a FileType
Igosuki
committed
cbdc9c7
Enable registering an avro table in the sql parsing
Igosuki
committed
7d6078e
Change package name for datafusion/avro
Igosuki
committed
44cfa87
Implement Avro datasource tests and fix avro_rs::Value resolution to Arrow types
Igosuki
committed
6ef92d8
Test for AvroExec::try_from_path
Igosuki
committed
01cbb18
external table avro test
Igosuki
committed
537577a
Basic schema conversion tests
Igosuki
committed
75566d1
Complete test for avro_to_arrow_reader on alltypes_dictionnary
Igosuki
committed
7fdf233
fix_stable: .rewind is 'unstable'
Igosuki
committed
697e3a9
Fix license files and remove the unused avro-converter crate
Igosuki
committed
e1d6df8
fix example test in avro_to_arrow
Igosuki
committed
66a5901
add avro_sql test to default workflow
Igosuki
committed
9ea942c
Adress clippies
Igosuki
committed
84ee28a
Enable avro as a valid datasource for client execution
Igosuki
committed
9edac57
Add avro to available logical plan nodes
Igosuki
committed
e8a6206
Add ToTimestampMillis as a scalar function in protos
Igosuki
committed
45eac7c
Allow Avro in PhysicalPlan nodes
Igosuki
committed
b4340ac
Remove remaining confusing references to 'json' in avro mod
Igosuki
committed
408f759
rename 'parquet' words in avro test and examples
Igosuki
committed
f34b995
Handle Union of nested lists in arrow reader
Igosuki
committed
6ce0904
test timestamp arrays
Igosuki
committed
e1e40ef
remove debug statement
Igosuki
committed
1c79ffb
Make avro optional
Igosuki
committed
c96b781
Remove debug statement
Igosuki
committed
060c644
Remove GetField usage (see #628)
Igosuki
committed
72ad35c
Fix docstring in parser tests
Igosuki
committed
3c8f6ce
Test batch output rather than just rows individually
Igosuki
committed
0db3013
Remove 'csv' from error strings in physical_plan::avro
Igosuki
committed
faa2152
Avro sample sql and explain queries tests in sql.rs
Igosuki
committed
a6cd7f1
Activate avro feature for cargo tests in github workflow
Igosuki
committed
8b4fcae
Add a test for avro registering multiple files in a single table
Igosuki
committed
b2b7915
Switch to Result instead of Option for resolve_string
Igosuki
committed
8acf062
Address missing clippy warning should_implement_trait in arrow_to_avro/reader
Igosuki
committed
65c9b9e
Add fmt display implementation for AvroExec
Igosuki
committed
1668b96
ci: fix cargo sql run example, use datafusion/avro feature instead of 'avro'
Igosuki
committed
c9218e2
license: missing license file for avro_to_arrow/schema.rs
Igosuki
committed
be09a64
only run avro datasource tests if features have 'avro'
Igosuki
committed
68df268
refactor: rename infer_avro_schema_from_reader to read_avro_schema_from_reader
Igosuki
committed
09c4ecb
Pass None as props to avro schema schema_to_field_with_props until further notice
Igosuki
committed
94b9dcb
Change schema inferance to FixedSizeBinary(16) for Uuid
Igosuki
committed
a03a3c2
schema: prefix metadata coming from avro with 'avro'
Igosuki
committed
405c63a
make num traits optional and part of the avro feature flag
Igosuki
committed
291637a
Fix avro schema tests regarding external props
Igosuki
committed
00f109c
split avro physical plan test feature wise and add a non-implemented test
Igosuki
committed
ea31e02
submodule: switch back to apache/arrow-testing
Igosuki
committed
a264858
fix_test: columns are now prefixed in the plan
Igosuki
committed
c62d931
avro_test: fix clippy warning cmp-owned
Igosuki
committed
aa45189
avro: move statistics to the physical plan
Igosuki
committed
0b746db

Commits on Sep 15, 2021

Increase min stack size for cargo tests
Igosuki
committed
767c469