One binary. One syntax. Every format.
DX replaces jq, yq, dasel, miller, xsv, and custom scripts with a single, fast Rust binary. Query, convert, diff, validate, and transform data across 16 formats with one consistent syntax.
| Problem | Before | With DX |
|---|---|---|
| Query JSON | jq '.users[] | select(.age > 25)' | dx query data.json '.users[] | select(.age > 25)' |
| Query YAML | yq '.database.host' config.yaml | dx query config.yaml '.database.host' |
| Convert YAML to TOML | yq -o json | python3 -c "..." | dx convert config.yaml --to toml |
| Diff two configs | diff <(yq ...) <(toml ...) | dx diff staging.yaml prod.toml |
| Validate against schema | ajv validate -s schema.json -d data.json | dx validate data.json --schema schema.json |
| CSV field extraction | csvcut -c name,email data.csv | dx pick data.csv name email |
One tool. Every format. No more tool-per-format chaos.
| Feature | jq | yq | dasel | miller | DX |
|---|---|---|---|---|---|
| JSON | Yes | Yes | Yes | Yes | Yes |
| YAML | - | Yes | Yes | - | Yes |
| TOML | - | Yes | Yes | - | Yes |
| CSV | - | - | Yes | Yes | Yes |
| XML | - | Yes | Yes | - | Yes |
| NDJSON | Yes | - | - | Yes | Yes |
| MessagePack / CBOR / BSON | - | - | - | - | Yes |
| INI / HCL / Properties | - | - | - | - | Yes |
| JSON5 / RON / Plist | - | - | - | - | Yes |
| Query String | - | - | - | - | Yes |
| Cross-format convert | - | Partial | Yes | - | Yes |
| Cross-format diff | - | - | - | - | Yes |
| Schema validation | - | - | - | - | Yes |
| Schema inference | - | - | - | - | Yes |
| Total formats | 1 | 4 | 4 | 3 | 16 |
From crates.io:
cargo install dx-cliFrom prebuilt binaries:
Download the latest release for your platform from GitHub Releases.
Homebrew (macOS/Linux):
brew install aronriley24/tap/dxFrom source:
git clone https://github.com/aronriley24/Data-eXchange.git
cd Data-eXchange
cargo build --release
# Binary at target/release/dxdx --version
dx --help# Simple field access
dx query config.yaml '.database.host'# => "localhost"# Array filtering
dx query users.json '.[] | select(.age > 25)'# Object construction
dx query users.json '.[] | {name: .name, email: .email}'# Aggregation
dx query sales.json '.[] | sum'# Sort and group
dx query users.json '. | sort_by(.age) | reverse'
dx query users.json '. | group_by(.active)'# String operations
dx query users.json '.[] | .name | upper'# Pipe chains — same syntax works on any format
dx query users.csv '.[] | select(.active) | {name: .name, age: .age} | sort_by(.age)'# Text formats
dx convert config.yaml --to toml
dx convert users.json --to csv
dx convert settings.ini --to yaml
dx convert infrastructure.hcl --to json
# Binary formats (output requires -o or piping)
dx convert data.json --to msgpack -o data.msgpack
dx convert data.msgpack --to json
# CSV options
dx convert data.csv --to json # with type inference
dx convert data.csv --to json --csv-no-infer # keep as strings
dx convert data.tsv --to json --csv-delimiter '\t'# TSV support# JSONC is transparent — comments and trailing commas just work
dx convert config.jsonc --to yaml
# Stdin piping
cat data.json | dx convert - --to yaml
# NDJSON streaming
dx convert logs.ndjson --to json --stream# Compare two files (any format mix)
dx diff config.json config_modified.json
# Cross-format diff — this just works
dx diff staging.yaml prod.toml
# Keys only (hide values)
dx diff old.json new.json --keys-only
# Ignore array ordering
dx diff expected.json actual.json --ignore-order
# Machine-readable JSON output
dx diff old.json new.json --output-format json# Validate data against a schema
dx validate users.json --schema user_schema.json
# Verbose error output
dx validate data.json --schema schema.json --verbose
# Machine-readable JSON errors
dx validate data.json --schema schema.json --output-format json
# Exit code 0 = valid, 1 = invalid
dx validate data.json --schema schema.json &&echo"Valid!"# Infer schema from data
dx schema users.json
# Specify JSON Schema draft version
dx schema users.json --draft 2020-12# Extract specific fields
dx pick users.json name email
# => [{"name":"Alice","email":"alice@example.com"}, ...]# Nested field access
dx pick config.yaml database.host database.port# Count array elements
dx count users.json
# => 5# Count CSV rows (excluding header)
dx count data.csv
# Count object keys
dx count config.yaml
# Count at a nested path
dx count data.json '.users'# Flatten nested data
dx flatten config.yaml
# => {"database.host":"localhost","database.port":5432,...}# Custom separator
dx flatten config.yaml --separator '/'# Unflatten back to nested
dx unflatten flat.json# Merge two configs (later values win)
dx merge base.yaml overrides.yaml
# Merge multiple files (left to right)
dx merge base.toml local.toml production.toml
# Array strategies
dx merge a.json b.json --arrays concat # concatenate arrays (default)
dx merge a.json b.json --arrays replace # later array replaces
dx merge a.json b.json --arrays union # concat + deduplicate# Pretty-print JSON
dx fmt data.json
# Format in-place
dx fmt config.json --in-place
# Compact output
dx fmt data.json --compact
# Custom indentation
dx fmt data.json --indent 4DX supports 16 formats — 13 text and 3 binary:
| Format | Extensions | Read | Write | Notes |
|---|---|---|---|---|
| JSON | .json, .jsonc | Yes | Yes | JSONC (comments + trailing commas) handled transparently |
| YAML | .yaml, .yml | Yes | Yes | |
| TOML | .toml | Yes | Yes | No null type; keys sorted alphabetically |
| CSV | .csv, .tsv | Yes | Yes | Type inference, custom delimiters |
| NDJSON | .ndjson, .jsonl | Yes | Yes | Streaming support with --stream |
| XML | .xml | Yes | Yes | |
| JSON5 | .json5 | Yes | - | Parse-only (emit as JSON instead) |
| RON | .ron | Yes | Yes | Rusty Object Notation |
| Plist | .plist | Yes | Yes | Apple property lists |
| MessagePack | .msgpack, .mp | Yes | Yes | Binary — requires -o or pipe |
| CBOR | .cbor | Yes | Yes | Binary — requires -o or pipe |
| BSON | .bson | Yes | Yes | Binary — requires -o or pipe |
| INI | .ini | Yes | Yes | Sectioned config files |
| HCL | .hcl, .tf | Yes | Yes | HashiCorp Configuration Language |
| Query String | .qs | Yes | Yes | URL-encoded key-value pairs |
| Properties | .properties | Yes | Yes | Java properties files |
DX automatically detects the input format from:
--fromflag (highest priority)- File extension
- Content sniffing (JSON:
{/[, YAML:---, TOML:[section], etc.)
| Flag | Description |
|---|---|
--from <fmt> | Override input format detection |
--to <fmt> | Override output format |
--pretty | Force pretty-printed output |
--compact | Force compact output (no whitespace) |
--raw | Raw string output (no quotes) |
--lines | Output each array element on its own line |
--no-color | Disable colored output |
--color=always | Force colored output (even in pipes) |
-o <file> | Write output to file instead of stdout |
DX uses a jq-inspired query language that works identically across all 16 formats.
| Operator | Description | Example |
|---|---|---|
. | Identity (current value) | dx query data.json '.' |
.field | Field access | .name |
.field? | Optional field (null if missing) | .maybe_field? |
.[n] | Array index | .[0] |
.[n:m] | Array slice | .[1:3] |
.[] | Array iteration | .[] |
| | Pipe | .users | .[] | .name |
+, -, *, /, % | Arithmetic | .price * .quantity |
==, !=, <, >, <=, >= | Comparison | .age > 25 |
=~ | Regex match | .email =~ "@example\\.com$" |
and, or, not | Boolean logic | .active and .age > 18 |
| Category | Functions |
|---|---|
| Access | length, keys, values, type, has(key) |
| Transform | pick(f1,...), omit(f1,...), map(expr), flat_map(expr), flatten |
| Sort/Group | sort_by(expr), group_by(expr), unique_by(expr), reverse |
| Aggregate | sum, min, max, avg, first, last |
| String | upper, lower, trim, split(d), join(d), contains(s), starts_with(s), ends_with(s) |
| Type | to_int, to_float, to_string |
| Logic | not (also prefix operator) |
| Selection | select(condition) |
# Default: format-appropriate output
dx query data.json '.users'# Table view (arrays of objects become ASCII tables)
dx query data.json '.users' --to table
# Raw strings (no quotes)
dx query data.json '.name' --raw
# One element per line (shell-friendly)
dx query data.json '.names' --lines# Bash
dx completions bash >~/.local/share/bash-completion/completions/dx
# Zsh
dx completions zsh >~/.zfunc/_dx
# Fish
dx completions fish >~/.config/fish/completions/dx.fish
# PowerShell
dx completions powershell > dx.ps1
# Elvish
dx completions elvish > dx.elv| Code | Meaning |
|---|---|
0 | Success |
1 | Validation failure (dx validate only) |
2 | Runtime error (parse failure, I/O error, etc.) |
- Binary formats (MessagePack, CBOR, BSON) are supported for all commands that read input when the format is set via
--fromor a recognized file extension (e.g..msgpack). Use--from msgpackfor stdin or when the extension is ambiguous. - JSON5 is parse-only (no emit); use
--to jsonor another format for output. - Pickle (Python serialization) is unsafe for untrusted data; see SECURITY.
DX is built as a Cargo workspace with 6 crates:
dx/
├── dx-core # DxValue IR, error types, Format enum, traits
├── dx-formats # Parsers/emitters for all 16 formats
├── dx-query # jq-inspired query language (lexer, parser, evaluator)
├── dx-diff # Structural diff engine
├── dx-validate # JSON Schema validation + inference
└── dx-cli # CLI binary (presentation, clap, colored output)
- Single IR: All formats parse to
DxValue, a unified intermediate representation. Queries, diffs, and validation operate onDxValue, never on raw format data. This means cross-format operations (likedx diff staging.yaml prod.toml) work automatically. - Crate boundaries: Library crates produce structured data. Only
dx-clihandles presentation (colors, tables, terminal detection). - Integer/Float distinction:
DxValuepreserves the difference between42(integer) and42.0(float) through all operations. - Key ordering: Uses
IndexMapto preserve insertion order across all formats that support it. - Factory dispatch:
parser_for()/emitter_for()centralize format dispatch — no N x M match blocks.
git clone https://github.com/aronriley24/Data-eXchange.git
cd Data-eXchange
cargo build --release
# The binary is at target/release/dx
./target/release/dx --version
# Run all 1128 tests
cargo test --workspace
# Run clippy (zero warnings required)
cargo clippy --workspace -- -D warnings
# Format check
cargo fmt --all -- --check
# Supply chain audit
cargo deny checkContributions are welcome! See CONTRIBUTING.md for:
- How to run checks (fmt, clippy, tests,
cargo deny) - Architecture and crate boundaries
- Code style and where to add tests
Check the issues for "good first issue" labels if you're looking for a place to start.
Licensed under either of:
at your option.