Add tsfile-cli: inspect and import .tsfile from the command line

[SpriCoder]

Summary

Adds tsfile-cli, a single pipe-friendly C++ command-line tool (cpp/tools/) for working with .tsfile files from the shell. Built entirely on the existing storage::TsFileReader / TsFileTableWriter APIs; no storage-engine changes.

Inspect / export (read-only)

• ls — list devices (tree model) or tables (table model)
• schema — per-series datatype / encoding / compression
• meta — file-level summary: model, counts, global time range, file size
• stats — per-series count, start, end, min, max, first, last, sum (from statistics, no page scan)
• count — per-series row counts + total
• head / cat — preview / stream rows, with projection (-m), time range (--start/--end), --offset / -n
• sample — deterministic reservoir sample (--seed)
• Output csv|tsv|json|table (TTY-adaptive); data → stdout, diagnostics → stderr; exit codes 0/1/2/3

Import (write)

• write — import CSV/TSV rows into a new table-model .tsfile
• Explicit --columns name:TYPE:tag|field schema (no type inference); first input column is the timestamp
• Reads a file or stdin (-); -o output; -f csv|tsv; optional --no-header / --header-match
• Silent on success (-v prints a one-line summary), Unix-style

Other

• ReadFile::open errors now go to stderr (were stdout) so read output stays pipe-clean
• Unit + in-process E2E tests: arg parsing, formatters, statistics, CSV/TSV parsing, and a write→read round-trip

Test plan

• cd cpp && bash build.sh -t=Debug builds bin/tsfile-cli + TsFile_Test (add --disable-antlr4 on CMake >= 4)
• CLI suites pass: InputFormatTest, ParseArgsTest, RunCliTest, CliE2E, RowWriterTest, StatTableTest
• tsfile-cli meta|ls|schema|stats|count <file.tsfile>
• round-trip: printf 'time,id,v\n0,d,1\n' | tsfile-cli write --table t --columns "id:STRING:tag,v:INT64:field" -o out.tsfile - then tsfile-cli count -f tsv out.tsfile

[Copilot]

Pull request overview

Adds a new C++ command-line tool (tsfile-cli) under cpp/tools/ for inspecting/exporting .tsfile contents and importing CSV/TSV into a new table-model .tsfile, with accompanying formatters, command implementations, and a new tool-focused test suite integrated into the existing CMake test target.

Changes:

• Introduces tsfile-cli executable + CLI dispatch/arg parsing, read commands (ls/schema/meta/stats/count/head/cat/sample), and CSV/TSV import (write).
• Adds output/input formatting helpers (CSV/TSV/NDJSON/table) and statistics aggregation helpers for metadata-driven commands.
• Integrates tool build + tests into CMake, and adjusts ReadFile::open() diagnostics to go to stderr.

Reviewed changes

Copilot reviewed 37 out of 38 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
cpp/tools/tools_main.cc	Adds main() entrypoint that forwards argv to run_cli.
cpp/tools/CMakeLists.txt	Adds tsfile-cli build target and install rule.
cpp/tools/README.md	Documents build/usage/options and examples for tsfile-cli.
cpp/tools/skills/tsfile-cli/SKILL.md	Adds a machine-readable skill doc describing CLI usage.
cpp/tools/cli/cli_args.h	Defines ParsedArgs struct for CLI arguments.
cpp/tools/cli/cli_args.cc	Implements argument parsing for all commands/options.
cpp/tools/cli/exit_codes.h	Defines standardized exit codes for CLI.
cpp/tools/cli/run_cli.h	Declares run_cli entrypoint for CLI execution.
cpp/tools/cli/run_cli.cc	Implements top-level dispatch, validation, and help/version handling.
cpp/tools/commands/commands.h	Declares per-command handlers and shared helpers.
cpp/tools/commands/cmd_ls.cc	Implements ls for devices/tables.
cpp/tools/commands/cmd_schema.cc	Implements schema listing for tree/table models.
cpp/tools/commands/cmd_meta.cc	Implements file-level summary output (meta).
cpp/tools/commands/cmd_stats.cc	Implements per-series statistics output (stats).
cpp/tools/commands/cmd_count.cc	Implements per-series counts + total (count).
cpp/tools/commands/cmd_head.cc	Implements head via shared row query helper.
cpp/tools/commands/cmd_cat.cc	Implements cat via shared row query helper.
cpp/tools/commands/cmd_sample.cc	Implements reservoir sampling (sample).
cpp/tools/commands/row_query.cc	Implements shared querying logic used by row-returning commands.
cpp/tools/commands/cmd_write.cc	Implements CSV/TSV import into new table-model .tsfile (write).
cpp/tools/commands/stat_table.h	Defines structures/helpers for stats + meta summary extraction.
cpp/tools/commands/stat_table.cc	Implements metadata/statistics collection used by stats/meta/count.
cpp/tools/format/output_format.h	Defines output format enum and RowWriter.
cpp/tools/format/output_format.cc	Implements CSV/TSV/NDJSON/table formatting and escaping.
cpp/tools/format/input_format.h	Defines column-spec parsing and delimited-line parsing helpers.
cpp/tools/format/input_format.cc	Implements --columns parsing, CSV quote splitting, bool parsing.
cpp/tools/format/result_set_format.h	Declares helpers for converting result sets to output.
cpp/tools/format/result_set_format.cc	Implements result-set streaming and reservoir-sampled output.
cpp/test/CMakeLists.txt	Wires tool tests into TsFile_Test when BUILD_TOOLS is on; adjusts GTest include handling.
cpp/test/tools/cli_test_util.h	Adds helpers for CLI fixture generation and temp file naming.
cpp/test/tools/cli_args_test.cc	Adds unit tests for arg parsing and CLI-level validation.
cpp/test/tools/input_format_test.cc	Adds unit tests for column spec parsing and CSV/TSV splitting.
cpp/test/tools/output_format_test.cc	Adds unit tests for escaping, formatting, and table alignment.
cpp/test/tools/stat_table_test.cc	Adds unit tests for statistic-to-cell conversion logic.
cpp/test/tools/command_e2e_test.cc	Adds in-process E2E tests including a write→read round-trip.
cpp/src/file/read_file.cc	Routes ReadFile::open() diagnostics to stderr instead of stdout.
cpp/CMakeLists.txt	Adds BUILD_TOOLS option and includes cpp/tools subdirectory.
.gitignore	Ignores local AI tooling dirs and test-run artifacts.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[ColinLeeo]

When handling exceptions in the CLI, we should return readable error messages instead of generic descriptions or error codes.

[SpriCoder]

@ColinLeeo Addressed in 7bb6b62. The CLI now translates storage-engine error codes into readable phrases via a query_error_text() helper — e.g. Error: query failed: table does not exist instead of query failed (code 49) — used by both the row-query and sample paths, and the result-read failure path now prints a readable message too. The other diagnostics (file open/corrupt, bad field count, type/overflow, header mismatch) were already human-readable. If a specific path is still surfacing a raw code, point me at it and I'll map it.

[ColinLeeo]

include_directories(
${LIBRARY_INCLUDE_DIR}
${THIRD_PARTY_INCLUDE}
)

do not use ${CMAKE_CURRENT_SOURCE_DIR} pls

[SpriCoder]

Addressed in a8c38e3. tsfile_cli_obj now includes library headers from the staged ${LIBRARY_INCLUDE_DIR} + ${THIRD_PARTY_INCLUDE} instead of reaching into ${PROJECT_SOURCE_DIR}/src; ${CMAKE_CURRENT_SOURCE_DIR} stays only for the tool's own cli/ commands/ format/ headers. This matches what test/CMakeLists.txt already does (include_directories(${LIBRARY_INCLUDE_DIR} ${THIRD_PARTY_INCLUDE}) for the library + -I .../tools for tool headers). Verified the Debug build still resolves all headers from the staged include tree.

[ColinLeeo]

remove CMAKE_CURRENT_SOURCE_DIR

[SpriCoder]

Done in #835 — both ${CMAKE_CURRENT_SOURCE_DIR} usages are replaced with ${CMAKE_SOURCE_DIR}/tools (for tsfile_cli_obj and the tsfile_cli executable), matching the convention already used in cpp/test/CMakeLists.txt. Note the executable line is not redundant: $<TARGET_OBJECTS:tsfile_cli_obj> only pulls in object files and does not propagate the object library's PUBLIC include dirs, so tools_main.cc still needs tools/ on its include path. Verified the Debug build links tsfile_cli and resolves all headers from the staged include tree.

[ColinLeeo]

Good Job！