Skip to content

Repository files navigation

loq

Fast, friendly CLI log analyzer built with Bun. Think "jq for logs."

CIDocumentationLicense: MITBun

Features

  • Auto-detects log formats - JSON, Apache, Nginx, Syslog, CLF
  • Simple query syntax - loq app.log where level=error
  • SQL-like queries - loq "SELECT * FROM app.log WHERE status >= 400"
  • Powerful filtering - equality, comparison, contains, regex
  • Aggregations - count, group by, sum, avg, min, max
  • Multiple outputs - colorized, table, JSON, CSV
  • Streaming - handles large files efficiently
  • Pipe support - cat *.log | loq where level=error
  • Tail mode - loq -f app.log where level=error
  • Extensible - add custom log formats via config

Installation

Using Bun

# Install Bun if you haven't
curl -fsSL https://bun.sh/install | bash
# Clone and install
git clone https://github.com/code-with-auto/loq.git
cd loq
bun install
bun link

Global binary

# After bun link, loq is available at ~/.bun/bin/loq# Add to your PATH or use directly:~/.bun/bin/loq --help

Quick Start

# View all logs with colorized output
loq app.log
# Filter by level
loq app.log where level=error
# Filter HTTP logs by status
loq access.log where "status >= 400"# Search in messages
loq app.log where message contains "timeout"# Regex matching
loq app.log where path matches "^/api/v[0-9]+"# Boolean logic
loq app.log where level=error and message contains "database"
loq app.log where level=error or level=warn
# Aggregations
loq app.log count by level
loq access.log count by status
# Limit results
loq app.log where level=error limit 10
# Tail mode (like tail -f)
loq -f app.log where level=error
# Pipe support
cat logs/*.log | loq where level=error
kubectl logs pod/my-app | loq where level=error
# Output formats
loq app.log -o json where level=error
loq app.log -o csv where level=error
loq app.log -o table

Query Syntax

DSL Syntax

loq <file> [where <conditions>] [count [by <field>]] [limit <n>]

Operators:

  • =, != - equality
  • >, <, >=, <= - comparison
  • contains - substring match
  • matches - regex match

Boolean:

  • and, or, not
  • Parentheses for grouping: (level=error or level=warn) and status>=500

Time filters:

  • after yesterday
  • before today
  • between "10:00" and "11:00" today

SQL Syntax

loq "SELECT * FROM app.log WHERE level='error' LIMIT 10"
loq "SELECT level, COUNT(*) FROM app.log GROUP BY level"
loq "SELECT path, AVG(response_time) FROM access.log GROUP BY path"

Supported Log Formats

Auto-detected formats

FormatExample
JSON{"level":"info","message":"Server started"}
Apache/Nginx192.168.1.1 - - [20/Dec/2024:10:00:00 +0000] "GET /api HTTP/1.1" 200 1234
SyslogDec 20 12:34:56 myhost myapp[1234]: Message
CLF127.0.0.1 - - [10/Oct/2000:13:55:36 -0700] "GET / HTTP/1.0" 200 2326
Plain textFallback - entire line as message, no field extraction

Note: Plain text mode has limited query support. Use message contains "ERROR" instead of level=error. For full query capabilities, use structured logging (JSON) or create a custom format.

Custom formats

Create ~/.loqrc or ./loq.config.ts:

// loq.config.tsexportdefault{formats: [{name: 'my-app',detect: /^\[\d{4}-\d{2}-\d{2}/,parse: {pattern: /^\[(?<timestamp>[^\]]+)\](?<level>\w+):(?<message>.+)$/,fields: {timestamp: 'timestamp',level: 'level',message: 'message',},},},{name: 'nginx-json',detect: (line)=>{try{constobj=JSON.parse(line);return'request_uri'inobj;}catch{returnfalse;}},parse: (line)=>{constobj=JSON.parse(line);return{timestamp: obj.time_iso8601,level: obj.status>=400 ? 'error' : 'info',message: `${obj.request_method}${obj.request_uri}`,fields: obj,};},},],aliases: {errors: 'where level=error',slow: 'where response_time>1000',},};

Or use JSON (~/.loqrc):

{
"formats": [
{
"name": "bracketed",
"detect": "^\\[\\d{4}",
"parse": {
"pattern": "^\\[([^\\]]+)\\] (\\w+): (.+)$",
"fields": {
"timestamp": 1,
"level": 2,
"message": 3
}
}
}
]
}

CLI Options

Options:
-f, --follow Tail mode (like tail -f)
-o, --output Output format: color, table, json, csv
--format Force log format: json, apache, syslog, clf
-n, --limit Limit number of results
-h, --help Show help
-v, --version Show version

Development

# Install dependencies
bun install
# Run in development mode
bun run dev
# Run tests
bun test# Run tests with coverage
bun test --coverage
# Type check
bun run typecheck
# Build standalone binary
bun build src/index.ts --compile --outfile loq
# Generate documentation
bun run docs

Documentation

API documentation is automatically generated using TypeDoc and hosted on GitHub Pages.

# Generate docs locally
bun run docs
# Serve docs locally
bun run docs:serve
# Then open http://localhost:8080

Project Structure

loq/
├── src/
│ ├── index.ts # CLI entry point
│ ├── cli/
│ │ └── args.ts # Argument parsing
│ ├── config/
│ │ └── loader.ts # Config file loading
│ ├── parser/
│ │ ├── types.ts # Types & plugin system
│ │ ├── auto-detect.ts # Format detection
│ │ └── formats/ # Built-in parsers
│ ├── query/
│ │ ├── lexer.ts # Tokenizer
│ │ ├── parser.ts # Query parser
│ │ ├── ast.ts # AST types
│ │ └── executor.ts # Query execution
│ ├── output/
│ │ ├── formatter.ts # Output formatting
│ │ └── colors.ts # Terminal colors
│ └── utils/
│ └── time.ts # Time utilities
├── tests/ # Test files
├── .github/workflows/ # CI/CD
└── package.json

Adding Custom Formats

Option 1: Config file

Create ~/.loqrc with your format definitions (see above).

Option 2: Contribute a parser

  1. Create a parser in src/parser/formats/:
// src/parser/formats/myformat.tsimporttype{LogEntry,LogParser}from'../types';exportconstmyFormatParser: LogParser={name: 'myformat',detect(line: string): boolean{// Return true if this line matches your formatreturnline.startsWith('MYFORMAT:');},parse(line: string): LogEntry|null{// Parse the line and return a LogEntryconstmatch=line.match(/^MYFORMAT:\[(.+?)\](\w+)-(.+)$/);if(!match)returnnull;return{raw: line,timestamp: match[1],level: match[2],message: match[3],fields: {/* any additional fields */},};},};
  1. Register in src/parser/auto-detect.ts:
import{myFormatParser}from'./formats/myformat';constbuiltinParsers: LogParser[]=[myFormatParser,// Add herejsonParser,// ...];
  1. Add tests in tests/parsers/myformat.test.ts

  2. Submit a PR!

License

MIT

Contributing

Contributions welcome! See CONTRIBUTORS.md for guidelines.

Releases

Packages

Contributors

Languages