klogstream

klogstream is a Go library that implements Kubernetes log streaming with multi-pod filtering, JSON formatting, and error handling. The library extends standard Kubernetes clients by adding regex-based filtering and concurrent log processing. It provides a handler interface for custom log processing, automatic reconnection mechanisms, and built-in support for multiline log assembly. Developers can filter logs by pod names, namespaces, and labels while streaming from multiple containers simultaneously.

Features

• Concurrent log streaming across multiple pods/containers using goroutines
• Regex-based filtering for pod/container names
• Namespace and label-based log filtering
• Multiline log reassembly (e.g., Java stack traces)
• Flexible log formatting with JSON and custom formats
• Pluggable log handler system
• Automatic reconnection with exponential backoff
• Direct Kubernetes clientset injection support for testing

Installation

go get github.com/archsyscall/klogstream/pkg/klogstream

Log Handler Interface

klogstream provides a LogHandler interface for flexible log processing:

typeLogHandlerinterface {
// OnLog is called whenever a new log message arrivesOnLog(messageLogMessage)
// OnError is called when an error occurs during streamingOnError(errerror)
// OnEnd is called when the streaming endsOnEnd()
}

LogMessage structure:

typeLogMessagestruct {
Timestamp time.TimePodNamestringContainerNamestringMessagestringLabelsmap[string]string
}

Quick Start

package main
import (
"context""fmt""time"// Other imports..."github.com/archsyscall/klogstream/pkg/klogstream"
)
funcmain() {
ctx, cancel:=context.WithCancel(context.Background())
defercancel()
// Set up context and signal handling...// Create a streamer using the builder pattern - this is the main part!streamer, err:=klogstream.NewBuilder().
WithNamespace("default"). // Stream logs from the default namespaceWithPodRegex("my-app.*"). // Stream logs from pods matching regexWithContainerRegex(".*"). // Stream logs from all containersWithHandler(&ConsoleHandler{}). // Use a custom log handlerBuild()
iferr!=nil {
// Error handling...
}
// Start streaming logsiferr:=streamer.Start(ctx); err!=nil&&err!=context.Canceled {
// Error handling...
}
}
// ConsoleHandler is a simple handler that prints logs to the consoletypeConsoleHandlerstruct{}
func (h*ConsoleHandler) OnLog(message klogstream.LogMessage) {
fmt.Printf("[%s] %s/%s: %s\n", message.Timestamp.Format(time.RFC3339),
message.PodName,
message.ContainerName,
message.Message)
}
// OnError and OnEnd implementations...

Development Status

This project is under active development and not yet production-ready. APIs may change without notice.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

MIT