pythonlogs

High-performance Python logging library with file rotation and optimized caching for better performance

Table of Contents

• Features
• Installation
• Logger Types
  • Basic Logger
  • Size Rotating Logger
  • Timed Rotating Logger
• Context Manager Support
• Using With Multiple Log Levels and Files
• Environment Variables
  • Settings Cache Management
• Flexible Configuration Options
• Development and Testing
  • Create DEV Environment and Running Tests
  • Update DEV Environment Packages
  • Building Wheel
  • Optionals
• License
• Support

Features

✨ Factory Pattern - Easy logger creation with centralized configuration
🚀 High Performance - Optimized caching for 90%+ performance improvements
🔄 File Rotation - Automatic rotation by size or time with compression
🎯 Type Safety - Enum-based configuration with IDE support
⚙️ Flexible Configuration - Environment variables, direct parameters, or defaults
📍 Location Tracking - Optional filename and line number in logs
🌍 Timezone Support - Full timezone handling including localtime and UTC
💾 Memory Efficient - Logger registry and settings caching
🔒 Context Manager Support - Automatic resource cleanup and exception safety
🧵 Thread Safe - Concurrent access protection for all operations
🔧 Resource Management - Automatic handler cleanup and memory leak prevention

Installation

pip install pythonlogs

Logger Types

Tip: All logger types support both string values (e.g., level="debug") and type-safe enums (e.g., level=LogLevel.DEBUG).
See Flexible Configuration Options for all available enums.

Basic Logger

Console-only logging without file output. Perfect for development and simple applications.

Usage

frompythonlogsimportBasicLoglogger=BasicLog(
name="my_app",
level="debug", # "debug", "info", "warning", "error", "critical"timezone="America/Sao_Paulo",
showlocation=False,
)
logger.warning("This is a warning example")

Example Output

[2024-10-08T19:08:56.918-0300]:[WARNING]:[my_app]:This is a warning example

Size Rotating Logger

File-based logging with automatic rotation when files reach a specified size. Rotated files are compressed as .gz.

• Rotation: Based on file size (maxmbytes parameter)
• Naming: Rotated logs have sequence numbers: app.log_1.gz, app.log_2.gz
• Cleanup: Old logs deleted based on daystokeep (default: 30 days)

Usage

frompythonlogsimportSizeRotatingLoglogger=SizeRotatingLog(
name="my_app",
level="debug", # "debug", "info", "warning", "error", "critical"directory="/app/logs",
filenames=["main.log", "app1.log"],
maxmbytes=5,
daystokeep=7,
timezone="America/Chicago",
streamhandler=True,
showlocation=False,
)
logger.warning("This is a warning example")

Example Output

[2024-10-08T19:08:56.918-0500]:[WARNING]:[my_app]:This is a warning example

Timed Rotating Logger

File-based logging with automatic rotation based on time intervals. Rotated files are compressed as .gz.

• Rotation: Based on time (when parameter, defaults to midnight)
• Naming: Rotated logs have date suffix: app_20240816.log.gz
• Cleanup: Old logs deleted based on daystokeep (default: 30 days)
• Supported Intervals: midnight, hourly, daily, W0-W6 (weekdays, 0=Monday)

Usage

frompythonlogsimportTimedRotatingLoglogger=TimedRotatingLog(
name="my_app",
level="debug", # "debug", "info", "warning", "error", "critical"directory="/app/logs",
filenames=["main.log", "app2.log"],
when="midnight", # "midnight", "H", "D", "W0"-"W6"daystokeep=7,
timezone="UTC",
streamhandler=True,
showlocation=False,
)
logger.warning("This is a warning example")

Example Output

[2024-10-08T19:08:56.918-0000]:[WARNING]:[my_app]:This is a warning example

Context Manager Support

All logger types support context managers for automatic resource cleanup and exception safety.

Usage Examples

frompythonlogsimportLogLevelfrompythonlogs.basic_logimportBasicLogfrompythonlogs.size_rotatingimportSizeRotatingLogfrompythonlogs.timed_rotatingimportTimedRotatingLog# Automatic cleanup with context managerswithBasicLog(name="app", level=LogLevel.INFO) aslogger:
logger.info("This is automatically cleaned up")
# Handlers are automatically closed on exitwithSizeRotatingLog(name="app", directory="/logs", filenames=["app.log"]) aslogger:
logger.info("File handlers cleaned up automatically")
# File handlers closed and resources freed# Exception safety - cleanup happens even if exceptions occurtry:
withTimedRotatingLog(name="app", directory="/logs") aslogger:
logger.error("Error occurred")
raiseValueError("Something went wrong")
exceptValueError:
pass# Logger was still cleaned up properly

Using With Multiple Log Levels and Files

frompythonlogsimportSizeRotatingLog, TimedRotatingLog, LogLevel, RotateWhen# Application loggerapp_logger=SizeRotatingLog(
name="production_app",
directory="/var/log/myapp",
filenames=["app.log"],
maxmbytes=50, # 50MB filesdaystokeep=30, # Keep 30 dayslevel=LogLevel.INFO,
streamhandler=True, # Also log to consoleshowlocation=True, # Show file:function:linetimezone="UTC",
)
# Error logger with longer retentionerror_logger=SizeRotatingLog(
name="production_errors",
directory="/var/log/myapp",
filenames=["errors.log"],
maxmbytes=10,
daystokeep=90, # Keep errors longerlevel=LogLevel.ERROR,
streamhandler=False,
)
# Audit logger with daily rotationaudit_logger=TimedRotatingLog(
name="audit_log", directory="/var/log/myapp", filenames=["audit.log"], when=RotateWhen.MIDNIGHT, level=LogLevel.INFO
)
# Use the loggersapp_logger.info("Application started")
error_logger.error("Database connection failed")
audit_logger.info("User admin logged in")

Env Variables (Optional)

The .env variables file can be used by leaving all options blank when calling the class.
If not specified inside the .env file, it will use the default value.
This is a good approach for production environments, since options can be changed easily.

frompythonlogsimportTimedRotatingLoglog=TimedRotatingLog()

LOG_LEVEL=DEBUG
LOG_TIMEZONE=UTC
LOG_ENCODING=UTF-8
LOG_APPNAME=app
LOG_FILENAME=app.log
LOG_DIRECTORY=/app/logs
LOG_DAYS_TO_KEEP=30
LOG_DATE_FORMAT=%Y-%m-%dT%H:%M:%S
LOG_STREAM_HANDLER=True
LOG_SHOW_LOCATION=False
LOG_MAX_LOGGERS=50
LOG_LOGGER_TTL_SECONDS=1800
# SizeRotatingLog
LOG_MAX_FILE_SIZE_MB=10
# TimedRotatingLog
LOG_ROTATE_WHEN=midnight
LOG_ROTATE_FILE_SUFIX="%Y%m%d"

Settings Cache Management

Use get_log_settings() to inspect current configuration and clear_settings_cache() to reload configuration from environment variables:

frompythonlogsimportget_log_settings, clear_settings_cache# Inspect current settingssettings=get_log_settings()
print(settings.level) # Current log levelprint(settings.timezone) # Current timezone# Clear cache and reload .env on next access (default)clear_settings_cache()
# Clear cache but keep current .env valuesclear_settings_cache(reload_env=False)

Flexible Configuration Options

You can use either enums (for type safety) or strings (for simplicity):

frompythonlogsimportLogLevel, RotateWhen# Option 1: Type-safe enums (recommended)LogLevel.DEBUG# "DEBUG"LogLevel.INFO# "INFO"LogLevel.WARNING# "WARNING"LogLevel.ERROR# "ERROR"LogLevel.CRITICAL# "CRITICAL"# Option 2: String values (case-insensitive)"debug"# Same as LogLevel.DEBUG"info"# Same as LogLevel.INFO"warning"# Same as LogLevel.WARNING"warn"# Same as LogLevel.WARN (alias)"error"# Same as LogLevel.ERROR"critical"# Same as LogLevel.CRITICAL"crit"# Same as LogLevel.CRIT (alias)# Also supports: "DEBUG", "Info", "Warning", etc.# RotateWhen valuesRotateWhen.MIDNIGHT# "midnight"RotateWhen.HOURLY# "H"RotateWhen.DAILY# "D"RotateWhen.MONDAY# "W0"# ... through SUNDAY # "W6"# String equivalents: "midnight", "H", "D", "W0"-"W6"

Development and Testing

Must have UV installed.

Create DEV Environment and Running Tests

uv sync --all-extras --all-groups
poe tests

Update DEV Environment Packages

This will update all packages dependencies

poe updatedev

Building Wheel

This will update all packages, run linter, both unit and integration tests and finally build the wheel

poe build

Optionals

Create a cprofile.prof file from unit tests

poe profile

License

Released under the MIT License

Support

If you find this project helpful, consider supporting development.