💮 Collectivist

AI-powered curator for intentional collectors.
[Work in Progress] Exploring how to transform semantically coherent hoards into beautiful living documentation substrates with LLM-powered organization and curation.

🌱 Planned Features

• Domain-specific intelligence: Git metadata, ID3 tags, EXIF data, citations, and semantic understanding
• LLM-powered analysis: Bespoke categorization and rich description generation (perfect context for agents)
• Self-healing documentation: Collection indices that adapt to filesystem changes using semantic diffing and persisted structural memory
• Flexible configuration: All OpenAI compatible providers supported - local (LM Studio, Ollama), cloud (OpenRouter, Pollinations), or custom endpoints
• Multi-format outputs: Markdown documentation, interactive HTML index, JSON export
• Collection types: Repositories, Obsidian vaults, documents, media files, research papers, creative projects, datasets

🚧 Current Status

Repository collections are partially implemented with basic indexing and description generation. The broader vision of multi-format collection support, plugin architecture, and distributed coordination is in active development.

See .kiro/specs/collectivist/ for detailed requirements and design documentation.

Installation

🧺 Experimental Build (Repository Collections Only)

Clone repo to try the current repository indexing implementation.

Note: Only repository collections are currently functional. Other collection types are planned.

---

🎮 Full Installation (Coming Eventually)

# Planned: pip install collectivist# collectivist init ~/my-collection --standard

The complete multi-collection system with interactive UI, curation loops, and automation is in development.

🤖 Configuration

• Current implementation works with LM Studio and other OpenAI-compatible endpoints. Configuration is evolving as the system develops.

⚠️ Development Warning

This is experimental software. Expect breaking changes and incomplete features as the architecture evolves.

Collection Schema (Planned)

Collection Types (In Development)

• Repositories: Git-aware scanning, commit summaries, category taxonomy (partially implemented)
• Research: Citation extraction, topic clustering, reading status (planned)
• Media: Timeline organization, mood/genre inference, EXIF metadata (planned)
• Creative: Version tracking, asset linking, format intelligence (planned)
• Datasets: Schema inference, sample previews, data provenance (planned)
• Documents: Text extraction, metadata parsing, content analysis (planned)

The collection.yaml schema is evolving as different collection types are implemented.

Repository Collection Example:

collection_type: repositoriesstatus: "✓"# Status glyph for up-to-date reposname: my-repospath: /path/to/reposcategories:
- phext_hyperdimensional # Multi-dimensional text systems
- ai_llm_agents # AI agents and LLM infrastructure
- terminal_ui # Terminal UI frameworks and components
- creative_aesthetic # Art, music, visualization tools
- dev_tools # Development utilities and build tools
- esoteric_experimental # Experimental and occult systems
- system_infrastructure # System-level networking tools
- utilities_misc # General utilities and miscellaneousexclude_hidden: truescanner_config:
always_pull: {} # Repos to auto-pull: {repo_name: true}fetch_timeout: 30# Git fetch timeout in seconds

Media Collection Example:

collection_type: mediastatus: "🎵"# Status glyph for media collectionsname: my-mediapath: /path/to/mediacategories:
- music_albums # Full album collections
- music_singles # Individual tracks
- video_movies # Movie files
- video_series # TV series and episodes
- images_photos # Photography collections
- images_artwork # Digital art and graphicsexclude_hidden: truescanner_config:
extract_metadata: true # Extract ID3, EXIF, etc.supported_formats: # File extensions to scanaudio: [".mp3", ".flac", ".wav", ".m4a"]video: [".mp4", ".mkv", ".avi", ".mov"]image: [".jpg", ".png", ".gif", ".webp"]

🔌 Plugin Architecture (In Development)

Planned Plugin System: Collectivist will include domain-specific plugins for different collection types. The plugin architecture is being designed to automatically select appropriate plugins based on collection type detection.

Development Status:

• repositories - Basic implementation exists (functional but evolving)
• obsidian vaults - Planned
• media libraries - Planned
• document libraries - Planned

Plugin Discovery: The automatic collection type detection and plugin routing system is in development.

Custom Plugins: Plugin interface is being designed - see specs for current architecture plans.

🔗 Workflow Stages

Stage 1: Indexer (deterministic)

Initialize for a specific type of library or get a custom schema.

Discovers items and note metadata using domain-specific plugins.

Stage 2: Describer (LLM)

Generates LLM descriptions and collection overview with concurrent workers.

Contributing

Issues and PRs welcome! This is an exploration of collection-first AI curation.

License

Private research tool - not for distribution.

---

Infrastructure as palimpsest, data as compiled breath 🜍