Skip to content
BranchesTags

Repository files navigation

Latch

File attachments for Crystal. Cache, promote, process, and serve uploads with pluggable storage, metadata extraction, and file variant generation.

  • Two-stage uploads. Cache first, promote later for safer form handling.
  • File processing. Constrain originals, create variants, run in parallel.
  • Avram integration. Attach files to models with a single macro.
  • Pluggable storage. FileSystem, S3, and Memory out of the box.
  • Metadata extraction. Filename, MIME type, size, and image dimensions.
  • Framework-agnostic. Built-in Lucky support, adaptable to Kemal or any other Crystal framework.

The name is short for Lucky Attachment. While originally created for Lucky, Latch can be used with any Crystal framework.

CIGitHub tag

Table of contents

Quick start

Set up your uploader:

# src/uploaders/avatar_uploader.crstructAvatarUploaderincludeLatch::UploaderstructVersionsProcessorincludeLatch::Processor::Magick
original resize:"2000x2000>"
variant thumb, resize:"200x200", crop:"200x200+0+0", gravity:"center"end
extract dimensions, using:Latch::Extractor::DimensionsFromMagick
process versions, using:VersionsProcessorend# src/models/user.crclassUser < BaseModelincludeLatch::Avram::Model
table do
attach avatar : AvatarUploader::StoredFile?
endend# src/operations/save_user.crclassUser::SaveOperation < User::BaseOperation
attach avatar, process:trueend

Upload a file:

user =User::SaveOperation.create!(avatar_file: uploaded_file)
user.avatar.url # => "/uploads/user/1/avatar/a1b2c3d4.jpg"
user.avatar.versions_thumb.url # => "/uploads/user/1/avatar/a1b2c3d4/versions_thumb.jpg"
user.avatar.width # => 2000

Installation

  1. Add the dependency to your shard.yml:

    dependencies:
latch:
github: wout/latch

  2. Run shards install

  3. Require Latch with your framework integration:

    require"latch"require"latch/lucky/avram"# Lucky + Avram

    Other combinations:

    require"latch/lucky/uploaded_file"# Lucky without Avramrequire"latch/avram/model"# Avram without Lucky

Configuration

Latch.configure do |settings|
settings.storages["cache"] =Latch::Storage::FileSystem.new(
directory:"uploads", prefix:"cache"
)
settings.storages["store"] =Latch::Storage::FileSystem.new(
directory:"uploads"
)
settings.path_prefix =":model/:id/:attachment"end

For tests, use the in-memory backend:

Latch.configure do |settings|
settings.storages["cache"] =Latch::Storage::Memory.new
settings.storages["store"] =Latch::Storage::Memory.new
end

Uploaders

An uploader defines how files are stored and what metadata is extracted.

structImageUploaderincludeLatch::Uploaderend

Every uploader automatically extracts filename, mime_type, and size. These are available as methods on the returned StoredFile.

Uploading files

# Cache (temporary storage, e.g. between form submissions)
cached =ImageUploader.cache(uploaded_file)
# Promote from cache to permanent storage
stored =ImageUploader.promote(cached)
# Or store directly
stored =ImageUploader.store(uploaded_file)

Custom upload locations

structImageUploaderincludeLatch::Uploaderdefgenerate_location(uploaded_file, metadata, **options) : String
date =Time.utc.to_s("%Y/%m/%d")
File.join("images", date, super)
endend

Custom storage keys

By default, uploaders use "cache" and "store". Override with the storages macro:

structImageUploaderincludeLatch::Uploader
storages cache:"tmp", store:"offsite"end

Avram integration

Latch integrates with Avram for model-level file attachments with automatic caching, promotion, and cleanup.

Model setup

Use the attach macro inside a table block. The column should be a jsonb type in your migration:

classUser < BaseModelincludeLatch::Avram::Model
table do
attach avatar : ImageUploader::StoredFile?
endend
# In your migration
add avatar : JSON::Any?

SaveOperation setup

The attach macro registers a file attribute and lifecycle hooks:

classUser::SaveOperation < User::BaseOperation
attach avatar
end

The file attribute defaults to avatar_file. A custom name can be provided:

attach avatar, field_name:"avatar_upload"

For nilable attachments, a delete_avatar attribute is added automatically:

User::SaveOperation.update!(user, delete_avatar:true)

Processing after upload

To run processors after promotion, pass process: true:

attach avatar, process:true

For background processing, pass a block instead. For example, using Mel:

attach avatar do |record|
User::AvatarProcessingJob.run(record_id: record.id)
end

The background job:

structUser::AvatarProcessingJobincludeMel::Job::Nowdefinitialize(@record_id : Int64)
enddefrun
user =UserQuery.find(@record_id)
# For nilable attachments:
user.avatar.try(&.process)
# Otherwise simply:
user.avatar.process
endend

Validating attachments

Validate file size and MIME type in a before_save block:

classUser::SaveOperation < User::BaseOperation
attach avatar
before_save do
validate_file_size_of avatar_file, max:5_000_000
validate_file_mime_type_of avatar_file, in:%w[image/png image/jpeg image/webp]endend

MIME types can also be validated with a pattern:

validate_file_mime_type_of avatar_file, with:/image\/.*/

Upload lifecycle

  1. Before save the file is cached to temporary storage
  2. After commit the cached file is promoted to permanent storage
  3. After promotion processors run (if configured)
  4. On update the old file is replaced
  5. On delete the attached file is removed

Processors

Processors transform uploaded files into variants and can optionally modify the original. Processing is decoupled from uploading, runs in parallel for variants, and can be triggered inline or in a background job.

ImageMagick processor

The built-in Latch::Processor::Magick module wraps magick convert. Define variants with compile-time validated options:

structAvatarProcessorincludeLatch::Processor::Magick
original resize:"2000x2000>"
variant large, resize:"800x800"
variant thumb, resize:"200x200", crop:"200x200+0+0", gravity:"center"end

Typos are caught at compile time. All built-in options are optional, but custom processors can declare required options.

Available options
  • auto_orient: true (Bool) → fix orientation from EXIF data
  • background: "white" (String) → background color, e.g. "white", "transparent"
  • colorspace: "sRGB" (String) → convert color model, e.g. "sRGB", "Gray"
  • crop: "200x200+10+10" (String) → cut a region
  • density: 72 (Int32 | String) → resolution in DPI, e.g. 72 or "72x72"
  • extent: "800x600" (String) → pad/canvas size
  • flatten: true (Bool) → merge layers into one
  • gaussian_blur: "0x3" (String) → blur effect
  • gravity: "center" (String) → anchor point, e.g. "center", "north"
  • interlace: "Plane" (String) → progressive rendering
  • quality: 85 (Int32 | String) → compression quality
  • resize: "800x600" (String) → scale to fit, e.g. "800x600", "200x200>"
  • rotate: 90 (Int32 | String) → rotate by degrees
  • sampling_factor: "4:2:0" (String) → chroma subsampling
  • sharpen: "0x1" (String) → sharpen
  • strip: true (Bool) → remove all metadata and profiles
  • thumbnail: "200x200" (String) → like resize but strips profiles for smaller files

Important

Requires ImageMagick to be installed.

FFmpeg processor

The built-in Latch::Processor::FFmpeg module wraps ffmpeg for video and audio transformations:

structVideoProcessorincludeLatch::Processor::FFmpeg
original video_codec:"libx264", crf:"23", preset:"fast"
variant preview, scale:"640:-1", video_codec:"libx264", crf:"28"
variant thumb, frames:"1", format:"image2", scale:"320:-1"end
Available options
  • audio_bitrate: "128k" (String) → audio bitrate
  • audio_codec: "aac" (String) → audio codec, e.g. "aac", "libopus"
  • audio_filter: "volume=0.5" (String) → custom audio filter
  • crf: 23 (Int32 | String) → constant rate factor (quality)
  • duration: 10 (Int32 | String) → max duration, e.g. 10 or "00:01:30"
  • format: "webm" (String) → output format, e.g. "mp4", "webm", "image2"
  • frame_rate: 30 (Int32 | String) → output frame rate
  • frames: 1 (Int32 | String) → number of frames to output (for thumbnails)
  • no_audio: true (Bool) → strip audio track
  • preset: "fast" (String) → encoding speed/quality, e.g. "fast", "slow"
  • scale: "1280:720" (String) → resize, e.g. "1280:720", "-1:480"
  • start: "00:00:05" (String) → start time
  • video_bitrate: "1M" (String) → video bitrate, e.g. "1M", "500k"
  • video_codec: "libx264" (String) → video codec, e.g. "libx264", "libx265"
  • video_filter: "transpose=1" (String) → custom video filter

Important

Requires FFmpeg to be installed.

Vips processor

The built-in Latch::Processor::Vips module uses vipsthumbnail for resize operations and vips copy for metadata/format changes:

structAvatarProcessorincludeLatch::Processor::Vips
original resize:"2000x2000>", strip:true
variant large, resize:"800x800"
variant thumb, resize:"200x200", crop:true, quality:85end
Available options
  • auto_orient: true (Bool) → fix orientation from EXIF data
  • crop: true (Bool) → crop to fill instead of shrink-to-fit
  • format: "webp" (String) → output format, e.g. "webp", "png"
  • linear: true (Bool) → process in linear color space (higher quality)
  • quality: 85 (Int32 | String) → JPEG/WebP compression quality (1-100)
  • resize: "200x200" (String) → bounding box, e.g. "200x200", "800x", "2000x2000>"
  • smartcrop: "attention" (String) → smart crop mode, e.g. "attention", "entropy"
  • strip: true (Bool) → remove all metadata and profiles

Important

Requires libvips to be installed.

Processing the original

The original macro processes the uploaded file in place without creating a copy. Variants are always processed first so they use the maximum available quality.

structAvatarProcessorincludeLatch::Processor::Magick
original resize:"2000x2000>"end

Note

If original is not declared, the uploaded file remains as-is.

Registering and running processors

Register a processor on an uploader with the process macro:

structAvatarUploaderincludeLatch::Uploader
process versions, using:AvatarProcessorend

Processing runs separately from uploading:

stored =AvatarUploader.store(uploaded_file)
stored.process

Variant accessors are generated on StoredFile, prefixed with the processor name:

stored.versions_large.url # => "/uploads/abc123/versions_large.jpg"
stored.versions_thumb.url # => "/uploads/abc123/versions_thumb.jpg"
stored.versions_thumb.exists? # => true

Nilable accessors are also available, returning nil if the variant hasn't been processed yet:

# Returns nil before processing, the StoredFile after
stored.versions_thumb?.try(&.url)
# Useful in templatesif thumb = user.avatar.versions_thumb?
img src: thumb.url
end# The non-nilable accessor always returns a StoredFile,# even if the file doesn't exist in storage yet
user.avatar.versions_thumb.url

Error handling

Processing errors are wrapped in Latch::ProcessingError with the variant and processor name. The original exception is available via cause:

begin
stored.process
rescue ex : Latch::ProcessingError
ex.message # => "Failed to process variant 'thumb' for processor 'versions'"
ex.cause # => the underlying Latch::CliToolNotFound, IO::Error, etc.end

Custom processors

Create a module with @[Latch::VariantOptions(...)] and use the process macro to define per-variant logic. The block should return an IO:

@[Latch::VariantOptions(quality:Int32)]
moduleMyQualityProcessorincludeLatch::Processor
process do
do_your_thing_with_the(tempfile, variant_options) # return an IOendendstructQualityProcessorincludeMyQualityProcessor
variant high, quality:95
variant low, quality:30end

The block runs with stored_file, storage, name, tempfile, variant_name, and variant_options in scope.

For full control, bypass the process macro and generate self.process directly with an included macro:

@[Latch::VariantOptions(quality:Int32)]
moduleMyQualityProcessorincludeLatch::Processormacroincludeddefself.process(
stored_file : Latch::StoredFile,
storage : Latch::Storage,
name : String,
**options,
) : Nil
stored_file.download do |tempfile|
VARIANTS.each do |variant_name, variant_options|
location = stored_file.variant_location("\#{name}_\#{variant_name}")
io = do_your_thing_with_the(tempfile, variant_options)
storage.upload(io, location)
endendendendend

Storage backends

FileSystem

Latch::Storage::FileSystem.new(
directory:"uploads",
prefix:"cache", # optional subdirectoryclean:true, # clean empty parent dirs on delete (default)permissions:File::Permissions.new(0o644),
directory_permissions:File::Permissions.new(0o755)
)

S3

Works with AWS S3 and any S3-compatible service (RustFS, Tigris, Cloudflare R2):

Note

RustFS is the open-source successor to MinIO, whose repository has been archived.

Latch::Storage::S3.new(
bucket:"my-bucket",
region:"eu-west-1",
access_key_id:ENV["AWS_ACCESS_KEY_ID"],
secret_access_key:ENV["AWS_SECRET_ACCESS_KEY"],
endpoint:"http://localhost:9000", # optional, for S3-compatible servicesprefix:"uploads", # optional key prefixpublic:false, # set to true for public-read ACLupload_options: { # optional default headers"Cache-Control" => "max-age=31536000",
}
)

Note

S3 storage requires the awscr-s3 shard. Add it to your shard.yml:

dependencies:
awscr-s3:
github: taylorfinnell/awscr-s3

Presigned URLs are supported:

stored_file.url(expires_in:1.hour)

Memory

In-memory storage for testing:

storage =Latch::Storage::Memory.new(
base_url:"https://cdn.example.com"# optional
)
storage.clear! # reset between tests

Custom storage

Inherit from Latch::Storage and implement five methods:

classMyStorage < Latch::Storagedefupload(io : IO, id : String, **options) : Nilenddefopen(id : String, **options) : IOenddefexists?(id : String) : Boolenddefurl(id : String, **options) : Stringenddefdelete(id : String) : Nilendend

Metadata extractors

Built-in extractors

Every uploader registers three extractors by default:

  • FilenameFromIO (filename) → Original filename from the upload
  • MimeFromIO (mime_type) → MIME type from the Content-Type header
  • SizeFromIO (size) → File size in bytes

Additional extractors can be registered with the extract macro:

  • MimeFromExtension (mime_type) → MIME type from the file extension
  • MimeFromFile (mime_type) → requires file CLI tool
  • DimensionsFromMagick (width, height) → requires magick or identify
  • DimensionsFromVips (width, height) → requires vipsheader
structImageUploaderincludeLatch::Uploader
extract mime_type, using:Latch::Extractor::MimeFromFile
extract dimensions, using:Latch::Extractor::DimensionsFromMagickend

Custom extractors

Create a struct that includes Latch::Extractor:

structPageCountExtractorincludeLatch::Extractordefextract(uploaded_file, metadata, **options) : Int32?
count_pages(uploaded_file.tempfile)
endend

Register it and access the value on the stored file:

structPdfUploaderincludeLatch::Uploader
extract pages, using:PageCountExtractorend
stored =PdfUploader.store(uploaded_file)
stored.pages # => 24

An extractor can also write multiple values to metadata directly. Use the @[Latch::MetadataMethods] annotation to generate typed accessor methods for each value:

@[Latch::MetadataMethods(width : Int32, height : Int32)]
structDimensionsExtractorincludeLatch::Extractordefextract(uploaded_file, metadata, **options) : Nil
metadata["width"] =800
metadata["height"] =600endend
stored =ImageUploader.store(uploaded_file)
stored.width # => 800
stored.height # => 600

Working with stored files

StoredFile objects are JSON-serializable and provide convenience methods for accessing, downloading, and streaming files:

stored.url # storage URL
stored.exists? # check existence
stored.extension # file extension
stored.delete # remove from storage
stored.open { |io| io.gets_to_end } # read content
stored.download { |tempfile| tempfile.path } # download to tempfile
stored.stream(response.output) # stream to IO

Each uploader generates its own StoredFile subclass, which can be extended with custom methods:

structImageUploaderincludeLatch::Uploader# This extractor extracts `width` and `height` and creates methods for them
extract dimensions, using:Latch::Extractor::DimensionsFromMagickclassStoredFiledefratio : Float64
width.to_f / height
endendend
stored =ImageUploader.store(uploaded_file)
stored.ratio # => 1.5

StoredFile serializes to a format compatible with Shrine. Values from registered extractors are also stored in the metadata object:

{
"id": "uploads/a1b2c3d4.jpg",
"storage": "store",
"metadata": {
"filename": "photo.jpg",
"size": 102400,
"mime_type": "image/jpeg",
"width": 2000,
"height": 1333
}
}

Other frameworks

Latch works with any Crystal framework. Implement the Latch::UploadedFile module on your framework's upload class:

moduleLatch::UploadedFileabstractdeftempfile : Fileabstractdeffilename : String# Optional overrides with sensible defaults:# def path : String -> tempfile.path# def content_type : String? -> nil# def size : UInt64 -> tempfile.sizeend

Kemal example

require"kemal"require"latch"structKemal::FileUploadincludeLatch::UploadedFiledeffilename : String@filename||"upload"enddefcontent_type : String?
headers["Content-Type"]?
endend
post "/upload"do |env|
upload = env.params.files["image"]
stored =ImageUploader.store(upload)
stored.url
end

API docs

Online API documentation is available at wout.github.io/latch.

Contributing

  1. Fork it (https://github.com/wout/latch/fork)
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create a new Pull Request

Contributors

  • Wout - creator and maintainer

About

File attachment toolkit for Crystal apps with pluggable storage backends, metadata extraction, file processing, and a two-stage upload workflow.

Topics

avramcrystalffmpegimagemagickkemalluckyframeworkmetadatarustfss3-storageuploaduploadervips

Resources

Readme
MIT license
Activity

Stars

8 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Contributors

Languages