Skip to content

Repository files navigation

Marquery

CIVersion

A compile-time markdown file query engine for Crystal. Drop your markdown files in a directory, define a query class, and get a type-safe, filterable, pagination-ready collection baked right into your binary. No database, no runtime parsing.

  • Low runtime overhead. Markdown is parsed at compile time and embedded in the binary.
  • Frontmatter support. YAML frontmatter for metadata like title, description, tags, and custom fields.
  • Chainable queries. Filter, sort, reverse, shuffle, navigate, and more.
  • Built-in HTML rendering. Cmark GFM out of the box, or bring your own renderer.
  • Framework-friendly. Works with Lucky, Kemal, or any Crystal app.

Note

The original repository is hosted at Codeberg. The GitHub repo is just a mirror.

Tip

Looking for the Ruby version? See marquery.

Quick start

require"marquery"classBlog::PostQueryincludeMarquery::Queryend
query =Blog::PostQuery.new
query.filter(&.active?).sort_by(&.title).all
query.find("my-first-post").to_html

Add your markdown files to marquery/blog_post/ and you're good to go:

marquery/blog_post/20260320_my_first_post.md
marquery/blog_post/20260323_another_post.md

That's it.

Installation

  1. Add the dependency to your shard.yml:

    dependencies:
    marquery:
    codeberg: fluck/marquery.cr
  2. Run shards install

Markdown files

Entries are markdown files with a date-prefixed filename:

marquery/blog_post/20260320_first_post.md

The date (YYYYMMDD) and name are extracted from the filename. The name becomes the slug (hyphenated) and title (humanized). An optional YAML frontmatter block can override these and add custom fields:

---title: The very first postdescription: >- This is the first post.active: truetags:
- crystal
- web---
The body of the post goes here.

Note

Supported frontmatter types: Bool, Int32, Float64, String, Time, and Array(String).

Models

By default, entries are deserialized into Marquery::Entry. For custom fields, define a struct that includes Marquery::Model:

structBlog::PostincludeMarquery::Modelgetter tags : Array(String) = [] of Stringend

The base fields provided by Marquery::Model are:

  • slug (String)
  • title (String)
  • description (String?)
  • content (String)
  • date (Time)
  • active? (Bool, defaults to true).

The original markdown file path is also available if you add it to your model:

structBlog::PostincludeMarquery::Modelgetter source : Stringend

Queries

Create a query class and include Marquery::Query:

classBlog::PostQueryincludeMarquery::Query
model Blog::Postend

The data directory is derived from the class name (without the Query suffix):

  • Blog::PostQuery -> marquery/blog_post/*.md
  • ItemQuery -> marquery/item/*.md
  • News::ArticleQuery -> marquery/news_article/*.md

Querying

query =Blog::PostQuery.new
query.all # all entries
query.first # first entry (raises if empty)
query.first? # first entry or nil
query.last # last entry (raises if empty)
query.last? # last entry or nil
query.find("first-post") # find by slug (raises if not found)
query.find?("first-post") # find by slug or nil
query.previous(post) # previous entry or nil
query.next(post) # next entry or nil

Error handling

Query methods raise typed exceptions that you can rescue from:

beginBlog::PostQuery.new.find("nonexistent")
rescue ex : Marquery::EntryNotFoundputs ex.message # => "Entry not found: nonexistent"end

All library exceptions inherit from Marquery::Error, so you can also catch them all:

rescue ex : Marquery::Error# ...end

Filtering, sorting, and chaining

filter, sort_by, reverse, and shuffle all return self, so they chain naturally:

Blog::PostQuery.new
.filter(&.active?)
.filter { |post| post.date >=1.month.ago }
.filter(&.tags.includes?("crystal"))
.sort_by(&.title)
.all

Note

Since filter is just a thin wrapper around Array#select, you can express any condition.

Default sort order

Entries are sorted by date descending by default. Use order_by to change the default:

classBlog::PostQueryincludeMarquery::Query
order_by date, Marquery::Order::ASCend

Or sort by a different field:

classBlog::PostQueryincludeMarquery::Query
order_by title
end

Pagination

The all method returns a plain Array, so it works with any array-based pagination solution.

Lucky has built-in array pagination with paginate_array:

classBlog::Index < BrowserAction
get "/blog"do
pages, posts = paginate_array(Blog::PostQuery.new.all)
html Blog::IndexPage, posts: posts, pages: pages
endend

For Kemal and other frameworks, pager is a good option:

require"pager/collections/array"
get "/blog"do |env|
current_page = env.params.query["page"]?.try(&.to_i) ||0
posts =Blog::PostQuery.new.all.paginate(current_page, 10)
# ...end

Index pages

Each query can have an index page with metadata for the collection itself (e.g. page title, meta description). Create a _index.md file in the data directory:

marquery/blog_post/_index.md
marquery/blog_post/_index/og-image.png
---title: Blogdescription: Thoughts on Crystal and web development.---
Welcome to the blog.

Access it via the query class:

Blog::PostQuery.index.title # => "Blog"Blog::PostQuery.index.description # => "Thoughts on Crystal and web development."Blog::PostQuery.index.to_html # => "<p>Welcome to the blog.</p>\n"

If no _index.md exists, the index returns empty defaults.

Custom index model

For custom fields, create a struct that includes Marquery::Collection and assign it with the index macro:

structBlog::PostIndexincludeMarquery::Collectiongetter subtitle : String?
endclassBlog::PostQueryincludeMarquery::Query
index Blog::PostIndexend

HTML rendering

Model instances have a to_html method that renders the content field to HTML using Cmark (GitHub Flavored Markdown) by default:

post =Blog::PostQuery.new.find("first-post")
post.to_html # => "<p>The body of the post goes here.</p>\n"

Custom renderer

To use a different markdown renderer, create a struct that includes Marquery::MarkdownToHtml:

structMyRendererincludeMarquery::MarkdownToHtmldefmarkdown_to_html(content : String) : StringMyMarkdownLib.render(content)
endend

Then declare it on your model with to_html:

structBlog::PostincludeMarquery::Model
to_html MyRendererend

Pre-processing content

Marquery passes raw markdown through process_content before handing it to the renderer. The default implementation rewrites asset:KEY URI references to their resolved paths, so you can write asset references inline using ordinary markdown link and image syntax:

![Hero](asset:hero.jpg)
See the [spec sheet](asset:specs.pdf) for details.

These get rewritten to whatever asset("hero.jpg") and asset("specs.pdf") resolve to. Unknown keys raise Marquery::AssetNotFound, matching the behavior of the asset helper.

Plugging in a templating engine

Override process_content on your model to do more, such as running the content through a templating engine. The hook runs inside the model instance, so self, frontmatter fields, and asset(...) are all available.

For example, with Crinja:

require"crinja"structBlog::PostincludeMarquery::Modeldefprocess_content(raw : String) : StringCrinja.new.from_string(raw).render({"entry" => self})
endend

Then in your markdown:

---title: First post---# {{ entry.title }}![Hero]({{ entry.asset("hero.jpg") }})

Marquery deliberately stays out of the templating decision. Use Crinja, Liquid, ECR (compile-time only), or roll your own. Whatever process_content returns is what gets passed to the markdown renderer.

Markdown in pages and components

Include Marquery::MarkdownHelper in pages or components of your app to get a convenient markdown method:

classBlog::ShowPageincludeMarquery::MarkdownHelperdefcontent
div do# Accepts a String
markdown "**Something**"# Or an instance of Marquery::Model
markdown post
endendend

Note

In Lucky apps, the output is automatically wrapped in raw.

To use a custom renderer in pages, use the markdown_renderer macro:

classBlog::ShowPageincludeMarquery::MarkdownHelper
markdown_renderer MyRendererend

Assets

Place images and media files in a directory matching the markdown filename (without the .md extension):

marquery/blog_post/20260320_first_post.md
marquery/blog_post/20260320_first_post/hero.png
marquery/blog_post/20260320_first_post/diagram.svg

The assets are available on the entry at compile time:

post =Blog::PostQuery.new.find("first-post")
post.asset("hero.png") # => "/marquery/blog_post/20260320_first_post/hero.png"
post.asset?("missing") # => nil
post.assets # => {"hero.png" => "marquery/blog_post/...", ...}

Note

Supported file types: .avif, .gif, .jpeg, .jpg, .mp3, .mp4, .ogg, .pdf, .png, .svg, .webm, .webp. Dotfiles are ignored.

Serving assets

To serve asset files over HTTP, add Marquery::AssetHandler to your middleware:

require"marquery/asset_handler"# LuckyclassAppServer < Lucky::BaseAppServerdefmiddleware : Array(HTTP::Handler)
[
# ... other handlers ...Marquery::AssetHandler.new(Blog::PostQuery.dir, News::ArticleQuery.dir),
# ...
] of HTTP::Handlerendend# Kemal
add_handler Marquery::AssetHandler.new(Blog::PostQuery.dir)

The handler serves files from the configured directories and falls through for anything else.

Configuring the data directory

The default data directory is marquery/. To change it globally, annotate the Marquery module:

# e.g. config/marquery.cr
@[Marquery::Dir("data")]
moduleMarqueryend

Individual query classes can override the directory:

@[Marquery::Dir("db/content")]
classBlog::PostQueryincludeMarquery::Queryend

Multi-language content

For multilingual sites, create a query class per language and extract shared configuration into a mixin:

moduleBlog::PostQueryMethods
model Blog::Post
order_by date
endclassBlog::Post::EnQueryincludeMarquery::QueryincludeBlog::PostQueryMethodsendclassBlog::Post::NlQueryincludeMarquery::QueryincludeBlog::PostQueryMethodsend

Each class gets its own directory derived from the class name:

marquery/blog_post_en/20260320_first_post.md
marquery/blog_post_nl/20260320_eerste_bericht.md

Then select the query based on the current locale:

query =case locale
when"nl"thenBlog::Post::NlQuery.new
elseBlog::Post::EnQuery.new
end

To avoid duplicating assets across locales, use the @[Marquery::Assets] annotation to point to a shared assets directory:

@[Marquery::Assets("marquery/blog_post")]
classBlog::Post::EnQueryincludeMarquery::QueryincludeBlog::PostQueryMethodsend
@[Marquery::Assets("marquery/blog_post")]
classBlog::Post::NlQueryincludeMarquery::QueryincludeBlog::PostQueryMethodsend

Assets are organized by date (matching the entry filename prefix) and an optional _shared directory for assets common to all entries:

marquery/blog_post/
├── _shared/logo.svg
├── 20260320/hero.png
└── 20260325/diagram.svg

These are merged with any per-entry assets. When names collide, per-entry assets take precedence over date-based, which take precedence over _shared.

To serve shared assets over HTTP, pass assets_dir to the handler:

Marquery::AssetHandler.new(
Blog::Post::EnQuery.dir,
Blog::Post::EnQuery.assets_dir,
)

Contributing

We use conventional commits for our commit messages, so please adhere to that pattern.

  1. Fork it (https://codeberg.org/fluck/marquery.cr/fork)
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'feat: new feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create a new Pull Request

Contributors

  • Wout - creator and maintainer

About

Turn markdown files with frontmatter into queryable, type-safe collections. Parsed at compile time, baked into your binary.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages