40ANTS-DOC Documentation Generator

About this fork

This system is a fork of MGL-PAX.

There are a few reasons, why I've created the fork.

The main goal is to extract a core features into the 40ants-doc system with as little dependencies as possible. This is important, because with MGL-PAX's style, you define documentation sections in your library's code, which makes it dependent on the documentation system. However, heavy weight dependencies like IRONCLAD, 3BMD or SWANK should not be required.

The seconds goal was to refactor a 3.5k lines of pax.lisp file into a smaller modules to make navigation easier. This will help any person who will decide to learn how the documentation builder works. Also, granular design will make it possible loading subsystems like SLIME or SLY integration.

The third goal was to make documentation processing more sequential and hackable. To introduce hooks for adding new markup languages, and HTML themes.

Why this fork is different

Here are features already implemented in this fork:

• Core system 40ants-doc now has only two dependencies on NAMED-READTABLES and PYTHONIC-STRING-READER. If you want to compile a documentation, load 40ants-doc-full system which will download such dependencies as markdown parser and more.
• Now you don't have to import any locative symbols into your package. Import only a defsection macro and it will be enough to define documentation for your library!
• Added a warning mechanism, which will issue such warnings on words which looks like a symbol, but when real symbol or reference is absent:

WARNING: Unable to find target for reference #<XREF "FIND-SOURCE" GENERIC-FUNCTION> mentioned at 40Ants Doc Manual / Extension API / Reference Based Extensions

• Documentation processing now uses CommonDoc as intermediate format, and markup languages other than Markdown can be supported.
• Added a JS search index which will work when you are hosting pages on a static website like GitHub pages.
• It is possible to render pages in multiple formats and having cross references between them. See Multiple Formats.

I'm planning to extend this fork even more. Read todo section to learn about proposed features or start a new discussion on the GitHub to suggest a new feature.

See full list of changes in the ChangeLog section.

Full Documentation

Read full documentation at site 40ants.com/doc/.

Tutorial

40ants-doc provides an extremely poor man's Explorable Programming environment. Narrative primarily lives in so called sections that mix markdown docstrings with references to functions, variables, etc, all of which should probably have their own docstrings.

The primary focus is on making code easily explorable by using SLIME's M-. (slime-edit-definition). See how to enable some fanciness in Emacs Integration. Generating documentation from sections and all the referenced items in Markdown or HTML format is also implemented.

With the simplistic tools provided, one may accomplish similar effects as with Literate Programming, but documentation is generated from code, not vice versa and there is no support for chunking yet. Code is first, code must look pretty, documentation is code.

When the code is loaded into the lisp, pressing M-. in SLIME on the name of the section will take you there. Sections can also refer to other sections, packages, functions, etc and you can keep exploring.

Here is an example of how it all works together:

(uiop:define-package #:foo-random
(:nicknames #:40ants-doc-full/tutorial)
(:documentation "This package provides various utilities for
random. See @FOO-RANDOM-MANUAL.")
(:use #:common-lisp
#:40ants-doc)
(:import-from #:40ants-doc/ignored-words
#:ignore-words-in-package)
(:export #:foo-random-state
#:state
#:*foo-state*
#:gaussian-random
#:uniform-random))
(in-package foo-random)
(defsection @foo-random-manual (:title "Foo Random manual"
:ignore-words ("FOO"))
"Here you describe what's common to all the referenced (and
exported) functions that follow. They work with *FOO-STATE*,
and have a :RANDOM-STATE keyword arg. Also explain when to
choose which."
(foo-random-state class)
(state (reader foo-random-state))
"Hey we can also print states!"
(print-object (method () (foo-random-state t)))
(*foo-state* variable)
(gaussian-random function)
(uniform-random function)
;; this is a subsection
(@foo-random-examples section))
(defclass foo-random-state ()
((state :reader state
:documentation "Returns random foo's state.")))
(defmethod print-object ((object foo-random-state) stream)
(print-unreadable-object (object stream :type t)))
(defvar *foo-state* (make-instance 'foo-random-state)
"Much like *RANDOM-STATE* but uses the FOO algorithm.")
(defun uniform-random (limit &key (random-state *foo-state*))
"Return a random number from the between 0 and LIMIT (exclusive)
uniform distribution."
(declare (ignore limit random-state))
nil)
(defun gaussian-random (stddev &key (random-state *foo-state*))
"Return not a random number from a zero mean normal distribution with
STDDEV."
(declare (ignore stddev random-state))
nil)
(defsection @foo-random-examples (:title "Examples")
"Let's see the transcript of a real session of someone working
with FOO:
```cl-transcript
(values (princ :hello) (list 1 2))
.. HELLO
=> :HELLO
=> (1 2)
(make-instance 'foo-random-state)
==> #<FOO-RANDOM-STATE >
```")

Generating documentation in a very stripped down markdown format is easy:

(40ants-doc-full/builder:render-to-string
@foo-random-manual
:format:markdown)

For this example, the generated markdown would look like this:

<aid="x-28FOO-RANDOM-3A-3A-40FOO-RANDOM-MANUAL-2040ANTS-DOC-2FLOCATIVES-3ASECTION-29"></a>
# Foo Random manual
Here you describe what's common to all the referenced (and
exported) functions that follow. They work with [`*foo-state*`][2133],
and have a `:RANDOM-STATE` keyword arg. Also explain when to
choose which.
<aid="x-28FOO-RANDOM-3AFOO-RANDOM-STATE-20CLASS-29"></a>
## [class](ebf3)`foo-random:foo-random-state` ()
<aid="x-28FOO-RANDOM-3ASTATE-20-2840ANTS-DOC-2FLOCATIVES-3AREADER-20FOO-RANDOM-3AFOO-RANDOM-STATE-29-29"></a>
## [reader](01ce)`foo-random:state` (foo-random-state) ()
Returns random foo's state.
Hey we can also print states!
<aid="x-28PRINT-OBJECT-20-28METHOD-20NIL-20-28FOO-RANDOM-3AFOO-RANDOM-STATE-20T-29-29-29"></a>
## [method](7656)`common-lisp:print-object` (object foo-random-state) stream
<aid="x-28FOO-RANDOM-3A-2AFOO-STATE-2A-20-28VARIABLE-29-29"></a>
## [variable](1a03)`foo-random:*foo-state*` #<foo-random-state >
Much like `*RANDOM-STATE*` but uses the `FOO` algorithm.
<aid="x-28FOO-RANDOM-3AGAUSSIAN-RANDOM-20FUNCTION-29"></a>
## [function](19bc)`foo-random:gaussian-random` stddev &key (random-state \*foo-state\*)
Return not a random number from a zero mean normal distribution with
`STDDEV`.
<aid="x-28FOO-RANDOM-3AUNIFORM-RANDOM-20FUNCTION-29"></a>
## [function](2e1e)`foo-random:uniform-random` limit &key (random-state \*foo-state\*)
Return a random number from the between 0 and `LIMIT` (exclusive)
uniform distribution.
<aid="x-28FOO-RANDOM-3A-3A-40FOO-RANDOM-EXAMPLES-2040ANTS-DOC-2FLOCATIVES-3ASECTION-29"></a>
## Examples
Let's see the transcript of a real session of someone working
with `FOO`:
```cl-transcript(values (princ :hello) (list 1 2)).. HELLO=> :HELLO=> (1 2)(make-instance 'foo-random-state)==> #<FOO-RANDOM-STATE >```[2133]: #x-28FOO-RANDOM-3A-2AFOO-STATE-2A-20-28VARIABLE-29-29[ebf3]: https://github.com/40ants/doc/blob/709d3f2b7ab67bc617bff93be5219465d2d096f2/full/tutorial.lisp#L35[01ce]: https://github.com/40ants/doc/blob/709d3f2b7ab67bc617bff93be5219465d2d096f2/full/tutorial.lisp#L36[7656]: https://github.com/40ants/doc/blob/709d3f2b7ab67bc617bff93be5219465d2d096f2/full/tutorial.lisp#L39[1a03]: https://github.com/40ants/doc/blob/709d3f2b7ab67bc617bff93be5219465d2d096f2/full/tutorial.lisp#L42[2e1e]: https://github.com/40ants/doc/blob/709d3f2b7ab67bc617bff93be5219465d2d096f2/full/tutorial.lisp#L45[19bc]: https://github.com/40ants/doc/blob/709d3f2b7ab67bc617bff93be5219465d2d096f2/full/tutorial.lisp#L51

MGL-PAX supported the plain text format which was more readble when viewed from a simple text editor, but I've dropped support for plain text in this fork because most time documentation are read in the browser these days.

To render into the files, use 40ants-doc-full/builder:render-to-files and 40ants-doc-full/builder:update-asdf-system-docs functions.

Last one can even generate documentation for different, but related libraries at the same time with the output going to different files, but with cross-page links being automatically added for symbols mentioned in docstrings. See Generating Documentation for some convenience functions to cover the most common cases.

Note how (*FOO-STATE* VARIABLE) in the defsection form includes its documentation in @FOO-RANDOM-MANUAL. The symbols variable and function are just two instances of 'locatives' which are used in defsection to refer to definitions tied to symbols. See Locative Types.

The transcript in the code block tagged with cl-transcript is automatically checked for up-to-dateness. See Transcripts.

TODO

• Refactor code and make a core package with only a few dependencies.
• Add warnings on UPPERCASED symbols in docstrings which aren't found in the package and can't be cross referenced.
• Support SLY and make both SLIME and SLY integrations optional.
• Add a search facility which will build an index for static file like Sphinx does.
• Separate markup parsing and result rendering code to support markups other than Markdown and HTML.
• Add a new section type to render ChangeLog.
• Support custom HTML themes.
• Generate RSS or Atom feed out of changelog items, defined with 40ants-doc/changelog:defchangelog macro.
• Make some warnings compile-time for defsection and show them in the Emacs, if possible.

---

[generated by 40ANTS-DOC]