Snipsync

Snipsync makes sure your documented code snippets are always in sync with your Github repo source files.

Prerequisites

This tool requires Node v15.0.0 or above (recommended 15.2.1) and Yarn.

Install

Yarn:

yarn add snipsync

Configure

Create a file called "snipsync.config.yaml" in the project root. This file specifies the following:

• origins: The Github repositories or local files where the tool will look for source code snippets.
• targets: The local directories that contain the files to be spliced with the code snippets.

The origins property is a list of objects that have one of the following 2 formats:

1. owner, repo, and optionally ref: Pull snippets from a GitHub repo
2. files: a set of strings:

• pattern: Relative path to load snippets from. Supports glob syntax.
• owner: GitHub repo owner name, to be used in the source snippets links
• repo: Name of the repo snipsync is being used in, to link to the source snippets
• ref: (Optional, defaults to main) Used for writing source snippet links.

If the ref key is left blank or not specified, then the most recent commit from the main branch will be used. If the enable_source_link key in features is not specified, then it will default to true. If the enable_code_block key in features is not specified, then it will default to true.

The allowed_target_extensions key in features lets you set a list of extensions to scan. Specify extensions like [.md,.txt]. If the allowed_target_extensions key in features is not specified, then it defaults to an empty array ([]) and all files are scanned.

The enable_code_dedenting key in features lets you remove leading spaces from indented code snippets. This is handy when you're including a snippet of code within a class or function and don't want to include the leading indentation. This is false by default.

Example of a complete snipsync.config.yaml:

origins:
- owner: temporaliorepo: go-samplesref: 6880b0d09ddb6edf150e3095c90522602022578f
- owner: temporaliorepo: java-samples
- files:
pattern: ./sample-apps/typescript/*.tsowner: temporaliorepo: documentationref: maintargets:
- docs
- blogfeatures:
enable_source_link: falseenable_code_block: falseallowed_target_extensions: [.md]enable_code_dedenting: false

Example of a bare minimum snipsync.config.yaml:

origins:
- owner: temporaliorepo: go-samplestargets:
- docs

Comment wrappers

Use comments to identify code snippets and the locations where they should be merged.

Source code

In the source repo, wrap the code snippets in comments with a unique snippet identifier like this:

// @@@SNIPSTART hellouniversefuncHelloUniverse() {
fmt.Println("Hello Universe!")
}
// @@@SNIPEND

In the example above, "hellouniverse" is the unique identifier for the code snippet.

Unique identifiers can contain letters, numbers, hyphens, and underscores.

Target files

In the target files wrap the location with comments that reference the identifier of the code snippet that will be placed there:

<!--SNIPSTART hellouniverse--><!--SNIPEND-->

In the example above, the "hellouniverse" code snippet will be spliced between the comments. Any text inside of the placeholders will be replaced by the code snippet when the tool runs. The tool will automatically specify the code type for markdown rendering. For example, if the source file ends in ".go" then the code section will be written like this: ```go

Per-snip features

To customize how a single snip is rendered, add a JSON feature configuration in the snip start line.

<!--SNIPSTART hellouniverse {"enable_source_link": false, "enable_code_block": false}--><!--SNIPEND-->

Selected lines

A single source code snippet may be used in multiple places. If so, you may wish to customize which lines are rendered. Add a "selected" configuration to the snip start line.

<!--SNIPSTART hellouniverse {"selectedLines": ["1", "3-5"]}-->

The line numbers are relative to the snippet, not the source file.

The feature supports multiple selections as either a single line or a range.

Highlighed lines

Some frameworks support highlighting code lines in code blocks. If so, you can add a "highlights" configuration to the snip start line.

<!--SNIPSTART hellouniverse {"highlightedLines": "{1, 3-4}"}-->

The line numbers are relative to the published snippet. That means that if selectedLines is used, the line numbers to highlight are relative to the pared down selection that is merged into the Markdown file.

If you use Docusuarus, you just need to add some additional CSS: https://docusaurus.io/docs/markdown-features/code-blocks#line-highlighting

Regex snipping

Instead of specifying a set of line numbers to snip, you can use regular expression patterns to mark the start and end of a snip. Specify a startPattern and an endPattern:

<!--SNIPSTART hellouniverse {"startPattern" : "const \\{ greet", "endPattern": "\\}\\)"} -->

Specifying a source file

If the named snippet you want to extract exists in multiple source repositories, provide the path to only that source file after the snippet name, followed by an @:

<!--SNIPSTART money-transfer-project-template-go-workflow @https://github.com/temporalio/money-transfer-project-template-go/workflow.go -->

Run

From the root directory of your project run the following command:

yarn snipsync

Sync only some files

Pass --target with a glob to splice a subset of your target files:

yarn snipsync --target "docs/develop/dotnet/index.mdx"
yarn snipsync --target "docs/develop/dotnet/**/*.mdx"

Origins are still downloaded and all snippets extracted, so only the splice step gets faster. Scoping to a single page in the Temporal documentation repository takes a run from about 90 seconds to under 20.

Notes on the glob:

• It resolves from the directory you run the command in.
• It replaces targets rather than filtering it, so it can match files outside your configured target directories.
• allowed_target_extensions still applies.
• Quote it and pass it as a separate argument, so your shell does not expand it.

Remove snippets

In some cases, you may want to remove the snippets from your target files. Use the --clear flag to do that:

yarn snipsync --clear

--clear takes --target too:

yarn snipsync --clear --target "docs/develop/dotnet/index.mdx"

Development

The snipsync tool is set up to run its own functionality during development. Git ignores the snipsync.config.yaml file and the /docs directory within the package itself.

While developing, you can add files to /docs define the snipsync.config.yaml file, and run yarn dev to run snipsync from the root of the repo.

To clear the snippets run yarn dev --clear

Testing

Run yarn test from the root of the repo to run the testing suites.