rudof

This repo contains an RDF and Knowledge Graphs processing tool implemented in Rust.

The implementation supports RDF and property graphs data, as well as ShEx, SHACL, DCTap, PGSchema, and conversions between different RDF data modeling formalisms.

The code can be used as a Rust library but it also contains a binary called rudof which can be used as an RDF playground.

We provide binaries for Linux, Windows, Mac and Docker (see releases), as well as Python bindings.

• Documentation
• Using rudof as Jupyter notebooks
• Installation
• List of issues
• Discussion
• FAQ
• How to guides
• Roadmap

Features

rudof currently supports the following:

• RDF and RDF 1.2 parsing, conversion and visualization.
• SPARQL querying to RDF data and endpoints
• Parsing SPARQL service description
• ShEx
• SHACL
• DCTAP
• Property graphs and Schemas for Property Graphs (PGSchemahttps://dl.acm.org/doi/10.1145/3589778)

Future features we are planning to add:

• rdf-config
• LinkML

Installation

Official releases

You can download a binary from the latest release page. There you will also find the compiled packages for the installation on your system using a package manager.

Published Debian packages

rudof is available as a Debian package for Debian-based distributions (like Ubuntu).

It is also possible to download the binary from https://github.com/rudof-project/rudof/releases and install the .deb package running the following commands after replacing X.X.X by the latest version:

wget https://github.com/rudof-project/rudof/releases/download/vX.X.X/rudof_vX.X.X_amd64.deb
sudo dpkg -i rudof_vX.X.X_amd64.deb

Windows

The binary can be downloaded from https://github.com/rudof-project/rudof/releases.

Mac

The binary is available at: https://github.com/rudof-project/rudof/releases so you can download the corresponding binary to your machine.

The usual way to run/install a binary in Mac is to download it in a folder, add that folder to your PATH and activating the binary using:

chmod +x <binary_file>

After that, the processor may complain the first time about security and you have to agree to use it. Once you agree, it should work.

Compiling from source

Compiling from source

rudof has been implemented in Rust and is compiled using cargo. The command cargo run can be used to compile and run locally the code.

For example:

cargo run -- validate examples/user.ttl --schema examples/user.shex --shapemap examples/user.sm

Troubleshooting

If the example doesn’t work as expected, here are a few things you can try:

• Use the --release flag to compile in release mode, which can resolve some build issues and improve performance:

cargo run --release -- validate examples/user.ttl --schema examples/user.shex --shapemap examples/user.sm

• Run the command inside WSL (Windows Subsystem for Linux). If you're using Windows, compiling the project in WSL can help resolve environment-related issues, as Rust tends to compile more reliably and efficiently in Linux-based systems.

Compiling from source and installing the binary (Debian)

Install cargo deb (only the first time)

cargo install cargo-deb

Create the .deb package by:

cargo deb

And run:

sudo dpkg -i target/debian/rudof_<VERSION>_amd64.deb

Alternative for Linux

To create a binary for Linux with debug information:

cargo build --target x86_64-unknown-linux-gnu

The binary will be created in: target/x86_64-unknown-linux-gnu/debug/rudof

If you want a release binary which is more optimized, you can run:

cargo build --target x86_64-unknown-linux-gnu

In this case, the binary will be target/x86_64-unknown-linux-gnu/release/rudof

Docker

The library is also published as a Docker image (angelip2303/rudof:latest).

Usage

Some examples

The folder examples contains several example files with ShEx schemas and RDF data.

Validate a simple RDF file with a ShEx schema using a ShapeMap

rudof validate examples/user.ttl --schema examples/user.shex --shapemap examples/user.sm

We maintain a Wiki page with some common Usage scenarios and How-to guides.

Debugging information

It is possible to change the debug level information with:

export RUST_LOG=info

where value can be info, debug or trace to show more information. It is also possible to have more control about the logs using more complex filters. For example, to show trace for the elements in crate shacl_validation, and supress any logs for crates hyper and reqwest (which can be quite verbose with traces), you can use:

export RUST_LOG=info,shacl_validation=trace,hyper=off,reqwest=off

In case you use nu_shell, you can use:

$env.RUST_LOG = 'info,shacl_validation=trace,hyper=off,reqwest=off'

Command line usage

RDF and Knowledge Graphs processing tool
Usage: rudof [OPTIONS] [COMMAND]
Commands:
mcp Export rudof as an MCP server
shapemap Show information about ShEx ShapeMaps
shex Show information about ShEx schemas
pgschema Show information about Property Graph Schemas
validate Validate RDF data using ShEx or SHACL
shex-validate Validate RDF using ShEx schemas
shacl-validate Validate RDF data using SHACL shapes
data Show information about RDF data
node Show information about a node in an RDF Graph
shacl Show information about SHACL shapes
dctap Arguments for the `dctap`command
convert Arguments for the `convert`command
compare Compare two shapes (which can be in different formats)
rdf-config Show information about rdf config
service Show information about SPARQL service
query Run SPARQL queries
generate Generate synthetic RDF data from ShEx or SHACL schemas
materialize Materialize an RDF graph from a ShEx schema and Map semantic-action state
pgschema-validate Validate Property Graph data using PGSchema
completion Generates a shell completion script for the specified shell
config Dump the effective configuration rudof is using as TOML
help Print this message or the help of the given subcommand(s)
Options:
-d, --debug... Increase logging verbosity
-h, --help Print help
-V, --version Print version

You can see the manual

Main modules

The repo is divided in the following modules:

• rudof_iri defines simple IRIs.
• srdf simple RDF model which will be used for validation.
• prefixmap Prefix maps implementation.
• shapemap ShapeMap implementation.
• shex_ast defines the ShEx Abstract syntax
• shex_compact contains the code required to handle ShEx compact syntax.
• shex_validation contains the code required to validate RDF using ShEx.
• shex_testsuite contains the code required to run the ShEx testsuite.
• shacl_ast defines the SHACL core Abstract syntax.
• shacl_validation contains the code required to validate RDF using SHACL.
• dctap contains the code required to do handle DCTAP files.
• shapes_converter contains the code required to do conversion between different shapes formalisms.

Publishing the crates

cargo workspaces publish

Worskpaces

The project is using cargo workspaces wihch can be installed with:

cargo install cargo-workspaces

Unit-testing

In order to test all the sub-projects

cargo test --all

Testing one specific subproject:

cargo test -p shex_validation

Using the ShEx test-suite

The ShEx testsuite is included in a git submodule. In order to obtain it, it is necessary to do:

git submodule update --init --recursive
cargo run -p shex_testsuite

Usage: shex_testsuite [OPTIONS]
Options:
-m, --manifest <Manifest FILE (.jsonld)>
Name of Manifest file [default: shex_testsuite/shexTest/validation/manifest.jsonld]
-c, --config <Config file>
[default: shex_testsuite/config.toml]
-x, --run_mode <MANIFEST_RUN_MODE>
[default: collect-errors] [possible values: collect-errors, fail-first-error]
-f, --manifest_mode <MANIFEST_MODE>
[possible values: schemas, validation, negative-syntax, negative-structure]
-p, --print_result_mode <PRINT_RESULT_MODE>
[default: basic] [possible values: basic, failed, passed, not-implemented, all]
-e, --entry <Entry names>
-t, --trait <Trait names>
-h, --help
Print help
-V, --version
Print version

Validation conformance tests for ShEx

cargo run -p shex_testsuite -- -m shex_testsuite/shexTest/validation/manifest.jsonld 

If you want to see the failed tests, you can run:

cargo run -p shex_testsuite -- -m shex_testsuite/shexTest/validation/manifest.jsonld -p failed

Schemas tests

cargo run -p shex_testsuite -- -m shex_testsuite/shexTest/schemas/manifest.jsonld -f schemas -p failed

Benchmarks

Performance benchmarks live in benchmarks/. They are not published to crates.io and are not executed in CI. Run them locally with Criterion.

License

Licensed under either of

• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Contributors

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Local development

Precommit has been configured to run formatters and linters before each commit. It is recommended to use it to ensure that the code is formatted and linted correctly before pushing it to the repository.

In order to install it, you can run:

# Installs pre-commit it self
pip install pre-commit
# Installs the pre-commit hooks defined in .pre-commit-config.yaml
pre-commit install

Nix

If you have Nix with flakes enabled, the repository provides a ready-to-use development shell that includes all required dependencies.

Simply run the following command from the root of the repository:

nix develop

This drops you into a shell with everything pre-configured. No manual installation of Rust, cargo, or any other tool is needed.

You can also run a one-off command without entering the shell interactively:

nix develop --command cargo test

If you use direnv, add a .envrc file with use flake to activate the development environment automatically whenever you cd into the repository:

echo"use flake"> .envrc && direnv allow