Markdown-LD Knowledge Bank

Markdown-LD Knowledge Bank is a .NET 10 library for turning Markdown knowledge-base files into an in-memory RDF graph that can be searched, queried with read-only SPARQL, validated with SHACL, exported as RDF, and rendered as a diagram.

The recommended entry point is MarkdownKnowledgeBank. It is a single facade for build, source-change planning, chunk evaluation, ranked search, optional semantic indexing, and cited answers. Lower-level pipeline and graph APIs remain available when a host needs more control.

The runtime is local and in-memory: no localhost server, no Azure Functions host, no database server, and no hosted graph service are required.

Use it when you want plain Markdown notes to become a queryable knowledge graph without making your application depend on a specific model provider, graph server, or hosted indexing service.

What It Does

flowchart LR
Source["Markdown / MDX / text\nJSON / YAML / CSV"] --> Bank["MarkdownKnowledgeBank\nfacade API"]
Bank --> Plan["Source manifest\nchange planning"]
Bank --> Converter["KnowledgeSourceDocumentConverter"]
Converter --> Parser["MarkdownDocumentParser\n→ MarkdownDocument"]
Parser --> Mode["Extraction mode\nAuto / None / ChatClient / Tiktoken"]
Parser --> ChunkEval["MarkdownChunkEvaluator"]
Mode --> None["None\nmetadata only"]
Mode --> Chat["ChatClientKnowledgeFactExtractor\nIChatClient"]
Mode --> Token["Tiktoken token-distance extractor\nMicrosoft.ML.Tokenizers"]
None --> Merge["KnowledgeFactMerger\n→ canonical facts"]
Chat --> Merge
Token --> Merge
Merge --> Normalize["KnowledgeGraphNormalizer\n→ clean facts + warnings"]
Normalize --> Builder["KnowledgeGraphBuilder\n→ RDF + ontology + SKOS graph"]
Builder --> Search["SearchBySchemaAsync"]
Builder --> Ranked["SearchRankedAsync\nGraph / BM25 / Semantic / Hybrid RRF\nDocument-aware via build result"]
Ranked --> Answer["AnswerAsync\ncited IChatClient answer"]
Builder --> Sparql["ExecuteSelectAsync\nExecuteAskAsync"]
Builder --> FederatedSearch["SearchBySchemaFederatedAsync"]
Builder --> Shacl["ValidateShacl\nSHACL report"]
Builder --> Snap["ToSnapshot\nsemantic/operator graph"]
Builder --> Complete["ToCompleteSnapshot\nretrieval diagnostics"]
Builder --> Cycles["FindCycles\nbounded SCC analysis"]
Builder --> Diagram["SerializeMermaidFlowchart\nSerializeDotGraph"]
Builder --> Export["SerializeTurtle\nSerializeJsonLd"]

Loading

Extraction is explicit:

• Auto uses IChatClient when one is supplied, otherwise extracts no facts and reports a diagnostic.
• None builds document metadata only.
• ChatClient builds facts only from structured Microsoft.Extensions.AI.IChatClient output.
• Tiktoken builds a local corpus graph from Tiktoken token IDs, section/segment structure, explicit front matter entity hints, and local keyphrase topics using Microsoft.ML.Tokenizers.

Tiktoken mode is deterministic and network-free. It uses lexical token-distance search rather than semantic embedding search. Its default local weighting is subword TF-IDF; raw term frequency and binary presence are also available. Token-distance search can opt into fuzzy query correction over corpus words before Tiktoken encoding, which helps typo-heavy same-language queries without treating model-specific token IDs as editable text. It creates schema:DefinedTerm topic nodes, explicit front matter hint entities, and schema:hasPart / schema:about / schema:mentions edges. Retrieval-only section, segment, and n-gram topic nodes stay available through ToCompleteSnapshot() and RDF serialization, but the default operator projection excludes them.

Graph outputs:

• MarkdownKnowledgeBank — recommended facade for build, change planning, chunk evaluation, ranked search, optional semantic indexing, and cited answers
• MarkdownKnowledgeBankBuild.SearchAsync(...) — ranked graph search through one build object
• MarkdownKnowledgeBankBuild.AnswerAsync(...) — answer a question with citations from the built Markdown graph through IChatClient
• MarkdownKnowledgeBankBuild.BuildSemanticIndexAsync(...) — optional semantic index through IEmbeddingGenerator<string, Embedding<float>>
• MarkdownKnowledgeBank.PlanChanges(...) — compare source SHA256 fingerprints and return changed, unchanged, and removed paths before a build
• MarkdownKnowledgeBank.EvaluateChunks(...) — deterministic chunk-size, expected-answer coverage, and quality-sample report
• ToSnapshot() — stable semantic/operator KnowledgeGraphSnapshot with retrieval internals removed
• ToSemanticSnapshot() — explicit equivalent of the default semantic projection
• ToCompleteSnapshot() — complete RDF snapshot including Tiktoken retrieval internals
• FindCycles(...) — bounded strongly connected components for selected relationship predicates
• MarkdownKnowledgeBuildResult.Normalization — structured warnings for removed duplicate, invalid, self-loop, symmetric, cycle-causing, invalid-provenance, and invalid-node facts plus normalized confidence
• SerializeMermaidFlowchart() — Mermaid graph LR diagram
• SerializeDotGraph() — Graphviz DOT diagram
• SerializeTurtle() — Turtle RDF serialization
• SerializeJsonLd() — JSON-LD serialization
• LoadJsonLd(jsonLd) — load JSON-LD text into a searchable in-memory graph
• SaveToStoreAsync(store, location, options) — persist the graph through a graph-store abstraction
• SaveToFileAsync(path, options) — persist the graph as RDF
• SaveJsonLdToStoreAsync(store, location) / SaveJsonLdToFileAsync(path) — persist JSON-LD explicitly, including opaque storage keys
• LoadFromStoreAsync(store, location, options) — load a graph from a graph-store abstraction
• LoadFromFileAsync(path, options) — load a graph from one RDF file
• LoadFromDirectoryAsync(path, options) — load and merge RDF files from a directory
• LoadJsonLdFromStoreAsync(store, location) / LoadJsonLdFromFileAsync(path) — load JSON-LD explicitly, including opaque storage keys
• MarkdownKnowledgeBuildResult.Contract — self-describing graph contract with schema introspection and search-profile validation
• KnowledgeGraphContract.SerializeJson() / SerializeYaml() — portable contract artifacts for preprocessing and search handoff
• KnowledgeGraphContract.LoadJson(json) / LoadYaml(yaml) — reload contract artifacts alongside generated JSON-LD
• KnowledgeGraphContract.GenerateShacl() — generate SHACL Turtle from the contract search profile
• LoadFromLinkedDataFragmentsAsync(endpoint, options) — materialize a Linked Data Fragments source into a local graph
• ExecuteSelectAsync(sparql) — read-only SPARQL SELECT returning SparqlQueryResult
• ExecuteAskAsync(sparql) — read-only SPARQL ASK returning bool
• ExecuteFederatedSelectAsync(sparql, options) — explicit federated read-only SPARQL SELECT with endpoint diagnostics
• ExecuteFederatedAskAsync(sparql, options) — explicit federated read-only SPARQL ASK with endpoint diagnostics
• ValidateShacl() — SHACL validation against the built-in Markdown-LD Knowledge Bank shapes
• ValidateShacl(shapesTurtle) — SHACL validation against caller-supplied Turtle shapes
• DescribeSchema(prefixes) — inspect actual RDF types, predicates, literal predicates, and resource predicates in a graph
• ValidateSchemaSearchProfile(profile) — validate a schema-aware search profile against the actual graph shape
• SearchBySchemaAsync(term, profile) — recommended schema-aware SPARQL search over caller-defined predicates, relationships, type filters, and focused graph expansion
• SearchBySchemaFederatedAsync(term, profile, options) — schema-aware SPARQL search compiled into explicit SERVICE blocks for allowlisted federated endpoints
• SearchFocusedAsync(term, options) — sparse graph search that can use a schema-aware profile and returns primary, related, and next-step matches plus a bounded focused graph snapshot
• SearchAsync(term) — compatibility helper for simple schema:name / schema:description lookup; use schema-aware SPARQL search for application search
• KnowledgeGraph.Diff(other) — compare graph snapshots and report added, removed, and changed literal edges
• BuildIncrementalAsync(...) — rebuild deterministically while returning a source manifest, changed paths, unchanged paths, removed paths, and optional graph diff
• MaterializeInferenceAsync(options) — explicit RDFS / SKOS / N3-rule materialization
• BuildFullTextIndexAsync(options) — optional Lucene-backed graph full-text index
• ToDynamicSnapshot() — optional dynamic graph access over dotNetRDF dynamic types

All async methods accept an optional CancellationToken.

What To Use When

Goal	Use	Details
Start from one library API	MarkdownKnowledgeBank	Unified API
Build a graph from Markdown files	MarkdownKnowledgeBank.BuildFromDirectoryAsync(...)	Build From Files
Ask questions with source citations	MarkdownKnowledgeBankBuild.AnswerAsync(...)	Unified API
Search with graph, BM25, optional fuzzy BM25, semantic, or hybrid RRF ranking	MarkdownKnowledgeBankBuild.SearchAsync(...)	Unified API
Evaluate chunking quality	MarkdownKnowledgeBank.EvaluateChunks(...)	Unified API
Generate portable graph output	SerializeJsonLd(), SaveJsonLdToFileAsync(...), SerializeTurtle()	Generate JSON-LD Files, Export The Graph
Load a preprocessed graph from another system	KnowledgeGraph.LoadJsonLd(...), LoadJsonLdFromFileAsync(...)	Generate JSON-LD Files
Keep graph creation and search rules together	KnowledgeGraphBuildProfile, MarkdownKnowledgeBuildResult.Contract	Schema-Aware SPARQL Search, Graph Production Pipeline
Validate an externally generated graph	KnowledgeGraphContract.GenerateShacl(), ValidateShacl(...)	Validate With SHACL
Search custom JSON-LD/RDF shapes	SearchBySchemaAsync(term, profile)	Schema-Aware SPARQL Search
Search across graph slices or endpoints	SearchBySchemaFederatedAsync(...), ExecuteFederatedSelectAsync(...)	Federated SPARQL Execution
Explain why a result matched	KnowledgeGraphSchemaSearchResult.Explain, Evidence, SourceContexts	Schema-Aware SPARQL Search
Compare graph versions	KnowledgeGraph.Diff(other)	Graph Production Pipeline
Rebuild and know which source files changed	MarkdownKnowledgeBank.PlanChanges(...), BuildIncrementalAsync(...)	Graph Production Pipeline
Add AI extraction without provider lock-in	IChatClient, MarkdownKnowledgeExtractionMode.ChatClient	Optional AI Extraction
Add optional semantic retrieval	IEmbeddingGenerator<string, Embedding<float>>	Unified API
Use deterministic local extraction	MarkdownKnowledgeExtractionMode.Tiktoken	Local Tiktoken Extraction
Render an operator knowledge graph	ToSnapshot()	Graph Normalization
Inspect retrieval internals deliberately	ToCompleteSnapshot()	Graph Normalization

The most important split is local graph search versus federated graph search. SearchBySchemaAsync searches one in-memory graph. SearchBySchemaFederatedAsync and ExecuteFederatedSelectAsync are explicit opt-in federation paths that require allowlisted SERVICE endpoints.

Install

dotnet add package ManagedCode.MarkdownLd.Kb --version 0.2.8

For local repository development:

dotnet add reference ./src/MarkdownLd.Kb/MarkdownLd.Kb.csproj

Project Structure

The production source tree now follows feature-oriented slices instead of a mostly flat technical grouping:

• src/MarkdownLd.Kb/DocumentsModels, Parsing, and Chunking
• src/MarkdownLd.Kb/MarkdownKnowledgeBank* high-level facade for the common build/search/answer/evaluation flow
• src/MarkdownLd.Kb/ExtractionChat, Cache, and Processing
• src/MarkdownLd.Kb/Pipeline orchestration-only files such as MarkdownKnowledgePipeline
• src/MarkdownLd.Kb/GraphBuild and Runtime
• src/MarkdownLd.Kb/Tokenization local Tiktoken graph extraction
• src/MarkdownLd.Kb/QuerySearch, Sparql, NaturalLanguage, and Answering
• src/MarkdownLd.Kb/Rdf low-level RDF helpers and serialization

This layout mirrors docs/Architecture.md and keeps orchestration separate from parsing, extraction, graph runtime, and query capabilities.

For production graph handoff, contract artifacts, generated SHACL, explainable SPARQL evidence, presets, diffing, and incremental rebuilds, see docs/Features/GraphProductionPipeline.md.

Unified API

Use MarkdownKnowledgeBank when you want one object to own the normal knowledge-bank flow. It wraps the deterministic pipeline and keeps optional AI services behind Microsoft.Extensions.AI abstractions.

usingManagedCode.MarkdownLd.Kb;usingManagedCode.MarkdownLd.Kb.Pipeline;usingManagedCode.MarkdownLd.Kb.Query;usingMicrosoft.Extensions.AI;internalstaticclassKnowledgeBankDemo{publicstaticasyncTaskRunAsync(IReadOnlyList<MarkdownSourceDocument>documents,IChatClientchatClient,IEmbeddingGenerator<string,Embedding<float>>embeddings,KnowledgeGraphSourceManifest?previousManifest=null){varbank=newMarkdownKnowledgeBank(newMarkdownKnowledgeBankOptions{PipelineOptions=newMarkdownKnowledgePipelineOptions{BaseUri=newUri("https://kb.example/"),ExtractionMode=MarkdownKnowledgeExtractionMode.None,},ChatClient=chatClient,EmbeddingGenerator=embeddings,});varchangeSet=bank.PlanChanges(documents,previousManifest);varchunkReport=bank.EvaluateChunks(documents[0].Content,documents[0].Path,[newMarkdownChunkCoverageExpectation("How do I restore cache?","cache restore verification")]);varbuild=awaitbank.BuildAsync(documents);awaitbuild.BuildSemanticIndexAsync();varmatches=awaitbuild.SearchAsync("restore cache manifest",newKnowledgeGraphRankedSearchOptions{Mode=KnowledgeGraphSearchMode.Hybrid,HybridFusionStrategy=KnowledgeGraphHybridFusionStrategy.ReciprocalRank,MaxResults=5,});varanswer=awaitbuild.AnswerAsync("What should I use to restore cache?",newKnowledgeGraphRankedSearchOptions{Mode=KnowledgeGraphSearchMode.Bm25,EnableFuzzyTokenMatching=true,MaxResults=3,});Console.WriteLine(changeSet.ChangedPaths.Count);Console.WriteLine(chunkReport.CoverageRate);Console.WriteLine(matches[0].Label);Console.WriteLine(answer.Answer);Console.WriteLine(answer.Citations[0].SourcePath);}}

The facade does not hide missing optional services. AnswerAsync requires an IChatClient; BuildSemanticIndexAsync requires an IEmbeddingGenerator<string, Embedding<float>>. If those services are absent, the call fails explicitly instead of silently falling back to a weaker path. Facade search and cited answers are document-aware: document node candidates include parsed Markdown body chunks, so BM25 and optional semantic search can retrieve evidence that is only present in body text. BM25 can also opt into bounded fuzzy token matching for typo-tolerant lexical retrieval; the default remains exact token matching. EvaluateChunks uses the facade pipeline chunking options by default, including the Han, Japanese kana, and Korean Hangul-aware token estimate, so build and evaluation share the same chunk budget unless a caller passes explicit evaluation options.

Minimal Example

usingManagedCode.MarkdownLd.Kb;usingManagedCode.MarkdownLd.Kb.Pipeline;internalstaticclassMinimalGraphDemo{privateconststringSearchTerm="RDF SPARQL Markdown graph";privateconststringArticleMarkdown="""---title: Zero Cost Knowledge Graphdescription: Markdown notes can become a queryable graph.tags: - markdown - rdfauthor: - Ada Lovelace---# Zero Cost Knowledge GraphMarkdown-LD Knowledge Bank links [RDF](https://www.w3.org/RDF/) and [SPARQL](https://www.w3.org/TR/sparql11-query/).""";publicstaticasyncTaskRunAsync(){varbank=newMarkdownKnowledgeBank(newMarkdownKnowledgeBankOptions{PipelineOptions=newMarkdownKnowledgePipelineOptions{ExtractionMode=MarkdownKnowledgeExtractionMode.None,},});varresult=awaitbank.BuildFromMarkdownAsync(ArticleMarkdown);varsearch=awaitresult.SearchAsync(SearchTerm,newKnowledgeGraphRankedSearchOptions{Mode=KnowledgeGraphSearchMode.Bm25,});Console.WriteLine(search[0].Label);}}

Build From Files

usingManagedCode.MarkdownLd.Kb;usingManagedCode.MarkdownLd.Kb.Pipeline;internalstaticclassFileGraphDemo{privateconststringFilePath="/absolute/path/to/content/article.md";privateconststringDirectoryPath="/absolute/path/to/content";privateconststringMarkdownSearchPattern="*.md";publicstaticasyncTaskRunAsync(){varbank=newMarkdownKnowledgeBank();varsingleFile=awaitbank.BuildFromFileAsync(FilePath);vardirectory=awaitbank.BuildFromDirectoryAsync(DirectoryPath,searchPattern:MarkdownSearchPattern);Console.WriteLine(singleFile.Graph.TripleCount);Console.WriteLine(directory.Documents.Count);}}

KnowledgeSourceDocumentConverter supports Markdown and other text-like knowledge inputs: .md, .markdown, .mdx, .txt, .text, .log, .csv, .json, .jsonl, .yaml, and .yml. Files with unknown or missing extensions are still accepted when their bytes decode as text, and they are treated as text/plain. Truly unreadable binary files are either skipped during directory loads or fail explicitly with InvalidDataException when the caller disables skipping.

Graph loading and persistence support RDF files in Turtle (.ttl), JSON-LD (.jsonld, .json), RDF/XML (.rdf, .xml), N-Triples (.nt), Notation3 (.n3), TriG (.trig), and N-Quads (.nq). Use the explicit JSON-LD helpers when a file path or storage key has no extension or uses an opaque name.

Relative Markdown links, image links, and same-document fragment links are resolved from the current source path before base URI composition. For example, a link from content/guides/setup/intro.md to ../runbooks/cache-restore.md#steps resolves to https://kb.example/guides/runbooks/cache-restore/#steps when https://kb.example/ is the base URI, and [Steps](#steps) in the same file resolves to https://kb.example/guides/setup/intro/#steps.

You do not need to pass a base URI for normal use. Document identity is resolved in this order:

• KnowledgeDocumentConversionOptions.CanonicalUri when you provide one
• the file path, normalized deterministically: content/notes/rdf.md becomes a stable document IRI
• the generated inline document path when BuildFromMarkdownAsync is called without a path

The library uses urn:managedcode:markdown-ld-kb:/ as an internal default base URI only to create valid RDF IRIs when the source does not provide KnowledgeDocumentConversionOptions.CanonicalUri. Configure MarkdownKnowledgeBankOptions.PipelineOptions.BaseUri only when you want generated document/entity IRIs to live under your own domain.

Capability Graph Rules

Markdown can include deterministic graph rules in front matter. These rules are useful for capability catalogs, tool catalogs, workflow graphs, and any corpus where related and next-step nodes matter more than broad top-N search.

---title: Story Delete Toolsummary: Delete a story after the caller identifies the exact story item.graph_groups:
- Story tools
- Delete operationgraph_related:
- https://kb.example/tools/story-feed-detail/graph_next_steps:
- https://kb.example/tools/story-comments/---# Story Delete Tool
Use this capability to remove an existing story.

graph_groups creates kb:memberOf edges. graph_related creates kb:relatedTo edges. graph_next_steps creates kb:nextStep edges. For advanced graphs, use graph_entities and graph_edges to add explicit nodes and predicates. Absolute IRIs are preserved; plain labels become stable entity IRIs under the pipeline base URI.

usingManagedCode.MarkdownLd.Kb.Pipeline;internalstaticclassCapabilityGraphDemo{publicstaticasyncTaskRunAsync(IReadOnlyList<MarkdownSourceDocument>documents){varpipeline=newMarkdownKnowledgePipeline(newUri("https://kb.example/"),extractionMode:MarkdownKnowledgeExtractionMode.Tiktoken);varresult=awaitpipeline.BuildAsync(documents);varfocused=awaitresult.Graph.SearchFocusedAsync("remove the selected story from the feed",newKnowledgeGraphFocusedSearchOptions{MaxPrimaryResults=1,MaxRelatedResults=3,MaxNextStepResults=3,});varprimary=focused.PrimaryMatches[0];varmermaid=KnowledgeGraph.SerializeMermaidFlowchart(focused.FocusedGraph);Console.WriteLine(primary.Label);Console.WriteLine(mermaid);}}

Use BuildAsync(documents, KnowledgeGraphBuildOptions) when graph rules are assembled by the host application instead of authored in Markdown front matter.

Entities with the same schema:sameAs target are merged before assertions are emitted, including cases where a later entity uses that sameAs target as its direct ID. Assertion endpoints are rewritten to the chosen canonical entity IRI. Duplicate assertions keep all source provenance before graph materialization, so cited answers and SPARQL evidence can still point at the best Markdown source. This keeps the graph sparse without losing useful source attribution when callers provide multiple labels, IDs, or rule sources for the same outside resource.

Ontology And SKOS Layers

KnowledgeGraphBuilder now builds one additive graph, not a flat triple dump:

• document and entity instance triples
• a SKOS concept scheme / concept layer for graph concepts
• repository-owned ontology declarations for kb: classes and properties

The implementation uses dotNetRdf, dotNetRdf.Ontology, and dotNetRdf.Skos as the semantic building blocks. Markdown remains the source of truth; the library owns the mapping from Markdown/front matter/rules into the RDF graph.

By default, semantic layers are enabled through KnowledgeGraphBuildOptions.SemanticLayers.

usingManagedCode.MarkdownLd.Kb.Pipeline;varresult=awaitpipeline.BuildAsync(documents,newKnowledgeGraphBuildOptions{SemanticLayers=newKnowledgeGraphSemanticLayerOptions{IncludeOntologyLayer=true,IncludeSkosLayer=true,ConceptSchemeLabel="Operations Capability Scheme",},});

Graph Runtime Lifecycle

Once a Markdown file or directory has been built into a KnowledgeGraph, the same public runtime can persist it through a graph-store abstraction, reload it, materialize inference, expose a full-text index, expose a dynamic snapshot, or materialize a Linked Data Fragments source into the same local graph model.

The runtime now uses dotNetRdf, dotNetRdf.Ontology, dotNetRdf.Skos, dotNetRdf.Inferencing, dotNetRdf.Dynamic, dotNetRdf.Query.FullText, and dotNetRdf.Ldf through repository-owned adapters instead of a hand-rolled RDF stack. RDF serialization remains repository-owned; filesystem/blob access is delegated to ManagedCode.Storage.

usingManagedCode.MarkdownLd.Kb.Pipeline;internalstaticclassGraphRuntimeLifecycleDemo{privateconststringFilePath="/absolute/path/to/content/query-federation-runbook.md";privateconststringTurtlePath="/absolute/path/to/output/runtime-graph.ttl";privateconststringStorageLocation="graphs/runtime/runtime-graph.ttl";privateconststringSchemaPath="/absolute/path/to/runtime-schema.ttl";privateconststringRulesPath="/absolute/path/to/runtime-rules.n3";publicstaticasyncTaskRunAsync(){varpipeline=newMarkdownKnowledgePipeline(newUri("https://kb.example/"));varbuilt=awaitpipeline.BuildFromFileAsync(FilePath);varmemoryStore=newInMemoryKnowledgeGraphStore();awaitbuilt.Graph.SaveToStoreAsync(memoryStore,StorageLocation);varfromMemory=awaitKnowledgeGraph.LoadFromStoreAsync(memoryStore,StorageLocation);awaitbuilt.Graph.SaveToFileAsync(TurtlePath);varreloaded=awaitKnowledgeGraph.LoadFromFileAsync(TurtlePath);varinference=awaitfromMemory.MaterializeInferenceAsync(newKnowledgeGraphInferenceOptions{AdditionalSchemaFilePaths=[SchemaPath],AdditionalN3RuleFilePaths=[RulesPath],});usingvarfullText=awaitinference.Graph.BuildFullTextIndexAsync();varmatches=awaitfullText.SearchAsync("federated wikidata workflow");dynamicdynamicGraph=inference.Graph.ToDynamicSnapshot();dynamicdynamicDocument=dynamicGraph["https://kb.example/query-federation-runbook/"];Console.WriteLine(inference.InferredTripleCount);Console.WriteLine(matches.Count);Console.WriteLine(dynamicDocument["https://schema.org/name"].Count);Console.WriteLine(reloaded.TripleCount);}}

The built-in graph-store implementations are:

• FileSystemKnowledgeGraphStore — local file paths, internally backed by ManagedCode.Storage.FileSystem
• StorageKnowledgeGraphStore — any configured ManagedCode.Storage.Core.IStorage backend, including blob/object providers
• InMemoryKnowledgeGraphStore — process-local graph persistence without files

DI helpers are available for hosts that want one or more configured stores:

usingManagedCode.MarkdownLd.Kb.Pipeline;usingManagedCode.Storage.FileSystem;usingManagedCode.Storage.FileSystem.Extensions;usingMicrosoft.Extensions.DependencyInjection;varservices=newServiceCollection();services.AddFileSystemKnowledgeGraphStoreAsDefault(options =>{options.BaseFolder="/absolute/path/to/storage-root";options.CreateContainerIfNotExists=true;});services.AddFileSystemStorage("archive", options =>{options.BaseFolder="/absolute/path/to/archive-root";options.CreateContainerIfNotExists=true;});services.AddKeyedStorageBackedKnowledgeGraphStore<IFileSystemStorage>("archive");

Use new InMemoryKnowledgeGraphStore() for process-local persistence, or AddVirtualFileSystemKnowledgeGraphStore() after AddVirtualFileSystem(...) when the host already standardizes on a VFS overlay.

The same runtime can also materialize a read-only Triple Pattern Fragments source into a local graph:

usingManagedCode.MarkdownLd.Kb.Pipeline;varldfGraph=awaitKnowledgeGraph.LoadFromLinkedDataFragmentsAsync(newUri("https://example.org/tpf"));

If the host needs custom transport settings, pass a caller-owned HttpClient through KnowledgeGraphLinkedDataFragmentsOptions. Host apps may source that client from IHttpClientFactory; the core library intentionally accepts the configured client instance instead of depending on IHttpClientFactory.

After materialization, callers use the normal local ExecuteSelectAsync, ExecuteAskAsync, SearchBySchemaAsync, ValidateShacl, persistence, and inference APIs.

Generate JSON-LD Files

JSON-LD is the portable RDF file format for exchanging a built graph with another process. The graph can be generated from deterministic Markdown metadata, from IChatClient extraction, or by an external preprocessing pipeline that writes JSON-LD for this library to load later.

usingManagedCode.MarkdownLd.Kb.Pipeline;usingMicrosoft.Extensions.AI;internalstaticclassJsonLdPreprocessingDemo{privateconststringMarkdownRoot="/absolute/path/to/content";privateconststringMarkdownPattern="*.md";privateconststringOutputJsonLdPath="/absolute/path/to/output/knowledge-bank.jsonld";privateconststringOpaqueJsonLdPath="/absolute/path/to/output/knowledge-bank.payload";privateconststringExternalJsonLdPath="/absolute/path/to/external/preprocessed.jsonld";publicstaticasyncTaskGenerateFromMarkdownMetadataAsync(){varpipeline=newMarkdownKnowledgePipeline(newUri("https://kb.example/"),extractionMode:MarkdownKnowledgeExtractionMode.None);varresult=awaitpipeline.BuildFromDirectoryAsync(MarkdownRoot,searchPattern:MarkdownPattern);awaitresult.Graph.SaveJsonLdToFileAsync(OutputJsonLdPath);}publicstaticasyncTaskGenerateAfterAiExtractionAsync(IChatClientchatClient){varpipeline=newMarkdownKnowledgePipeline(newMarkdownKnowledgePipelineOptions{BaseUri=newUri("https://kb.example/"),ChatClient=chatClient,ChatModelId="host-selected-model",ExtractionMode=MarkdownKnowledgeExtractionMode.ChatClient,ExtractionCache=newFileKnowledgeExtractionCache("/absolute/path/to/extraction-cache"),});varresult=awaitpipeline.BuildFromDirectoryAsync(MarkdownRoot,searchPattern:MarkdownPattern);awaitresult.Graph.SaveJsonLdToFileAsync(OpaqueJsonLdPath);}publicstaticasyncTaskLoadExternalPreprocessedJsonLdAsync(){vargraph=awaitKnowledgeGraph.LoadJsonLdFromFileAsync(ExternalJsonLdPath);varprofile=newKnowledgeGraphSchemaSearchProfile{Prefixes=newDictionary<string,string>(StringComparer.Ordinal){["ex"]="https://kb.example/vocab/",},TypeFilters=["ex:Capability"],TextPredicates=[newKnowledgeGraphSchemaTextPredicate("schema:name",Weight:1.2d),newKnowledgeGraphSchemaTextPredicate("ex:intent",Weight:1.5d),newKnowledgeGraphSchemaTextPredicate("skos:prefLabel",Weight:1.1d),],RelationshipPredicates=[newKnowledgeGraphSchemaRelationshipPredicate("ex:requires",["ex:symptom","skos:prefLabel"],Weight:0.9d),],};varmatches=awaitgraph.SearchBySchemaAsync("restore cache",profile);Console.WriteLine(matches.Matches.Count);Console.WriteLine(matches.GeneratedSparql);}}

When another system performs preprocessing for you, its JSON-LD must be parseable RDF. SPARQL can query any RDF shape the file contains. For application search, define a KnowledgeGraphSchemaSearchProfile that names the exact RDF types, literal predicates, relationship predicates, and expansion predicates your JSON-LD emits. SearchAsync stays available as a compatibility helper, but it is intentionally sparse and should not be the main search strategy for custom schemas.

For directory imports, LoadFromDirectoryAsync can merge .ttl, .jsonld, .json, .rdf, .xml, .nt, .n3, .trig, and .nq files. For a single JSON-LD payload with an opaque file name or object-storage key, prefer LoadJsonLdFromFileAsync or LoadJsonLdFromStoreAsync so format selection does not depend on extension inference.

Loaded JSON-LD graphs can also participate in federated SPARQL as local service bindings. This is useful when each preprocessing job emits one JSON-LD file and the host wants to query across them without merging first:

usingManagedCode.MarkdownLd.Kb.Pipeline;varpolicyGraph=awaitKnowledgeGraph.LoadJsonLdFromFileAsync("/absolute/path/to/policy.payload");varrunbookGraph=awaitKnowledgeGraph.LoadJsonLdFromFileAsync("/absolute/path/to/runbook.payload");varfederation=newFederatedSparqlExecutionOptions{AllowedServiceEndpoints=[newUri("https://kb.example/services/policy"),newUri("https://kb.example/services/runbook"),],LocalServiceBindings=[newFederatedSparqlLocalServiceBinding(newUri("https://kb.example/services/policy"),policyGraph),newFederatedSparqlLocalServiceBinding(newUri("https://kb.example/services/runbook"),runbookGraph),],};varrows=awaitpolicyGraph.ExecuteFederatedSelectAsync(""" PREFIX schema: <https://schema.org/> SELECT ?policy ?runbook WHERE { SERVICE <https://kb.example/services/policy> { ?policy schema:name ?policyTitle . } SERVICE <https://kb.example/services/runbook> { ?runbook schema:name ?runbookTitle . } } """,federation);Console.WriteLine(rows.Result.Rows.Count);

This local federation path is network-free. Remote federation still uses SPARQL SERVICE calls and must be explicitly allowlisted through FederatedSparqlExecutionOptions or a named profile.

Optional AI Extraction

AI extraction builds graph facts from entities and assertions returned by an injected Microsoft.Extensions.AI.IChatClient. The package stays provider-neutral: it does not reference OpenAI, Azure OpenAI, Anthropic, or any other model-specific SDK. If no chat client is provided, Auto mode extracts no facts and reports a diagnostic; choose Tiktoken mode explicitly for local token-distance extraction.

Chat extraction is chunk-based. The pipeline parses Markdown into deterministic chunks, sends each chunk through the structured extractor in order, and merges the resulting facts into one canonical graph. Optional cache reuse can be enabled through MarkdownKnowledgePipelineOptions.ExtractionCache.

usingManagedCode.MarkdownLd.Kb.Pipeline;usingMicrosoft.Extensions.AI;internalstaticclassAiGraphDemo{privateconststringArticlePath="content/entity-extraction.md";privateconststringArticleMarkdown="""---title: Entity Extraction RDF Pipeline---# Entity Extraction RDF PipelineThe article mentions Markdown-LD Knowledge Bank, SPARQL, RDF, and entity extraction.""";privateconststringAskQuery="""PREFIX schema: <https://schema.org/>ASK WHERE { ?article a schema:Article ; schema:name "Entity Extraction RDF Pipeline" ; schema:mentions ?entity . ?entity schema:name ?name .}""";publicstaticasyncTaskRunAsync(IChatClientchatClient){varpipeline=newMarkdownKnowledgePipeline(chatClient:chatClient);varresult=awaitpipeline.BuildFromMarkdownAsync(ArticleMarkdown,path:ArticlePath);varhasAiFacts=awaitresult.Graph.ExecuteAskAsync(AskQuery);Console.WriteLine(hasAiFacts);}}

The built-in chat extractor requests structured output through GetResponseAsync<T>(), normalizes the returned entity/assertion payload, and then builds the same in-memory RDF graph used by search and SPARQL. Tests use one local non-network IChatClient implementation so the full extraction-to-graph flow is covered without a live model. When cache reuse is enabled, the cache key includes document identity, chunk fingerprints, chunker profile, prompt version, and model identity so stale reuse stays explicit and controllable.

Ranked Search And Cited Answers

Ranked search has four modes:

• Graph — graph-native label, description, and related-label ranking
• Bm25 — in-memory lexical ranking over graph candidate text
• Semantic — optional in-memory semantic index built through IEmbeddingGenerator<string, Embedding<float>>
• Hybrid — graph plus semantic ranking, with default canonical-first ordering or opt-in reciprocal-rank fusion

KnowledgeGraph.SearchRankedAsync works when callers only have an RDF graph, so it ranks graph-native labels, descriptions, and related labels. MarkdownKnowledgeBuildResult.SearchRankedAsync, MarkdownKnowledgeBankBuild.SearchAsync, and cited answers add parsed Markdown chunks to document candidates. Use the build-result or facade API when body-only evidence matters.

varbank=newMarkdownKnowledgeBank(newMarkdownKnowledgeBankOptions{PipelineOptions=newMarkdownKnowledgePipelineOptions{ExtractionMode=MarkdownKnowledgeExtractionMode.None,},ChatClient=chatClient,});varbuild=awaitbank.BuildFromDirectoryAsync("/absolute/path/to/content");varbm25=awaitbuild.SearchAsync("cache restore manifest",newKnowledgeGraphRankedSearchOptions{Mode=KnowledgeGraphSearchMode.Bm25,MaxResults=5,});varanswer=awaitbuild.AnswerAsync("Which runbook restores cache manifests?",newKnowledgeGraphRankedSearchOptions{Mode=KnowledgeGraphSearchMode.Bm25,MaxResults=3,});Console.WriteLine(bm25[0].Label);Console.WriteLine(answer.Answer);Console.WriteLine(answer.Citations[0].SourcePath);

Cited answers use the built Markdown documents and graph matches to create citation snippets, then call IChatClient for the final grounded response. Snippets are bounded and focus around matched query text when evidence appears deep in a chunk. Source scopes intersect any existing candidate-node filter instead of widening it. Duplicate document URIs and graph nodes with multiple entity or assertion provenance sources are resolved by the best available Markdown evidence, so citations point at the useful source document instead of just the first provenance edge. Body snippets can support graph labels through shared description context, but a single weak overlap token is not enough to override stronger label evidence. Follow-up question rewriting is optional and uses caller-supplied conversation messages; the library does not store chat history or own session policy.

Chunk Evaluation And Change Planning

MarkdownKnowledgeBank.EvaluateChunks(...) reports chunk size distribution, expected-answer coverage, and deterministic quality samples. Invalid threshold ranges and empty coverage expectations fail explicitly so evaluation reports do not silently overstate quality. PlanChanges(...) compares source fingerprints before a build so host adapters can skip expensive downstream work without adding an indexer, database, or background service to the core library. Chunk overlap is opt-in through MarkdownChunkingOptions.ChunkOverlapTokenTarget; overlap copies whole trailing blocks into the next chunk and leaves the default chunker non-overlapping. Chunk budgeting treats Han ideographs, Japanese kana, Korean Hangul, CJK symbols, and fullwidth forms as denser token ranges than Latin text.

varbank=newMarkdownKnowledgeBank();varreport=bank.EvaluateChunks(markdown,"content/runbooks/cache-restore.md",[newMarkdownChunkCoverageExpectation("How do I restore cache?","cache restore verification")]);varoverlapReport=bank.EvaluateChunks(markdown,"content/runbooks/cache-restore.md",options:newMarkdownChunkEvaluationOptions{ParsingOptions=newMarkdownParsingOptions{Chunking=newMarkdownChunkingOptions{ChunkTokenTarget=512,ChunkOverlapTokenTarget=50,},},});varchangeSet=bank.PlanChanges(documents,previousManifest);Console.WriteLine(report.CoverageRate);Console.WriteLine(overlapReport.SizeDistribution.Total);Console.WriteLine(changeSet.ChangedPaths.Count);Console.WriteLine(changeSet.UnchangedPaths.Count);

Local Tiktoken Extraction

usingManagedCode.MarkdownLd.Kb.Pipeline;internalstaticclassTiktokenGraphDemo{privateconststringMarkdown="""The observatory stores telescope images in a cold archive near the mountain lab.River sensors use cached forecasts to protect orchards from frost.""";publicstaticasyncTaskRunAsync(){varpipeline=newMarkdownKnowledgePipeline(extractionMode:MarkdownKnowledgeExtractionMode.Tiktoken);varresult=awaitpipeline.BuildFromMarkdownAsync(Markdown);varmatches=awaitresult.Graph.SearchByTokenDistanceAsync("telescop image archive",newTokenDistanceSearchOptions{EnableFuzzyQueryCorrection=true,});Console.WriteLine(matches[0].Text);}}

Tiktoken mode uses Microsoft.ML.Tokenizers to encode section/paragraph text into token IDs, builds normalized sparse vectors, and calculates token-distance ranking from cached squared magnitudes and dot products. The default weighting is SubwordTfIdf, fitted over the current build corpus and reused for query vectors. TermFrequency uses raw token counts, and Binary uses token presence/absence.

SearchByTokenDistanceAsync keeps exact token-distance behavior by default. Pass TokenDistanceSearchOptions with EnableFuzzyQueryCorrection = true when user queries may contain typos. The correction step checks words that are absent from the indexed corpus vocabulary, finds close corpus terms with the bounded edit-distance matcher, appends the best corrections to the query, and only then runs Tiktoken vector search. This improves recall for misspelled words in the query or corpus text while leaving the Tiktoken vector space as the ranking signal.

Tiktoken mode also builds a corpus graph:

• heading or loose document sections and paragraph/line segments become schema:CreativeWork nodes
• local Unicode word n-gram keyphrases become schema:DefinedTerm topic nodes
• explicit front matter entity_hints / entityHints become graph entities with stable hash IDs and preserved sameAs links
• containment uses schema:hasPart
• segment/topic membership uses schema:about
• document/entity-hint membership uses schema:mentions
• segment similarity uses kb:relatedTo

The local lexical design uses subword tokenization plus TF-IDF instead of manually curated tokenization, stop words, or stemming rules. It is designed for same-language lexical retrieval. Cross-language semantic retrieval requires a translation or embedding layer owned by the host application.

The current test corpus validates top-1 token-distance retrieval across English, Ukrainian, French, and German. Same-language queries hit the expected segment at 10/10 for each language in the test corpus. Sampled cross-language aligned hits stay low at 3/40, which matches the lexical design.

Query The Graph

usingManagedCode.MarkdownLd.Kb.Pipeline;internalstaticclassQueryGraphDemo{privateconststringSelectQuery="""PREFIX schema: <https://schema.org/>SELECT ?article ?title WHERE { ?article a schema:Article ; schema:name ?title ; schema:mentions ?entity . ?entity schema:name "RDF" .}LIMIT 100""";privateconststringSearchTerm="sparql";privateconststringArticleKey="article";privateconststringTitleKey="title";publicstaticasyncTaskRunAsync(MarkdownKnowledgeBuildResultresult){varrows=awaitresult.Graph.ExecuteSelectAsync(SelectQuery);varsearch=awaitresult.Graph.SearchBySchemaAsync(SearchTerm);foreach(varrowinrows.Rows){Console.WriteLine(row.Values[ArticleKey]);Console.WriteLine(row.Values[TitleKey]);}Console.WriteLine(search.Matches.Count);}}

SPARQL execution is intentionally read-only. SELECT and ASK are allowed; mutation forms such as INSERT, DELETE, LOAD, CLEAR, DROP, and CREATE are rejected before execution.

Schema-Aware SPARQL Search

Use SearchBySchemaAsync when search must follow a caller-defined RDF/JSON-LD schema instead of the compatibility SearchAsync helper. The profile is compiled into SPARQL, so custom predicates, relationship evidence, type filters, and expansion rules stay explicit.

usingManagedCode.MarkdownLd.Kb.Pipeline;vargraph=awaitKnowledgeGraph.LoadJsonLdFromFileAsync("/absolute/path/to/corpus.jsonld");varprofile=newKnowledgeGraphSchemaSearchProfile{Prefixes=newDictionary<string,string>(StringComparer.Ordinal){["ex"]="https://kb.example/vocab/",},TypeFilters=["ex:Capability"],TextPredicates=[newKnowledgeGraphSchemaTextPredicate("schema:name",Weight:1.2d),newKnowledgeGraphSchemaTextPredicate("ex:intent",Weight:1.5d),newKnowledgeGraphSchemaTextPredicate("skos:prefLabel",Weight:1.1d),],RelationshipPredicates=[newKnowledgeGraphSchemaRelationshipPredicate("ex:requires",["ex:symptom","skos:prefLabel"],Weight:0.9d),],ExpansionPredicates=[newKnowledgeGraphSchemaExpansionPredicate("ex:requires",KnowledgeGraphSchemaSearchRole.Related,Score:0.8d),newKnowledgeGraphSchemaExpansionPredicate("ex:next",KnowledgeGraphSchemaSearchRole.NextStep,Score:0.7d),],};varresult=awaitgraph.SearchBySchemaAsync("restore cache",profile);Console.WriteLine(result.Matches[0].Label);Console.WriteLine(result.Matches[0].Evidence[0].PredicateId);Console.WriteLine(result.GeneratedSparql);

RelationshipPredicates let a source node match because a related node contains the evidence literal, for example ?capability ex:requires ?system and ?system ex:symptom "stale shard checksum". ExpansionPredicates return related and next-step nodes with the local focused graph so the caller can show the smallest useful subgraph around the hit.

For federated schema search, put endpoint URIs in FederatedServiceEndpoints and execute with SearchBySchemaFederatedAsync. The generated query uses explicit SPARQL SERVICE blocks and the same FederatedSparqlExecutionOptions allowlist used by raw federated SPARQL:

varfederatedProfile=profilewith{FederatedServiceEndpoints=[newUri("https://kb.example/services/policy"),newUri("https://kb.example/services/runbook"),],};varfederationOptions=newFederatedSparqlExecutionOptions{AllowedServiceEndpoints=federatedProfile.FederatedServiceEndpoints,};varfederated=awaitgraph.SearchBySchemaFederatedAsync("restore cache",federatedProfile,federationOptions);Console.WriteLine(federated.GeneratedSparql);Console.WriteLine(federated.ServiceEndpointSpecifiers[0]);

Unknown prefixes and missing federated endpoints fail before query execution. See Schema-Aware SPARQL Search for the full JSON-LD preprocessing and federation contract.

Build profiles let graph creation and search travel together:

varpipeline=newMarkdownKnowledgePipeline(newMarkdownKnowledgePipelineOptions{ExtractionMode=MarkdownKnowledgeExtractionMode.None,BuildProfile=newKnowledgeGraphBuildProfile{Name="capability-workflow",BuildOptions=newKnowledgeGraphBuildOptions(),SearchProfile=profile,},});varbuild=awaitpipeline.BuildFromMarkdownAsync(markdown);KnowledgeGraphContractcontract=build.Contract;KnowledgeGraphSchemaDescriptionschema=build.Graph.DescribeSchema(profile.Prefixes);KnowledgeGraphSchemaSearchProfileValidationvalidation=build.Graph.ValidateSchemaSearchProfile(profile);

Use contract.SearchProfile for application search when the graph was built by the pipeline. Use DescribeSchema and ValidateSchemaSearchProfile when JSON-LD was produced by another preprocessing job and you need to verify what it contains before searching.

The supported query surface is intentionally narrow:

• local read-only queries: ExecuteSelectAsync for SELECT and ExecuteAskAsync for ASK
• explicit federated read-only queries: ExecuteFederatedSelectAsync for SELECT and ExecuteFederatedAskAsync for ASK
• unsupported query types: CONSTRUCT, DESCRIBE, and all mutation/update forms

The default public SPARQL contract remains local and in-memory. Local ExecuteSelectAsync / ExecuteAskAsync reject top-level SERVICE clauses. Federated queries are explicit through ExecuteFederatedSelectAsync / ExecuteFederatedAskAsync, require an allowlist or named profile, and currently ship caller-visible endpoint diagnostics through FederatedSparqlSelectResult / FederatedSparqlAskResult.

Cross-endpoint access is expressed with SPARQL SERVICE clauses and endpoint policy stays explicit at the caller boundary. The library ships ready-made profiles for the WDQS main/scholarly split introduced on 9 May 2025:

• FederatedSparqlProfiles.WikidataMain allowlists https://query.wikidata.org/sparql
• FederatedSparqlProfiles.WikidataScholarly allowlists https://query-scholarly.wikidata.org/sparql
• FederatedSparqlProfiles.WikidataMainAndScholarly allowlists both endpoints for multi-endpoint federated queries

usingManagedCode.MarkdownLd.Kb.Pipeline;varfederated=awaitresult.Graph.ExecuteFederatedSelectAsync(""" SELECT ?item WHERE { SERVICE <https://query.wikidata.org/sparql> { ?item ?p ?o } } """,FederatedSparqlProfiles.WikidataMain);Console.WriteLine(federated.ServiceEndpointSpecifiers[0]);

Use ExecuteFederatedAskAsync the same way when the caller needs a read-only federated ASK query instead of a result set.

For deterministic multi-graph federation inside the same process, bind allowlisted endpoint URIs to other in-memory KnowledgeGraph instances:

usingManagedCode.MarkdownLd.Kb.Pipeline;varlocalOptions=newFederatedSparqlExecutionOptions{AllowedServiceEndpoints=[newUri("https://kb.example/services/policy"),newUri("https://kb.example/services/runbook"),],LocalServiceBindings=[newFederatedSparqlLocalServiceBinding(newUri("https://kb.example/services/policy"),policyGraph),newFederatedSparqlLocalServiceBinding(newUri("https://kb.example/services/runbook"),runbookGraph),],};varresult=awaitrootGraph.ExecuteFederatedSelectAsync(""" PREFIX schema: <https://schema.org/> SELECT ?policyTitle ?runbookTitle WHERE { SERVICE <https://kb.example/services/policy> { ?policy schema:name ?policyTitle . } SERVICE <https://kb.example/services/runbook> { ?runbook schema:name ?runbookTitle . } } """,localOptions);

This path still uses SPARQL SERVICE and the same allowlist checks, but it stays fully in-memory and network-free for test fixtures or host-managed multi-graph workflows.

For more complete federation examples, including schema-aware SERVICE generation, ASK checks, failure handling, local binding policy, and remote endpoint profiles, see Federated SPARQL Execution.

Validate With SHACL

usingManagedCode.MarkdownLd.Kb.Pipeline;internalstaticclassShaclValidationDemo{publicstaticvoidRun(MarkdownKnowledgeBuildResultresult){KnowledgeGraphShaclValidationReportreport=result.ValidateShacl();if(!report.Conforms){foreach(varissueinreport.Results){Console.WriteLine(issue.FocusNode);Console.WriteLine(issue.Message);}}Console.WriteLine(report.ReportTurtle);}}

ValidateShacl() uses default Markdown-LD Knowledge Bank shapes backed by dotNetRdf.Shacl. The default shapes validate article names, entity names, schema:sameAs IRIs, provenance IRIs, and assertion confidence metadata when reified assertion metadata is present.

Graph assertions always remain direct RDF edges for existing SPARQL and search callers. Reified assertion metadata is now an explicit throughput trade-off:

• default builds keep only the direct RDF edges, which is the fast path for large Markdown corpora and tokenized graphs
• opt in to RDF reification when the caller needs rdf:Statement metadata with rdf:subject, rdf:predicate, rdf:object, kb:confidence, and optional prov:wasDerivedFrom

Use KnowledgeGraphBuildOptions.IncludeAssertionReification = true when assertion-level provenance and confidence triples must be queryable:

varpipeline=newMarkdownKnowledgePipeline(newMarkdownKnowledgePipelineOptions{BaseUri=newUri("https://kb.example/"),BuildOptions=newKnowledgeGraphBuildOptions{IncludeAssertionReification=true,},});

Pass custom Turtle shapes when the host application needs stricter rules:

conststringShapes="""@prefix sh: <http://www.w3.org/ns/shacl#> .@prefix schema: <https://schema.org/> .<urn:shape:ArticleDatePublished> a sh:NodeShape ; sh:targetClass schema:Article ; sh:property [ sh:path schema:datePublished ; sh:minCount 1 ; sh:message "Every Article must have a schema:datePublished." ; ] .""";varreport=result.Graph.ValidateShacl(Shapes);

Invalid values loaded directly through RDF/JSON-LD remain available to SHACL so the report can expose the exact violation. Extracted and graph-rule entity sameAs values pass through graph normalization first: malformed targets, duplicates, and self-links are removed with caller-visible warnings.

Export The Graph

usingManagedCode.MarkdownLd.Kb.Pipeline;internalstaticclassExportGraphDemo{publicstaticasyncTaskRunAsync(MarkdownKnowledgeBuildResultresult){KnowledgeGraphSnapshotsnapshot=result.Graph.ToSnapshot();stringmermaid=result.Graph.SerializeMermaidFlowchart();stringdot=result.Graph.SerializeDotGraph();stringturtle=result.Graph.SerializeTurtle();stringjsonLd=result.Graph.SerializeJsonLd();KnowledgeGraphloadedFromJsonLd=KnowledgeGraph.LoadJsonLd(jsonLd);awaitresult.Graph.SaveJsonLdToFileAsync("knowledge-graph.payload");KnowledgeGraphloadedFromFile=awaitKnowledgeGraph.LoadJsonLdFromFileAsync("knowledge-graph.payload");varsearch=awaitloadedFromFile.SearchBySchemaAsync("rdf");Console.WriteLine(snapshot.Nodes.Count);Console.WriteLine(snapshot.Edges.Count);Console.WriteLine(mermaid);Console.WriteLine(dot);Console.WriteLine(turtle.Length);Console.WriteLine(jsonLd.Length);Console.WriteLine(loadedFromJsonLd.TripleCount);Console.WriteLine(search.Matches.Count);}}

ToSnapshot() returns the stable semantic/operator graph used by UI, JSON endpoint, and diagram callers. It removes Tiktoken section, segment, and n-gram topic nodes plus every incident retrieval edge while retaining authored node IDs for relationship navigation. Use ToCompleteSnapshot() only for retrieval diagnostics or callers that deliberately need the complete RDF projection. URI node labels are resolved from schema:name when available, so diagram output is readable by default.

SerializeJsonLd() generates JSON-LD text directly. The explicit SaveJsonLdToFileAsync, SaveJsonLdToStoreAsync, LoadJsonLdFromFileAsync, and LoadJsonLdFromStoreAsync helpers force JSON-LD format even when a storage key does not have a .jsonld extension. Loaded JSON-LD becomes a normal in-memory KnowledgeGraph, so SPARQL and search APIs work the same way as they do on the original graph.

Focused graph snapshots can also be exported directly:

varsearch=awaitresult.Graph.SearchBySchemaAsync("rdf",result.Contract.SearchProfile);stringfocusedJsonLd=search.FocusedGraph.SerializeJsonLd();stringfocusedTurtle=search.FocusedGraph.SerializeTurtle();stringfocusedMermaid=search.FocusedGraph.SerializeMermaidFlowchart();stringfocusedDot=search.FocusedGraph.SerializeDotGraph();

Example Mermaid output shape:

graph LR
n0["Zero Cost Knowledge Graph"]
n1["RDF"]
n0 -->|"schema:mentions"| n1

Loading

Example DOT output shape:

digraphKnowledgeGraph {
rankdir=LR;
"n0" [label="Zero Cost Knowledge Graph"];
"n1" [label="RDF"];
"n0"->"n1" [label="schema:mentions"];
}

Thread Safety

KnowledgeGraph is safe for shared in-memory read/write use through its public API. Search, read-only SPARQL, snapshot export, diagram serialization, and RDF serialization run under a read lock; MergeAsync snapshots a built graph and merges it under a write lock.

Use this when many workers convert Markdown independently and publish their results into one graph:

varshared=awaitpipeline.BuildFromMarkdownAsync(string.Empty);varnext=awaitpipeline.BuildFromMarkdownAsync(markdown,path:"content/note.md");awaitshared.Graph.MergeAsync(next.Graph);varsearch=awaitshared.Graph.SearchBySchemaAsync("rdf");

Key Types

Type	Purpose
MarkdownKnowledgeBank	Recommended facade for build, change planning, chunk evaluation, ranked search, optional semantic indexing, and cited answers.
MarkdownKnowledgeBankBuild	Build-session wrapper with Graph, SearchAsync(...), AnswerAsync(...), and BuildSemanticIndexAsync(...).
MarkdownKnowledgePipeline	Lower-level pipeline. Orchestrates parsing, extraction, merge, and graph build.
MarkdownKnowledgeBuildResult	Holds Documents, Facts, and the built Graph.
KnowledgeGraph	In-memory dotNetRDF graph with query, search, SHACL validation, export, and merge.
KnowledgeGraphSnapshot	Immutable view with Nodes (KnowledgeGraphNode) and Edges (KnowledgeGraphEdge).
KnowledgeGraphContract	Build-result contract with actual schema description, search profile, and validation diagnostics.
KnowledgeGraphBuildProfile	Pipeline-level bundle of build options, search profile, and optional SHACL shapes.
KnowledgeGraphSchemaDescription	RDF shape summary with types, predicates, literal predicates, and resource predicates.
KnowledgeGraphShaclValidationReport	SHACL conformance result with flattened issues and Turtle report output.
KnowledgeGraphShaclValidationIssue	Caller-readable SHACL result fields such as focus node, path, value, severity, and message.
MarkdownDocument	Pipeline parsed document: FrontMatter, Body, and Sections.
MarkdownFrontMatter	Typed front matter model used by the low-level Markdown parser.
KnowledgeExtractionResult	Merged collection of KnowledgeEntityFact and KnowledgeAssertionFact.
SparqlQueryResult	Query result with Variables and Rows of SparqlRow.
KnowledgeGraphSchemaSearchProfile	Schema-aware SPARQL search profile with prefixes, predicates, type filters, expansions, federation endpoints, and limits.
KnowledgeGraphSchemaSearchResult	Schema-aware search result with matches, evidence, focused graph, generated SPARQL, and optional federated endpoint diagnostics.
KnowledgeGraphSchemaSearchEvidence	Explanation record naming the predicate, matched text, related node, relationship predicate, score, and service endpoint.
KnowledgeGraphRankedSearchOptions	Ranked search options for graph, BM25, semantic, and hybrid reciprocal-rank retrieval.
KnowledgeAnswerRequest	Cited answer request with question, optional conversation history, ranked-search options, semantic index, and source filters.
KnowledgeAnswerCitation	Caller-visible citation with document URI, source path, heading path, snippet, match label, score, and search source.
MarkdownChunkEvaluator	Deterministic chunk-size, expected-answer coverage, and review-sample helper.
KnowledgeGraphSourceManifest	Source fingerprint manifest and changed/unchanged/removed path planner.
KnowledgeSourceDocumentConverter	Converts files and directories into pipeline-ready source documents.
ChatClientKnowledgeFactExtractor	AI extraction adapter behind IChatClient.
TiktokenKnowledgeGraphOptions	Options for explicit Tiktoken token-distance extraction.
TokenVectorWeighting	Local token weighting mode: SubwordTfIdf, TermFrequency, or Binary.
TokenDistanceSearchOptions	Token-distance search options, including opt-in fuzzy query correction.
TokenDistanceSearchResult	Search result returned by SearchByTokenDistanceAsync.

Markdown Conventions

---title: Markdown-LD Knowledge Bankdescription: A Markdown knowledge graph note.datePublished: 2026-04-11tags:
- markdown
- rdfauthor:
- Ada Lovelaceabout:
- Knowledge Graph---# Markdown-LD Knowledge Bank
Use [RDF](https://www.w3.org/RDF/) and [SPARQL](https://www.w3.org/TR/sparql11-query/).

Recognized front matter keys:

Key	RDF property	Type
title	schema:name	string
description / summary	schema:description	string
datePublished	schema:datePublished	string (ISO date)
dateModified	schema:dateModified	string (ISO date)
author	schema:author	string or list
tags / keywords	schema:keywords	list
about	schema:about	list
entryType / entry_type	compatibility metadata plus optional additional schema.org article subtype typing	string or list
sourceProject / source_project	kb:sourceProject	string or list
canonicalUrl / canonical_url	low-level Markdown parser document identity; use KnowledgeDocumentConversionOptions.CanonicalUri for pipeline identity	string (URL)
entity_hints / entityHints	explicit graph entities in Tiktoken mode; parsed as front matter metadata otherwise	list of {label, type, sameAs}

Generic RDF front matter mapping is also supported for richer document metadata beyond article-only defaults:

Key	Purpose	Type
rdf_prefixes / rdfPrefixes	additional vocabulary prefixes	object
rdf_types / rdfTypes	additional RDF types for the document node	string or list
rdf_properties / rdfProperties	arbitrary predicate/value mappings for the document node	object

Example:

rdf_prefixes:
dcterms: http://purl.org/dc/terms/skos: http://www.w3.org/2004/02/skos/core#rdf_types:
- schema:HowTo
- skos:ConceptSchemerdf_properties:
schema:isPartOf:
id: https://example.com/projects/ai-memexdcterms:issued:
value: 2026-04-21datatype: xsd:dateskos:prefLabel: Flexible Graph Spec

Scalar values become literals by default. Object values may use id to emit a URI node or value plus optional datatype to emit a typed literal. Unknown prefixes fail explicitly instead of being silently guessed.

Predicate normalization for explicit chat/token facts:

• mentions becomes schema:mentions
• about becomes schema:about
• author becomes schema:author
• creator becomes schema:creator
• sameas becomes schema:sameAs
• relatedTo becomes kb:relatedTo
• prefixed predicates such as schema:mentions, kb:relatedTo, prov:wasDerivedFrom, and rdf:type are preserved
• absolute predicate URIs are preserved when valid

Markdown links, wikilinks, and arrow assertions are not implicitly converted into graph facts. Use IChatClient extraction or explicit Tiktoken mode when you want body content to produce graph nodes and edges.

Architecture Choices

• Markdig parses Markdown structure.
• YamlDotNet parses front matter.
• dotNetRDF builds the RDF graph, runs local SPARQL, and serializes Turtle/JSON-LD.
• Schema-aware search compiles caller profiles into local or federated SPARQL and keeps generated queries/evidence visible to callers.
• Ranked search can use graph-native ranking, in-memory BM25, optional fuzzy BM25 token matching, optional semantic ranking, or hybrid reciprocal-rank fusion.
• Exact BM25 counts selected query terms with span-based lookup and pooled per-query statistics; fuzzy BM25 stays opt-in because it must enumerate typo candidates.
• Cited answers use IChatClient plus ranked graph retrieval and return source citations without storing conversation history.
• Chunk evaluation and source-change planning are deterministic local helpers, not hosted indexing services.
• dotNetRdf.Shacl validates built graphs with default or caller-supplied SHACL shapes.
• Microsoft.Extensions.AI.IChatClient is the only AI boundary in the core pipeline.
• The production source tree is organized by feature slices: Documents, Extraction, Pipeline, Graph, Tokenization, Query, and Rdf.
• Microsoft.ML.Tokenizers powers the explicit Tiktoken token-distance mode.
• Subword TF-IDF is the default local token weighting because it downweights corpus-common tokens without adding language-specific preprocessing or model runtime dependencies.
• Local topic graph construction uses Unicode word n-gram keyphrases and RDF schema:DefinedTerm, schema:hasPart, and schema:about edges.
• Embeddings are not required for the core graph/search flow; optional semantic ranking uses IEmbeddingGenerator<string, Embedding<float>> supplied by the host.
• Microsoft Agent Framework is treated as host-level orchestration, not a core package dependency.

Algorithm References

• Optional fuzzy lexical matching is shared by BM25 typo-tolerant ranking and Tiktoken fuzzy query correction. It uses bounded edit distance with common-affix trimming, stack-backed bit-vector masks for short residual tokens, and a pooled bounded banded dynamic-programming fallback for longer residual tokens. It is not a naive full-matrix Levenshtein implementation and does not use platform-specific SIMD intrinsics.
• The bit-vector path is guided by Gene Myers, "A fast bit-vector algorithm for approximate string matching based on dynamic programming", Journal of the ACM, 1999, DOI: https://doi.org/10.1145/316542.316550.
• The bounded-threshold behavior is guided by Esko Ukkonen, "Algorithms for approximate string matching", Information and Control, 1985, DOI: https://doi.org/10.1016/S0019-9958(85)80046-2.
• Thanks to biegehydra/MyersBitParallelDotnet for inspiring the practical direction we took for fast short-token typo matching.

See docs/Architecture.md, ADR-0001, ADR-0002, ADR-0003, ADR-0006, Graph Runtime Lifecycle, Graph Creation Contracts, JSON-LD Graph Round Trip, Schema-Aware SPARQL Search, Graph SHACL Validation, Federated SPARQL Execution, and Performance Benchmarks.

Development

dotnet restore MarkdownLd.Kb.slnx
dotnet build MarkdownLd.Kb.slnx --configuration Release --no-restore
dotnet test --solution MarkdownLd.Kb.slnx --configuration Release
dotnet format MarkdownLd.Kb.slnx --verify-no-changes
dotnet test --solution MarkdownLd.Kb.slnx --configuration Release -- --coverage --coverage-output-format cobertura --coverage-output "$PWD/TestResults/TUnitCoverage/coverage.cobertura.xml" --coverage-settings "$PWD/CodeCoverage.runsettings"

Coverage is collected through Microsoft.Testing.Extensions.CodeCoverage. Cobertura is the XML output format used for line and branch reporting; the test project does not reference Coverlet.

BenchmarkDotNet performance runs are separate from TUnit correctness tests. Commands, workload profiles, profiler options, and full result tables live in Performance Benchmarks. The build/test/pack validation job stays separate; PR validation, release validation, and the dedicated benchmark workflow run the complete BenchmarkDotNet suite as parallel suite jobs and upload suite-specific benchmarkdotnet-results-* artifacts.

Current local headline numbers from the May 4, 2026 BenchmarkDotNet 0.15.8 run on Apple M2 Pro with .NET 10.0.5:

Area	Current local result
Full suite	118 BenchmarkDotNet cases using the Required job; local sequential pass completed in 5 minutes 41 seconds (real 341.12s)
Graph build	LargeCorpus builds in 151.12 ms with 58.75 MB allocated
Low-latency search	ShortDocuments exact ranked graph search is 1.143 ms / 2.15 MB; BM25 is 2.503 ms / 2.14 MB
Typo-tolerant search	BM25 fuzzy stays opt-in; ShortDocuments typo fuzzy search is 7.366 ms / 2.77 MB
RDF query paths	ShortDocuments exact schema SPARQL is 94.422 ms / 61.25 MB; local federated schema search is 92.469 ms / 63.24 MB
Tiktoken search	LongDocuments exact token-distance search is 145.52 μs / 107.23 KB; typo correction is 250.10 μs / 110.17 KB
Persistence	LargeCorpus Turtle file load is 118.305 ms / 28.10 MB; JSON-LD file load is 163.824 ms / 75.51 MB
Lifecycle	Build/search/save/load/export is 154.3 ms / 53.7 MB
Fuzzy edit distance	Long insertion is 381.38x faster than naive Levenshtein; long no-match is 178.03x faster, both with 0 B allocated

These numbers are local diagnostics, not a cross-machine performance contract.