Some function should confirm CASE- and UCO-prefixed concepts used in instance data exist

[ajnelson-nist]

Background

One of the issues with using an ontology is that the concept names are prone to being typo'd as programming and example drafting happens. Some mechanism is needed to catch typos.

SHACL does not provide the ability to turn a namespace into a closed concept set. There is an ability to declare a node shape to be sh:closed, but the effect of this is preventing any properties outside of the sh:property-associated properties being used, regardless of namespace. CASE and UCO are choosing to not use the sh:closed property. (One example why: even rdfs:comment would trigger non-conformance.)

This leaves the current state of ontology-encoded enforcement unable to report that use of this concept in instance data is an error:

<https://ontology.unifiedcyberontology.org/uco/core/name_WITH_TYPO>

Hence, checking for concept typos may be delegated from the ontology to a downstream validation mechanism.

One form of solution for this problem is, in the abstract, to run a set-difference between the CASE and UCO prefixed concepts that are in the ontology versus in the instance data. The CASE-Examples-QC repository did this, with a combination of shell scripting and Python. That repository, until the 0.4.0 release of this repository, was tracked as a submodule to import that functionality as part of unit tests.

Unfortunately, tracking that repository as a dependency (i.e. Git submodule) of this repository introduced a circular Git submodule dependency, eventually looping back to this repository. With the 0.4.0 release of case_utils, that submodule is gone, but so is its "IRI typo check" functionality that, with all due irony, helped catch a bug as the CASE 0.6.0 release branch of this repository was being drafted.

Solution Summary

No firm solution is offered quite yet. I have two ideas that could be explored, and I'm not sure offhand if they are mutually exclusive:

1. A RDFLib DefinedNamespace could be used to catch typos at object instantation time within Python.
   • For example, trying to refer to rdflib.RDFS.typo yields the runtime error AttributeError: term 'typo' not in namespace 'http://www.w3.org/2000/01/rdf-schema#'.
   • This might be an unpleasant user experience. A user could be handed CASE data, try to read it, and get a traceback while parsing the graph.
2. An extra step within case_validate might be able to generate SHACL sh:ValidationResults by running an independent test like the CASE-Examples-QC test had been doing.
   • However, it's not clear if this can be "Slipped into" the pySHACL functions that generate the output graph. It might be possible to modify the rdflib.Graph holding the sh:ValidationReport, but it's not clear if the human output could also be influenced.

[ajnelson-nist]

From discussion on the Adoption Committee call this morning, the better detection point for nonexistent concepts seemed to be at data-validation time. Runtime concept review would make it harder to generate incorrect CASE data, but only for adopters of that module.

[ajnelson-nist]

@plbt5 , you said you might have experience with this particular topic: Something needs to be adjusted with the call to pyshacl.validate, here.

[plbt5]

@ajnelson-nist What, exactly, is going wrong there. Can you create a MWE, and in relation to that, what it should be doing in stead?

[ajnelson-nist]

Hm. This ask might be a scope expansion beyond pyshacl.validate.

We need to check for typo'd concepts. This is not something we can do with SHACL. Yet, it would be nice if we could use SHACL's results framework to provide a consistent user experience, treating typos as sh:Violations.

It would be nice if we could extend pyshacl.validate to generate sh:Violation results on encountering a typo'd concept. We can manipulate the graph generated by pyshacl.validate. However, we can't manipulate the human-readable output that is also generated by that function.

Maybe there's a deeper function we can reach into that pyshacl.validate calls, that we could use to re-process the validation results graph? The end framework might look like:

• Run pyshacl.validate, get its results graph.
• Augment that results graph with any typo finds detected by a set-difference of UCO- and CASE-prefixed concepts, from the ontology vs. the instance data.
• Re-run the part of pyshacl.validate that renders the sh:ValidationReport graph into human-readable output, except have it run on our augmented results graph.

Before programming things, what do you think about that approach, particularly any bends of the SHACL spec by making our own sh:Violations?

[plbt5]

First, this is different than with what I have experience with. No problem, just saying.
Next, I think your suggested flow can be rather tedious to achieve. The assumption / investigation objective becomes whether we can (i) find (reverse engineer) and (ii) use the call to render human-readable output. The latter might even be harder than the former, since you might run into dependencies that are required to be fulfilled which we cannot fulfil that easily. If anything, we will be needing the help from the original developers; it'll depend on whether they consider this architecturally allowable and/or sufficiently interesting to put effort into.

Having put some (minimal) thought into this, I wonder why we want to solve the problem with the SHACL tool. The nature of the problem is not about a shape but, due to the typo, about a non-existing IRI. I appreciate your drift to slip this into the case_validate framework, however, if the tool isn't fit for the task, the task becomes more troublesome by using the tool. The one exception to this is the use of a screwdriver to open a can of paint, but I'm not sure the subject problem/tool combination belongs to that exception.

How about using rdflib for this: https://stackoverflow.com/questions/64056183/validating-ttl-elements-with-rdflib

[plbt5]

@ajnelson-nist : How to proceed with this?