Make all docstrings Sphinx readable

[rgerkin]

This will allow the auto-generated API documentation to contain more information.

[rgerkin]

@appukuttan-shailesh This is the issue I was referring to in the call. We can coordinate here unless you have an open issue that you would prefer to use. I guess the first thing to do is coordinate on a docstring format? What does HBP use internally?

[appukuttan-shailesh]

@rgerkin : Sure, we can use ticket this to coordinate our ideas.
The HBP doesn't enforce a particular format, but we have generally employed numpydoc docstring format. I tend to prefer it as 'NumPy style tends be easier to read for long and in-depth docstrings' (source).

sphinx.ext.napoleon is a handy sphinx extension for use with this format.

Like I mentioned on the call, I have been thinking of making a custom Sphinx extension to help streamline the documentation of sciunit based validation test units. From what I have explored recently, this might not be very trivial a task (mainly because I lack any experience with this). But I still intend to give it a go, and see if it seems worth pursuing. Shall keep you posted here (though progress might be a bit slow).

[rgerkin]

What do you mean by "streamline the documentation of sciunit based validation test units."

I prefer Google docstring style -- do you think it will be important that we all use the same format?

Would it be helpful to use type hints? I could imagine that these could be exploited to avoid specifying expected types in the docstrings themselves, as well as for unit testing and other purposes.

[appukuttan-shailesh]

My intention is to have a Sphinx extension managing SciUnit entities such as Capabilities, Test, Test Suites, Models, Scores etc by adding SciUnit specific 'directives'. For lack of a better example at the moment... something similar to sphinxcontrib.httpdomain that make it simpler and more efficient to document REST APIs.

I don't think it is necessary to enforce a single docstring style, and it might actually be wise to allow developers that freedom of choice. AFAIK, the compiled output would appear the same, and that's what matters.

I was actually not aware of 'type hints', and so started reading up on it following your post. I do agree that it might be useful. Sphinx certainly can exploit this via extensions such as https://pypi.org/project/sphinx-autodoc-typehints/. Let's go ahead with this.

[rgerkin]

On Thursday we will have a lab hackathon and one or more of us will work on updating our docstrings. There are also some attributes that could be used with some of the directives you propose, for example I have attributes for observation schema (defines what the observation should look like), params_schema (defines what kind of parameters the test can use) and default_params (default values for those parameters). The first two of these are formatted for use with cerberus, and so they are proper schemas. Presumably it should be possible to auto-generate something for them.

[rgerkin]

@appukuttan-shailesh If you aren't doing anything late Thursday (in your time zone; we will start Thursday morning in ours) you should join us.

[appukuttan-shailesh]

Sounds interesting! Can you tell me about the schedule for Thursday? We are in the middle of pretty disruptive transportation protests here in Paris, and so very few options for commuting to work. It gets particularly bad on the way back. So I will try to join in, maybe atleast for a bit, based on the timing.

[rgerkin]

We will be working from 9am-4pm in our time zone, which is 5pm-midnight for you. No set schedule yet, as we usually focus on a few issues (like this one) and discuss them and then start working. If you are around near the beginning maybe we could have a Skype or Hangouts call to discuss.

[appukuttan-shailesh]

Sure, sounds good! I think I can be around for the first 1-1.5 hours at the least. We could use hangouts like for our Monday conference calls.

[appukuttan-shailesh]

Jotting down my thoughts on this...

We have an individual page each for the following (taking neuronunit as example):

• all tests
• all capabilities
• all scores

Each test entry should link to its associated entries on the capabilities and scores pages. E.g.

We should draft a minimal doc schema for each of the three entities. The following is to be updated and is just a preliminary sketch:

Example doc for test:

Test: InputResistanceTest

Description: Test the input resistance of a cell.

Score-Type: 			Z-Score

Required Capabilites: 		ProducesMembranePotential
				ReceivesSquareCurrent

Example doc for capability:

Capability: ReceivesSquareCurrent

Description: Indicate that somatic current can be injected into the model as a square pulse.

Return data type: 		dict
Return data structure:		{
					'amplitude' : -10.0*pq.pA,
			                         'delay' : 100*pq.ms,
			                         'duration' : 500*pq.ms}
				}
				where 'pq' is the quantities package

Potential issues (in random order):

• some capabilities and scores may actually be imported from other packages (e.g. ZScore from sciunit being used in neuronunit); we naturally want to encourage such re-use. Might have to explore how to "borrow" documentation for modules from other packages within our package.

[rgerkin]

@appukuttan-shailesh @ChihweiLHBird
Shailesh, allow me to introduce Zhiwei, who was sitting with me during our call at the hackathon.

So my current understanding, based on the last HBP call, is that you intend to build a sphinx extension which will create new docstrings from a combination of A) original docstrings in the .py files and B) automatic analysis of attributes, methods, etc. of the corresponding classes. These new docstrings would not be inserted into code anywhere, but they would be the basis of the html documentation. So someone going to e.g. readthedocs would see the new, rich docstring, while a developer working with the code would still see the simpler docstring in (A).

If that is so, then we should try to define what belongs in (A), and assume that everything else can either be computed by (B), or is unimportant. In some cases, something might in principle be computable by (B), but we would still want a developer browsing the code to see it in (A) anyway, because it so essential. So we could further break this down into:

1. Things that must go into the original docstring, because only humans could write them (e.g. plain language descriptions of things).
2. Things that should go into the original docstring (even though they can be computed from code), because developers deserve to see them while browsing code in an IDE.
3. Things that should not go into original docstring, because the developer does not need to see it in their IDE (and can rely on the html documentation instead), but should be inserted into the new docstring programmatically by the extension. Some of these may require new conventions (like type hints).
4. Things that should not go into the original docstring or be inserted into the new docstring.

If we populate these 4 categories with various kinds of information, that should tell us what the scope of the extension should be, provides guidelines for writing SciUnit classes, and provides guidelines for filling in manual docstrings.

[appukuttan-shailesh]

That's a good idea. I agree that the above setup will help in identifying what needs to be documented, and also how and where.

You are correct about:

create new docstrings from a combination of A) original docstrings in the .py files and B) automatic analysis of attributes, methods, etc. of the corresponding classes. These new docstrings would not be inserted into code anywhere, but they would be the basis of the html documentation. So someone going to e.g. readthedocs would see the new, rich docstring, while a developer working with the code would still see the simpler docstring in (A).

But after exploring a bit, I think we might infact not have to undertake the more arduous task of developing a Sphinx extension, but rather can use the concept of metaclass in Python to achieve the same. Andrew has used this recently for another task, and I am now trying to adapt it to our requirement. I should be able to have a prototype of the proposed implementation by early next week at the latest. I would be away for a few weeks after Christmas (but I will be able to keep track of this ticket from time to time). @ChihweiLHBird could possibly extend this implementation, if he is around during this period. Once I have the prototype ready, I could propose what parts he could start with.

[appukuttan-shailesh]

Creating a separate post to classify the documentation bits into categories you proposed above.

Type A: Things that must go into the original docstring, because only humans could write them (e.g. plain language descriptions of things).

Type B: Things that should go into the original docstring (even though they can be computed from code), because developers deserve to see them while browsing code in an IDE.

Type C: Things that should not go into original docstring, because the developer does not need to see it in their IDE (and can rely on the html documentation instead), but should be inserted into the new docstring programmatically by the extension. Some of these may require new conventions (like type hints).

Type D: Things that should not go into the original docstring nor be inserted into the new docstring.

Unclassified: Temporary list to hold parameters that we are undecided about the above category (just to ensure we don't omit any parameter that we come across).

Also, we probably should identify the main entities that require to be documented. I can list: Test, Capability, Score. Let me know if I have missed out on any. As these entities will have different documentation requirements, it might be handy to handle each separately.

---

Test

Type A:

• single line description
• paragraph description

Type B:

• <>

Type C:

• required_capabilities
• score_type

Type D:

• <>

Unclassified:

• <>

---

Capability

Type A:

• single line description
• paragraph description

Type B:

• <>

Type C:

• <>

Type D:

• <>

Unclassified:

• <>

---

Score

Type A:

• single line description
• paragraph description

Type B:

• <>

Type C:

• <>

Type D:

• <>

Unclassified:

• <>

---

I believe you would be able to edit this post to add info.
(feel free to change to a better system if you find appropriate)