ARROW-13637: [Python] Fix docstrings by amol- · Pull Request #11245 · apache/arrow

amol- · 2021-09-27T12:54:27Z

Address all docstrings to make sure they pass archery numpydoc --allow-rule PR01

github-actions · 2021-09-27T12:54:48Z

https://issues.apache.org/jira/browse/ARROW-13637

jorisvandenbossche

Thanks for your effort here! Added a few comments

jorisvandenbossche · 2021-09-28T10:35:41Z

+ {0} : optional
+ Parameter for {1} constructor. Either `options`
+ or `{0}` can be passed, but not both at the same time.
+ """.format(p.name, option_class.__name__))


Nice! (now we should still have a way to automatically add actual explanation of the keyword, but that's for another JIRA :))

See https://issues.apache.org/jira/browse/ARROW-10317

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

jorisvandenbossche

Looked at the additional commits

jorisvandenbossche · 2021-09-30T08:24:19Z

+ num_record_batches : number of record batches.
+ num_dictionary_batches : number of dictionary batches.
+ num_dictionary_deltas : delta of dictionaries.
+ num_replaced_dictionaries : number of replaced dictionaries.


In practice those are never created by the user, so not sure how useful such a docstring is (except for passing the check ..)

Although those can of course still be accessed by the user (so strictly speaking it might actually be more logical to list those as "Attributes" instead of "Parameters", but OK :))

pitrou · 2021-09-30T10:02:23Z

 the size of individual record batches or table chunks.
 Minimum valid value for block size is 1
- skip_rows: int, optional (default 0)
+ skip_rows: int, optional (default 0)


Just for the record, did you have to do these by hand or is there a mechanical fixer?

Did those by hand, I wanted to check what issues the docstrings had and thus had to look at them anyway. At that point fixing them was a minor deal.
If it doesn't already exist I guess that an automation to fix them wouldn't be too hard to build, but given that @kszucs has a PR to prevent broken/missing docstrings from happening again I hope this is the only time we have to do this work.

@amol- yes, #7732 should be ready depending on which rules do we want to enable at first.

pitrou · 2021-09-30T12:53:13Z

+ Range of the rows,
+ The i-th row spans from `indptr[i]` to `indptr[i+1]` in the data.
+ indices : numpy.ndarray
+ Column indices of the corresponding non-zero values.


It seems you have copy-pasted all this from the previous from_numpy methods, but it doesn't really correspond to what's expected. Below you can see that these are lists of ndarrays (CSF is a n-dimensional generalization of the idea behind CSR and CSC, there's an explanation here: https://www.boristhebrave.com/2021/01/01/compressed-sparse-fibers-explained/).

We already have some text here https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs#L162-L200

You are right, I copy/pasted from CSR / CSC and didn't notice in this case was a multidimensional Tensor.
I would have expected that for CSF Tensor the arguments were multiple arrays but they still are indptr+indices which confused me, I'll edit the docstring accordingly.

I tweaked the docstrings to match what we have in https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs
Hopefully this should do

pitrou · 2021-09-30T12:55:27Z

+ shape : tuple
+ Shape of the matrix.
+ axis_order : list, optional
+ The order of the axis.


I would say for example "The dimensions corresponding to each array in indptr and indices"

Changed docstring to match what we have in https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

Address all docstrings to make sure they pass `archery numpydoc --allow-rule PR01` Closesapache#11245 from amol-/ARROW-13637 Authored-by: Alessandro Molina <amol@turbogears.org> Signed-off-by: Antoine Pitrou <antoine@python.org>

amol- · 2021-09-27T12:54:27Z

Address all docstrings to make sure they pass archery numpydoc --allow-rule PR01

github-actions · 2021-09-27T12:54:48Z

https://issues.apache.org/jira/browse/ARROW-13637

jorisvandenbossche

Thanks for your effort here! Added a few comments

jorisvandenbossche · 2021-09-28T10:35:41Z

+ {0} : optional
+ Parameter for {1} constructor. Either `options`
+ or `{0}` can be passed, but not both at the same time.
+ """.format(p.name, option_class.__name__))


Nice! (now we should still have a way to automatically add actual explanation of the keyword, but that's for another JIRA :))

See https://issues.apache.org/jira/browse/ARROW-10317

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

jorisvandenbossche

Looked at the additional commits

jorisvandenbossche · 2021-09-30T08:24:19Z

+ num_record_batches : number of record batches.
+ num_dictionary_batches : number of dictionary batches.
+ num_dictionary_deltas : delta of dictionaries.
+ num_replaced_dictionaries : number of replaced dictionaries.


In practice those are never created by the user, so not sure how useful such a docstring is (except for passing the check ..)

Although those can of course still be accessed by the user (so strictly speaking it might actually be more logical to list those as "Attributes" instead of "Parameters", but OK :))

pitrou · 2021-09-30T10:02:23Z

 the size of individual record batches or table chunks.
 Minimum valid value for block size is 1
- skip_rows: int, optional (default 0)
+ skip_rows: int, optional (default 0)


Just for the record, did you have to do these by hand or is there a mechanical fixer?

Did those by hand, I wanted to check what issues the docstrings had and thus had to look at them anyway. At that point fixing them was a minor deal.
If it doesn't already exist I guess that an automation to fix them wouldn't be too hard to build, but given that @kszucs has a PR to prevent broken/missing docstrings from happening again I hope this is the only time we have to do this work.

@amol- yes, #7732 should be ready depending on which rules do we want to enable at first.

pitrou · 2021-09-30T12:53:13Z

+ Range of the rows,
+ The i-th row spans from `indptr[i]` to `indptr[i+1]` in the data.
+ indices : numpy.ndarray
+ Column indices of the corresponding non-zero values.


It seems you have copy-pasted all this from the previous from_numpy methods, but it doesn't really correspond to what's expected. Below you can see that these are lists of ndarrays (CSF is a n-dimensional generalization of the idea behind CSR and CSC, there's an explanation here: https://www.boristhebrave.com/2021/01/01/compressed-sparse-fibers-explained/).

We already have some text here https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs#L162-L200

You are right, I copy/pasted from CSR / CSC and didn't notice in this case was a multidimensional Tensor.
I would have expected that for CSF Tensor the arguments were multiple arrays but they still are indptr+indices which confused me, I'll edit the docstring accordingly.

I tweaked the docstrings to match what we have in https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs
Hopefully this should do

pitrou · 2021-09-30T12:55:27Z

+ shape : tuple
+ Shape of the matrix.
+ axis_order : list, optional
+ The order of the axis.


I would say for example "The dimensions corresponding to each array in indptr and indices"

Changed docstring to match what we have in https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

Address all docstrings to make sure they pass `archery numpydoc --allow-rule PR01` Closesapache#11245 from amol-/ARROW-13637 Authored-by: Alessandro Molina <amol@turbogears.org> Signed-off-by: Antoine Pitrou <antoine@python.org>

amol- · 2021-09-27T12:54:27Z

Address all docstrings to make sure they pass archery numpydoc --allow-rule PR01

github-actions · 2021-09-27T12:54:48Z

https://issues.apache.org/jira/browse/ARROW-13637

jorisvandenbossche

Thanks for your effort here! Added a few comments

jorisvandenbossche · 2021-09-28T10:35:41Z

+ {0} : optional
+ Parameter for {1} constructor. Either `options`
+ or `{0}` can be passed, but not both at the same time.
+ """.format(p.name, option_class.__name__))


Nice! (now we should still have a way to automatically add actual explanation of the keyword, but that's for another JIRA :))

See https://issues.apache.org/jira/browse/ARROW-10317

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

jorisvandenbossche

Looked at the additional commits

jorisvandenbossche · 2021-09-30T08:24:19Z

+ num_record_batches : number of record batches.
+ num_dictionary_batches : number of dictionary batches.
+ num_dictionary_deltas : delta of dictionaries.
+ num_replaced_dictionaries : number of replaced dictionaries.


In practice those are never created by the user, so not sure how useful such a docstring is (except for passing the check ..)

Although those can of course still be accessed by the user (so strictly speaking it might actually be more logical to list those as "Attributes" instead of "Parameters", but OK :))

pitrou · 2021-09-30T10:02:23Z

 the size of individual record batches or table chunks.
 Minimum valid value for block size is 1
- skip_rows: int, optional (default 0)
+ skip_rows: int, optional (default 0)


Just for the record, did you have to do these by hand or is there a mechanical fixer?

Did those by hand, I wanted to check what issues the docstrings had and thus had to look at them anyway. At that point fixing them was a minor deal.
If it doesn't already exist I guess that an automation to fix them wouldn't be too hard to build, but given that @kszucs has a PR to prevent broken/missing docstrings from happening again I hope this is the only time we have to do this work.

@amol- yes, #7732 should be ready depending on which rules do we want to enable at first.

pitrou · 2021-09-30T12:53:13Z

+ Range of the rows,
+ The i-th row spans from `indptr[i]` to `indptr[i+1]` in the data.
+ indices : numpy.ndarray
+ Column indices of the corresponding non-zero values.


It seems you have copy-pasted all this from the previous from_numpy methods, but it doesn't really correspond to what's expected. Below you can see that these are lists of ndarrays (CSF is a n-dimensional generalization of the idea behind CSR and CSC, there's an explanation here: https://www.boristhebrave.com/2021/01/01/compressed-sparse-fibers-explained/).

We already have some text here https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs#L162-L200

You are right, I copy/pasted from CSR / CSC and didn't notice in this case was a multidimensional Tensor.
I would have expected that for CSF Tensor the arguments were multiple arrays but they still are indptr+indices which confused me, I'll edit the docstring accordingly.

I tweaked the docstrings to match what we have in https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs
Hopefully this should do

pitrou · 2021-09-30T12:55:27Z

+ shape : tuple
+ Shape of the matrix.
+ axis_order : list, optional
+ The order of the axis.


I would say for example "The dimensions corresponding to each array in indptr and indices"

Changed docstring to match what we have in https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

Address all docstrings to make sure they pass `archery numpydoc --allow-rule PR01` Closesapache#11245 from amol-/ARROW-13637 Authored-by: Alessandro Molina <amol@turbogears.org> Signed-off-by: Antoine Pitrou <antoine@python.org>

amol- · 2021-09-27T12:54:27Z

Address all docstrings to make sure they pass archery numpydoc --allow-rule PR01

github-actions · 2021-09-27T12:54:48Z

https://issues.apache.org/jira/browse/ARROW-13637

jorisvandenbossche

Thanks for your effort here! Added a few comments

jorisvandenbossche · 2021-09-28T10:35:41Z

+ {0} : optional
+ Parameter for {1} constructor. Either `options`
+ or `{0}` can be passed, but not both at the same time.
+ """.format(p.name, option_class.__name__))


Nice! (now we should still have a way to automatically add actual explanation of the keyword, but that's for another JIRA :))

See https://issues.apache.org/jira/browse/ARROW-10317

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

jorisvandenbossche

Looked at the additional commits

jorisvandenbossche · 2021-09-30T08:24:19Z

+ num_record_batches : number of record batches.
+ num_dictionary_batches : number of dictionary batches.
+ num_dictionary_deltas : delta of dictionaries.
+ num_replaced_dictionaries : number of replaced dictionaries.


In practice those are never created by the user, so not sure how useful such a docstring is (except for passing the check ..)

Although those can of course still be accessed by the user (so strictly speaking it might actually be more logical to list those as "Attributes" instead of "Parameters", but OK :))

pitrou · 2021-09-30T10:02:23Z

 the size of individual record batches or table chunks.
 Minimum valid value for block size is 1
- skip_rows: int, optional (default 0)
+ skip_rows: int, optional (default 0)


Just for the record, did you have to do these by hand or is there a mechanical fixer?

Did those by hand, I wanted to check what issues the docstrings had and thus had to look at them anyway. At that point fixing them was a minor deal.
If it doesn't already exist I guess that an automation to fix them wouldn't be too hard to build, but given that @kszucs has a PR to prevent broken/missing docstrings from happening again I hope this is the only time we have to do this work.

@amol- yes, #7732 should be ready depending on which rules do we want to enable at first.

pitrou · 2021-09-30T12:53:13Z

+ Range of the rows,
+ The i-th row spans from `indptr[i]` to `indptr[i+1]` in the data.
+ indices : numpy.ndarray
+ Column indices of the corresponding non-zero values.


It seems you have copy-pasted all this from the previous from_numpy methods, but it doesn't really correspond to what's expected. Below you can see that these are lists of ndarrays (CSF is a n-dimensional generalization of the idea behind CSR and CSC, there's an explanation here: https://www.boristhebrave.com/2021/01/01/compressed-sparse-fibers-explained/).

We already have some text here https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs#L162-L200

You are right, I copy/pasted from CSR / CSC and didn't notice in this case was a multidimensional Tensor.
I would have expected that for CSF Tensor the arguments were multiple arrays but they still are indptr+indices which confused me, I'll edit the docstring accordingly.

I tweaked the docstrings to match what we have in https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs
Hopefully this should do

pitrou · 2021-09-30T12:55:27Z

+ shape : tuple
+ Shape of the matrix.
+ axis_order : list, optional
+ The order of the axis.


I would say for example "The dimensions corresponding to each array in indptr and indices"

Changed docstring to match what we have in https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

Address all docstrings to make sure they pass `archery numpydoc --allow-rule PR01` Closesapache#11245 from amol-/ARROW-13637 Authored-by: Alessandro Molina <amol@turbogears.org> Signed-off-by: Antoine Pitrou <antoine@python.org>

amol- · 2021-09-27T12:54:27Z

Address all docstrings to make sure they pass archery numpydoc --allow-rule PR01

github-actions · 2021-09-27T12:54:48Z

https://issues.apache.org/jira/browse/ARROW-13637

jorisvandenbossche

Thanks for your effort here! Added a few comments

jorisvandenbossche · 2021-09-28T10:35:41Z

+ {0} : optional
+ Parameter for {1} constructor. Either `options`
+ or `{0}` can be passed, but not both at the same time.
+ """.format(p.name, option_class.__name__))


Nice! (now we should still have a way to automatically add actual explanation of the keyword, but that's for another JIRA :))

See https://issues.apache.org/jira/browse/ARROW-10317

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

jorisvandenbossche

Looked at the additional commits

jorisvandenbossche · 2021-09-30T08:24:19Z

+ num_record_batches : number of record batches.
+ num_dictionary_batches : number of dictionary batches.
+ num_dictionary_deltas : delta of dictionaries.
+ num_replaced_dictionaries : number of replaced dictionaries.


In practice those are never created by the user, so not sure how useful such a docstring is (except for passing the check ..)

Although those can of course still be accessed by the user (so strictly speaking it might actually be more logical to list those as "Attributes" instead of "Parameters", but OK :))

pitrou · 2021-09-30T10:02:23Z

 the size of individual record batches or table chunks.
 Minimum valid value for block size is 1
- skip_rows: int, optional (default 0)
+ skip_rows: int, optional (default 0)


Just for the record, did you have to do these by hand or is there a mechanical fixer?

Did those by hand, I wanted to check what issues the docstrings had and thus had to look at them anyway. At that point fixing them was a minor deal.
If it doesn't already exist I guess that an automation to fix them wouldn't be too hard to build, but given that @kszucs has a PR to prevent broken/missing docstrings from happening again I hope this is the only time we have to do this work.

@amol- yes, #7732 should be ready depending on which rules do we want to enable at first.

pitrou · 2021-09-30T12:53:13Z

+ Range of the rows,
+ The i-th row spans from `indptr[i]` to `indptr[i+1]` in the data.
+ indices : numpy.ndarray
+ Column indices of the corresponding non-zero values.


It seems you have copy-pasted all this from the previous from_numpy methods, but it doesn't really correspond to what's expected. Below you can see that these are lists of ndarrays (CSF is a n-dimensional generalization of the idea behind CSR and CSC, there's an explanation here: https://www.boristhebrave.com/2021/01/01/compressed-sparse-fibers-explained/).

We already have some text here https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs#L162-L200

You are right, I copy/pasted from CSR / CSC and didn't notice in this case was a multidimensional Tensor.
I would have expected that for CSF Tensor the arguments were multiple arrays but they still are indptr+indices which confused me, I'll edit the docstring accordingly.

I tweaked the docstrings to match what we have in https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs
Hopefully this should do

pitrou · 2021-09-30T12:55:27Z

+ shape : tuple
+ Shape of the matrix.
+ axis_order : list, optional
+ The order of the axis.


I would say for example "The dimensions corresponding to each array in indptr and indices"

Changed docstring to match what we have in https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

Address all docstrings to make sure they pass `archery numpydoc --allow-rule PR01` Closesapache#11245 from amol-/ARROW-13637 Authored-by: Alessandro Molina <amol@turbogears.org> Signed-off-by: Antoine Pitrou <antoine@python.org>

amol- · 2021-09-27T12:54:27Z

Address all docstrings to make sure they pass archery numpydoc --allow-rule PR01

github-actions · 2021-09-27T12:54:48Z

https://issues.apache.org/jira/browse/ARROW-13637

jorisvandenbossche

Thanks for your effort here! Added a few comments

jorisvandenbossche · 2021-09-28T10:35:41Z

+ {0} : optional
+ Parameter for {1} constructor. Either `options`
+ or `{0}` can be passed, but not both at the same time.
+ """.format(p.name, option_class.__name__))


Nice! (now we should still have a way to automatically add actual explanation of the keyword, but that's for another JIRA :))

See https://issues.apache.org/jira/browse/ARROW-10317

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

jorisvandenbossche

Looked at the additional commits

jorisvandenbossche · 2021-09-30T08:24:19Z

+ num_record_batches : number of record batches.
+ num_dictionary_batches : number of dictionary batches.
+ num_dictionary_deltas : delta of dictionaries.
+ num_replaced_dictionaries : number of replaced dictionaries.


In practice those are never created by the user, so not sure how useful such a docstring is (except for passing the check ..)

Although those can of course still be accessed by the user (so strictly speaking it might actually be more logical to list those as "Attributes" instead of "Parameters", but OK :))

pitrou · 2021-09-30T10:02:23Z

 the size of individual record batches or table chunks.
 Minimum valid value for block size is 1
- skip_rows: int, optional (default 0)
+ skip_rows: int, optional (default 0)


Just for the record, did you have to do these by hand or is there a mechanical fixer?

Did those by hand, I wanted to check what issues the docstrings had and thus had to look at them anyway. At that point fixing them was a minor deal.
If it doesn't already exist I guess that an automation to fix them wouldn't be too hard to build, but given that @kszucs has a PR to prevent broken/missing docstrings from happening again I hope this is the only time we have to do this work.

@amol- yes, #7732 should be ready depending on which rules do we want to enable at first.

pitrou · 2021-09-30T12:53:13Z

+ Range of the rows,
+ The i-th row spans from `indptr[i]` to `indptr[i+1]` in the data.
+ indices : numpy.ndarray
+ Column indices of the corresponding non-zero values.


It seems you have copy-pasted all this from the previous from_numpy methods, but it doesn't really correspond to what's expected. Below you can see that these are lists of ndarrays (CSF is a n-dimensional generalization of the idea behind CSR and CSC, there's an explanation here: https://www.boristhebrave.com/2021/01/01/compressed-sparse-fibers-explained/).

We already have some text here https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs#L162-L200

You are right, I copy/pasted from CSR / CSC and didn't notice in this case was a multidimensional Tensor.
I would have expected that for CSF Tensor the arguments were multiple arrays but they still are indptr+indices which confused me, I'll edit the docstring accordingly.

I tweaked the docstrings to match what we have in https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs
Hopefully this should do

pitrou · 2021-09-30T12:55:27Z

+ shape : tuple
+ Shape of the matrix.
+ axis_order : list, optional
+ The order of the axis.


I would say for example "The dimensions corresponding to each array in indptr and indices"

Changed docstring to match what we have in https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

Address all docstrings to make sure they pass `archery numpydoc --allow-rule PR01` Closesapache#11245 from amol-/ARROW-13637 Authored-by: Alessandro Molina <amol@turbogears.org> Signed-off-by: Antoine Pitrou <antoine@python.org>

amol- · 2021-09-27T12:54:27Z

Address all docstrings to make sure they pass archery numpydoc --allow-rule PR01

github-actions · 2021-09-27T12:54:48Z

https://issues.apache.org/jira/browse/ARROW-13637

jorisvandenbossche

Thanks for your effort here! Added a few comments

jorisvandenbossche · 2021-09-28T10:35:41Z

+ {0} : optional
+ Parameter for {1} constructor. Either `options`
+ or `{0}` can be passed, but not both at the same time.
+ """.format(p.name, option_class.__name__))


Nice! (now we should still have a way to automatically add actual explanation of the keyword, but that's for another JIRA :))

See https://issues.apache.org/jira/browse/ARROW-10317

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

jorisvandenbossche

Looked at the additional commits

jorisvandenbossche · 2021-09-30T08:24:19Z

+ num_record_batches : number of record batches.
+ num_dictionary_batches : number of dictionary batches.
+ num_dictionary_deltas : delta of dictionaries.
+ num_replaced_dictionaries : number of replaced dictionaries.


In practice those are never created by the user, so not sure how useful such a docstring is (except for passing the check ..)

Although those can of course still be accessed by the user (so strictly speaking it might actually be more logical to list those as "Attributes" instead of "Parameters", but OK :))

pitrou · 2021-09-30T10:02:23Z

 the size of individual record batches or table chunks.
 Minimum valid value for block size is 1
- skip_rows: int, optional (default 0)
+ skip_rows: int, optional (default 0)


Just for the record, did you have to do these by hand or is there a mechanical fixer?

Did those by hand, I wanted to check what issues the docstrings had and thus had to look at them anyway. At that point fixing them was a minor deal.
If it doesn't already exist I guess that an automation to fix them wouldn't be too hard to build, but given that @kszucs has a PR to prevent broken/missing docstrings from happening again I hope this is the only time we have to do this work.

@amol- yes, #7732 should be ready depending on which rules do we want to enable at first.

pitrou · 2021-09-30T12:53:13Z

+ Range of the rows,
+ The i-th row spans from `indptr[i]` to `indptr[i+1]` in the data.
+ indices : numpy.ndarray
+ Column indices of the corresponding non-zero values.


It seems you have copy-pasted all this from the previous from_numpy methods, but it doesn't really correspond to what's expected. Below you can see that these are lists of ndarrays (CSF is a n-dimensional generalization of the idea behind CSR and CSC, there's an explanation here: https://www.boristhebrave.com/2021/01/01/compressed-sparse-fibers-explained/).

We already have some text here https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs#L162-L200

You are right, I copy/pasted from CSR / CSC and didn't notice in this case was a multidimensional Tensor.
I would have expected that for CSF Tensor the arguments were multiple arrays but they still are indptr+indices which confused me, I'll edit the docstring accordingly.

I tweaked the docstrings to match what we have in https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs
Hopefully this should do

pitrou · 2021-09-30T12:55:27Z

+ shape : tuple
+ Shape of the matrix.
+ axis_order : list, optional
+ The order of the axis.


I would say for example "The dimensions corresponding to each array in indptr and indices"

Changed docstring to match what we have in https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

Address all docstrings to make sure they pass `archery numpydoc --allow-rule PR01` Closesapache#11245 from amol-/ARROW-13637 Authored-by: Alessandro Molina <amol@turbogears.org> Signed-off-by: Antoine Pitrou <antoine@python.org>

amol- · 2021-09-27T12:54:27Z

Address all docstrings to make sure they pass archery numpydoc --allow-rule PR01

github-actions · 2021-09-27T12:54:48Z

https://issues.apache.org/jira/browse/ARROW-13637

jorisvandenbossche

Thanks for your effort here! Added a few comments

jorisvandenbossche · 2021-09-28T10:35:41Z

+ {0} : optional
+ Parameter for {1} constructor. Either `options`
+ or `{0}` can be passed, but not both at the same time.
+ """.format(p.name, option_class.__name__))


Nice! (now we should still have a way to automatically add actual explanation of the keyword, but that's for another JIRA :))

See https://issues.apache.org/jira/browse/ARROW-10317

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

jorisvandenbossche

Looked at the additional commits

jorisvandenbossche · 2021-09-30T08:24:19Z

+ num_record_batches : number of record batches.
+ num_dictionary_batches : number of dictionary batches.
+ num_dictionary_deltas : delta of dictionaries.
+ num_replaced_dictionaries : number of replaced dictionaries.


In practice those are never created by the user, so not sure how useful such a docstring is (except for passing the check ..)

Although those can of course still be accessed by the user (so strictly speaking it might actually be more logical to list those as "Attributes" instead of "Parameters", but OK :))

pitrou · 2021-09-30T10:02:23Z

 the size of individual record batches or table chunks.
 Minimum valid value for block size is 1
- skip_rows: int, optional (default 0)
+ skip_rows: int, optional (default 0)


Just for the record, did you have to do these by hand or is there a mechanical fixer?

Did those by hand, I wanted to check what issues the docstrings had and thus had to look at them anyway. At that point fixing them was a minor deal.
If it doesn't already exist I guess that an automation to fix them wouldn't be too hard to build, but given that @kszucs has a PR to prevent broken/missing docstrings from happening again I hope this is the only time we have to do this work.

@amol- yes, #7732 should be ready depending on which rules do we want to enable at first.

pitrou · 2021-09-30T12:53:13Z

+ Range of the rows,
+ The i-th row spans from `indptr[i]` to `indptr[i+1]` in the data.
+ indices : numpy.ndarray
+ Column indices of the corresponding non-zero values.


It seems you have copy-pasted all this from the previous from_numpy methods, but it doesn't really correspond to what's expected. Below you can see that these are lists of ndarrays (CSF is a n-dimensional generalization of the idea behind CSR and CSC, there's an explanation here: https://www.boristhebrave.com/2021/01/01/compressed-sparse-fibers-explained/).

We already have some text here https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs#L162-L200

You are right, I copy/pasted from CSR / CSC and didn't notice in this case was a multidimensional Tensor.
I would have expected that for CSF Tensor the arguments were multiple arrays but they still are indptr+indices which confused me, I'll edit the docstring accordingly.

I tweaked the docstrings to match what we have in https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs
Hopefully this should do

pitrou · 2021-09-30T12:55:27Z

+ shape : tuple
+ Shape of the matrix.
+ axis_order : list, optional
+ The order of the axis.


I would say for example "The dimensions corresponding to each array in indptr and indices"

Changed docstring to match what we have in https://github.com/apache/arrow/blob/master/format/SparseTensor.fbs

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

Address all docstrings to make sure they pass `archery numpydoc --allow-rule PR01` Closesapache#11245 from amol-/ARROW-13637 Authored-by: Alessandro Molina <amol@turbogears.org> Signed-off-by: Antoine Pitrou <antoine@python.org>

amol- added 7 commits September 24, 2021 11:44

numpydoc fix for _json module
1e9d6ed

numpy doc _csv module
f5584ba

Document types
323a823

Fix compute functions docstring
a92e6aa

Parquet docstrings
758df9d

Address _fs docstrings
62c46cd

Fix pyarrow.ipc docstrings
cefdcf8

github-actionsBot added the Component: Python label Sep 27, 2021

amol- added 2 commits September 27, 2021 15:01

Fix feather and IPC
c5b034d

Document tensor and matrixes
d0d90ac

jorisvandenbossche reviewed Sep 28, 2021
View reviewed changes

amol-and others added 9 commits September 28, 2021 16:51

More docstrings
95bf519

Only compute functions pending
ffb8736

Remaining modules
f5ed74b

Apply suggestions from code review
b6f36ae
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

More docstrings for dataset and fs
472339a

More work on docstrings
0eea359

Last docstrings
e3fa416

review comments
52a48d6

Lint
8461025

amol- marked this pull request as ready for review September 30, 2021 07:57

jorisvandenbossche reviewed Sep 30, 2021
View reviewed changes

pitrou reviewed Sep 30, 2021
View reviewed changes

amol-and others added 6 commits September 30, 2021 15:24

Apply suggestions from code review
e93d044
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

Some review feedback
b282f37

More feedback
de68f95

More feedback
74d5749

Further code review feedback
ee5f3ff

lint
a3a0274

Address docstring for CSFTensor
0aa9141

pitrou closed this in 0dfe592Oct 4, 2021

asfimport mentioned this pull request Oct 4, 2021
[Python][Doc] Make docstrings conform to same style #29276
Closed

amol- added 7 commits September 24, 2021 11:44

numpydoc fix for _json module
1e9d6ed

numpy doc _csv module
f5584ba

Document types
323a823

Fix compute functions docstring
a92e6aa

Parquet docstrings
758df9d

Address _fs docstrings
62c46cd

Fix pyarrow.ipc docstrings
cefdcf8

github-actionsBot added the Component: Python label Sep 27, 2021

amol- added 2 commits September 27, 2021 15:01

Fix feather and IPC
c5b034d

Document tensor and matrixes
d0d90ac

jorisvandenbossche reviewed Sep 28, 2021
View reviewed changes

amol-and others added 9 commits September 28, 2021 16:51

More docstrings
95bf519

Only compute functions pending
ffb8736

Remaining modules
f5ed74b

Apply suggestions from code review
b6f36ae
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

More docstrings for dataset and fs
472339a

More work on docstrings
0eea359

Last docstrings
e3fa416

review comments
52a48d6

Lint
8461025

amol- marked this pull request as ready for review September 30, 2021 07:57

jorisvandenbossche reviewed Sep 30, 2021
View reviewed changes

pitrou reviewed Sep 30, 2021
View reviewed changes

amol-and others added 6 commits September 30, 2021 15:24

Apply suggestions from code review
e93d044
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

Some review feedback
b282f37

More feedback
de68f95

More feedback
74d5749

Further code review feedback
ee5f3ff

lint
a3a0274

Address docstring for CSFTensor
0aa9141

pitrou closed this in 0dfe592Oct 4, 2021

asfimport mentioned this pull request Oct 4, 2021
[Python][Doc] Make docstrings conform to same style #29276
Closed

amol- added 7 commits September 24, 2021 11:44

numpydoc fix for _json module
1e9d6ed

numpy doc _csv module
f5584ba

Document types
323a823

Fix compute functions docstring
a92e6aa

Parquet docstrings
758df9d

Address _fs docstrings
62c46cd

Fix pyarrow.ipc docstrings
cefdcf8

github-actionsBot added the Component: Python label Sep 27, 2021

amol- added 2 commits September 27, 2021 15:01

Fix feather and IPC
c5b034d

Document tensor and matrixes
d0d90ac

jorisvandenbossche reviewed Sep 28, 2021
View reviewed changes

amol-and others added 9 commits September 28, 2021 16:51

More docstrings
95bf519

Only compute functions pending
ffb8736

Remaining modules
f5ed74b

Apply suggestions from code review
b6f36ae
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

More docstrings for dataset and fs
472339a

More work on docstrings
0eea359

Last docstrings
e3fa416

review comments
52a48d6

Lint
8461025

amol- marked this pull request as ready for review September 30, 2021 07:57

jorisvandenbossche reviewed Sep 30, 2021
View reviewed changes

pitrou reviewed Sep 30, 2021
View reviewed changes

amol-and others added 6 commits September 30, 2021 15:24

Apply suggestions from code review
e93d044
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

Some review feedback
b282f37

More feedback
de68f95

More feedback
74d5749

Further code review feedback
ee5f3ff

lint
a3a0274

Address docstring for CSFTensor
0aa9141

pitrou closed this in 0dfe592Oct 4, 2021

asfimport mentioned this pull request Oct 4, 2021
[Python][Doc] Make docstrings conform to same style #29276
Closed

amol- added 7 commits September 24, 2021 11:44

numpydoc fix for _json module
1e9d6ed

numpy doc _csv module
f5584ba

Document types
323a823

Fix compute functions docstring
a92e6aa

Parquet docstrings
758df9d

Address _fs docstrings
62c46cd

Fix pyarrow.ipc docstrings
cefdcf8

github-actionsBot added the Component: Python label Sep 27, 2021

amol- added 2 commits September 27, 2021 15:01

Fix feather and IPC
c5b034d

Document tensor and matrixes
d0d90ac

jorisvandenbossche reviewed Sep 28, 2021
View reviewed changes

amol-and others added 9 commits September 28, 2021 16:51

More docstrings
95bf519

Only compute functions pending
ffb8736

Remaining modules
f5ed74b

Apply suggestions from code review
b6f36ae
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

More docstrings for dataset and fs
472339a

More work on docstrings
0eea359

Last docstrings
e3fa416

review comments
52a48d6

Lint
8461025

amol- marked this pull request as ready for review September 30, 2021 07:57

jorisvandenbossche reviewed Sep 30, 2021
View reviewed changes

pitrou reviewed Sep 30, 2021
View reviewed changes

amol-and others added 6 commits September 30, 2021 15:24

Apply suggestions from code review
e93d044
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

Some review feedback
b282f37

More feedback
de68f95

More feedback
74d5749

Further code review feedback
ee5f3ff

lint
a3a0274

Address docstring for CSFTensor
0aa9141

pitrou closed this in 0dfe592Oct 4, 2021

asfimport mentioned this pull request Oct 4, 2021
[Python][Doc] Make docstrings conform to same style #29276
Closed

amol- added 7 commits September 24, 2021 11:44

numpydoc fix for _json module
1e9d6ed

numpy doc _csv module
f5584ba

Document types
323a823

Fix compute functions docstring
a92e6aa

Parquet docstrings
758df9d

Address _fs docstrings
62c46cd

Fix pyarrow.ipc docstrings
cefdcf8

github-actionsBot added the Component: Python label Sep 27, 2021

amol- added 2 commits September 27, 2021 15:01

Fix feather and IPC
c5b034d

Document tensor and matrixes
d0d90ac

jorisvandenbossche reviewed Sep 28, 2021
View reviewed changes

amol-and others added 9 commits September 28, 2021 16:51

More docstrings
95bf519

Only compute functions pending
ffb8736

Remaining modules
f5ed74b

Apply suggestions from code review
b6f36ae
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

More docstrings for dataset and fs
472339a

More work on docstrings
0eea359

Last docstrings
e3fa416

review comments
52a48d6

Lint
8461025

amol- marked this pull request as ready for review September 30, 2021 07:57

jorisvandenbossche reviewed Sep 30, 2021
View reviewed changes

pitrou reviewed Sep 30, 2021
View reviewed changes

amol-and others added 6 commits September 30, 2021 15:24

Apply suggestions from code review
e93d044
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

Some review feedback
b282f37

More feedback
de68f95

More feedback
74d5749

Further code review feedback
ee5f3ff

lint
a3a0274

Address docstring for CSFTensor
0aa9141

pitrou closed this in 0dfe592Oct 4, 2021

asfimport mentioned this pull request Oct 4, 2021
[Python][Doc] Make docstrings conform to same style #29276
Closed

amol- added 7 commits September 24, 2021 11:44

numpydoc fix for _json module
1e9d6ed

numpy doc _csv module
f5584ba

Document types
323a823

Fix compute functions docstring
a92e6aa

Parquet docstrings
758df9d

Address _fs docstrings
62c46cd

Fix pyarrow.ipc docstrings
cefdcf8

github-actionsBot added the Component: Python label Sep 27, 2021

amol- added 2 commits September 27, 2021 15:01

Fix feather and IPC
c5b034d

Document tensor and matrixes
d0d90ac

jorisvandenbossche reviewed Sep 28, 2021
View reviewed changes

amol-and others added 9 commits September 28, 2021 16:51

More docstrings
95bf519

Only compute functions pending
ffb8736

Remaining modules
f5ed74b

Apply suggestions from code review
b6f36ae
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

More docstrings for dataset and fs
472339a

More work on docstrings
0eea359

Last docstrings
e3fa416

review comments
52a48d6

Lint
8461025

amol- marked this pull request as ready for review September 30, 2021 07:57

jorisvandenbossche reviewed Sep 30, 2021
View reviewed changes

pitrou reviewed Sep 30, 2021
View reviewed changes

amol-and others added 6 commits September 30, 2021 15:24

Apply suggestions from code review
e93d044
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

Some review feedback
b282f37

More feedback
de68f95

More feedback
74d5749

Further code review feedback
ee5f3ff

lint
a3a0274

Address docstring for CSFTensor
0aa9141

pitrou closed this in 0dfe592Oct 4, 2021

asfimport mentioned this pull request Oct 4, 2021
[Python][Doc] Make docstrings conform to same style #29276
Closed

amol- added 7 commits September 24, 2021 11:44

numpydoc fix for _json module
1e9d6ed

numpy doc _csv module
f5584ba

Document types
323a823

Fix compute functions docstring
a92e6aa

Parquet docstrings
758df9d

Address _fs docstrings
62c46cd

Fix pyarrow.ipc docstrings
cefdcf8

github-actionsBot added the Component: Python label Sep 27, 2021

amol- added 2 commits September 27, 2021 15:01

Fix feather and IPC
c5b034d

Document tensor and matrixes
d0d90ac

jorisvandenbossche reviewed Sep 28, 2021
View reviewed changes

amol-and others added 9 commits September 28, 2021 16:51

More docstrings
95bf519

Only compute functions pending
ffb8736

Remaining modules
f5ed74b

Apply suggestions from code review
b6f36ae
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

More docstrings for dataset and fs
472339a

More work on docstrings
0eea359

Last docstrings
e3fa416

review comments
52a48d6

Lint
8461025

amol- marked this pull request as ready for review September 30, 2021 07:57

jorisvandenbossche reviewed Sep 30, 2021
View reviewed changes

pitrou reviewed Sep 30, 2021
View reviewed changes

amol-and others added 6 commits September 30, 2021 15:24

Apply suggestions from code review
e93d044
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

Some review feedback
b282f37

More feedback
de68f95

More feedback
74d5749

Further code review feedback
ee5f3ff

lint
a3a0274

Address docstring for CSFTensor
0aa9141

pitrou closed this in 0dfe592Oct 4, 2021

asfimport mentioned this pull request Oct 4, 2021
[Python][Doc] Make docstrings conform to same style #29276
Closed

amol- added 7 commits September 24, 2021 11:44

numpydoc fix for _json module
1e9d6ed

numpy doc _csv module
f5584ba

Document types
323a823

Fix compute functions docstring
a92e6aa

Parquet docstrings
758df9d

Address _fs docstrings
62c46cd

Fix pyarrow.ipc docstrings
cefdcf8

github-actionsBot added the Component: Python label Sep 27, 2021

amol- added 2 commits September 27, 2021 15:01

Fix feather and IPC
c5b034d

Document tensor and matrixes
d0d90ac

jorisvandenbossche reviewed Sep 28, 2021
View reviewed changes

amol-and others added 9 commits September 28, 2021 16:51

More docstrings
95bf519

Only compute functions pending
ffb8736

Remaining modules
f5ed74b

Apply suggestions from code review
b6f36ae
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

More docstrings for dataset and fs
472339a

More work on docstrings
0eea359

Last docstrings
e3fa416

review comments
52a48d6

Lint
8461025

amol- marked this pull request as ready for review September 30, 2021 07:57

jorisvandenbossche reviewed Sep 30, 2021
View reviewed changes

pitrou reviewed Sep 30, 2021
View reviewed changes

amol-and others added 6 commits September 30, 2021 15:24

Apply suggestions from code review
e93d044
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

Some review feedback
b282f37

More feedback
de68f95

More feedback
74d5749

Further code review feedback
ee5f3ff

lint
a3a0274

Address docstring for CSFTensor
0aa9141

pitrou closed this in 0dfe592Oct 4, 2021

asfimport mentioned this pull request Oct 4, 2021
[Python][Doc] Make docstrings conform to same style #29276
Closed

Uh oh!

Conversation

amol- commented Sep 27, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actionsBot commented Sep 27, 2021

Uh oh!

jorisvandenbossche left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jorisvandenbossche left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amol-Oct 1, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

amol- commented Sep 27, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actionsBot commented Sep 27, 2021

Uh oh!

jorisvandenbossche left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

amol- commented Sep 27, 2021•
edited
Loading

amol-Oct 1, 2021•
edited
Loading

amol- commented Sep 27, 2021•
edited
Loading

amol-Oct 1, 2021•
edited
Loading