Scraper function for sphinx-gallery, in plotly.io by emmanuelle · Pull Request #1577 · plotly/plotly.py

emmanuelle · 2019-05-24T06:49:53Z

This PR proposes the addition of a scraper function in plotly.io, to be used by the sphinx-gallery sphinx extension when a sphinx-based doc contains a gallery of examples. At the moment sphinx-gallery has a scraper for matplotlib figures, but some open-source projects have expressed their interest in having figures with plotly.py instead.

The PR also contains a minimal sphinx doc with two gallery examples (one for 2D, one for 3D). It's not necessary to merge this but I thought that having gallery examples in the code base could be useful.

Note that this PR does not require a sphinx-gallery dependency, because it will be sphinx-gallery which calls the scraper function and not the other way around.

One thing to decide is whether the function used in examples is plot or iplot. Do most users work in notebooks and therefore use iplot? The function is monkey-patched anyway so it's easy to change.

scraper function + skeleton of sphinx doc with minimal gallery (two examples)

emmanuelle · 2019-05-24T11:01:49Z

I'm struggling with installing the orca excutable on the CI optional build. Any idea of how to solve this?

jonmmease · 2019-05-27T15:42:21Z

Hi @emmanuelle, thanks this is very interesting.

A couple of high-level thoughts. Rather than use monkey-patching it would be nice to handle this use-case with a custom renderer (Renderers were introduced in #1474).

I'm also a little hesitant to introduce this logic into the plotly.py repo. Would it make sense to add this to sphinx gallery?

jonmmease · 2019-05-27T15:45:10Z

I'm struggling with installing the orca excutable on the CI optional build. Any idea of how to solve this?

So far, I've had all of the orca-dependent tests run in the orca builds. These builds install orca using conda

emmanuelle · 2019-05-27T21:02:26Z

Thanks for the comments @jonmmease ! Yes, a custom renderer would be nicer, I just read through #1474 and it seems well adapted.

I initially proposed the scraper function to sphinx-gallery as a PR (sphinx-gallery/sphinx-gallery#493) but their policy is that each plotting library should maintain its own scraper, which kinda makes sense since there is a large number of plotting libraries. So if we want that other projects use plotly in their documentation when using sphinx-gallery (it was mentioned on the scikit-learn mailing-list that there is interest for this), I think we have to include it and maintain it...

jonmmease · 2019-05-27T23:07:20Z

Thanks for the background @emmanuelle, having something in the plotly.py repo is fine in that case. Looking over the code, I think we could replace the _patched_plotly_plot logic with a custom renderer.

Would it be possible to get away with only having a custom renderer and not the plotly_sg_scraper and figure_rst logic? Can we make the output of plotly look just like matplotlib so that scraper built into sphinx gallery picks it up? It doesn't look like there's anything in these two functions that's specific to how plotly.py works, so it would be nice to avoid them if possible. Thanks!

emmanuelle · 2019-05-30T13:05:17Z

Thanks @jonmmease. The way sphing-gallery handles matplotlib examples is quite different, because it loops through the list of existing matplotlib figures with plt.get_fignums (https://github.com/sphinx-gallery/sphinx-gallery/blob/master/sphinx_gallery/scrapers.py#L87), and we don't have such a list in plotly.py if I understand well. Therefore for plotly figures what I did instead was to save the figure as files and then move the files to the right location. And the figure_rst looks very much like the figure_rst of sphinx-gallery except that it's html and not a png file so another rst directive is needed. But I could ask the sphinx-gallery team if they would accept a PR for extending their figure_rst function to other formats, they didn't want to maintain the scrapers but a figure_rst function with html support might be ok for them.

emmanuelle · 2019-05-31T11:54:06Z

@jonmmease here is a version with a custom renderer. I can add more documentation and comments but first tell me whether this was what you had in mind. I also changed the examples so that now they call plotly.io.show() since this is the syntax we'll be pushing now.

emmanuelle · 2019-05-31T11:54:45Z

Oh and the tests failure seem to be due to something else than the tests added by this PR.

jonmmease · 2019-06-01T11:54:44Z

+# This module defines an image scraper for sphinx-gallery
+# https://sphinx-gallery.github.io/
+# which can be used by projects using plotly in their documentation.
+# This module monkey-patches plotly.offline.plot, so we do not


We can now remove mention of monkey patching right?

jonmmease · 2019-06-01T11:59:50Z

Thanks for making the renderer update @emmanuelle, this looks nice. I left a couple of comments inline.

But I could ask the sphinx-gallery team if they would accept a PR for extending their figure_rst function to other formats, they didn't want to maintain the scrapers but a figure_rst function with html support might be ok for them.

Sounds like it would be worth asking, as I'd expect this to be useful for other libraries as well. But it's not a lot of logic, so not a huge deal.

Oh and the tests failure seem to be due to something else than the tests added by this PR.

Yeah, this has been fixed on master so if you merge/rebase it should go away.

Regarding the example in doc/conf.py, I think I'd rather have this moved to a separate repository that has only the sphinx gallery example. If that sounds alright, could you move this to a new repo in the plotly org and remove it from this PR? Thanks!

+ changed name of renderer sphinx -> sphinx-gallery, and changed wording of documentation

emmanuelle · 2019-06-02T12:34:49Z

Hurray it's green! I think I addressed all your requests, I moved the sphinx files and gallery examples to plotly/plotly-sphinx-gallery.

emmanuelle · 2019-06-04T14:17:27Z

@jonmmease no hurry but since you're always very reactive, I just wanted to make sure that this hasn't fallen off your radar :-).

jonmmease · 2019-06-04T14:51:02Z

haha, thanks for the ping. I haven't forgotten 🙂 I should be able to take a look tomorrow. BTW, are you fine leaving this for v4 or would you like to get it out into the word before that?

emmanuelle · 2019-06-04T14:57:40Z

v4 is fine, it should be in a month or so right? I'm used to slower development cycles so no pb :-).

jonmmease

This looks great @emmanuelle, a couple of small comments/requests below but otherwise good to go. Thanks!

emmanuelle · 2019-07-02T16:07:56Z

@jonmmease sorry to have let this idle for some time. I think the tests failure are not related to this PR. Shall I try to make a new PR to the v4_integration branch instead?

jonmmease · 2019-07-03T13:59:24Z

Looks, good. Thanks!

jonmmease · 2019-07-04T12:36:43Z

@emmanuelle, could you double check that your sphinx-gallery example works with the plotly==4.0.0a9 alpha release? Thanks!

emmanuelle · 2019-07-05T11:44:29Z

Yes it works! Thank you very much @jonmmease :-)

* scraper function for sphinx-gallery, in plotly.io. scraper function + skeleton of sphinx doc with minimal gallery (two examples) * added psutil dep to tox.ini * python 2 compatibility * added orca dep to package.json for tests * removed typo * corrected bug * python 2 compatibility * added electron * try to configure orca * Moved test file so that it's tested by the orca build only. * use custom renderer for sphinx-gallery * removed sphinx files which are now in plotly-sphinx-gallery + changed name of renderer sphinx -> sphinx-gallery, and changed wording of documentation * Modified test with new renderer name * try to fix py2 compatibility * write_html instead of offline.plot * change import order

emmanuelle added 9 commits May 24, 2019 08:37

scraper function for sphinx-gallery, in plotly.io
f6a5af8
scraper function + skeleton of sphinx doc with minimal gallery (two examples)

added psutil dep to tox.ini
343f2b2

python 2 compatibility
0e7e209

added orca dep to package.json for tests
dd4cc35

removed typo
2ea2fea

corrected bug
6909d76

python 2 compatibility
795061a

added electron
6654b46

try to configure orca
b976752

emmanuelle mentioned this pull request May 30, 2019
Extending figure_rst to support html figures? sphinx-gallery/sphinx-gallery#498
Closed

emmanuelle added 2 commits May 31, 2019 08:14

Moved test file so that it's tested by the orca build only.
65ea73f

use custom renderer for sphinx-gallery
6433175

jonmmease reviewed Jun 1, 2019
View reviewed changes

Comment threadplotly/io/_renderers.py Outdated

jonmmease reviewed Jun 1, 2019
View reviewed changes

emmanuelle added 4 commits June 2, 2019 08:35

Merge branch 'master' into sphinxgallery
c76736e

removed sphinx files which are now in plotly-sphinx-gallery
c353bc4
+ changed name of renderer sphinx -> sphinx-gallery, and changed wording of documentation

Modified test with new renderer name
b210bc5

try to fix py2 compatibility
e6f183e

emmanuelle changed the title ~~[WIP] scraper function for sphinx-gallery, in plotly.io~~Scraper function for sphinx-gallery, in plotly.ioJun 2, 2019

jonmmease suggested changes Jun 5, 2019
View reviewed changes

Comment threadplotly/tests/test_optional/test_jupyter/package.json Outdated
Comment threadplotly/io/_base_renderers.py Outdated
Comment threadplotly/io/_base_renderers.py Outdated
Comment threadplotly/io/_base_renderers.py Outdated

jonmmease added the V4 label Jun 18, 2019

emmanuelle added 2 commits July 2, 2019 16:36

write_html instead of offline.plot
8b87a18

change import order
3d881ab

jonmmease approved these changes Jul 3, 2019
View reviewed changes

jonmmease merged commit 3be34f7 into plotly:masterJul 3, 2019

mkcor mentioned this pull request Jul 12, 2021
disable calls to plotly.io.show when running on Azure scikit-image/scikit-image#5446
Merged

Uh oh!

Uh oh!

Conversation

emmanuelle commented May 24, 2019

Uh oh!

emmanuelle commented May 24, 2019

Uh oh!

jonmmease commented May 27, 2019

Uh oh!

jonmmease commented May 27, 2019

Uh oh!

emmanuelle commented May 27, 2019

Uh oh!

jonmmease commented May 27, 2019

Uh oh!

emmanuelle commented May 30, 2019

Uh oh!

emmanuelle commented May 31, 2019

Uh oh!

emmanuelle commented May 31, 2019

Uh oh!

Uh oh!

jonmmeaseJun 1, 2019

Choose a reason for hiding this comment

Uh oh!

jonmmease commented Jun 1, 2019

Uh oh!

emmanuelle commented Jun 2, 2019

Uh oh!

emmanuelle commented Jun 4, 2019

Uh oh!

jonmmease commented Jun 4, 2019

Uh oh!

emmanuelle commented Jun 4, 2019

Uh oh!

jonmmease left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

emmanuelle commented Jul 2, 2019

Uh oh!

jonmmease commented Jul 3, 2019

Uh oh!

jonmmease commented Jul 4, 2019

Uh oh!

emmanuelle commented Jul 5, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants