Skip to content

Zeppelin-275: Add SpringXD Stream and Job Interpreters - #291

Closed
tzolov wants to merge 11 commits into
apache:masterfrom
tzolov:ZEPPELIN-275
Closed

Zeppelin-275: Add SpringXD Stream and Job Interpreters#291
tzolov wants to merge 11 commits into
apache:masterfrom
tzolov:ZEPPELIN-275

Conversation

@tzolov

Copy link
Copy Markdown
Contributor

1. SpringXD Stream Interpreter:

  • %xd.stream - identifies the SpringXD Streams interpreter.
  • Use (Ctrl+.) for auto-completion.
  • Streams must have names. Follow the convention: (stream name = stream definition). Streams without names are ignored.
  • New Run destroys any previous streams created in the same paragraph. The previous streams are destroyed even if their names have been changed or removed.
  • New Run will Create and automatically Deploy all streams defined in the Paragraph.
  • If one stream fails to create or deploy all streams in the paragraph are destroyed
  • Zeppelin returns after the streams deployment (e.g. Zeppelin goes into Finished state).
  • When streams have successfully been deployed the result contains a button that lists the just deployed streams and status 'DEPLOYED'
  • To destroy streams in the paragraph press the Annular button in the result section. Button state should switch from 'DEPLOYED' to 'DESTROYED'.
  • Streams can refer other streams or jobs in any paragraph and even different notebooks

2. SpringXD Job Interpreter:

  • %xd.job - identifies the SpringXD Job interpreter.
  • Use (Ctrl+.) for auto-completion.
  • Jobs must have names. Follow the convention: (job name = job definition). Streams without names are ignored.
  • New Run destroys any previous jobs created in the same paragraph. The previous jobs are destroyed even if their names have been changed or removed.
  • New Run will Create and automatically Deploy and Start all jobs defined in the Paragraph.
  • If one job deployment fail all jobs in the paragraph are destroyed
  • Zeppelin returns after the jobs have started (e.g. Zeppelin goes into Finished state).
  • When job have successfully been deployed the result contains a button that lists the just deployed jobs and status 'DEPLOYED'
  • To destroy all jobs in the paragraph, press the Annular button in the result section. Button state should switch from 'DEPLOYED' to 'DESTROYED'.
  • Jobs can refer other streams or jobs in any paragraph and even different notebooks

@tzolov
tzolovforce-pushed the ZEPPELIN-275 branch 3 times, most recently from c7d38ba to 6774d15CompareSeptember 13, 2015 12:04
@bzz

bzz commented Jan 5, 2016

Copy link
Copy Markdown
Member

Looks like this guy diverged from master as well as having some commits that are not part of ZEPPELIN-275

@tzolov

Copy link
Copy Markdown
ContributorAuthor

Good spot @bzz, I will try to clear the ghost commits.

@tzolov

Copy link
Copy Markdown
ContributorAuthor

@bzz, i've resolved the commits mix-up, please review.
Also i've rebased the ZEPPELIN-275 branch to the master.

@bzz

bzz commented Jan 15, 2016

Copy link
Copy Markdown
Member

Great!
Quick question - do we need to add this interpreters to the list of interpreters in docs? (They now live in master, under ./docs)

Also, as they become part of releases - are there any new dependency that it brings, that need to be added to zeppelin-distribution/src/bin_license/LICENSE ?

@tzolov

Copy link
Copy Markdown
ContributorAuthor
  • Sure the interpreter should have documentation - I will add such to the ./docs
  • The interpreter brings 2 deps: spring-xd-rest-client (under Apache2 License) which i will add to the LICENSE file and Guava - already there

@bzz

bzz commented Jan 16, 2016

Copy link
Copy Markdown
Member

Sounds great, thank you very much!

On Sat, Jan 16, 2016 at 3:14 AM, Christian Tzolov notifications@github.com
wrote:

  • Sure the interpreter should have documentation - I will add such to
    the ./docs
  • The interpreter brings 2 deps: spring-xd-rest-client (under Apache2
    License) which i will add to the LICENSE file and Guava - already there


Reply to this email directly or view it on GitHub
#291 (comment)
.

Kind regards,
Alexander.

@tzolov
tzolovforce-pushed the ZEPPELIN-275 branch 2 times, most recently from fde1cba to 6b746eaCompareJanuary 17, 2016 16:39
@tzolov

Copy link
Copy Markdown
ContributorAuthor

Cheers @bzz,
I've updated the LICENSE file with the springxd dependecy. Also i've put together a basic springxd documentation page and listed it in the interpreter's menu. Let me know if we are still missing tasks.

Comment threaddocs/interpreter/springxd.md Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that back quotation marks are not working in the table tag. Like this below image.
screen shot 2016-01-18 at 5 53 55 pm

So, why don't you use <code> tag instead of the back quotations? : )

@tzolov

Copy link
Copy Markdown
ContributorAuthor

Thanks @AhyoungRyu! I've just commited a fix for the abovementioned misspellings.

Comment threadconf/zeppelin-site.xml.template Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Were == and < intentionally added here?

@bzz

bzz commented Jan 19, 2016

Copy link
Copy Markdown
Member

Looks great to me, thank you! Just needed to be updated on top of the latest master.

@tzolov
tzolovforce-pushed the ZEPPELIN-275 branch 3 times, most recently from 600707c to 9102439CompareJanuary 27, 2016 16:12
@corneadoug

Copy link
Copy Markdown
Contributor

@tzolov@bzz
Do we want to rebase and try to merge this PR?

@asfgitasfgit closed this in c38a0a0May 9, 2018
asfgit pushed a commit that referenced this pull request May 9, 2018
close#83close#86close#125close#133close#139close#146close#193close#203close#246close#262close#264close#273close#291close#299close#320close#347close#389close#413close#423close#543close#560close#658close#670close#728close#765close#777close#782close#783close#812close#822close#841close#843close#878close#884close#918close#989close#1076close#1135close#1187close#1231close#1304close#1316close#1361close#1385close#1390close#1414close#1422close#1425close#1447close#1458close#1466close#1485close#1492close#1495close#1497close#1536close#1545close#1561close#1577close#1600close#1603close#1678close#1695close#1739close#1748close#1765close#1767close#1776close#1783close#1799
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@tzolov@bzz@corneadoug@AhyoungRyu