Skip to content

ZEPPELIN-5536 Update protoc and grpc for Linux ARM64 - #4233

Closed
martin-g wants to merge 1 commit into
apache:masterfrom
martin-g:update-protobuf-and-grpc
Closed

ZEPPELIN-5536 Update protoc and grpc for Linux ARM64#4233
martin-g wants to merge 1 commit into
apache:masterfrom
martin-g:update-protobuf-and-grpc

Conversation

@martin-g

Copy link
Copy Markdown
Member

What is this PR for?

Update protoc and grpc to their first versions that support Linux ARM64 architecture.
With this the build on Linux ARM64 passes!

What type of PR is it?

Bug Fix

What is the Jira issue?

ZEPPELIN-5536

How should this be tested?

Run the build on Linux ARM64, e.g. at TravisCI or even better CircleCI on ARM64 hardware!
Let me know if you want me to setup either of those!

Questions:

  • Does the licenses files need update? - NO
  • Is there breaking changes for older versions? - Hopefully not! There is one API break in Grpc APIs (usePlaintext()), but there might be problems in reading serialized messages with the old versions
  • Does this needs documentation? - NO

Update protoc and grpc to their first versions that support Linux ARM64 architecture.
With this the build on Linux ARM64 passes!
@martin-g

martin-g commented Sep 24, 2021

Copy link
Copy Markdown
MemberAuthor

Actually the build passes only if I disable the tests for the web app module, because they use PhantomJS which has no binary for Linux aarch64, and since PhantomJS is a dead project it will never have such officially.

@jongyoul

Copy link
Copy Markdown
Member

@martin-g Hello, I might not understand what you are saying but could you please explain more about PhantomJS and the result of this CI? It looks like all passed without disabling anything.

@martin-g

Copy link
Copy Markdown
MemberAuthor

The CI passed because GitHub Actions run on x86_64 machines.
This PR is about improving the situation on Linux aarch64 (ARM64).
Officially PhantomJS provides only x86_64 binaries. Trying to npm install phantomjs on non-x86_64 would fail.
Since PhantomJS is not maintained since few years some projects have replaced it with headless chrome/chromium.

@jongyoul

jongyoul commented Sep 24, 2021

Copy link
Copy Markdown
Member

@martin-g Thank you for explaining it. I also think PhantomJS is quite old but we don't have many resources to fix/improve it. Do you have any idea to improve it?

@zjffdu

Copy link
Copy Markdown
Contributor

@martin-g Do you know whether GitHub Actions support ARM64 ? If so, we should enable it to avoid regression.

@martin-g

Copy link
Copy Markdown
MemberAuthor

Thank you for explaining it. I also think PhantomJS is quite old but we don't have many resources to fix/improve it. Do you have any idea to improve it?

I could try to replace it with headless Chromium! I'll keep you posted!

@jongyoul

Copy link
Copy Markdown
Member

I could try to replace it with headless Chromium! I'll keep you posted!

Sounds like a great idea!! Thank you!!

@martin-g

Copy link
Copy Markdown
MemberAuthor

Do you know whether GitHub Actions support ARM64 ? If so, we should enable it to avoid regression.

GitHub Actions does not provide Linux ARM64 VMs at the moment (actions/runner-images#2536).
But there are few ways to do it:

  • self-hosted runner - There are security concerns with it! GHA does not recommend using self-hosted runners for public repositories. Apache AirFlow uses custom GHA runner with better security. If you go this way then using Oracle Cloud ARM64 for the runner is very easy!
  • Remote Kubernetes - I have blogged about it here. The issue here is that the setup becomes a bit more complex

Other projects use TravisCI or CircleCI (my recommendation) for testing on Linux ARM64/PPC64/s390x. The issue is - one more CI to maintain and watch over.
Yet another option is to use Apache Jenkins or Buildbot.

@zjffdu

zjffdu commented Sep 26, 2021

Copy link
Copy Markdown
Contributor

Thanks @martin-g for the investigation, I checked CircleCI. But it seems not free like github action. We have to think about other options. It could be done in another PR future. Regarding this PR, LGTM +1

@martin-g

martin-g commented Sep 26, 2021

Copy link
Copy Markdown
MemberAuthor

But it seems not free like github action.

CircleCI is free for public repos.
And resource type arm64.medium is free. arm64.large is paid though!

@martin-g

Copy link
Copy Markdown
MemberAuthor

Sounds like a great idea!! Thank you!!

@jongyoul I will work on it in a separate PR!

@zjffdu

Copy link
Copy Markdown
Contributor

CircleCI is free for public repos

ha, right, CircleCI has free plan for open source project.

@martin-g

Copy link
Copy Markdown
MemberAuthor

@zjffdu I will prepare a PR for testing on Linux ARM64 at CircleCI!

@zjffdu

Copy link
Copy Markdown
Contributor

Thanks @martin-g I will merge this PR if no more comment

asfgit pushed a commit that referenced this pull request Sep 28, 2021
### What is this PR for?
Update protoc and grpc to their first versions that support Linux ARM64 architecture.
With this the build on Linux ARM64 passes!
### What type of PR is it?
Bug Fix
### What is the Jira issue?
[ZEPPELIN-5536](https://issues.apache.org/jira/browse/ZEPPELIN-5536)
### How should this be tested?
Run the build on Linux ARM64, e.g. at TravisCI or even better CircleCI on ARM64 hardware!
Let me know if you want me to setup either of those!
### Questions:
* Does the licenses files need update? - NO
* Is there breaking changes for older versions? - Hopefully not! There is one API break in Grpc APIs (`usePlaintext()`), but there might be problems in reading serialized messages with the old versions
* Does this needs documentation? - NO
Author: Martin Tzvetanov Grigorov <mgrigorov@apache.org>
Closes#4233 from martin-g/update-protobuf-and-grpc and squashes the following commits:
60b0c1d [Martin Tzvetanov Grigorov] ZEPPELIN-5536 Update protoc and grpc for Linux ARM64
(cherry picked from commit a1fe668)
Signed-off-by: Jeff Zhang <zjffdu@apache.org>
zlosim pushed a commit to zlosim/zeppelin that referenced this pull request Oct 22, 2021
### What is this PR for?
Update protoc and grpc to their first versions that support Linux ARM64 architecture.
With this the build on Linux ARM64 passes!
### What type of PR is it?
Bug Fix
### What is the Jira issue?
[ZEPPELIN-5536](https://issues.apache.org/jira/browse/ZEPPELIN-5536)
### How should this be tested?
Run the build on Linux ARM64, e.g. at TravisCI or even better CircleCI on ARM64 hardware!
Let me know if you want me to setup either of those!
### Questions:
* Does the licenses files need update? - NO
* Is there breaking changes for older versions? - Hopefully not! There is one API break in Grpc APIs (`usePlaintext()`), but there might be problems in reading serialized messages with the old versions
* Does this needs documentation? - NO
Author: Martin Tzvetanov Grigorov <mgrigorov@apache.org>
Closesapache#4233 from martin-g/update-protobuf-and-grpc and squashes the following commits:
60b0c1d [Martin Tzvetanov Grigorov] ZEPPELIN-5536 Update protoc and grpc for Linux ARM64
zlosim pushed a commit to zlosim/zeppelin that referenced this pull request Oct 22, 2021
### What is this PR for?
Update protoc and grpc to their first versions that support Linux ARM64 architecture.
With this the build on Linux ARM64 passes!
### What type of PR is it?
Bug Fix
### What is the Jira issue?
[ZEPPELIN-5536](https://issues.apache.org/jira/browse/ZEPPELIN-5536)
### How should this be tested?
Run the build on Linux ARM64, e.g. at TravisCI or even better CircleCI on ARM64 hardware!
Let me know if you want me to setup either of those!
### Questions:
* Does the licenses files need update? - NO
* Is there breaking changes for older versions? - Hopefully not! There is one API break in Grpc APIs (`usePlaintext()`), but there might be problems in reading serialized messages with the old versions
* Does this needs documentation? - NO
Author: Martin Tzvetanov Grigorov <mgrigorov@apache.org>
Closesapache#4233 from martin-g/update-protobuf-and-grpc and squashes the following commits:
60b0c1d [Martin Tzvetanov Grigorov] ZEPPELIN-5536 Update protoc and grpc for Linux ARM64
JasonLeeCoding pushed a commit to JasonLeeCoding/zeppelin that referenced this pull request Jan 5, 2022
### What is this PR for?
Update protoc and grpc to their first versions that support Linux ARM64 architecture.
With this the build on Linux ARM64 passes!
### What type of PR is it?
Bug Fix
### What is the Jira issue?
[ZEPPELIN-5536](https://issues.apache.org/jira/browse/ZEPPELIN-5536)
### How should this be tested?
Run the build on Linux ARM64, e.g. at TravisCI or even better CircleCI on ARM64 hardware!
Let me know if you want me to setup either of those!
### Questions:
* Does the licenses files need update? - NO
* Is there breaking changes for older versions? - Hopefully not! There is one API break in Grpc APIs (`usePlaintext()`), but there might be problems in reading serialized messages with the old versions
* Does this needs documentation? - NO
Author: Martin Tzvetanov Grigorov <mgrigorov@apache.org>
Closesapache#4233 from martin-g/update-protobuf-and-grpc and squashes the following commits:
60b0c1d [Martin Tzvetanov Grigorov] ZEPPELIN-5536 Update protoc and grpc for Linux ARM64
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@martin-g@jongyoul@zjffdu