Skip to content

feat(deducer): slim down the python bundle, support cross-platfrom deployment - #163

Merged
jianzs merged 3 commits into
pluto-lang:mainfrom
jianzs:feat-slim-python-bundle
Mar 27, 2024
Merged

feat(deducer): slim down the python bundle, support cross-platfrom deployment#163
jianzs merged 3 commits into
pluto-lang:mainfrom
jianzs:feat-slim-python-bundle

Conversation

@jianzs

Copy link
Copy Markdown
Contributor

1. Does this PR affect any open issues?(Y/N) and add issue references (e.g. "fix#123", "re #123".):

  • N
  • Y

fix#155

2. What is the scope of this PR (e.g. component or file name):

  • Pyright Deducer
  • Infra SDK

3. Provide a description of the PR(e.g. more details, effects, motivations or doc link):

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Other

When creating the Lambda deployment package, it's essential to bundle dependencies. Previously, we built the dependency graph for a single closure, identified the directories containing the dependent packages, and copied them into the deployment package. However, this approach struggles with cross-architecture deployment and may include unnecessary files.

Now, we utilize pip install to install directly dependent packages for a closure. If the target runtime or architecture differs from the local environment, we employ Docker to handle dependency installation before packaging. While this method offers greater reliability, it's slower compared to the previous approach.

4. Are there any breaking changes?(Y/N) and describe the breaking changes(e.g. more details, motivations or doc link):

  • N
  • Y

5. Are there test cases for these changes?(Y/N) select and add more details, references or doc links:

  • Unit test
  • Integration test
  • Benchmark (add benchmark stats below)
  • Manual test (add detailed scripts or steps below)
  • Other

@jianzsjianzs added deducer Deducer sdk Client SDK and Infra SDK labels Mar 27, 2024
@changeset-bot

changeset-botBot commented Mar 27, 2024

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: e68c218

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 10 packages
NameType
@plutolang/pyright-deducerPatch
@plutolang/basePatch
@plutolang/cliPatch
@plutolang/pulumi-adapterPatch
@plutolang/simulator-adapterPatch
@plutolang/static-deducerPatch
@plutolang/graphviz-generatorPatch
@plutolang/static-generatorPatch
@plutolang/pluto-infraPatch
@plutolang/plutoPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

…ocal packages
When creating the Lambda deployment package, it's essential to bundle dependencies. Previously, we built the dependency graph for a single closure, identified the directories containing the dependent packages, and copied them into the deployment package. However, this approach struggles with cross-architecture deployment and may include unnecessary files.
Now, we utilize `pip install` to install directly dependent packages for a closure. If the target runtime or architecture differs from the local environment, we employ Docker to handle dependency installation before packaging. While this method offers greater reliability, it's slower compared to the previous approach.
@jianzs
jianzsforce-pushed the feat-slim-python-bundle branch from ab76f9e to 1dbad17CompareMarch 27, 2024 07:36
@jianzs
jianzs requested a review from PeefyMarch 27, 2024 07:45
@jianzs
jianzsforce-pushed the feat-slim-python-bundle branch from 1dbad17 to e68c218CompareMarch 27, 2024 07:48

@PeefyPeefy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jianzs
jianzs merged commit 1ee9494 into pluto-lang:mainMar 27, 2024
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deducerDeducersdkClient SDK and Infra SDK

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support deploying Python projects across different environments

2 participants

@jianzs@Peefy