Skip to content

ci: s3 cache - #1167

Merged
shumkov merged 82 commits into
v0.25-devfrom
ci/s3_cache
Jul 11, 2023
Merged

ci: s3 cache#1167
shumkov merged 82 commits into
v0.25-devfrom
ci/s3_cache

Conversation

@shumkov

@shumkovshumkov commented Jun 15, 2023

Copy link
Copy Markdown
Collaborator

Issue being fixed or feature implemented

Self-hosted runners are expensive so we need to utilize cache as much as possible to reduce costs.
This PR is using s3 store as a cache backend for NPM, Cargo, Docker build and Rust build caches in CI

There is some work needed to support ARM builds in a proper way

What was done?

  • Disabled sccache by default
  • Enabled docker mount cache
  • Enabled docker layer cache
  • Use independent envs to setup caching for each service in Dashmate
  • Utilized s3 for the docker layer and mount caches in CI

This PR contains prelumeary work from #1105 and #953

How Has This Been Tested?

Running CI and locally

Breaking Changes

None

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have made corresponding changes to the documentation

For repository code-owners and collaborators only

  • I have assigned this pull request to a milestone

@shumkov
shumkov requested a review from antouhou as a code ownerJune 15, 2023 03:28
@shumkovshumkov changed the title ci: s3 cache[WIP] ci: s3 cacheJun 15, 2023
@shumkovshumkov added this to the v0.24.x milestone Jun 23, 2023
@shumkovshumkov changed the title [WIP] ci: s3 cacheci: s3 cacheJul 5, 2023

@strophystrophy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking really good! Let's make a few more changes for readability

Comment thread.github/actions/docker/action.yaml Outdated
Comment thread.github/workflows/all-packages.yml Outdated
Comment thread.github/workflows/all-packages.yml Outdated
Comment thread.github/workflows/js-checks.yml Outdated
Comment thread.github/workflows/rs-drive-abci.yml Outdated

@strophystrophy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@shumkov
shumkov merged commit c435a3e into v0.25-devJul 11, 2023
@shumkov
shumkov deleted the ci/s3_cache branch July 11, 2023 13:35
@strophystrophy mentioned this pull request Jul 13, 2023
5 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@shumkov@strophy