Skip to content

tools: automate zlib update#47417

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nodejs-github-bot merged 1 commit into from
Apr 19, 2023
Merged

tools: automate zlib update #47417

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/tools.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -198,6 +198,14 @@ jobs:
cat temp-output
tail -n1 temp-output | grep "NEW_VERSION=" >> "$GITHUB_ENV" || true
rm temp-output
- id: zlib
subsystem: deps
label: dependencies
run: |
./tools/dep_updaters/update-zlib.sh > temp-output
cat temp-output
tail -n1 temp-output | grep "NEW_VERSION=" >> "$GITHUB_ENV" || true
rm temp-output
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0
with:
Expand Down
64 changes: 64 additions & 0 deletions tools/dep_updaters/update-zlib.sh
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
#!/bin/sh
set -e
# Shell script to update zlib in the source tree to a specific version

BASE_DIR=$(cd "$(dirname "$0")/../.." && pwd)
DEPS_DIR="$BASE_DIR/deps"

CURRENT_VERSION=$(grep "#define ZLIB_VERSION" "$DEPS_DIR/zlib/zlib.h" | sed -n "s/^.*VERSION \"\(.*\)\"/\1/p")

NEW_VERSION_ZLIB_H=$(curl -s "https://chromium.googlesource.com/chromium/src/+/refs/heads/main/third_party/zlib/zlib.h?format=TEXT" | base64 --decode)

@lpincalpincaApr 5, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works but it might remain the same for a very long time. It is not very useful. We refer to the tip of the remote main branch to check if an update is needed. However, afaik, we store the current commit SHA only in the commit message.

@marco-ippolitomarco-ippolitoApr 5, 2023

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I havent found any other solution to check what's the latest version, I think this might be a reasonable compromise and its not very expensive

@lpincalpincaApr 5, 2023•
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fine but it probably won't find a new version for years. See https://github.com/madler/zlib/tags.

@mhdawsonmhdawsonApr 18, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lpinca are you saying that we typically update to something that is not a new release? If so then an option in the script/action to say generate PR based on commit X instead of a new version might make sense.

@lpincalpincaApr 18, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are you saying that we typically update to something that is not a new release?

Yes, exactly.

@marco-ippolitomarco-ippolitoApr 18, 2023

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should probably change it in the future to something like @tniessen did in this PR #47482


NEW_VERSION=$(printf '%s' "$NEW_VERSION_ZLIB_H" | grep "#define ZLIB_VERSION" | sed -n "s/^.*VERSION \"\(.*\)\"/\1/p")

echo "Comparing $NEW_VERSION with $CURRENT_VERSION"

if [ "$NEW_VERSION" = "$CURRENT_VERSION" ]; then
echo "Skipped because zlib is on the latest version."
exit 0
fi

echo "Making temporary workspace..."

WORKSPACE=$(mktemp -d 2> /dev/null || mktemp -d -t 'tmp')

cd "$WORKSPACE"

mkdir zlib

ZLIB_TARBALL=zlib.tar.gz

echo "Fetching zlib source archive"
curl -sL -o $ZLIB_TARBALL https://chromium.googlesource.com/chromium/src/+archive/refs/heads/main/third_party/$ZLIB_TARBALL

gzip -dc "$ZLIB_TARBALL" | tar xf - -C zlib/

rm "$ZLIB_TARBALL"

cp "$DEPS_DIR/zlib/zlib.gyp" "$DEPS_DIR/zlib/GN-scraper.py" "$DEPS_DIR/zlib/win32/zlib.def" "$DEPS_DIR"

rm -rf "$DEPS_DIR/zlib" zlib/.git

mv zlib "$DEPS_DIR/"

mv "$DEPS_DIR/zlib.gyp" "$DEPS_DIR/GN-scraper.py" "$DEPS_DIR/zlib/"

mkdir "$DEPS_DIR/zlib/win32"

mv "$DEPS_DIR/zlib.def" "$DEPS_DIR/zlib/win32"

perl -i -pe 's|^#include "chromeconf.h"|//#include "chromeconf.h"|' deps/zlib/zconf.h

echo "All done!"
echo ""
echo "Make sure to update the deps/zlib/zlib.gyp if any significant changes have occurred upstream"
echo ""
echo "Please git add zlib, commit the new version:"
echo ""
echo "$ git add -A deps/zlib"
echo "$ git commit -m \"deps: update zlib to $NEW_VERSION\""
echo ""

# The last line of the script should always print the new version,
# as we need to add it to $GITHUB_ENV variable.
echo "NEW_VERSION=$NEW_VERSION"