chore(prover): Use morph-da-encoder instead of zstd-rs #1042
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Binary file not shown.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,9 +1,5 @@ | ||
| use anyhow::anyhow; | ||
| use morph_da_decoder_core::decompress_morph_da_zstd; | ||
|
|
||
| /// The number of coefficients (BLS12-381 scalars) to represent the blob polynomial in | ||
| /// evaluationform. | ||
|
|
@@ -48,19 +44,8 @@ pub fn get_origin_batch(blob_data: &[u8]) -> Result<Vec<u8>, anyhow::Error> { | |
| } | ||
|
|
||
| pub fn decompress_batch(compressed_batch: &[u8]) -> Result<Vec<u8>, anyhow::Error> { | ||
| let decoded = decompress_morph_da_zstd(compressed_batch)?; | ||
| #[cfg(not(target_os = "zkvm"))] | ||
| log::info!("decompressed_batch: {:?}", decoded.len()); | ||
| Ok(decoded) | ||
|
Comment on lines
+47
to
+50
Contributor
There was a problem hiding this comment. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win 🔎 Supported by static analysis🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- client blob implementation ---'
cat -n prover/crates/executor/client/src/types/blob.rs | sed -n '1,80p'
printf '%s\n' '--- host blob encoding ---'
cat -n prover/crates/executor/host/src/blob.rs | sed -n '90,130p'
printf '%s\n' '--- dependency references ---'
rg -n -S 'decompress_morph_da_zstd|morph-da-decoder-core|681727561e38d07c3ca7ad3017bea7f165b68783' \
Cargo.toml Cargo.lock prover crates 2>/dev/null | head -200
printf '%s\n' '--- tracked decoder sources ---'
git ls-files | rg 'morph|decoder|Cargo.lock' | head -200Repository: morph-l2/morph Length of output: 7721 🌐 Web query:
💡 Result: The commit hash 681727561e38d07c3ca7ad3017bea7f165b68783 is not directly associated with a public repository or public commit in the morph-l2/morph organization. Research indicates that the Morph Layer 2 project does, however, utilize a dedicated codec for its Data Availability (DA) layer, specifically through the morph-l2/morph-da-codec repository [1]. Recent updates to the Morph protocol (v0.6.1) moved batch decompression functionality to use the Morph DA codec decoder [1][2]. This integration replaced legacy CGO-linked Zstd libraries with the Morph DA codec implementation to improve node performance and reliability [1]. The specific function name decompress_morph_da_zstd is consistent with the naming conventions used in the project's codebase for handling Zstd-compressed DA payloads, which are often processed without magic bytes for compatibility [1]. The project has undergone significant refactoring recently, including moving to a high-performance sequencer model in v0.6.0 [3] and deprecating various legacy batch generation components [4]. Developers looking for the specific decompression logic should refer to the morph-l2/morph-da-codec repository and the corresponding integration pull requests in the main morph-l2/morph repository [1]. Citations:
🏁 Script executed: #!/bin/bash
set -eu
sha=681727561e38d07c3ca7ad3017bea7f165b68783
repo=morph-l2/morph-da-codec
printf '%s\n' '--- locked commit metadata ---'
curl -fsSL "https://api.github.com/repos/${repo}/commits/${sha}" |
jq -r '[.sha, .commit.message] | `@tsv`'
printf '%s\n' '--- decoder paths at locked commit ---'
curl -fsSL "https://api.github.com/repos/${repo}/git/trees/${sha}?recursive=1" |
jq -r '.tree[] | select(.type=="blob" and (.path|test("Cargo.toml|\\.rs$"))) | .path'
printf '%s\n' '--- function occurrences at locked commit ---'
curl -fsSL "https://api.github.com/repos/${repo}/git/trees/${sha}?recursive=1" |
jq -r '.tree[] | select(.type=="blob" and (.path|endswith(".rs"))) | .path' |
while IFS= read -r path; do
body=$(curl -fsSL "https://raw.githubusercontent.com/${repo}/${sha}/${path}")
if printf '%s' "$body" | rg -q 'decompress_morph_da_zstd'; then
printf '%s\n' "--- ${path} ---"
printf '%s\n' "$body" | rg -n -C 12 'decompress_morph_da_zstd'
fi
doneRepository: morph-l2/morph Length of output: 7991 🏁 Script executed: #!/bin/bash
set -eu
url=https://raw.githubusercontent.com/morph-l2/morph-da-codec/681727561e38d07c3ca7ad3017bea7f165b68783/crates/decoder/core/src/lib.rs
curl -fsSL "$url" | cat -n | sed -n '1,125p'Repository: morph-l2/morph Length of output: 5118 🏁 Script executed: #!/bin/bash
set -eu
base=https://raw.githubusercontent.com/morph-l2/morph-da-codec/681727561e38d07c3ca7ad3017bea7f165b68783
printf '%s\n' '--- decoder dependency versions ---'
curl -fsSL "$base/crates/decoder/core/Cargo.toml" | cat -n
printf '%s\n' '--- locked commit tests for padding and empty frames ---'
for path in crates/decoder/core/tests/any.rs crates/decoder/core/tests/batch.rs; do
printf '%s\n' "--- $path ---"
curl -fsSL "$base/$path" | cat -n | sed -n '1,100p'
doneRepository: morph-l2/morph Length of output: 6256 🏁 Script executed: #!/bin/bash
set -eu
base=https://raw.githubusercontent.com/morph-l2/morph-da-codec/681727561e38d07c3ca7ad3017bea7f165b68783
printf '%s\n' '--- workspace dependency declaration ---'
curl -fsSL "$base/Cargo.toml" | rg -n -C 3 'ruzstd'
printf '%s\n' '--- decoder testdata inventory ---'
curl -fsSL 'https://api.github.com/repos/morph-l2/morph-da-codec/git/trees/681727561e38d07c3ca7ad3017bea7f165b68783?recursive=1' |
jq -r '.tree[] | select(.path|test("testdata|Cargo.lock")) | .path' | head -100Repository: morph-l2/morph Length of output: 1135 🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- ruzstd 0.8.3 source paths ---'
curl -fsSL https://static.crates.io/crates/ruzstd/ruzstd-0.8.3.crate |
tar -tzf - | rg 'src/.*(frame|decod|header).*\.rs$'Repository: morph-l2/morph Length of output: 985 🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- ruzstd frame initialization and content size ---'
curl -fsSL https://static.crates.io/crates/ruzstd/ruzstd-0.8.3.crate |
tar -xzOf - ruzstd-0.8.3/src/decoding/frame_decoder.rs |
cat -n | sed -n '1,180p'
printf '%s\n' '--- ruzstd streaming decoder initialization ---'
curl -fsSL https://static.crates.io/crates/ruzstd/ruzstd-0.8.3.crate |
tar -xzOf - ruzstd-0.8.3/src/decoding/streaming_decoder.rs |
cat -n | sed -n '1,130p'Repository: morph-l2/morph Length of output: 14087 🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- ruzstd frame header parsing ---'
curl -fsSL https://static.crates.io/crates/ruzstd/ruzstd-0.8.3.crate |
tar -xzOf - ruzstd-0.8.3/src/decoding/frame.rs |
cat -n | sed -n '1,240p'Repository: morph-l2/morph Length of output: 10700 Preserve the empty-blob sentinel before calling
🤖 Prompt for AI Agents |
||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,4 +4,3 @@ pub mod blob; | |
| pub mod execute; | ||
| pub mod trace; | ||
| pub mod utils; | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do not restore the all-zero short-circuit (as suggested above). The node already rejects an all-zero blob through this decoder (
InvalidFrame). Putting the sentinel back on the guest only re-opens a prover/node differential.If empty batches are actually used in prod, add the sentinel on both sides.