Skip to content

optimize compression settings for 7z and ZIP formats - #36

Merged
N6REJ merged 1 commit into
mainfrom
compression
Nov 4, 2025
Merged

optimize compression settings for 7z and ZIP formats#36
N6REJ merged 1 commit into
mainfrom
compression

Conversation

@jwaisner

@jwaisnerjwaisner commented Nov 2, 2025

Copy link
Copy Markdown
Contributor

User description

Improves build time by approximately 10%


PR Type

Enhancement


Description

  • Optimizes 7z compression with LZMA2 algorithm and increased dictionary size

  • Enhances ZIP compression with Deflate algorithm and multi-threading support

  • Adds diagnostic echo messages for compression format tracking

  • Reorders ZIP compression arguments for consistency and clarity


Diagram Walkthrough

flowchart LR
A["Compression Settings"] --> B["7z Format"]
A --> C["ZIP Format"]
B --> D["LZMA2 + 256MB Dictionary"]
B --> E["Multi-threading + Solid Archive"]
C --> F["Deflate + 15 Pass Compression"]
C --> G["Multi-threading Support"]
Loading

File Walkthrough

Relevant files
Enhancement
build-commons.xml
Enhanced compression algorithms and multi-threading configuration

build/build-commons.xml

  • Modified 7z compression: changed -mmt6 to -mmt and added -md=256m,
    -ms=on, -mfb=273 for enhanced compression ratio
  • Enhanced ZIP compression: added -tzip, -mx9, -mmt arguments and
    reordered parameters for consistency
  • Added diagnostic echo messages for both 7z and ZIP compression formats
  • Improved argument ordering in ZIP compression command for better
    readability
+11/-3

@qodo-code-review

qodo-code-reviewBot commented Nov 2, 2025

Copy link
Copy Markdown

PR Compliance Guide 🔍

(Compliance updated until commit 3ab2a8e)

Below is a summary of compliance checks for this PR:

Security Compliance
🟢
No security concerns identified No security vulnerabilities detected by AI analysis. Human verification advised for critical code.
Ticket Compliance
🎫 No ticket provided
  • Create ticket/issue
Codebase Duplication Compliance
Codebase context is not defined

Follow the guide to enable codebase context checks.

Custom Compliance
🟢
Generic: Meaningful Naming and Self-Documenting Code

Objective: Ensure all identifiers clearly express their purpose and intent, making code
self-documenting

Status: Passed

Generic: Secure Error Handling

Objective: To prevent the leakage of sensitive system information through error messages while
providing sufficient detail for internal debugging.

Status: Passed

Generic: Secure Logging Practices

Objective: To ensure logs are useful for debugging and auditing without exposing sensitive
information like PII, PHI, or cardholder data.

Status: Passed

Generic: Comprehensive Audit Trails

Objective: To create a detailed and reliable record of critical system actions for security analysis
and compliance.

Status:
Limited auditing: New echo messages provide minimal activity traces but lack user ID, timestamps, and
outcome context required for comprehensive audit trails.

Referred Code
 <echomessage="Compressing with 7z format (optimized settings)..."/>
<execexecutable="${7za}"failonerror="true">
<argvalue="a"/>
<argvalue="-t7z"/>
<argvalue="@{dest}"/>
<argvalue="@{src}"/>
<argvalue="-m0=LZMA2"/>
<argvalue="-mx9"/>
<argvalue="-mmt"/>
<argvalue="-md=256m"/>
<argvalue="-ms=on"/>
<argvalue="-mfb=273"/>
</exec>
</then>
<elseif>
<equalsarg1="@{format}"arg2="zip"/>
<then>
<echomessage="Compressing with ZIP format (optimized settings)..."/>
<execexecutable="${7za}"failonerror="true">
<argvalue="a"/>
<argvalue="-tzip"/>
... (clipped 8 lines)
Generic: Robust Error Handling and Edge Case Management

Objective: Ensure comprehensive error handling that provides meaningful context and graceful
degradation

Status:
Missing error context: While failonerror is enabled, added steps do not capture or log actionable error details
or handle edge cases like missing binaries or invalid paths.

Referred Code
 <execexecutable="${7za}"failonerror="true">
<argvalue="a"/>
<argvalue="-t7z"/>
<argvalue="@{dest}"/>
<argvalue="@{src}"/>
<argvalue="-m0=LZMA2"/>
<argvalue="-mx9"/>
<argvalue="-mmt"/>
<argvalue="-md=256m"/>
<argvalue="-ms=on"/>
<argvalue="-mfb=273"/>
</exec>
</then>
<elseif>
<equalsarg1="@{format}"arg2="zip"/>
<then>
<echomessage="Compressing with ZIP format (optimized settings)..."/>
<execexecutable="${7za}"failonerror="true">
<argvalue="a"/>
<argvalue="-tzip"/>
<argvalue="@{dest}"/>
... (clipped 8 lines)
Generic: Security-First Input Validation and Data Handling

Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent
vulnerabilities

Status:
Unvalidated inputs: The macro accepts external attributes like @{src}, @{dest}, and @{format} without visible
validation or sanitization before passing to exec.

Referred Code
<macrodefname="sevenzip">
<attributename="src"/>
<attributename="dest"/>
<attributename="format"default="7z"/>
<sequential>
<if>
<equalsarg1="@{format}"arg2="7z"/>
<then>
<echomessage="Compressing with 7z format (optimized settings)..."/>
<execexecutable="${7za}"failonerror="true">
<argvalue="a"/>
<argvalue="-t7z"/>
<argvalue="@{dest}"/>
<argvalue="@{src}"/>
<argvalue="-m0=LZMA2"/>
<argvalue="-mx9"/>
<argvalue="-mmt"/>
<argvalue="-md=256m"/>
<argvalue="-ms=on"/>
<argvalue="-mfb=273"/>
</exec>
... (clipped 16 lines)
Compliance status legend 🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

Previous compliance checks

Compliance check up to commit 3ab2a8e
Security Compliance
Resource exhaustion risk

Description: Setting 7z dictionary size to 256MB (-md=256m) and enabling maximum compression with
multi-threading can significantly increase memory/CPU usage, potentially causing build
agent resource exhaustion or denial of service on low-memory runners.
build-commons.xml [60-71]

Referred Code
<echomessage="Compressing with 7z format (optimized settings)..."/>
<execexecutable="${7za}"failonerror="true">
<argvalue="a"/>
<argvalue="-t7z"/>
<argvalue="@{dest}"/>
<argvalue="@{src}"/>
<argvalue="-m0=LZMA2"/>
<argvalue="-mx9"/>
<argvalue="-mmt"/>
<argvalue="-md=256m"/>
<argvalue="-ms=on"/>
<argvalue="-mfb=273"/>
Resource exhaustion risk

Description: Enabling multi-threading (-mmt) and high compression parameters for ZIP may overload
constrained CI runners, leading to potential build instability or denial of service due to
resource contention.
build-commons.xml [77-88]

Referred Code
<echomessage="Compressing with ZIP format (optimized settings)..."/>
<execexecutable="${7za}"failonerror="true">
<argvalue="a"/>
<argvalue="-tzip"/>
<argvalue="@{dest}"/>
<argvalue="@{src}"/>
<argvalue="-mm=Deflate"/>
<argvalue="-mx9"/>
<argvalue="-mfb=258"/>
<argvalue="-mpass=15"/>
<argvalue="-mmt"/>
<argvalue="-r"/>
Ticket Compliance
🎫 No ticket provided
  • Create ticket/issue
Codebase Duplication Compliance
Codebase context is not defined

Follow the guide to enable codebase context checks.

Custom Compliance
🟢
Generic: Meaningful Naming and Self-Documenting Code

Objective: Ensure all identifiers clearly express their purpose and intent, making code
self-documenting

Status: Passed

Generic: Secure Error Handling

Objective: To prevent the leakage of sensitive system information through error messages while
providing sufficient detail for internal debugging.

Status: Passed

Generic: Secure Logging Practices

Objective: To ensure logs are useful for debugging and auditing without exposing sensitive
information like PII, PHI, or cardholder data.

Status: Passed

Generic: Comprehensive Audit Trails

Objective: To create a detailed and reliable record of critical system actions for security analysis
and compliance.

Status:
Lacks audit logs: The new compression actions add echo messages but do not log user/context or outcomes,
which may be required for critical build operations.

Referred Code
 <echomessage="Compressing with 7z format (optimized settings)..."/>
<execexecutable="${7za}"failonerror="true">
<argvalue="a"/>
<argvalue="-t7z"/>
<argvalue="@{dest}"/>
<argvalue="@{src}"/>
<argvalue="-m0=LZMA2"/>
<argvalue="-mx9"/>
<argvalue="-mmt"/>
<argvalue="-md=256m"/>
<argvalue="-ms=on"/>
<argvalue="-mfb=273"/>
</exec>
</then>
<elseif>
<equalsarg1="@{format}"arg2="zip"/>
<then>
<echomessage="Compressing with ZIP format (optimized settings)..."/>
<execexecutable="${7za}"failonerror="true">
<argvalue="a"/>
<argvalue="-tzip"/>
... (clipped 9 lines)
Generic: Robust Error Handling and Edge Case Management

Objective: Ensure comprehensive error handling that provides meaningful context and graceful
degradation

Status:
Sparse error context: While exec uses failonerror="true", the added echo messages do not capture or
log failure reasons or edge cases such as missing binaries or invalid paths.

Referred Code
 <echomessage="Compressing with 7z format (optimized settings)..."/>
<execexecutable="${7za}"failonerror="true">
<argvalue="a"/>
<argvalue="-t7z"/>
<argvalue="@{dest}"/>
<argvalue="@{src}"/>
<argvalue="-m0=LZMA2"/>
<argvalue="-mx9"/>
<argvalue="-mmt"/>
<argvalue="-md=256m"/>
<argvalue="-ms=on"/>
<argvalue="-mfb=273"/>
</exec>
</then>
<elseif>
<equalsarg1="@{format}"arg2="zip"/>
<then>
<echomessage="Compressing with ZIP format (optimized settings)..."/>
<execexecutable="${7za}"failonerror="true">
<argvalue="a"/>
<argvalue="-tzip"/>
... (clipped 9 lines)
Generic: Security-First Input Validation and Data Handling

Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent
vulnerabilities

Status:
Unvalidated inputs: The macro attributes @{src}, @{dest}, and @{format} are passed to exec without visible
validation or sanitization against dangerous values.

Referred Code
<macrodefname="sevenzip">
<attributename="src"/>
<attributename="dest"/>
<attributename="format"default="7z"/>
<sequential>
<if>
<equalsarg1="@{format}"arg2="7z"/>
<then>
<echomessage="Compressing with 7z format (optimized settings)..."/>
<execexecutable="${7za}"failonerror="true">
<argvalue="a"/>
<argvalue="-t7z"/>
<argvalue="@{dest}"/>
<argvalue="@{src}"/>
<argvalue="-m0=LZMA2"/>
<argvalue="-mx9"/>
<argvalue="-mmt"/>
<argvalue="-md=256m"/>
<argvalue="-ms=on"/>
<argvalue="-mfb=273"/>
</exec>
... (clipped 17 lines)

@qodo-code-review

Copy link
Copy Markdown

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion Impact
Possible issue
Reduce memory usage during compression

To prevent potential out-of-memory errors on build systems, reduce the LZMA2
dictionary size for 7z compression from 256m to a more conservative 64m, which
lowers the required RAM.

build/build-commons.xml [66-71]

 <arg value="-m0=LZMA2"/>
<arg value="-mx9"/>
<arg value="-mmt"/>
-<arg value="-md=256m"/>+<arg value="-md=64m"/>
<arg value="-ms=on"/>
<arg value="-mfb=273"/>
  • Apply / Chat
Suggestion importance[1-10]: 8

__

Why: The suggestion correctly identifies that the -md=256m setting requires a large amount of RAM, which could cause build failures on resource-constrained systems, and proposes a more robust alternative.

Medium
  • More

@N6REJ
N6REJ merged commit 3b44376 into mainNov 4, 2025
2 checks passed
@N6REJ
N6REJ deleted the compression branch November 4, 2025 02:41
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement ✨Improve program

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@jwaisner@N6REJ