Skip to content

Speed up multipart callback dispatch - #317

Closed
Kludex wants to merge 4 commits into
mainfrom
speed-up-multipart-callback-dispatch
Closed

Speed up multipart callback dispatch#317
Kludex wants to merge 4 commits into
mainfrom
speed-up-multipart-callback-dispatch

Conversation

@Kludex

@KludexKludex commented Aug 9, 2026

Copy link
Copy Markdown
Owner

Summary

  • normalize omitted and explicit-None multipart callbacks to shared no-op functions once during parser initialization
  • cache callbacks in private dispatch slots and keep them synchronized through set_callback()
  • dispatch callbacks directly instead of constructing callback names and calling BaseParser.callback() for every event
  • pass resolved data callbacks into the existing mark/lookbehind helper

Performance

CodSpeed reports a 24.15% efficiency improvement for the corrected 100-field complete-parse benchmark, from 3.7 ms to 3.0 ms. The other four corrected benchmarks are unchanged. CodSpeed notes that the compared runs used different runtime environments, so the same-environment local results below provide additional context.

Same-environment local wall-time benchmarks on Python 3.13:

BenchmarkBaseHeadImprovement
Simple multipart form15.53 µs13.63 µs12.2%
100-field multipart form403.28 µs334.28 µs17.1%
1,000-field multipart form3,988.07 µs3,341.27 µs16.2%
8 MiB file upload605.73 µs599.75 µseffectively unchanged
1 MiB CR/LF-dense upload57.96 µs55.52 µs4.2%

A Starlette-shaped 100-field multipart parse improved from approximately 661 µs to 616 µs (about 6.8%).

Validation

  • 162 tests pass
  • 100% statement coverage
  • Ruff passes
  • mypy passes
  • source distribution check passes

Review in cubic

@codspeed-hq

codspeed-hqBot commented Aug 9, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 24.15%

⚠️Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 4 untouched benchmarks
⏩ 5 skipped benchmarks1

Performance Changes

ModeBenchmarkBASEHEADEfficiency
Simulationtest_parse_multipart_large_form3.7 ms3 ms+24.15%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing speed-up-multipart-callback-dispatch (d74fe57) with main (d9cb4c6)

Open in CodSpeed

Footnotes

  1. 5 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@chatgpt-codex-connectorchatgpt-codex-connectorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:0c875e1e81

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment threadpython_multipart/multipart.py Outdated
@Kludex

Kludex commented Aug 9, 2026

Copy link
Copy Markdown
OwnerAuthor

After rebasing onto the corrected benchmark lifecycle from #318, CodSpeed now reports a 24.15% efficiency improvement for the 100-field complete-parse benchmark (3.7 ms to 3.0 ms), with the other four corrected benchmarks unchanged. The report warns that the runs used different environments, while same-environment local measurements show a 17.1% improvement for the equivalent workload. The Cubic finding is marked addressed in d74fe57 and its thread is resolved; no further code change is needed.

@cubic-dev-aicubic-dev-aiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 2 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment threadtests/test_multipart.py Outdated
@Kludex

Copy link
Copy Markdown
OwnerAuthor

The current CodSpeed comparison is not representative because the benchmark fixture reuses an already-finished parser across measured iterations. I opened #318 to create and finalize a fresh parser per benchmark invocation. PR #317 should be rebased and evaluated again after #318 merges; the corrected baseline will provide a meaningful performance comparison.

@Kludex
Kludexforce-pushed the speed-up-multipart-callback-dispatch branch from c83a120 to d74fe57CompareAugust 9, 2026 12:29
@Kludex

Copy link
Copy Markdown
OwnerAuthor

Rebased onto main after #318 merged. CodSpeed will now compare PR #317 against fresh-parser, complete-request benchmark iterations. Also addressed the Cubic review in d74fe57 by testing set_callback("part_data", None) and resolving the thread.

@KludexKludex closed this Aug 10, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@Kludex