BranchesTags

Repository files navigation

UploadStream - high performance file upload streaming for dotnet

Installation

Nuget: https://www.nuget.org/packages/UploadStream

> install-package UploadStream

Background

A simplification & rewrite of recommended code for streaming multi-part file uploads as per Microsoft documentation.

Writeup on package and performance results can be read here - https://medium.com/@ma1f/file-streaming-performance-in-dotnet-4dee608dd953.

Updated for dotnet core v3.1.

Features

Optimise multi-part streaming file upload performance, offering 10x improvement in performance, and reduced memory allocation (10%-40%).

By default dotnet model form model binding loads the entire stream into memory using IEnumerable<IFormFile> - this is non-ideal for large files where processing of the stream should occur during streaming rather then buffering entire file(s) to memory/disk.

This package allows upload streams to be asynchronously processed via a delegate (StreamFiles<T>(Action<IFormFile> func), maintaining generic model binding functionality with ModelState validation.

Usage

[HttpPost("upload")]publicasyncTask<IActionResult>Upload(){// returns a generic typed model, alternatively non-generic overload if no model binding is requiredMyModelmodel=awaitthis.StreamFiles<MyModel>(async formFile =>{// implement processing of stream as required via an IFormFile interfaceusing(varstream=formfile.OpenReadStream())await ...});// ModelState is still validated from modelif(!ModelState.IsValid)
...}

Performance (Benchmark.Net)

Results

Results are normalised in comparison to default dotnet IFormFile model binding, the UploadStream package offers around 10x performance improvement and 10%-40% improvement in memory allocation.

Out of interest a comparison to file uploads via a base64 model was performed, interestingly this actually offers improved performance (1.5x - 2.5x) with a cost of increased memory allocations (0.3x - 0.1x) and increased memory heap trashing (gc gen1/gen2).

AliasFile sizesStreamFiles (us/alloc)Base64 (us/alloc)
Xs5.94 KB1.14x / 1.23x1.16x / 1.44x
Sm106.53 KB3.98x / 1.43x2.85x / 0.30x
Md865.37 KB8.66x / 1.24x2.36x / 0.09x
Lg6.04 MB9.86x / 1.09x1.97x / 0.11x
Xl21.91 MB9.16x / 1.08x1.72x / 0.14x

Upload Performance

Detailed Benchmark.Net results for file uploads, comparing base64, IFormFile and StreamFiles for a range of file sizes. Worth noting that the error and standard deviation range is much tighter for the StreamFiles method, meaning much more predictable performance.

MethodFilenameMeanErrorStdDevCI99.9% MarginGen 0Gen 1Gen 2Allocated
UploadBase64xs.png363.5 us36.83 us103.3 us36.83 us---190.4 KB
UploadBase64sm.jpg708.6 us63.66 us176.4 us63.66 us---1062.28 KB
UploadBase64md.jpg3,903.7 us125.31 us349.3 us125.31 us1000.00001000.00001000.00008100.71 KB
UploadBase64lg.jpg27,042.4 us746.13 us2,104.5 us746.13 us2000.00001000.00001000.000063570.73 KB
UploadBase64xl.exe114,938.1 us3,249.74 us9,219.0 us3,249.74 us7000.00004000.00002000.0000178189.37 KB
UploadFilexs.png422.0 us24.62 us69.05 us24.62 us---273.61 KB
UploadFilesm.jpg2,021.0 us130.21 us371.50 us130.21 us---313.6 KB
UploadFilemd.jpg9,196.8 us516.66 us1,498.93 us516.66 us---766.22 KB
UploadFilelg.jpg53,273.9 us1,071.62 us2,951.55 us1,071.62 us1000.0000--6821.3 KB
UploadFilexl.exe197,292.1 us4,373.54 us7,426.61 us4,373.54 us4000.00001000.0000-25336.69 KB
UploadStreamxs.png371.0 us16.29 us46.20 us16.29 us---222.6 KB
UploadStreamsm.jpg507.4 us25.21 us71.92 us25.21 us---219.88 KB
UploadStreammd.jpg1,061.9 us50.26 us139.27 us50.26 us---619.09 KB
UploadStreamlg.jpg5,404.5 us209.60 us601.39 us209.60 us1000.0000--6271.68 KB
UploadStreamxl.exe21,542.0 us429.47 us1,232.22 us429.47 us4000.00001000.0000-23537.77 KB

Key

  • UploadBase64 - default json model binding, convert from base64 to byte[]
  • UploadFile - default IFormFile model binding
  • UploadStream - custom StreamFiles stream processing and model binding

Multipart Load Performance - 1x Upload, 20x attached Files

Results of uploading 20x files in one upload with IEnumerable<IFormFile> compared to StreamFiles show below. Generally using the optimised StreamFiles method offers ~10-15x improvement in performance with slightly less memory allocation required.

Model binding - [DisableFormModelBinding]

Testing around Model binding appears to show that including this attribute is not required - if the model is defined in the method parameters then this is bound irrespective of if the attribute is in place or not. If no model is defined in the method parameters (which it shouldn't be when using StreamFiles) then no model binding is attempted.

About

high performance file upload streaming for dotnet

medium.com/@ma1f/file-streaming-performance-in-dotnet-4dee608dd953

Resources

Readme
MIT license
Activity

Stars

156 stars

Watchers

5 watching

Forks

36 forks
Report repository

Releases

Packages

Used by

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
BranchesTags
Repository files navigation
UploadStream - high performance file upload streaming for dotnet
Installation
Nuget: https://www.nuget.org/packages/UploadStream
> install-package UploadStream
Background
A simplification & rewrite of recommended code for streaming multi-part file uploads as per Microsoft documentation.
Writeup on package and performance results can be read here - https://medium.com/@ma1f/file-streaming-performance-in-dotnet-4dee608dd953.
Updated for dotnet core v3.1.
Features
Optimise multi-part streaming file upload performance, offering 10x improvement in performance, and reduced memory allocation (10%-40%).
By default dotnet model form model binding loads the entire stream into memory using IEnumerable<IFormFile> - this is non-ideal for large files
where processing of the stream should occur during streaming rather then buffering entire file(s) to memory/disk.
This package allows upload streams to be asynchronously processed via a delegate (StreamFiles<T>(Action<IFormFile> func),
maintaining generic model binding functionality with ModelState validation.
Usage
[HttpPost("upload")]publicasyncTask<IActionResult>Upload(){// returns a generic typed model, alternatively non-generic overload if no model binding is requiredMyModelmodel=awaitthis.StreamFiles<MyModel>(async formFile =>{// implement processing of stream as required via an IFormFile interfaceusing(varstream=formfile.OpenReadStream())await ...});// ModelState is still validated from modelif(!ModelState.IsValid)
...}
Performance (Benchmark.Net)
Results
Results are normalised in comparison to default dotnet IFormFile model binding, the UploadStream package offers around 10x performance
improvement and 10%-40% improvement in memory allocation.
Out of interest a comparison to file uploads via a base64 model was performed, interestingly this actually offers improved performance (1.5x - 2.5x)
with a cost of increased memory allocations (0.3x - 0.1x) and increased memory heap trashing (gc gen1/gen2).
AliasFile sizesStreamFiles (us/alloc)Base64 (us/alloc)
Xs5.94 KB1.14x / 1.23x1.16x / 1.44x
Sm106.53 KB3.98x / 1.43x2.85x / 0.30x
Md865.37 KB8.66x / 1.24x2.36x / 0.09x
Lg6.04 MB9.86x / 1.09x1.97x / 0.11x
Xl21.91 MB9.16x / 1.08x1.72x / 0.14x
Upload Performance
Detailed Benchmark.Net results for file uploads, comparing base64, IFormFile and StreamFiles for a range of file sizes.
Worth noting that the error and standard deviation range is much tighter for the StreamFiles method, meaning much more predictable performance.
MethodFilenameMeanErrorStdDevCI99.9% MarginGen 0Gen 1Gen 2Allocated
UploadBase64xs.png363.5 us36.83 us103.3 us36.83 us---190.4 KB
UploadBase64sm.jpg708.6 us63.66 us176.4 us63.66 us---1062.28 KB
UploadBase64md.jpg3,903.7 us125.31 us349.3 us125.31 us1000.00001000.00001000.00008100.71 KB
UploadBase64lg.jpg27,042.4 us746.13 us2,104.5 us746.13 us2000.00001000.00001000.000063570.73 KB
UploadBase64xl.exe114,938.1 us3,249.74 us9,219.0 us3,249.74 us7000.00004000.00002000.0000178189.37 KB
UploadFilexs.png422.0 us24.62 us69.05 us24.62 us---273.61 KB
UploadFilesm.jpg2,021.0 us130.21 us371.50 us130.21 us---313.6 KB
UploadFilemd.jpg9,196.8 us516.66 us1,498.93 us516.66 us---766.22 KB
UploadFilelg.jpg53,273.9 us1,071.62 us2,951.55 us1,071.62 us1000.0000--6821.3 KB
UploadFilexl.exe197,292.1 us4,373.54 us7,426.61 us4,373.54 us4000.00001000.0000-25336.69 KB
UploadStreamxs.png371.0 us16.29 us46.20 us16.29 us---222.6 KB
UploadStreamsm.jpg507.4 us25.21 us71.92 us25.21 us---219.88 KB
UploadStreammd.jpg1,061.9 us50.26 us139.27 us50.26 us---619.09 KB
UploadStreamlg.jpg5,404.5 us209.60 us601.39 us209.60 us1000.0000--6271.68 KB
UploadStreamxl.exe21,542.0 us429.47 us1,232.22 us429.47 us4000.00001000.0000-23537.77 KB
Key
  • UploadBase64 - default json model binding, convert from base64 to byte[]
  • UploadFile - default IFormFile model binding
  • UploadStream - custom StreamFiles stream processing and model binding
Multipart Load Performance - 1x Upload, 20x attached Files
Results of uploading 20x files in one upload with IEnumerable<IFormFile> compared to StreamFiles show below.
Generally using the optimised StreamFiles method offers ~10-15x improvement in performance with slightly less memory allocation required.
Model binding - [DisableFormModelBinding]
Testing around Model binding appears to show that including this attribute is not required - if the model is defined in the method parameters
then this is bound irrespective of if the attribute is in place or not. If no model is defined in the method parameters (which it shouldn't be
when using StreamFiles) then no model binding is attempted.
About
high performance file upload streaming for dotnet
medium.com/@ma1f/file-streaming-performance-in-dotnet-4dee608dd953
Resources
Readme
MIT license
Activity
Stars
156 stars
Watchers
5 watching
Forks
36 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
UploadStream - high performance file upload streaming for dotnet
Installation
Nuget: https://www.nuget.org/packages/UploadStream
> install-package UploadStream
Background
A simplification & rewrite of recommended code for streaming multi-part file uploads as per Microsoft documentation.
Writeup on package and performance results can be read here - https://medium.com/@ma1f/file-streaming-performance-in-dotnet-4dee608dd953.
Updated for dotnet core v3.1.
Features
Optimise multi-part streaming file upload performance, offering 10x improvement in performance, and reduced memory allocation (10%-40%).
By default dotnet model form model binding loads the entire stream into memory using IEnumerable<IFormFile> - this is non-ideal for large files
where processing of the stream should occur during streaming rather then buffering entire file(s) to memory/disk.
This package allows upload streams to be asynchronously processed via a delegate (StreamFiles<T>(Action<IFormFile> func),
maintaining generic model binding functionality with ModelState validation.
Usage
[HttpPost("upload")]publicasyncTask<IActionResult>Upload(){// returns a generic typed model, alternatively non-generic overload if no model binding is requiredMyModelmodel=awaitthis.StreamFiles<MyModel>(async formFile =>{// implement processing of stream as required via an IFormFile interfaceusing(varstream=formfile.OpenReadStream())await ...});// ModelState is still validated from modelif(!ModelState.IsValid)
...}
Performance (Benchmark.Net)
Results
Results are normalised in comparison to default dotnet IFormFile model binding, the UploadStream package offers around 10x performance
improvement and 10%-40% improvement in memory allocation.
Out of interest a comparison to file uploads via a base64 model was performed, interestingly this actually offers improved performance (1.5x - 2.5x)
with a cost of increased memory allocations (0.3x - 0.1x) and increased memory heap trashing (gc gen1/gen2).
AliasFile sizesStreamFiles (us/alloc)Base64 (us/alloc)
Xs5.94 KB1.14x / 1.23x1.16x / 1.44x
Sm106.53 KB3.98x / 1.43x2.85x / 0.30x
Md865.37 KB8.66x / 1.24x2.36x / 0.09x
Lg6.04 MB9.86x / 1.09x1.97x / 0.11x
Xl21.91 MB9.16x / 1.08x1.72x / 0.14x
Upload Performance
Detailed Benchmark.Net results for file uploads, comparing base64, IFormFile and StreamFiles for a range of file sizes.
Worth noting that the error and standard deviation range is much tighter for the StreamFiles method, meaning much more predictable performance.
MethodFilenameMeanErrorStdDevCI99.9% MarginGen 0Gen 1Gen 2Allocated
UploadBase64xs.png363.5 us36.83 us103.3 us36.83 us---190.4 KB
UploadBase64sm.jpg708.6 us63.66 us176.4 us63.66 us---1062.28 KB
UploadBase64md.jpg3,903.7 us125.31 us349.3 us125.31 us1000.00001000.00001000.00008100.71 KB
UploadBase64lg.jpg27,042.4 us746.13 us2,104.5 us746.13 us2000.00001000.00001000.000063570.73 KB
UploadBase64xl.exe114,938.1 us3,249.74 us9,219.0 us3,249.74 us7000.00004000.00002000.0000178189.37 KB
UploadFilexs.png422.0 us24.62 us69.05 us24.62 us---273.61 KB
UploadFilesm.jpg2,021.0 us130.21 us371.50 us130.21 us---313.6 KB
UploadFilemd.jpg9,196.8 us516.66 us1,498.93 us516.66 us---766.22 KB
UploadFilelg.jpg53,273.9 us1,071.62 us2,951.55 us1,071.62 us1000.0000--6821.3 KB
UploadFilexl.exe197,292.1 us4,373.54 us7,426.61 us4,373.54 us4000.00001000.0000-25336.69 KB
UploadStreamxs.png371.0 us16.29 us46.20 us16.29 us---222.6 KB
UploadStreamsm.jpg507.4 us25.21 us71.92 us25.21 us---219.88 KB
UploadStreammd.jpg1,061.9 us50.26 us139.27 us50.26 us---619.09 KB
UploadStreamlg.jpg5,404.5 us209.60 us601.39 us209.60 us1000.0000--6271.68 KB
UploadStreamxl.exe21,542.0 us429.47 us1,232.22 us429.47 us4000.00001000.0000-23537.77 KB
Key
  • UploadBase64 - default json model binding, convert from base64 to byte[]
  • UploadFile - default IFormFile model binding
  • UploadStream - custom StreamFiles stream processing and model binding
Multipart Load Performance - 1x Upload, 20x attached Files
Results of uploading 20x files in one upload with IEnumerable<IFormFile> compared to StreamFiles show below.
Generally using the optimised StreamFiles method offers ~10-15x improvement in performance with slightly less memory allocation required.
Model binding - [DisableFormModelBinding]
Testing around Model binding appears to show that including this attribute is not required - if the model is defined in the method parameters
then this is bound irrespective of if the attribute is in place or not. If no model is defined in the method parameters (which it shouldn't be
when using StreamFiles) then no model binding is attempted.
About
high performance file upload streaming for dotnet
medium.com/@ma1f/file-streaming-performance-in-dotnet-4dee608dd953
Resources
Readme
MIT license
Activity
Stars
156 stars
Watchers
5 watching
Forks
36 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
UploadStream - high performance file upload streaming for dotnet
Installation
Nuget: https://www.nuget.org/packages/UploadStream
> install-package UploadStream
Background
A simplification & rewrite of recommended code for streaming multi-part file uploads as per Microsoft documentation.
Writeup on package and performance results can be read here - https://medium.com/@ma1f/file-streaming-performance-in-dotnet-4dee608dd953.
Updated for dotnet core v3.1.
Features
Optimise multi-part streaming file upload performance, offering 10x improvement in performance, and reduced memory allocation (10%-40%).
By default dotnet model form model binding loads the entire stream into memory using IEnumerable<IFormFile> - this is non-ideal for large files
where processing of the stream should occur during streaming rather then buffering entire file(s) to memory/disk.
This package allows upload streams to be asynchronously processed via a delegate (StreamFiles<T>(Action<IFormFile> func),
maintaining generic model binding functionality with ModelState validation.
Usage
[HttpPost("upload")]publicasyncTask<IActionResult>Upload(){// returns a generic typed model, alternatively non-generic overload if no model binding is requiredMyModelmodel=awaitthis.StreamFiles<MyModel>(async formFile =>{// implement processing of stream as required via an IFormFile interfaceusing(varstream=formfile.OpenReadStream())await ...});// ModelState is still validated from modelif(!ModelState.IsValid)
...}
Performance (Benchmark.Net)
Results
Results are normalised in comparison to default dotnet IFormFile model binding, the UploadStream package offers around 10x performance
improvement and 10%-40% improvement in memory allocation.
Out of interest a comparison to file uploads via a base64 model was performed, interestingly this actually offers improved performance (1.5x - 2.5x)
with a cost of increased memory allocations (0.3x - 0.1x) and increased memory heap trashing (gc gen1/gen2).
AliasFile sizesStreamFiles (us/alloc)Base64 (us/alloc)
Xs5.94 KB1.14x / 1.23x1.16x / 1.44x
Sm106.53 KB3.98x / 1.43x2.85x / 0.30x
Md865.37 KB8.66x / 1.24x2.36x / 0.09x
Lg6.04 MB9.86x / 1.09x1.97x / 0.11x
Xl21.91 MB9.16x / 1.08x1.72x / 0.14x
Upload Performance
Detailed Benchmark.Net results for file uploads, comparing base64, IFormFile and StreamFiles for a range of file sizes.
Worth noting that the error and standard deviation range is much tighter for the StreamFiles method, meaning much more predictable performance.
MethodFilenameMeanErrorStdDevCI99.9% MarginGen 0Gen 1Gen 2Allocated
UploadBase64xs.png363.5 us36.83 us103.3 us36.83 us---190.4 KB
UploadBase64sm.jpg708.6 us63.66 us176.4 us63.66 us---1062.28 KB
UploadBase64md.jpg3,903.7 us125.31 us349.3 us125.31 us1000.00001000.00001000.00008100.71 KB
UploadBase64lg.jpg27,042.4 us746.13 us2,104.5 us746.13 us2000.00001000.00001000.000063570.73 KB
UploadBase64xl.exe114,938.1 us3,249.74 us9,219.0 us3,249.74 us7000.00004000.00002000.0000178189.37 KB
UploadFilexs.png422.0 us24.62 us69.05 us24.62 us---273.61 KB
UploadFilesm.jpg2,021.0 us130.21 us371.50 us130.21 us---313.6 KB
UploadFilemd.jpg9,196.8 us516.66 us1,498.93 us516.66 us---766.22 KB
UploadFilelg.jpg53,273.9 us1,071.62 us2,951.55 us1,071.62 us1000.0000--6821.3 KB
UploadFilexl.exe197,292.1 us4,373.54 us7,426.61 us4,373.54 us4000.00001000.0000-25336.69 KB
UploadStreamxs.png371.0 us16.29 us46.20 us16.29 us---222.6 KB
UploadStreamsm.jpg507.4 us25.21 us71.92 us25.21 us---219.88 KB
UploadStreammd.jpg1,061.9 us50.26 us139.27 us50.26 us---619.09 KB
UploadStreamlg.jpg5,404.5 us209.60 us601.39 us209.60 us1000.0000--6271.68 KB
UploadStreamxl.exe21,542.0 us429.47 us1,232.22 us429.47 us4000.00001000.0000-23537.77 KB
Key
  • UploadBase64 - default json model binding, convert from base64 to byte[]
  • UploadFile - default IFormFile model binding
  • UploadStream - custom StreamFiles stream processing and model binding
Multipart Load Performance - 1x Upload, 20x attached Files
Results of uploading 20x files in one upload with IEnumerable<IFormFile> compared to StreamFiles show below.
Generally using the optimised StreamFiles method offers ~10-15x improvement in performance with slightly less memory allocation required.
Model binding - [DisableFormModelBinding]
Testing around Model binding appears to show that including this attribute is not required - if the model is defined in the method parameters
then this is bound irrespective of if the attribute is in place or not. If no model is defined in the method parameters (which it shouldn't be
when using StreamFiles) then no model binding is attempted.
About
high performance file upload streaming for dotnet
medium.com/@ma1f/file-streaming-performance-in-dotnet-4dee608dd953
Resources
Readme
MIT license
Activity
Stars
156 stars
Watchers
5 watching
Forks
36 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
BranchesTags
Repository files navigation
UploadStream - high performance file upload streaming for dotnet
Installation
Nuget: https://www.nuget.org/packages/UploadStream
> install-package UploadStream
Background
A simplification & rewrite of recommended code for streaming multi-part file uploads as per Microsoft documentation.
Writeup on package and performance results can be read here - https://medium.com/@ma1f/file-streaming-performance-in-dotnet-4dee608dd953.
Updated for dotnet core v3.1.
Features
Optimise multi-part streaming file upload performance, offering 10x improvement in performance, and reduced memory allocation (10%-40%).
By default dotnet model form model binding loads the entire stream into memory using IEnumerable<IFormFile> - this is non-ideal for large files
where processing of the stream should occur during streaming rather then buffering entire file(s) to memory/disk.
This package allows upload streams to be asynchronously processed via a delegate (StreamFiles<T>(Action<IFormFile> func),
maintaining generic model binding functionality with ModelState validation.
Usage
[HttpPost("upload")]publicasyncTask<IActionResult>Upload(){// returns a generic typed model, alternatively non-generic overload if no model binding is requiredMyModelmodel=awaitthis.StreamFiles<MyModel>(async formFile =>{// implement processing of stream as required via an IFormFile interfaceusing(varstream=formfile.OpenReadStream())await ...});// ModelState is still validated from modelif(!ModelState.IsValid)
...}
Performance (Benchmark.Net)
Results
Results are normalised in comparison to default dotnet IFormFile model binding, the UploadStream package offers around 10x performance
improvement and 10%-40% improvement in memory allocation.
Out of interest a comparison to file uploads via a base64 model was performed, interestingly this actually offers improved performance (1.5x - 2.5x)
with a cost of increased memory allocations (0.3x - 0.1x) and increased memory heap trashing (gc gen1/gen2).
AliasFile sizesStreamFiles (us/alloc)Base64 (us/alloc)
Xs5.94 KB1.14x / 1.23x1.16x / 1.44x
Sm106.53 KB3.98x / 1.43x2.85x / 0.30x
Md865.37 KB8.66x / 1.24x2.36x / 0.09x
Lg6.04 MB9.86x / 1.09x1.97x / 0.11x
Xl21.91 MB9.16x / 1.08x1.72x / 0.14x
Upload Performance
Detailed Benchmark.Net results for file uploads, comparing base64, IFormFile and StreamFiles for a range of file sizes.
Worth noting that the error and standard deviation range is much tighter for the StreamFiles method, meaning much more predictable performance.
MethodFilenameMeanErrorStdDevCI99.9% MarginGen 0Gen 1Gen 2Allocated
UploadBase64xs.png363.5 us36.83 us103.3 us36.83 us---190.4 KB
UploadBase64sm.jpg708.6 us63.66 us176.4 us63.66 us---1062.28 KB
UploadBase64md.jpg3,903.7 us125.31 us349.3 us125.31 us1000.00001000.00001000.00008100.71 KB
UploadBase64lg.jpg27,042.4 us746.13 us2,104.5 us746.13 us2000.00001000.00001000.000063570.73 KB
UploadBase64xl.exe114,938.1 us3,249.74 us9,219.0 us3,249.74 us7000.00004000.00002000.0000178189.37 KB
UploadFilexs.png422.0 us24.62 us69.05 us24.62 us---273.61 KB
UploadFilesm.jpg2,021.0 us130.21 us371.50 us130.21 us---313.6 KB
UploadFilemd.jpg9,196.8 us516.66 us1,498.93 us516.66 us---766.22 KB
UploadFilelg.jpg53,273.9 us1,071.62 us2,951.55 us1,071.62 us1000.0000--6821.3 KB
UploadFilexl.exe197,292.1 us4,373.54 us7,426.61 us4,373.54 us4000.00001000.0000-25336.69 KB
UploadStreamxs.png371.0 us16.29 us46.20 us16.29 us---222.6 KB
UploadStreamsm.jpg507.4 us25.21 us71.92 us25.21 us---219.88 KB
UploadStreammd.jpg1,061.9 us50.26 us139.27 us50.26 us---619.09 KB
UploadStreamlg.jpg5,404.5 us209.60 us601.39 us209.60 us1000.0000--6271.68 KB
UploadStreamxl.exe21,542.0 us429.47 us1,232.22 us429.47 us4000.00001000.0000-23537.77 KB
Key
  • UploadBase64 - default json model binding, convert from base64 to byte[]
  • UploadFile - default IFormFile model binding
  • UploadStream - custom StreamFiles stream processing and model binding
Multipart Load Performance - 1x Upload, 20x attached Files
Results of uploading 20x files in one upload with IEnumerable<IFormFile> compared to StreamFiles show below.
Generally using the optimised StreamFiles method offers ~10-15x improvement in performance with slightly less memory allocation required.
Model binding - [DisableFormModelBinding]
Testing around Model binding appears to show that including this attribute is not required - if the model is defined in the method parameters
then this is bound irrespective of if the attribute is in place or not. If no model is defined in the method parameters (which it shouldn't be
when using StreamFiles) then no model binding is attempted.
About
high performance file upload streaming for dotnet
medium.com/@ma1f/file-streaming-performance-in-dotnet-4dee608dd953
Resources
Readme
MIT license
Activity
Stars
156 stars
Watchers
5 watching
Forks
36 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
UploadStream - high performance file upload streaming for dotnet
Installation
Nuget: https://www.nuget.org/packages/UploadStream
> install-package UploadStream
Background
A simplification & rewrite of recommended code for streaming multi-part file uploads as per Microsoft documentation.
Writeup on package and performance results can be read here - https://medium.com/@ma1f/file-streaming-performance-in-dotnet-4dee608dd953.
Updated for dotnet core v3.1.
Features
Optimise multi-part streaming file upload performance, offering 10x improvement in performance, and reduced memory allocation (10%-40%).
By default dotnet model form model binding loads the entire stream into memory using IEnumerable<IFormFile> - this is non-ideal for large files
where processing of the stream should occur during streaming rather then buffering entire file(s) to memory/disk.
This package allows upload streams to be asynchronously processed via a delegate (StreamFiles<T>(Action<IFormFile> func),
maintaining generic model binding functionality with ModelState validation.
Usage
[HttpPost("upload")]publicasyncTask<IActionResult>Upload(){// returns a generic typed model, alternatively non-generic overload if no model binding is requiredMyModelmodel=awaitthis.StreamFiles<MyModel>(async formFile =>{// implement processing of stream as required via an IFormFile interfaceusing(varstream=formfile.OpenReadStream())await ...});// ModelState is still validated from modelif(!ModelState.IsValid)
...}
Performance (Benchmark.Net)
Results
Results are normalised in comparison to default dotnet IFormFile model binding, the UploadStream package offers around 10x performance
improvement and 10%-40% improvement in memory allocation.
Out of interest a comparison to file uploads via a base64 model was performed, interestingly this actually offers improved performance (1.5x - 2.5x)
with a cost of increased memory allocations (0.3x - 0.1x) and increased memory heap trashing (gc gen1/gen2).
AliasFile sizesStreamFiles (us/alloc)Base64 (us/alloc)
Xs5.94 KB1.14x / 1.23x1.16x / 1.44x
Sm106.53 KB3.98x / 1.43x2.85x / 0.30x
Md865.37 KB8.66x / 1.24x2.36x / 0.09x
Lg6.04 MB9.86x / 1.09x1.97x / 0.11x
Xl21.91 MB9.16x / 1.08x1.72x / 0.14x
Upload Performance
Detailed Benchmark.Net results for file uploads, comparing base64, IFormFile and StreamFiles for a range of file sizes.
Worth noting that the error and standard deviation range is much tighter for the StreamFiles method, meaning much more predictable performance.
MethodFilenameMeanErrorStdDevCI99.9% MarginGen 0Gen 1Gen 2Allocated
UploadBase64xs.png363.5 us36.83 us103.3 us36.83 us---190.4 KB
UploadBase64sm.jpg708.6 us63.66 us176.4 us63.66 us---1062.28 KB
UploadBase64md.jpg3,903.7 us125.31 us349.3 us125.31 us1000.00001000.00001000.00008100.71 KB
UploadBase64lg.jpg27,042.4 us746.13 us2,104.5 us746.13 us2000.00001000.00001000.000063570.73 KB
UploadBase64xl.exe114,938.1 us3,249.74 us9,219.0 us3,249.74 us7000.00004000.00002000.0000178189.37 KB
UploadFilexs.png422.0 us24.62 us69.05 us24.62 us---273.61 KB
UploadFilesm.jpg2,021.0 us130.21 us371.50 us130.21 us---313.6 KB
UploadFilemd.jpg9,196.8 us516.66 us1,498.93 us516.66 us---766.22 KB
UploadFilelg.jpg53,273.9 us1,071.62 us2,951.55 us1,071.62 us1000.0000--6821.3 KB
UploadFilexl.exe197,292.1 us4,373.54 us7,426.61 us4,373.54 us4000.00001000.0000-25336.69 KB
UploadStreamxs.png371.0 us16.29 us46.20 us16.29 us---222.6 KB
UploadStreamsm.jpg507.4 us25.21 us71.92 us25.21 us---219.88 KB
UploadStreammd.jpg1,061.9 us50.26 us139.27 us50.26 us---619.09 KB
UploadStreamlg.jpg5,404.5 us209.60 us601.39 us209.60 us1000.0000--6271.68 KB
UploadStreamxl.exe21,542.0 us429.47 us1,232.22 us429.47 us4000.00001000.0000-23537.77 KB
Key
  • UploadBase64 - default json model binding, convert from base64 to byte[]
  • UploadFile - default IFormFile model binding
  • UploadStream - custom StreamFiles stream processing and model binding
Multipart Load Performance - 1x Upload, 20x attached Files
Results of uploading 20x files in one upload with IEnumerable<IFormFile> compared to StreamFiles show below.
Generally using the optimised StreamFiles method offers ~10-15x improvement in performance with slightly less memory allocation required.
Model binding - [DisableFormModelBinding]
Testing around Model binding appears to show that including this attribute is not required - if the model is defined in the method parameters
then this is bound irrespective of if the attribute is in place or not. If no model is defined in the method parameters (which it shouldn't be
when using StreamFiles) then no model binding is attempted.
About
high performance file upload streaming for dotnet
medium.com/@ma1f/file-streaming-performance-in-dotnet-4dee608dd953
Resources
Readme
MIT license
Activity
Stars
156 stars
Watchers
5 watching
Forks
36 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
UploadStream - high performance file upload streaming for dotnet
Installation
Nuget: https://www.nuget.org/packages/UploadStream
> install-package UploadStream
Background
A simplification & rewrite of recommended code for streaming multi-part file uploads as per Microsoft documentation.
Writeup on package and performance results can be read here - https://medium.com/@ma1f/file-streaming-performance-in-dotnet-4dee608dd953.
Updated for dotnet core v3.1.
Features
Optimise multi-part streaming file upload performance, offering 10x improvement in performance, and reduced memory allocation (10%-40%).
By default dotnet model form model binding loads the entire stream into memory using IEnumerable<IFormFile> - this is non-ideal for large files
where processing of the stream should occur during streaming rather then buffering entire file(s) to memory/disk.
This package allows upload streams to be asynchronously processed via a delegate (StreamFiles<T>(Action<IFormFile> func),
maintaining generic model binding functionality with ModelState validation.
Usage
[HttpPost("upload")]publicasyncTask<IActionResult>Upload(){// returns a generic typed model, alternatively non-generic overload if no model binding is requiredMyModelmodel=awaitthis.StreamFiles<MyModel>(async formFile =>{// implement processing of stream as required via an IFormFile interfaceusing(varstream=formfile.OpenReadStream())await ...});// ModelState is still validated from modelif(!ModelState.IsValid)
...}
Performance (Benchmark.Net)
Results
Results are normalised in comparison to default dotnet IFormFile model binding, the UploadStream package offers around 10x performance
improvement and 10%-40% improvement in memory allocation.
Out of interest a comparison to file uploads via a base64 model was performed, interestingly this actually offers improved performance (1.5x - 2.5x)
with a cost of increased memory allocations (0.3x - 0.1x) and increased memory heap trashing (gc gen1/gen2).
AliasFile sizesStreamFiles (us/alloc)Base64 (us/alloc)
Xs5.94 KB1.14x / 1.23x1.16x / 1.44x
Sm106.53 KB3.98x / 1.43x2.85x / 0.30x
Md865.37 KB8.66x / 1.24x2.36x / 0.09x
Lg6.04 MB9.86x / 1.09x1.97x / 0.11x
Xl21.91 MB9.16x / 1.08x1.72x / 0.14x
Upload Performance
Detailed Benchmark.Net results for file uploads, comparing base64, IFormFile and StreamFiles for a range of file sizes.
Worth noting that the error and standard deviation range is much tighter for the StreamFiles method, meaning much more predictable performance.
MethodFilenameMeanErrorStdDevCI99.9% MarginGen 0Gen 1Gen 2Allocated
UploadBase64xs.png363.5 us36.83 us103.3 us36.83 us---190.4 KB
UploadBase64sm.jpg708.6 us63.66 us176.4 us63.66 us---1062.28 KB
UploadBase64md.jpg3,903.7 us125.31 us349.3 us125.31 us1000.00001000.00001000.00008100.71 KB
UploadBase64lg.jpg27,042.4 us746.13 us2,104.5 us746.13 us2000.00001000.00001000.000063570.73 KB
UploadBase64xl.exe114,938.1 us3,249.74 us9,219.0 us3,249.74 us7000.00004000.00002000.0000178189.37 KB
UploadFilexs.png422.0 us24.62 us69.05 us24.62 us---273.61 KB
UploadFilesm.jpg2,021.0 us130.21 us371.50 us130.21 us---313.6 KB
UploadFilemd.jpg9,196.8 us516.66 us1,498.93 us516.66 us---766.22 KB
UploadFilelg.jpg53,273.9 us1,071.62 us2,951.55 us1,071.62 us1000.0000--6821.3 KB
UploadFilexl.exe197,292.1 us4,373.54 us7,426.61 us4,373.54 us4000.00001000.0000-25336.69 KB
UploadStreamxs.png371.0 us16.29 us46.20 us16.29 us---222.6 KB
UploadStreamsm.jpg507.4 us25.21 us71.92 us25.21 us---219.88 KB
UploadStreammd.jpg1,061.9 us50.26 us139.27 us50.26 us---619.09 KB
UploadStreamlg.jpg5,404.5 us209.60 us601.39 us209.60 us1000.0000--6271.68 KB
UploadStreamxl.exe21,542.0 us429.47 us1,232.22 us429.47 us4000.00001000.0000-23537.77 KB
Key
  • UploadBase64 - default json model binding, convert from base64 to byte[]
  • UploadFile - default IFormFile model binding
  • UploadStream - custom StreamFiles stream processing and model binding
Multipart Load Performance - 1x Upload, 20x attached Files
Results of uploading 20x files in one upload with IEnumerable<IFormFile> compared to StreamFiles show below.
Generally using the optimised StreamFiles method offers ~10-15x improvement in performance with slightly less memory allocation required.
Model binding - [DisableFormModelBinding]
Testing around Model binding appears to show that including this attribute is not required - if the model is defined in the method parameters
then this is bound irrespective of if the attribute is in place or not. If no model is defined in the method parameters (which it shouldn't be
when using StreamFiles) then no model binding is attempted.
About
high performance file upload streaming for dotnet
medium.com/@ma1f/file-streaming-performance-in-dotnet-4dee608dd953
Resources
Readme
MIT license
Activity
Stars
156 stars
Watchers
5 watching
Forks
36 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
BranchesTags
Repository files navigation
UploadStream - high performance file upload streaming for dotnet
Installation
Nuget: https://www.nuget.org/packages/UploadStream
> install-package UploadStream
Background
A simplification & rewrite of recommended code for streaming multi-part file uploads as per Microsoft documentation.
Writeup on package and performance results can be read here - https://medium.com/@ma1f/file-streaming-performance-in-dotnet-4dee608dd953.
Updated for dotnet core v3.1.
Features
Optimise multi-part streaming file upload performance, offering 10x improvement in performance, and reduced memory allocation (10%-40%).
By default dotnet model form model binding loads the entire stream into memory using IEnumerable<IFormFile> - this is non-ideal for large files
where processing of the stream should occur during streaming rather then buffering entire file(s) to memory/disk.
This package allows upload streams to be asynchronously processed via a delegate (StreamFiles<T>(Action<IFormFile> func),
maintaining generic model binding functionality with ModelState validation.
Usage
[HttpPost("upload")]publicasyncTask<IActionResult>Upload(){// returns a generic typed model, alternatively non-generic overload if no model binding is requiredMyModelmodel=awaitthis.StreamFiles<MyModel>(async formFile =>{// implement processing of stream as required via an IFormFile interfaceusing(varstream=formfile.OpenReadStream())await ...});// ModelState is still validated from modelif(!ModelState.IsValid)
...}
Performance (Benchmark.Net)
Results
Results are normalised in comparison to default dotnet IFormFile model binding, the UploadStream package offers around 10x performance
improvement and 10%-40% improvement in memory allocation.
Out of interest a comparison to file uploads via a base64 model was performed, interestingly this actually offers improved performance (1.5x - 2.5x)
with a cost of increased memory allocations (0.3x - 0.1x) and increased memory heap trashing (gc gen1/gen2).
AliasFile sizesStreamFiles (us/alloc)Base64 (us/alloc)
Xs5.94 KB1.14x / 1.23x1.16x / 1.44x
Sm106.53 KB3.98x / 1.43x2.85x / 0.30x
Md865.37 KB8.66x / 1.24x2.36x / 0.09x
Lg6.04 MB9.86x / 1.09x1.97x / 0.11x
Xl21.91 MB9.16x / 1.08x1.72x / 0.14x
Upload Performance
Detailed Benchmark.Net results for file uploads, comparing base64, IFormFile and StreamFiles for a range of file sizes.
Worth noting that the error and standard deviation range is much tighter for the StreamFiles method, meaning much more predictable performance.
MethodFilenameMeanErrorStdDevCI99.9% MarginGen 0Gen 1Gen 2Allocated
UploadBase64xs.png363.5 us36.83 us103.3 us36.83 us---190.4 KB
UploadBase64sm.jpg708.6 us63.66 us176.4 us63.66 us---1062.28 KB
UploadBase64md.jpg3,903.7 us125.31 us349.3 us125.31 us1000.00001000.00001000.00008100.71 KB
UploadBase64lg.jpg27,042.4 us746.13 us2,104.5 us746.13 us2000.00001000.00001000.000063570.73 KB
UploadBase64xl.exe114,938.1 us3,249.74 us9,219.0 us3,249.74 us7000.00004000.00002000.0000178189.37 KB
UploadFilexs.png422.0 us24.62 us69.05 us24.62 us---273.61 KB
UploadFilesm.jpg2,021.0 us130.21 us371.50 us130.21 us---313.6 KB
UploadFilemd.jpg9,196.8 us516.66 us1,498.93 us516.66 us---766.22 KB
UploadFilelg.jpg53,273.9 us1,071.62 us2,951.55 us1,071.62 us1000.0000--6821.3 KB
UploadFilexl.exe197,292.1 us4,373.54 us7,426.61 us4,373.54 us4000.00001000.0000-25336.69 KB
UploadStreamxs.png371.0 us16.29 us46.20 us16.29 us---222.6 KB
UploadStreamsm.jpg507.4 us25.21 us71.92 us25.21 us---219.88 KB
UploadStreammd.jpg1,061.9 us50.26 us139.27 us50.26 us---619.09 KB
UploadStreamlg.jpg5,404.5 us209.60 us601.39 us209.60 us1000.0000--6271.68 KB
UploadStreamxl.exe21,542.0 us429.47 us1,232.22 us429.47 us4000.00001000.0000-23537.77 KB
Key
  • UploadBase64 - default json model binding, convert from base64 to byte[]
  • UploadFile - default IFormFile model binding
  • UploadStream - custom StreamFiles stream processing and model binding
Multipart Load Performance - 1x Upload, 20x attached Files
Results of uploading 20x files in one upload with IEnumerable<IFormFile> compared to StreamFiles show below.
Generally using the optimised StreamFiles method offers ~10-15x improvement in performance with slightly less memory allocation required.
Model binding - [DisableFormModelBinding]
Testing around Model binding appears to show that including this attribute is not required - if the model is defined in the method parameters
then this is bound irrespective of if the attribute is in place or not. If no model is defined in the method parameters (which it shouldn't be
when using StreamFiles) then no model binding is attempted.
About
high performance file upload streaming for dotnet
medium.com/@ma1f/file-streaming-performance-in-dotnet-4dee608dd953
Resources
Readme
MIT license
Activity
Stars
156 stars
Watchers
5 watching
Forks
36 forks
Report repository
Releases
Packages
Used by
Contributors
Languages