Skip to content

FileSystem.Unix: improve CopyFile. - #59695

Merged
jeffhandley merged 9 commits into
dotnet:mainfrom
tmds:unix_filecopy
Nov 15, 2021
Merged

FileSystem.Unix: improve CopyFile.#59695
jeffhandley merged 9 commits into
dotnet:mainfrom
tmds:unix_filecopy

Conversation

@tmds

@tmdstmds commented Sep 28, 2021

Copy link
Copy Markdown
Member

Like the upcoming version of GNU coreutils 'cp' prefer a copy-on-write clone.
This shares the physical storage between files, which means no data needs to copied.
CoW-clones are supported by a number of Linux file systems, like Btrfs, XFS, and overlayfs.

Eliminate a 'stat' call that is always performed for checking if the target is a directory
by only performing the check when the 'open' syscall reports an error.

Eliminate a 'stat' call for retrieving the file size of the source by passing through
the length that was retrieved when checking the opened file is not a directory.

Create the destination with file permissions that match the source.
We still need to fchmod due to umask being applied to the open mode.

When performing a manual copy, limit the allocated buffer for small files.
And, avoid the last 'read' call by checking when we've copied the expected nr of bytes.

@ghostghost added community-contribution Indicates that the PR has been added by a community member area-System.IO labels Sep 28, 2021
@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

Issue Details

Like the upcoming version of GNU coreutils 'cp' prefer a copy-on-write clone.
This shares the physical storage between files, which means no data needs to copied.
CoW-clones are supported by a number of Linux file systems, like Btrfs, XFS, and overlayfs.

Eliminate a 'stat' call that is always performed for checking if the target is a directory
by only performing the check when the 'open' syscall reports an error.

Eliminate a 'stat' call for retrieving the file size of the source by passing through
the length that was retrieved when checking the opened file is not a directory.

Create the destination with file permissions that match the source.
We still need to fchmod due to umask being applied to the open mode.

When performing a manual copy, limit the allocated buffer for small files.
And, avoid the last 'read' call by checking when we've copied the expected nr of bytes.

Author:tmds
Assignees:-
Labels:

area-System.IO, community-contribution

Milestone:-

@adamsitnik

Copy link
Copy Markdown
Member

Hi @tmds

Could you please provide some benchmark results for small, medium and big files? The change is non-trivial, it would be good to make sure it's worth the complexity.

Thanks!

@tmds

tmds commented Oct 19, 2021

Copy link
Copy Markdown
MemberAuthor

For small files, we see an improvement due to eliminating syscalls.

For large files, this doesn't weigh in, and we see a huge gain on the filesystem that support CoW (my home partition uses Btrfs which supports it, while /tmp doesn't support it).

MethodJobToolchainSourceSizeBaseDirMeanErrorStdDevMedianRatioRatioSD
FileCopyJob-ABJWHG/orig/corerun0/home/tmds5.815 ms0.1136 ms0.1700 ms5.830 ms1.000.00
FileCopyJob-EQFSPF/pr/corerun0/home/tmds5.731 ms0.1138 ms0.0888 ms5.714 ms0.970.04
FileCopyJob-ABJWHG/orig/corerun0/tmp/1.470 ms0.0065 ms0.0061 ms1.469 ms1.000.00
FileCopyJob-EQFSPF/pr/corerun0/tmp/1.274 ms0.0086 ms0.0076 ms1.273 ms0.870.01
FileCopyJob-ABJWHG/orig/corerun1/home/tmds7.611 ms0.1449 ms0.2213 ms7.623 ms1.000.00
FileCopyJob-EQFSPF/pr/corerun1/home/tmds6.828 ms0.1350 ms0.1607 ms6.847 ms0.890.03
FileCopyJob-ABJWHG/orig/corerun1/tmp/1.970 ms0.0215 ms0.0191 ms1.967 ms1.000.00
FileCopyJob-EQFSPF/pr/corerun1/tmp/1.700 ms0.0085 ms0.0075 ms1.700 ms0.860.01
FileCopyJob-ABJWHG/orig/corerun100/home/tmds7.442 ms0.1417 ms0.1740 ms7.424 ms1.000.00
FileCopyJob-EQFSPF/pr/corerun100/home/tmds6.507 ms0.1284 ms0.2036 ms6.375 ms0.880.04
FileCopyJob-ABJWHG/orig/corerun100/tmp/1.945 ms0.0285 ms0.0266 ms1.951 ms1.000.00
FileCopyJob-EQFSPF/pr/corerun100/tmp/1.717 ms0.0341 ms0.0560 ms1.698 ms0.910.04
FileCopyJob-ABJWHG/orig/corerun512/home/tmds7.458 ms0.1465 ms0.1505 ms7.478 ms1.000.00
FileCopyJob-EQFSPF/pr/corerun512/home/tmds6.480 ms0.1269 ms0.1737 ms6.448 ms0.870.03
FileCopyJob-ABJWHG/orig/corerun512/tmp/1.892 ms0.0107 ms0.0100 ms1.893 ms1.000.00
FileCopyJob-EQFSPF/pr/corerun512/tmp/1.697 ms0.0086 ms0.0067 ms1.694 ms0.900.01
FileCopyJob-ABJWHG/orig/corerun1024/home/tmds7.435 ms0.1481 ms0.2670 ms7.448 ms1.000.00
FileCopyJob-EQFSPF/pr/corerun1024/home/tmds6.426 ms0.1271 ms0.1608 ms6.435 ms0.870.04
FileCopyJob-ABJWHG/orig/corerun1024/tmp/1.988 ms0.0136 ms0.0121 ms1.989 ms1.000.00
FileCopyJob-EQFSPF/pr/corerun1024/tmp/1.709 ms0.0138 ms0.0122 ms1.710 ms0.860.01
FileCopyJob-ABJWHG/orig/corerun4096/home/tmds7.241 ms0.1404 ms0.1725 ms7.182 ms1.000.00
FileCopyJob-EQFSPF/pr/corerun4096/home/tmds6.469 ms0.1280 ms0.2138 ms6.414 ms0.900.03
FileCopyJob-ABJWHG/orig/corerun4096/tmp/1.940 ms0.0190 ms0.0178 ms1.944 ms1.000.00
FileCopyJob-EQFSPF/pr/corerun4096/tmp/1.679 ms0.0063 ms0.0052 ms1.679 ms0.860.01
FileCopyJob-ABJWHG/orig/corerun1048576/home/tmds42.591 ms0.3892 ms0.3250 ms42.655 ms1.000.00
FileCopyJob-EQFSPF/pr/corerun1048576/home/tmds6.524 ms0.1250 ms0.1488 ms6.540 ms0.150.00
FileCopyJob-ABJWHG/orig/corerun1048576/tmp/30.270 ms0.2621 ms0.2451 ms30.248 ms1.000.00
FileCopyJob-EQFSPF/pr/corerun1048576/tmp/30.401 ms0.5868 ms0.8602 ms30.213 ms1.020.03
FileCopyJob-ABJWHG/orig/corerun10485760/home/tmds401.622 ms2.6749 ms2.2337 ms401.577 ms1.000.00
FileCopyJob-EQFSPF/pr/corerun10485760/home/tmds6.391 ms0.1265 ms0.2344 ms6.326 ms0.020.00
FileCopyJob-ABJWHG/orig/corerun10485760/tmp/368.813 ms6.8000 ms6.6785 ms368.475 ms1.000.00
FileCopyJob-EQFSPF/pr/corerun10485760/tmp/369.906 ms5.2226 ms4.8852 ms370.370 ms1.000.02

Benchmark:

usingSystem;usingSystem.Collections.Generic;usingSystem.IO;usingBenchmarkDotNet;usingBenchmarkDotNet.Attributes;namespaceFileCopyBenchmark{publicclassBenchmarks{privateconstintkB=1<<10;privateconstintMB=kB<<10;[Params(0,1,100,512,1*kB,4*kB,1*MB,10*MB)]publicintSourceSize;[ParamsSource(nameof(ValuesForBaseDir))]publicstringBaseDir{get;set;}publicIEnumerable<string>ValuesForBaseDir=>new[]{Path.GetTempPath(),Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments)};privatestringSourceFileName;privatestringDestinationFileName;[GlobalSetup]publicvoidGlobalSetup(){DestinationFileName=Path.Combine(BaseDir,Guid.NewGuid().ToString());SourceFileName=Path.Combine(BaseDir,Guid.NewGuid().ToString());usingFileStreamrandom=File.OpenRead("/dev/random");usingFileStreamsourceFile=File.OpenWrite(SourceFileName);Span<byte>buffer=stackallocbyte[4*kB];intlength=SourceSize;while(length>0){intread=random.Read(buffer.Slice(0,Math.Min(length,buffer.Length)));sourceFile.Write(buffer.Slice(0,read));length-=read;}}[GlobalCleanup]publicvoidGlobalCleanup(){File.Delete(SourceFileName);File.Delete(DestinationFileName);}[Benchmark]publicvoidFileCopy(){for(inti=0;i<100;i++){File.Delete(DestinationFileName);File.Copy(SourceFileName,DestinationFileName);}}}}

@stephentoub

Copy link
Copy Markdown
Member

we see a huge gain on the filesystem that support CoW

What is the impact on subsequent writes to the file?

@tmds

tmds commented Oct 19, 2021

Copy link
Copy Markdown
MemberAuthor

What is the impact on subsequent writes to the file?

The copies get avoided by the CoW, but then when you write to the file, you are making those copies again.
The copies are made at the block level, so only for the parts of the file that are written to.

Updating the benchmark to overwrite half of the file after File.Copy shows the 50% ratio in the benchmark results.

[Benchmark]publicvoidFileCopy(){Span<byte>buffer=stackallocbyte[4*kB];for(inti=0;i<100;i++){File.Delete(DestinationFileName);File.Copy(SourceFileName,DestinationFileName);usingSafeFileHandlewriteHandle=File.OpenHandle(DestinationFileName,FileMode.Open,FileAccess.Write,FileShare.Write);for(intoffset=SourceSize/2;offset<SourceSize;){RandomAccess.Write(writeHandle,buffer,offset);offset+=buffer.Length;}}}
MethodJobToolchainSourceSizeBaseDirMeanErrorStdDevRatio
FileCopyJob-XVSSYH/orig/corerun10485760/home/tmds974.1 ms6.15 ms5.14 ms1.00
FileCopyJob-QDDOUC/pr/corerun10485760/home/tmds484.8 ms3.47 ms3.25 ms0.50

Like the upcoming version of GNU coreutils 'cp' prefer a copy-on-write clone.
This shares the physical storage between files, which means no data needs to copied.
CoW-clones are supported by a number of Linux file systems, like Btrfs, XFS, and overlayfs.
Eliminate a 'stat' call that is always performed for checking if the target is a directory
by only performing the check when the 'open' syscall reports an error.
Eliminate a 'stat' call for retrieving the file size of the source by passing through
the length that was retrieved when checking the opened file is not a directory.
Create the destination with file permissions that match the source.
We still need to fchmod due to umask being applied to the open mode.
When performing a manual copy, limit the allocated buffer for small files.
And, avoid the last 'read' call by checking when we've copied the expected nr of bytes.
@tmds

tmds commented Oct 19, 2021

Copy link
Copy Markdown
MemberAuthor

I had messed up the PR by resolving conflicts in the GitHub UI.
I've rebased on my machine.

@tmds

tmds commented Oct 20, 2021

Copy link
Copy Markdown
MemberAuthor

I ran the benchmark once more to ensure there are no regressions. I've included some more sizes.

MethodJobToolchainSourceSizeBaseDirMeanErrorStdDevRatioRatioSD
FileCopyJob-JGOCUT/orig/corerun0/home/tmds3.744 ms0.0120 ms0.0100 ms1.000.00
FileCopyJob-WWULDC/pr/corerun0/home/tmds3.479 ms0.0244 ms0.0217 ms0.930.01
FileCopyJob-JGOCUT/orig/corerun0/tmp/1.358 ms0.0059 ms0.0053 ms1.000.00
FileCopyJob-WWULDC/pr/corerun0/tmp/1.150 ms0.0048 ms0.0042 ms0.850.00
FileCopyJob-JGOCUT/orig/corerun1/home/tmds4.770 ms0.0240 ms0.0224 ms1.000.00
FileCopyJob-WWULDC/pr/corerun1/home/tmds4.337 ms0.0149 ms0.0140 ms0.910.01
FileCopyJob-JGOCUT/orig/corerun1/tmp/1.762 ms0.0055 ms0.0049 ms1.000.00
FileCopyJob-WWULDC/pr/corerun1/tmp/1.584 ms0.0091 ms0.0085 ms0.900.01
FileCopyJob-JGOCUT/orig/corerun100/home/tmds4.753 ms0.0342 ms0.0286 ms1.000.00
FileCopyJob-WWULDC/pr/corerun100/home/tmds4.334 ms0.0239 ms0.0212 ms0.910.00
FileCopyJob-JGOCUT/orig/corerun100/tmp/1.746 ms0.0060 ms0.0053 ms1.000.00
FileCopyJob-WWULDC/pr/corerun100/tmp/1.561 ms0.0079 ms0.0070 ms0.890.00
FileCopyJob-JGOCUT/orig/corerun512/home/tmds4.835 ms0.0168 ms0.0157 ms1.000.00
FileCopyJob-WWULDC/pr/corerun512/home/tmds4.365 ms0.0277 ms0.0259 ms0.900.00
FileCopyJob-JGOCUT/orig/corerun512/tmp/1.764 ms0.0105 ms0.0093 ms1.000.00
FileCopyJob-WWULDC/pr/corerun512/tmp/1.530 ms0.0072 ms0.0067 ms0.870.01
FileCopyJob-JGOCUT/orig/corerun1024/home/tmds4.742 ms0.0266 ms0.0236 ms1.000.00
FileCopyJob-WWULDC/pr/corerun1024/home/tmds4.284 ms0.0160 ms0.0150 ms0.900.00
FileCopyJob-JGOCUT/orig/corerun1024/tmp/1.765 ms0.0033 ms0.0027 ms1.000.00
FileCopyJob-WWULDC/pr/corerun1024/tmp/1.570 ms0.0060 ms0.0053 ms0.890.00
FileCopyJob-JGOCUT/orig/corerun4096/home/tmds4.722 ms0.0144 ms0.0135 ms1.000.00
FileCopyJob-WWULDC/pr/corerun4096/home/tmds4.467 ms0.0349 ms0.0327 ms0.950.01
FileCopyJob-JGOCUT/orig/corerun4096/tmp/1.771 ms0.0061 ms0.0054 ms1.000.00
FileCopyJob-WWULDC/pr/corerun4096/tmp/1.566 ms0.0043 ms0.0040 ms0.880.00
FileCopyJob-JGOCUT/orig/corerun8192/home/tmds4.884 ms0.0133 ms0.0118 ms1.000.00
FileCopyJob-WWULDC/pr/corerun8192/home/tmds4.297 ms0.0137 ms0.0128 ms0.880.00
FileCopyJob-JGOCUT/orig/corerun8192/tmp/2.002 ms0.0168 ms0.0158 ms1.000.00
FileCopyJob-WWULDC/pr/corerun8192/tmp/1.814 ms0.0029 ms0.0022 ms0.910.01
FileCopyJob-JGOCUT/orig/corerun16384/home/tmds5.110 ms0.0267 ms0.0237 ms1.000.00
FileCopyJob-WWULDC/pr/corerun16384/home/tmds4.407 ms0.0148 ms0.0131 ms0.860.01
FileCopyJob-JGOCUT/orig/corerun16384/tmp/2.186 ms0.0072 ms0.0067 ms1.000.00
FileCopyJob-WWULDC/pr/corerun16384/tmp/2.013 ms0.0064 ms0.0056 ms0.920.00
FileCopyJob-JGOCUT/orig/corerun32768/home/tmds5.584 ms0.0166 ms0.0147 ms1.000.00
FileCopyJob-WWULDC/pr/corerun32768/home/tmds4.297 ms0.0162 ms0.0152 ms0.770.00
FileCopyJob-JGOCUT/orig/corerun32768/tmp/2.653 ms0.0074 ms0.0069 ms1.000.00
FileCopyJob-WWULDC/pr/corerun32768/tmp/2.477 ms0.0081 ms0.0072 ms0.930.00
FileCopyJob-JGOCUT/orig/corerun1048576/home/tmds33.937 ms0.5533 ms0.5175 ms1.000.00
FileCopyJob-WWULDC/pr/corerun1048576/home/tmds3.809 ms0.0079 ms0.0066 ms0.110.00
FileCopyJob-JGOCUT/orig/corerun1048576/tmp/29.520 ms0.3540 ms0.3311 ms1.000.00
FileCopyJob-WWULDC/pr/corerun1048576/tmp/29.888 ms0.4000 ms0.3742 ms1.010.01
FileCopyJob-JGOCUT/orig/corerun10485760/home/tmds418.623 ms5.8803 ms5.5004 ms1.0000.00
FileCopyJob-WWULDC/pr/corerun10485760/home/tmds3.742 ms0.0050 ms0.0047 ms0.0090.00
FileCopyJob-JGOCUT/orig/corerun10485760/tmp/404.851 ms4.3733 ms4.0908 ms1.000.00
FileCopyJob-WWULDC/pr/corerun10485760/tmp/401.042 ms5.0462 ms4.7202 ms0.990.02

Comment threadsrc/libraries/Native/Unix/System.Native/pal_io.c Outdated
Comment threadsrc/libraries/Native/Unix/System.Native/pal_io.c
Comment threadsrc/libraries/Native/Unix/System.Native/pal_io.c Outdated
Comment threadsrc/libraries/Native/Unix/System.Native/pal_io.c Outdated
Comment threadsrc/libraries/System.Private.CoreLib/src/System/IO/FileSystem.Unix.cs Outdated
@runfoapprunfoappBot mentioned this pull request Oct 28, 2021
@tmds

tmds commented Oct 28, 2021

Copy link
Copy Markdown
MemberAuthor

@stephentoub all feedback is addressed and CI is happy. This is good to merge.

@deeprobindeeprobin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few small suggestions about coding style, but nothing important.
LGTM

@tmds

tmds commented Nov 4, 2021

Copy link
Copy Markdown
MemberAuthor

@stephentoub @dotnet/area-system-io can you merge this?

@jeffhandley
jeffhandley merged commit 9b83294 into dotnet:mainNov 15, 2021
@jeffhandley

Copy link
Copy Markdown
Member

Thanks as always, @tmds!

Interop.Sys.Permissions filePermissions;
using SafeFileHandle src = SafeFileHandle.OpenReadOnly(sourceFullPath, FileOptions.None, out fileLength, out filePermissions);
using SafeFileHandle dst = SafeFileHandle.Open(destFullPath, overwrite ? FileMode.Create : FileMode.CreateNew,
FileAccess.ReadWrite, FileShare.None, FileOptions.None, preallocationSize: 0, openPermissions: filePermissions,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder whether it would be beneficial to provide preallocationSize: fileLength here. @tmds what do you think?

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-System.IOcommunity-contributionIndicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@tmds@adamsitnik@stephentoub@jeffhandley@deeprobin