Skip to content

Integrate github.com/docker/docker/pkg/system Sequential utilities - #121

Merged
thaJeztah merged 22 commits into
moby:mainfrom
thaJeztah:integrate_moby_sequential
Aug 29, 2022
Merged

Integrate github.com/docker/docker/pkg/system Sequential utilities#121
thaJeztah merged 22 commits into
moby:mainfrom
thaJeztah:integrate_moby_sequential

Conversation

@thaJeztah

@thaJeztahthaJeztah commented Aug 27, 2022

Copy link
Copy Markdown
Member

Integrate github.com/docker/docker/pkg/system Sequential utilities

Integrating the "Sequential" utilities from github.com/docker/docker/pkg/system,
taken from commit 3e8573a85a42d4882068bffab7d83f5d93d417e1

Steps taken:

# install filter-repo (https://github.com/newren/git-filter-repo/blob/main/INSTALL.md)
brew install git-filter-repo
# create a temporary clone of docker
cd ~/Projects
git clone https://github.com/docker/docker.git moby_sys
cd moby_sys
# commit taken from
git rev-parse --verify HEAD
3e8573a85a42d4882068bffab7d83f5d93d417e1
# remove all code, except for 'pkg/system/filesys_*.go', and rename to /sequential
git filter-repo \
--path-glob 'pkg/system/filesys_*.go' \
--path-rename pkg/system/filesys_unix.go:sequential/sequential_unix.go \
--path-rename pkg/system/filesys_windows.go:sequential/sequential_windows.go
# go to the target github.com/moby/sys repository
cd ~/go/src/github.com/moby/sys
# create a branch to work with
git checkout -b integrate_moby_sequential
# add the temporary repository as an upstream and make sure it's up-to-date
git remote add moby_sys ~/Projects/moby_sys
git fetch moby_sys
# merge the upstream code
git merge --allow-unrelated-histories --signoff -S moby_sys/master

sequential: initialize module, and remove unrelated functions

sequential: rename funcs to match their "os" equivalent

Rename the functions so that they can act as a drop-in replacement
for their "os" equivalents.

jhowardmsftand others added 20 commits April 30, 2015 11:59
jhowardmsft
Windows: mkdirall volume path aware
f7411d2
Signed-off-by: jhowardmsft <jhoward@microsoft.com>
John Howard
Windows: Fix use of IsAbs check
20a92d0
Signed-off-by: John Howard <jhoward@microsoft.com>
John Howard
Windows: create daemon root with ACL
1798aa0
Signed-off-by: John Howard <jhoward@microsoft.com>
John Howard
Windows: Use sequential file access
82d88ed
Signed-off-by: John Howard <jhoward@microsoft.com>
@vieux
skip api/types/container/ (like golint) and fix one pkg
a419ce5
Signed-off-by: Victor Vieux <victorvieux@gmail.com>
John Howard
Windows: Use sequential file access
aafb08f
Signed-off-by: John Howard <jhoward@microsoft.com>
John Howard
LCOW: Create layer folders with correct ACL
b291614
Signed-off-by: John Howard <jhoward@microsoft.com>
@tophj-ibm
[project] change syscall to /x/sys/unix|windows
b6bc917
Changes most references of syscall to golang.org/x/sys/
Ones aren't changes include, Errno, Signal and SysProcAttr
as they haven't been implemented in /x/sys/.
Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
[s390x] switch utsname from unsigned to signed
per golang/sys@33267e0
char in s390x in the /x/sys/unix package is now signed, so
change the buildtags
Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
@estesp
Merge pull request #33399 from tophj-ibm/move-all-of-pkg-to-syscall
28c30ad
[project] Switch most syscalls to golang.org/x/sys
@thaJeztah
Remove redundant build-tags
c24df86
Files that are suffixed with `_linux.go` or `_windows.go` are
already only built on Linux / Windows, so these build-tags
were redundant.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@yongtang
Merge pull request #35823 from thaJeztah/remove-dead-code
b425d2a
Remove dead code and redundant build tags
@dnephin
Add canonical import comment
3511ac8
Signed-off-by: Daniel Nephin <dnephin@docker.com>
John Howard
LCOW: Add SIDs to layer.vhd at creation
875002f
Signed-off-by: John Howard <jhoward@microsoft.com>
Some permissions corrections here. Also needs re-vendor of go-winio.
- Create the layer folder directory as standard, not with SDDL. It will inherit permissions from the data-root correctly.
- Apply the VM Group SID access to layer.vhd
Permissions after this changes
Data root:
```
PS C:\> icacls test
test BUILTIN\Administrators:(OI)(CI)(F)
NT AUTHORITY\SYSTEM:(OI)(CI)(F)
```
lcow subdirectory under dataroot
```
PS C:\> icacls test\lcow
test\lcow BUILTIN\Administrators:(I)(OI)(CI)(F)
NT AUTHORITY\SYSTEM:(I)(OI)(CI)(F)
```
layer.vhd in a layer folder for LCOW
```
.\test\lcow\c33923d21c9621fea2f990a8778f469ecdbdc57fd9ca682565d1fa86fadd5d95\layer.vhd NT VIRTUAL MACHINE\Virtual Machines:(R)
BUILTIN\Administrators:(I)(F)
NT AUTHORITY\SYSTEM:(I)(F)
```
And showing working
```
PS C:\> docker-ci-zap -folder=c:\test
INFO: Zapped successfully
PS C:\> docker run --rm alpine echo hello
Unable to find image 'alpine:latest' locally
latest: Pulling from library/alpine
8e402f1a9c57: Pull complete
Digest: sha256:644fcb1a676b5165371437feaa922943aaf7afcfa8bfee4472f6860aad1ef2a0
Status: Downloaded newer image for alpine:latest
hello
```
@thaJeztah
Allow system.MkDirAll() to be used as drop-in for os.MkDirAll()
12d814f
also renamed the non-windows variant of this file to be
consistent with other files in this package
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@zx2c4@thaJeztah
Use newer x/sys/windows SecurityAttributes struct
64d5004
This struct now has a properly typed member, so use the properly typed
functions with it.
Also update the vendor directory and hope nothing explodes.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah
pkg/system: normalize comment formatting
845ff9d
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah
pkg/system: simplify IsAbs()
88b47d5
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah
Update to Go 1.17.0, and gofmt with Go 1.17
de65c12
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@Juneezee
refactor: move from io/ioutil to io and os package
8b36fc0
The io/ioutil package has been deprecated in Go 1.16. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
@thaJeztah
Integrate github.com/docker/docker/pkg/system Sequential utilities
313aacd
Integrating the "Sequential" utilities from github.com/docker/docker/pkg/system,
taken from commit 3e8573a85a42d4882068bffab7d83f5d93d417e1
Steps taken:
# install filter-repo (https://github.com/newren/git-filter-repo/blob/main/INSTALL.md)
brew install git-filter-repo
# create a temporary clone of docker
cd ~/Projects
git clone https://github.com/docker/docker.git moby_sys
cd moby_sys
# commit taken from
git rev-parse --verify HEAD
3e8573a85a42d4882068bffab7d83f5d93d417e1
# remove all code, except for 'pkg/system/filesys_*.go', and rename to /sequential
git filter-repo \
--path-glob 'pkg/system/filesys_*.go' \
--path-rename pkg/system/filesys_unix.go:sequential/sequential_unix.go \
--path-rename pkg/system/filesys_windows.go:sequential/sequential_windows.go
# go to the target github.com/moby/sys repository
cd ~/go/src/github.com/moby/sys
# create a branch to work with
git checkout -b integrate_moby_sequential
# add the temporary repository as an upstream and make sure it's up-to-date
git remote add moby_sys ~/Projects/moby_sys
git fetch moby_sys
# merge the upstream code
git merge --allow-unrelated-histories --signoff -S moby_sys/master
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah
thaJeztahforce-pushed the integrate_moby_sequential branch from 22e3b6b to 0e1b894CompareAugust 27, 2022 11:29
@thaJeztah
sequential: initialize module, and remove unrelated functions
172bbe5
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah
thaJeztahforce-pushed the integrate_moby_sequential branch from 0e1b894 to d200fd6CompareAugust 27, 2022 11:38
@thaJeztah
sequential: rename funcs to match their "os" equivalent
a88a500
Rename the functions so that they can act as a drop-in replacement
for their "os" equivalents.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah
thaJeztahforce-pushed the integrate_moby_sequential branch from d200fd6 to a88a500CompareAugust 27, 2022 11:43
This was referenced Aug 27, 2022
Merged
Merged
Merged
@thaJeztah
thaJeztah marked this pull request as ready for review August 27, 2022 13:34
@thaJeztah

thaJeztah commented Aug 27, 2022

Copy link
Copy Markdown
MemberAuthor

@kolyshkin@kzys@AkihiroSuda@dims PTAL

dims
dims approved these changes Aug 27, 2022
View reviewed changes

@dimsdims left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @thaJeztah for saving the commit info from the other repo!

@thaJeztahthaJeztah mentioned this pull request Aug 29, 2022
Closed
AkihiroSuda
AkihiroSuda reviewed Aug 29, 2022
View reviewed changes
Comment threadsequential/doc.go
//
// The origin of these functions are the golang OS and windows packages,
// slightly modified to only cope with files, not directories due to the
// specific use case.

@AkihiroSudaAkihiroSudaAug 29, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not new in this PR, but probably we have to replicate the original copyright header

@thaJeztahthaJeztahAug 29, 2022

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, good point; perhaps extract the bits that were copied to a separate file (I'd have to check how much was copied; I think that's only in the CreateTemp, which uses similar /same logic as os.CreateTemp)

We can look at that in a follow up in deed 👍

@thaJeztahthaJeztahAug 29, 2022

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW; was also discussing with @crazy-max to see to what extend we still need that function; it may be that some parts were needed for "pre go1.9" code, which didn't work well with Windows, but some of those reasons may no longer be there, so possibly we can start to remove some of this.

I think it's still useful to first do the migration, and go from there though (proposing to tag it as v0.5, so that any deprecation that would happen could be done before v1.0.0).

@thaJeztah

thaJeztah commented Aug 29, 2022

Copy link
Copy Markdown
MemberAuthor

I'll bring this one in, and update the PR's in other projects that were opened to move to this module.

If those "look good", I'll tag a v0.5.0 release

@thaJeztah
thaJeztah merged commit b22ba8a into moby:mainAug 29, 2022
@thaJeztah
thaJeztah deleted the integrate_moby_sequential branch August 29, 2022 09:59
@thaJeztahthaJeztah mentioned this pull request Aug 29, 2022
Merged
@crazy-maxcrazy-max mentioned this pull request Sep 4, 2022
Merged
@thaJeztahthaJeztah mentioned this pull request Apr 6, 2023
Closed
@thaJeztahthaJeztah mentioned this pull request Oct 15, 2024
Open
79 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kzyskzyskzys approved these changes
@samuelkarpsamuelkarpsamuelkarp approved these changes
@AkihiroSudaAkihiroSudaAkihiroSuda approved these changes
+1 more reviewer
@dimsdimsdims approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

12 participants

@thaJeztah@kzys@dims@samuelkarp@AkihiroSuda@vieux@tophj-ibm@estesp@yongtang@dnephin@zx2c4@Juneezee