Add pinning of installed packages and documentation regarding pie.lock file

[cleboo]

Hi!

I wanted suggest an improvement of the docs of pie when it comes to pinning of dependencies.

The tl;dr is this:

• Add documentation on pie.lock files and the behaviour of pie install. Does it check against the lock-file? Or does it always get the newest version? What do I have to do to make sure that I build from the the exact same source code I installed from the last time, pinning some kind of checksum?

I am looking for a way to pin the code/artifacts that are downloaded and installed by pie to make sure, we don't unintentionally install new code. As far as I understand, right now pie allows us to

• pin to versions or version ranges - but that does not protect against the code being replaced underneath the tag (pinned tags on github+attestations may help here, but that has to be configured for every extension separately).
• install a specific git commit with pie install xdebug/xdebug:dev-master#af9280fff4eeb014309f8f66633c5e8f40e88b04 (this makes upgrading a hassle).

I noticed, that there is a pie.lock file generated in the $HOME/.pie/php<version>_<some hash> and that it contains a dist.shasum field (that is currently empty), as well as the exact version installed, but I could not find any documentation on whether this actually pins installed source code or not. When I run pie install, will it try to upgrade to newer code or install the code that is mentioned in pie.lock?

I guess that the idea of pinning the installed code is not new and that implementing an install/update behavior similar to composer is planned, since the lock file is already there. But adding some documentation around the pie install command for now that makes clear what it does when a pie.lock file is already present, would be good to avoid confusion.

Thanks for your time!

For reference: the discussion on verifying attestations during install is also an important aspect of this.

[asgrim]

• pin to versions or version ranges - but that does not protect against the code being replaced underneath the tag (pinned tags on github+attestations may help here, but that has to be configured for every extension separately).

• install a specific git commit with pie install xdebug/xdebug:dev-master#af9280fff4eeb014309f8f66633c5e8f40e88b04 (this makes upgrading a hassle).

The pinning should be done when you request the install initially; you already identified, you can request specific versions or ranges, but in practice if you want to pin to a specific hash, the second option should be used.

The pie.lock is mostly irrelevant; we don't actually keep sources or byproducts of the build after pie install completes (as sometimes the builds can be huge!)

pie install (without parameters) is a slightly different operation depending on context:

• if cwd is a PIE extension, it will install that extension
• if cwd is a PHP project, it will install any missing extensions that the project needs
• elsewhere, it is an error

A pie upgrade command is in the works (#561), which will upgrade all installed extensions, within the original constraints you specified. e.g.

pie install asgrim/example-pie-extension:^2.0 # a while ago might've installed 2.0.6# ... some time passes...
pie upgrade # will upgrade all PIE-controlled exts to the given constraint, so `asgrim/example-pie-extension might upgrade to 2.1.0, but not to 3.0.0

[cleboo]

Hmm - but isn't everything already in place to mimick composer's behvior? With that I mean, that it saves a reference to the actual commit in the source repo (or the zip file) to composer.lock and reinstalls from the same reference when composer install is run.

You have this section in the lockfile:

 {
"name": "xdebug/xdebug",
"version": "3.5.1",
"source": {
"type": "git",
"url": "https://github.com/xdebug/xdebug.git",
"reference": "af9280fff4eeb014309f8f66633c5e8f40e88b04"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/xdebug/xdebug/zipball/af9280fff4eeb014309f8f66633c5e8f40e88b04",
"reference": "af9280fff4eeb014309f8f66633c5e8f40e88b04",
"shasum": ""
},
...

So the required information is already there - same as in a composer.lock file.

The pie.lock is mostly irrelevant; we don't actually keep sources or byproducts of the build after pie install completes (as sometimes the builds can be huge!)

Yeah, I simply want to avoid that the source code that is downloaded can be replaced by simply changing where a tag at github points to. What happens during compilation, is a whole other story 🙂

[asgrim]

Understood; let me think on this, I think something could be done 👍

[asgrim]

Moved to #587 - this is scheduled for PIE 1.5.0