Add package validation command by landerweit-phytec · Pull Request #64 · phytec/partup

landerweit-phytec · 2023-08-02T08:07:59Z

Add a new command for package validation validate.
The goal of package validation is to make sure a package can be written to a device without writing to the device. Additionally, remove most validation steps from the install command to reduce the time i takes to write to the device.
Open tasks:

Check if URI points into archive
Size calculation for uncompressed archives
Option to use validate without passing a device (maybe use a loop device and pass eMMC size as argument)
Ensure the partitions fit the device
Test validate command
Add commit messages

mschwan-phytec

A first glimpse on your PR...

mschwan-phytec · 2023-09-22T12:38:24Z

You may want to consider squashing some commit fixing earlier stuff of your PR, e.g.:

Check file size and raw overlap only in validate

There is code in src/pu-emmc.c:pu_emmc_new that gets added earlier and then removed. Maybe you can squash/rearrange your commits, so changes are correct in the first place.

mschwan-phytec · 2023-09-22T12:44:28Z

+ -s, --skip-checksums Skip checksum verification for all input files
+ -w, --write-file Install the package to a file as an additional validation step
+ -k, --keep-file-as=FILENAME Keep the file. Implies --write-file


From a user perspective I think it is not clear, what you mean exactly with "file" in the options --write-file and --keep-file-as as the term collides with the input files.

mschwan-phytec · 2023-09-22T12:46:57Z

+ { "skip-checksums", 's', G_OPTION_FLAG_NONE, G_OPTION_ARG_NONE,
+ &arg_validate_skip_checksums, "Skip checksum verification for all input files", NULL },
+ { "size", 0, G_OPTION_FLAG_NONE, G_OPTION_ARG_STRING,
+ &arg_validate_file_size, "Size of loop device. Default is 1G", "SIZE" },


Again, a user may not be familiar with loop devices, so this description is not easy to understand without looking at the inner workings of partup.

Only append the source filename in case the destination is a directory. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

Add functions to create and remove loop devices. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

Separate input files from flash specific information as they only contain general information like filename, checksums and size on target. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

Add a list for all input files regardless of their destination. Use it to get the absolute path and size of all input files at once after config parsing instead of during writing. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

Add a new function for config validation. For now only verifies checksums. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

Checksum verification is done with the validate command. So remove it from the install command to save time when writing to a device. Update the docs accordingly. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

These steps are not necessary for package installation so execute them only when validating a package. Also, update the tests accordingly. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

Add an additional validation step to check if the input files of each partition fit into the corresponding partition. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

Check if the fixed size partitions fit into the device. This does not check expanding partitions. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

Add a new function pu_parse_size() which accepts a size string and returns the size in bytes. Accepts strings with the same format as ped_parse_unit, but only with absolute units. Also, add a corresponding unit test. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

The new command 'validate' validates a package either against a provided device or a new loop device. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

Writing the package to the loop device is an additional validation step. This is similar to installing the package to a real device. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

The generated image can be written with dd. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

landerweit-phytec added the enhancement New feature or request label Aug 2, 2023

landerweit-phytec added this to the v1.0.0 milestone Aug 2, 2023

landerweit-phytec self-assigned this Aug 2, 2023

landerweit-phytec force-pushed the wip-validate-command branch 4 times, most recently from 812ecaa to 92ddeb8Compare August 14, 2023 15:03

landerweit-phytec force-pushed the wip-validate-command branch 10 times, most recently from 418f8e1 to 2458213Compare August 17, 2023 09:47

landerweit-phytec requested a review from mschwan-phytec August 17, 2023 09:47

landerweit-phytec marked this pull request as ready for review August 17, 2023 11:02

landerweit-phytec force-pushed the wip-validate-command branch from 2458213 to 5cc4bbcCompare August 17, 2023 12:24

landerweit-phytec force-pushed the wip-validate-command branch 5 times, most recently from 319aac1 to ad3f15fCompare September 8, 2023 09:13

mschwan-phytec reviewed Sep 19, 2023
View reviewed changes

Comment threadsrc/pu-utils.c

mschwan-phytec requested changes Sep 22, 2023
View reviewed changes

Comment threadsrc/pu-utils.c Outdated
Comment threadtests/emmc.c
Comment threadsrc/pu-input.c
Comment threadsrc/pu-input.c Outdated
Comment threadsrc/pu-input.c Outdated
Comment threadsrc/pu-input.c Outdated
Comment threadsrc/pu-input.h Outdated
Comment threadsrc/pu-emmc.c Outdated

mschwan-phytec requested changes Sep 22, 2023
View reviewed changes

landerweit-phytec force-pushed the wip-validate-command branch from ad3f15f to 794acc2Compare September 22, 2023 13:29

landerweit-phytec added 18 commits January 31, 2024 09:19

src: utils: Allow copying to file destinations
4d7e856
Only append the source filename in case the destination is a directory. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

src: Add loopdev functions
75c241d
Add functions to create and remove loop devices. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

tests: Add test for loopdev
ec7a97b
Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

src: Make input files independent from emmc
8fd03ea
Separate input files from flash specific information as they only contain general information like filename, checksums and size on target. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

tests: Add unit test for pu-input
b381b1d
Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

src: Add config validation function
4dbcb76
Add a new function for config validation. For now only verifies checksums. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

src: Remove checksum verification from install command
c2b6c9f
Checksum verification is done with the validate command. So remove it from the install command to save time when writing to a device. Update the docs accordingly. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

Check file size and raw overlap only in validate
c1dc6b8
These steps are not necessary for package installation so execute them only when validating a package. Also, update the tests accordingly. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

src: emmc: Check if input files fit into partition
d46f7c8
Add an additional validation step to check if the input files of each partition fit into the corresponding partition. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

src: emmc: Check if partitions fit into device
b5d6aa7
Check if the fixed size partitions fit into the device. This does not check expanding partitions. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

tests: emmc: Add unit tests for validation
860574d
Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

src: main: Add validate command
4512cfa
The new command 'validate' validates a package either against a provided device or a new loop device. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

src: main: Add option to write package to loop device
6a5da15
Writing the package to the loop device is an additional validation step. This is similar to installing the package to a real device. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

src: main: Add option to build flashable image from package
d70fff1
The generated image can be written with dd. Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

doc: usage: Add validate command
e2ca74e
Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

github: workflows: system-tests: Add package validation step
b056f7f
Signed-off-by: Leonard Anderweit <l.anderweit@phytec.de>

landerweit-phytec force-pushed the wip-validate-command branch from 8cc250f to b056f7fCompare January 31, 2024 09:10

mschwan-phytec force-pushed the main branch 9 times, most recently from a97a710 to ed48627Compare September 25, 2025 11:30

mschwan-phytec force-pushed the main branch 2 times, most recently from 9ab6f72 to 41bf69fCompare January 15, 2026 08:06

Navigation Menu

Uh oh!

Add package validation command - #64

Add package validation command#64
landerweit-phytec wants to merge 19 commits into
phytec:mainfrom
landerweit-phytec:wip-validate-command

landerweit-phytec commented Aug 2, 2023•
edited
Loading

Uh oh!

Uh oh!

mschwan-phytec left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mschwan-phytec commented Sep 22, 2023

Uh oh!

Uh oh!

mschwan-phytecSep 22, 2023

Uh oh!

mschwan-phytecSep 22, 2023

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

landerweit-phytec commented Aug 2, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

mschwan-phytec left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mschwan-phytec commented Sep 22, 2023

Uh oh!

Uh oh!

mschwan-phytecSep 22, 2023

Choose a reason for hiding this comment

Uh oh!

mschwan-phytecSep 22, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

landerweit-phytec commented Aug 2, 2023•
edited
Loading