Restore --tipset flag to snapshot export command

[jdjaustin]

Summary of changes

Changes introduced in this pull request:

• Restore --tipset flag functionality to allow snapshot export at a certain epoch.
• Added a step to print a newline to the terminal after snapshot export is complete to partially address issue forest-cli snapshot export bugs up the terminal #3171.

Reference issue to close (if applicable)

Closes#2999 and #3171

Other information and links

Change checklist

• I have performed a self-review of my own code,
• I have made corresponding changes to the documentation,
• I have added tests that prove my fix is effective or that my feature works (if possible),
• I have made sure the CHANGELOG is up-to-date. All user-facing changes should be reflected in this document.

[jdjaustin]

Forest and Lotus snapshot sizes are a few hundred bytes different, similar to the status of snapshots prior to PR #2540. Haven't pinpointed a cause yet.

[lemmih]

Forest and Lotus snapshot sizes are a few hundred bytes different, similar to the status of snapshots prior to PR #2540. Haven't pinpointed a cause yet.

Are you comparing the compressed files or uncompressed files?

[ruseinov]

Forest and Lotus snapshot sizes are a few hundred bytes different, similar to the status of snapshots prior to PR #2540. Haven't pinpointed a cause yet.

Could have something to do with this: #3015 ?

[jdjaustin]

Forest and Lotus snapshot sizes are a few hundred bytes different, similar to the status of snapshots prior to PR #2540. Haven't pinpointed a cause yet.

Are you comparing the compressed files or uncompressed files?

Uncompressed.

[jdjaustin]

Forest and Lotus snapshot sizes are a few hundred bytes different, similar to the status of snapshots prior to PR #2540. Haven't pinpointed a cause yet.

Could have something to do with this: #3015 ?

Yes, that might be it. I'll re-check now that that change has been merged.

[jdjaustin]

(Uncompressed) Forest snapshot is still larger than (uncompressed) Lotus snapshot by a few hundred bytes.

[lemmih]

(Uncompressed) Forest snapshot is still larger than (uncompressed) Lotus snapshot by a few hundred bytes.

Try using the ipfs tool to inspect the CAR files. Something like ipfs dag import and then ipfs dag stat.

[jdjaustin]

(Uncompressed) Forest snapshot is still larger than (uncompressed) Lotus snapshot by a few hundred bytes.

Try using the ipfs tool to inspect the CAR files. Something like ipfs dag import and then ipfs dag stat.

Do you know how long the ipfs dag import step should take for a calibnet snapshot? I let it run overnight and it still hadn't finished this morning. Since I already had an older version of ipfs installed and had some issues with getting this command running, I might just try on a fresh droplet.

[lemmih]

(Uncompressed) Forest snapshot is still larger than (uncompressed) Lotus snapshot by a few hundred bytes.

Try using the ipfs tool to inspect the CAR files. Something like ipfs dag import and then ipfs dag stat.

Do you know how long the ipfs dag import step should take for a calibnet snapshot? I let it run overnight and it still hadn't finished this morning. Since I already had an older version of ipfs installed and had some issues with getting this command running, I might just try on a fresh droplet.

I wouldn't expect it to take more than a few seconds.

[lemmih]

@jdjaustin You can also use this tool: https://github.com/ipld/go-car/tree/master/cmd/car

[ruseinov]

Perhaps this could be useful to check: https://ipld.io/specs/transport/car/carv1/#determinism, some of this could potentially influence the file size.

If it's only several hundred bytes - perhaps checking the headers might help, starting with HEAD.

[jdjaustin]

Today @aatifsyed and I did a binary diff and located the mismatch at byte a33fe8ec. We were able to inspect the mismatch by using modified code from PR #3085 . Here are the results:

Still need to investigate why the CIDs at this block do not match.

[ruseinov]

Today @aatifsyed and I did a binary diff and located the mismatch at byte a33fe8ec. We were able to inspect the mismatch by using modified code from PR #3085 . Here are the results:

Still need to investigate why the CIDs at this block do not match.

Nice, all that's left now is to debug it on both Lotus and Forest to see what the differences are in terms of stack traces and such.