git-binary-guard

Verify that git stored your firmware binaries byte for byte — before a fresh clone hands someone a corrupt image.

$ git-binary-guard check
ERROR EOL_FILTER_ON build/app.bin
gitattributes sets text=set on a binary file. Git will run its EOL filter over
it and store different bytes than are on disk. Mark it binary: add
'*.bin binary' to .gitattributes.
ERROR BLOB_EOL_MANGLED build/app.bin
the committed blob differs from the file on disk by CRLF/LF conversion only.
A fresh clone of this repo gets a corrupt binary. disk sha256=8b1a9953c4611296
blob sha256=e3b0c44298fc1c14 (135168 bytes on disk vs 134902 stored).
3 binary file(s) checked, 2 error(s), 0 warning(s).

Why this exists

A firmware repo had one line in .gitattributes:

* text eol=lf

Reasonable-looking. It also applies to *.bin and *.elf, and git's EOL clean filter does not care that your file is a firmware image. Every 0x0D 0x0A pair in it became a single 0x0A on the way into the object store.

Eighteen of twenty-five committed firmware binaries were corrupt in their blobs.

The part that makes this genuinely nasty: git status says the tree is clean. Git compares the working file after running the same filter, so it agrees with itself and reports nothing. Your local build works, because your local file is fine. The corruption only appears when someone clones fresh, or CI does — and what they get is a binary that's the right name, roughly the right size, and won't boot.

This tool checks the thing git doesn't: are the bytes in the object store the bytes on disk?

Install

pip install git-binary-guard

Use

git-binary-guard check

Exit 0 clean, 1 findings, 2 usage error.

git-binary-guard check --staged # only what's about to be committed
git-binary-guard check --ref HEAD # compare against a ref, not the index
git-binary-guard check --ext dfu --ext ld # extra extensions
git-binary-guard check --json # for CI

Fix it

git-binary-guard init # write the *.bin/*.elf/*.hex ... binary rules
git add --renormalize . # re-stage everything under the new rules
git commit -m "Mark firmware binaries binary"
git-binary-guard check # confirm

init appends to an existing .gitattributes rather than replacing it, and is idempotent.

Stop it happening again

git-binary-guard install-hook

Installs a pre-commit hook that runs check --staged and blocks the commit. It refuses to overwrite a hook it didn't write; if you already have one, add this line to it yourself:

git-binary-guard check --staged ||exit 1

In CI

- name: Binaries must be stored intactrun: | pip install git-binary-guard git-binary-guard check

What it checks

Code	Severity	Meaning
BLOB_EOL_MANGLED	error	Stored blob differs from disk by CRLF/LF conversion only. This is the corruption.
BLOB_MISMATCH	error	Stored blob differs from disk some other way while git reports the file unmodified. Something in the filter chain is rewriting it.
EOL_FILTER_ON	error	Gitattributes has text set on a binary file — the cause, whether or not damage has happened yet.
EOL_UNDECLARED	warning	No rule covers the file, so its fate depends on each developer's core.autocrlf. Works on your machine, breaks on theirs.

A file you have simply edited and not committed is not reported. The check distinguishes "you changed it" from "git changed it," which is the whole trick.

Default extensions cover firmware (binelfhexs19srecaxfoutuf2dfupkgimg), objects and libraries, images, archives and fonts. Add your own with --ext.

As a library

fromgit_binary_guardimportcheck_repo, renderresult=check_repo(".")
ifnotresult.ok:
print(render(result))
raiseSystemExit(1)

Limitations, stated plainly

• It cannot repair history. It finds the corruption and tells you how to stop it. Binaries already committed wrong stay wrong in those commits; --renormalize fixes the tip, not the past. If you need clean history, that's filter-repo territory and a rewrite everyone has to re-clone after.
• Extension-based, not content-based. A binary file with no extension, or a .dat you didn't declare, is not checked unless you pass --ext.
• It shells out to git. Needs git on PATH. Tested against modern git; the attribute and cat-file interfaces it uses are old and stable.
• LFS is out of scope. Files stored via git-lfs have a pointer as their blob, so the comparison is meaningless. Exclude them.

Testing

python -m pytest

32 tests. They build real repositories in a temp dir and reproduce the actual corruption — one test asserts that git status reports clean and the blob differs, which is the exact condition that made the original bug invisible. No mocks, so if git's behaviour changes the suite tells you.

License

MIT — see LICENSE.

Built by Kyros Engineering. We do embedded firmware, and the unglamorous tooling that keeps a delivery from being wrong in a way nobody notices for three weeks.