Uh oh!
There was an error while loading. Please reload this page.
SERVER-1410 - Track VERSION so the version doesn't depend on clone depth - #3
SERVER-1410 - Track VERSION so the version doesn't depend on clone depth#3AerospikeNate-L wants to merge 2 commits into
Conversation
configure.ac derives the version from `git describe`, and when that fails it falls through to a hardcoded 0.0.0-0-g0000... that gets compiled into the library. `describe` needs the nearest ancestor tag in the graph; this pin sits 5 commits past 4.5.0, so any clone shallower than depth 6 silently produces the bogus value. Builds from a source distribution with no .git at all hit the same path. configure only reaches the git branch when VERSION does not already exist, so a tracked file short-circuits it: shallow clones keep the committed value, and deep clones regenerate the identical string. The value here is the output of the same `git describe --long --abbrev=40` invocation configure runs. Bumping this pin means regenerating VERSION in the same commit. SERVER-1410 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
cinterloper
left a comment
There was a problem hiding this comment.
I reproduced the intended behavior in fresh full and depth-1 clones, but found two cases where using VERSION as both a tracked input and generated output breaks the determinism this change is meant to provide.
relcleandeletes the tracked version input.Makefile.in:487still runsrm -f $(objroot)VERSION. In an in-tree build,make relcleantherefore deletes the tracked file. The next configure changes the version to4.5.0-6-g54dda...in a full clone and to the bogus0.0.0-0-g0000...value in a depth-1 clone. It also leaves the checkout withVERSIONdeleted or modified.--with-versionpermanently overwrites the tracked default.configure.ac:1306writes the override to${objroot}VERSION, which is the tracked source file for an in-tree build. The new existence guard then prevents the next ordinary configure from restoring the committed value. I reproduced this with--with-version=4.5.0-99-gdeadbeef; a subsequent configure without--with-versionretained that override and leftVERSIONmodified.
I suggest tracking a distinct immutable input such as VERSION.src, keeping VERSION as generated output, and copying the canonical input into objroot/VERSION. That avoids collisions with both relclean and --with-version while preserving the shallow-clone and source-archive fix.
The current GitHub checks are security checks only and do not exercise these configure paths.
Using VERSION as both a tracked input and a generated output collides with `make relclean`, which deletes it, and with --with-version, which overwrites it and then loses to the existence guard on the next configure. Track VERSION.src instead and copy it into objroot/VERSION. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
AerospikeNate-L
commented
Aug 31, 2026
Good catch on both — you're right that using one path as tracked input and generated output is the root of it, and I reproduced both cases before changing anything. Applied your suggestion in de00f42.
Re-verified on top of the change:
|
Draft — opening for direction, not merge-readiness. @pvinh-spike, you're the reviewer I want on this because the answer depends on how much liberty we have to patch this fork.
Problem
configure.acderives jemalloc's version fromgit describeand, when that fails, falls through to a hardcoded bogus value that gets compiled into the library:describeneeds the nearest ancestor tag in the graph. Our pin (8c87a080) sits 5 commits past4.5.0, so any clone shallower than depth 6 silently produces that bogus string. A--depth 1fetch does bring 45 tags, but none of them is an ancestor of the grafted commit.This is live today: the QE/bob pipeline clones submodules at
--depth 1, GitHub Actions clones them in full, so the binaries QE validates carry a differentJEMALLOC_VERSIONthan the ones we ship. Same failure hits builds from the public.src.tar.bz2, which has no.gitat all (pkg/src/git-cp-files.shcopiesgit ls-filesoutput).Context: SERVER-1410 / QE-1079. jemalloc is the only submodule that derives a build value from git history — swept all 14, and the other two hits (icu's
dist.mk, json'sMakefile) are in targets the server build never invokes.Fix — two parts, both needed
1. Track a new file,
VERSION.src. Content is the verbatim output of the samegit describe --long --abbrev=40invocation configure runs.VERSIONitself stays gitignored and generated — see Why a separate file below.2. Let
VERSION.srcwin overgit describeinconfigure.ac. Part 1 alone does not close the gap — I measured it. Adding the file creates a commit, so the tracked value can never describe the commit that contains it; on a deep clonedescribesucceeds and overwrites the generatedVERSION:--depth 14.5.0-5-g8c87a080…4.5.0-5-g8c87a080…4.5.0-6-g54dda05b…← still divergent4.5.0-5-g8c87a080….git(source archive)4.5.0-5-g8c87a080…4.5.0-5-g8c87a080…The change is one new branch ahead of the existing
git describeblock, inside the arm that already only runs when--with-versionwasn't passed:--with-versionstill takes precedence and still hard-errors on a malformed value — both re-verified.Why a separate file rather than tracking
VERSIONThe first revision of this PR tracked
VERSIONitself. @cinterloper found that this makes one path both a tracked input and a generated output, which breaks in two places (thanks — both reproduced):Makefile.in:487—relcleanrunsrm -f $(objroot)VERSION. In an in-tree build that deletes the tracked file, and the next configure regenerates a different value fromdescribe.configure.ac:1306—--with-versionwrites the override to${objroot}VERSION, i.e. over the tracked file. The existence guard then keeps that override forever, and the checkout is left dirty.With
VERSION.srcas the tracked input,VERSIONkeeps its upstream role as a generated artifact, and both of those paths behave exactly as upstream intends:relcleandeletes a generated file,--with-versionoverwrites a generated file, and the next plain configure restores the canonical value fromVERSION.src.This reverts a 2010 upstream decision — knowingly
a40bc7af("Add release versioning support", Jason Evans, 2010-03-02) deleted the then-trackedVERSIONand added/VERSIONto.gitignorein the same commit. The file it removed contained a stale hand-maintained0.0.0, which is why he moved togit describe.Upstream's model after that:
VERSIONis generated inside a checkout and shipped inside the release tarball.Makefile.in:487only removes it underrelclean, neverdistclean, so tarball consumers keep it;INSTALL:40documents the precedence as--with-version→git describe→ existingVERSIONfile, that last tier existing for exactly those consumers.Both halves of that assumption fail for us:
pkg/src/git-cp-files.shcopyinggit ls-filesoutput — tracked files only. We never invoke jemalloc'smake dist, so the generated-then-shippedVERSIONa genuine jemalloc tarball carries never reaches ours.The
configure.acchange therefore inserts a new tier abovegit describe— a trackedVERSION.srcnow beatsdescribe. That is a deliberate fork divergence, not a bug fix, and it is the part I most want your read on. It is safe for us because we never consume this fork as an upstream-style dist tarball, and--with-versionstill overrides everything. Upstream's own three tiers are left intact underneath.Related:
Makefile:195GIT_CLEAN = git clean -fdx, run by the server'smake cleangit, deletes the generatedVERSIONtoday. The trackedVERSION.srcsurvives it, and configure regeneratesVERSIONfrom it.Verified
autoconf && ./configure --with-jemalloc-prefix=jem_ --with-lg-page=12on this branch:--depth 1clone, andgit archiveexport (no.git) all produce4.5.0-5-g8c87a080f0a88375169faeba0f4358b81dcc27bb, with no "bogus VERSION" messagemake include/jemalloc/jemalloc.h→#define JEMALLOC_VERSION "4.5.0-5-g8c87a080f0a88375169faeba0f4358b81dcc27bb"asd-4.5.0without this change) reproduces0.0.0-0-g0000…make relcleanin an in-tree deep clone:VERSIONremoved,VERSION.srcuntouched,git statusclean, reconfigure restores the canonical string--with-version=4.5.0-99-gdeadbeef…overrides and leavesgit statusclean; the next plain configure restores the canonical string--with-version=nonsenseerrors outThe server builds this in-tree (
Makefile:233→cd $(JEMALLOC) && ./configure), soobjroot=srcroot= empty, andMakefile:230runsautoconfat build time — theconfigure.acchange takes effect without a regeneratedconfigureneeding to be tracked.What this asks of us going forward
VERSION.srcnames the last code-bearing commit, not the tip. The commits adding it change no code, so the string stays accurate about what's compiled — but bumping this pin means regeneratingVERSION.srcin the same push, and if someone forgets, we ship a valid but stale string instead of a loud failure. That's the tradeoff I'd like your read on.The alternative I considered and rejected:
--with-version=…in the server'sJEM_CONFIG_OPT(make_in/Makefile.vars:53). Same staleness exposure, but the constant lives in a different repo from the commit it describes, so a pin bump silently desyncs. Keeping the value next to the code it names is the only real difference — worth it, I think, but it's the reason this touchesconfigure.acat all.If you'd rather not carry a local patch on the fork, say so and I'll take the
--with-versionroute instead and close this.Once this lands, the server-side pin bump follows, then both pipelines can align on
--depth 1(drops ICU from 445 MB → 61 MB and retires the source-tarball size ceiling that started this).🤖 Generated with Claude Code