PyPI resolver pins lexicographically-greatest version (e.g. setuptools 9.1.0) instead of highest semver when using the deps.dev API as data source

[miraeti]

Summary

When the PyPI resolver (util/resolve/pypi) is driven by the deps.dev API client (APIClient in util/resolve — this is the path osv-scanner uses by default via --data-source deps.dev), unpinned transitive dependencies resolve to the lexicographically greatest version string instead of the highest semver version. This produces false-positive vulnerability reports in security scans.

Reproduction

1. Create a requirements.txt containing:

   pyramid==2.0.2
   

   (pyramid depends on setuptools and webob with no version constraints)

2. Run osv-scanner (default data source), e.g. v2.3.8 or v2.5.1:

   osv-scanner --lockfile=requirements.txt
   

   Result: transitive deps are reported as setuptools 9.1.0 (with multiple advisories: PYSEC-2022-43012, PYSEC-2025-49, PYSEC-2026-1918, PYSEC-2026-3447, plus GHSA aliases) and webob 1.8.9.

3. Run the same scan with --data-source native (direct PyPI access):
   Result: correctly resolves setuptools 84.0.0 and webob 1.8.11; no false positives.

4. For reference, pip install --dry-run pyramid==2.0.2 also resolves setuptools 84.0.0.

Root cause

Three things combine:

1. The v3alpha GetPackage API returns the versions list sorted lexicographically by version string, not by semver. Example:

   curl -s https://api.deps.dev/v3alpha/systems/pypi/packages/setuptools
   # last five entries of versions[].versionKey.version:
   # "83.0.0", "84.0.0", "9.0.0", "9.0.1", "9.1.0"
   

   ("9.1.0" > "84.0.0" as a string comparison)

2. APIClient.MatchingVersions (util/resolve/api.go) returns MatchRequirement(vk, Versions(...)) without re-sorting, so the lexicographic order from the API is preserved. (By contrast, osv-scalibr's native PyPIRegistryClient.Versions explicitly sorts the list with semver.PyPI.Compare before returning it.)

3. The PyPI resolver assumes candidates are "stored in ascending order" and iterates them from the end (util/resolve/pypi/resolve.go, attemptToPinCriterion):

   // Try the candidates in descending order. They are stored in ascending
   // order, so we can iterate in reverse.
   for i := len(crit.candidates) - 1; i >= 0; i-- {

   With lexicographically ordered input, the first candidate tried is the lexicographic maximum: 9.1.0 for setuptools, 1.8.9 for webob ("1.8.9" > "1.8.11"). These ancient versions happen to have no runtime dependencies, so they are pinned immediately and the resolver never considers the actually-highest versions.

Impact

Any unpinned transitive dependency whose version set contains strings that compare lexicographically above the true latest release (e.g. 9.x vs 84.x, or 1.8.9 vs 1.8.11) resolves to an old version, producing false-positive vulnerability findings for every advisory fixed after that old version.

Suggested fix

Any one of the following would fix it; the first is the cheapest:

• Call SortVersions (util/resolve/match.go, which already sorts a version list in ascending semver order) in APIClient.Versions/MatchingVersions, mirroring what the native registry clients do; or
• have the deps.dev API return versions in semver order; or
• stop assuming input ordering in the PyPI resolver (sort candidates itself in findMatches/mergeIntoCriterion).

Workaround

Use osv-scanner --data-source native, or explicitly pin the affected packages in the requirements file.

[miraeti]

Impact on users via osv-scanner: google/osv-scanner#3018