feat(dependencies): collect pnpm packages

[kristinapathak]

TL;DR

• Add pnpm lockfile collection and npm license lookup for Node.js packages.
• Include Node.js packages in grouped dependency reports.

Additional Details

• Read resolved package entries, scoped names, and peer-dependency suffixes.
• Skip workspace links and local file dependencies.
• Allow COLLECT_DEPS_NO_NPM=1 to disable npm lookups.

For QA

• go test -C ./tools/collect-dependencies .
• Ran the collector against a pnpm v9 lockfile with npm lookup enabled.
• QA is not needed.

Issues

Closes#581

Checklist

• I am familiar with the Contributing Guidelines.
• I have signed off my commits for Developer Certificate of Origin (DCO) compliance.
• Tests cover this work.
• Documentation is current.

Summary by CodeRabbit

• New Features

  • Added Node.js dependency collection from pnpm lockfiles.
  • Included Node.js dependencies and license information in generated reports.
  • Added support for selecting Node.js scans individually or as part of all dependency scans.
  • Added handling for unresolved package licenses and workspace dependencies.

• Documentation

  • Updated dependency-generation instructions and clarified regeneration requirements.
  • Documented pnpm, Node.js license-resolution limitations, and revised command usage.

[coderabbitai]

📝 Walkthrough

Walkthrough

The dependency collector now supports Node.js dependencies from pnpm lockfiles. It resolves npm license metadata, reports Node.js packages, updates tests, and documents the new go run -C invocation.

Changes

pnpm Dependency Collection

Layer / File(s)	Summary
Node parsing and license resolution tools/collect-dependencies/common.go, tools/collect-dependencies/node-deps.go, tools/collect-dependencies/main_test.go	The collector parses pnpm package keys, excludes local dependencies, resolves npm license metadata, and tests version parsing and invalid lockfiles.
Node scanning and reporting tools/collect-dependencies/scan.go, tools/collect-dependencies/main.go, tools/collect-dependencies/main_test.go	Tree scanning collects pnpm dependencies. Reports include Node.js rows and counts. The node language and all selection include Node.js.
Collector invocation documentation dependencies.md, tools/collect-dependencies/README.md, tools/collect-dependencies/main.go, tools/collect-dependencies/main_test.go	Commands use go run -C ./tools/collect-dependencies .. Documentation describes pnpm regeneration, npm lookups, and unresolved-license cases.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
participant scanTree
participant pnpmLockfile
participant npmRegistry
participant dependencyReport
scanTree->>pnpmLockfile: parse pnpm dependencies
pnpmLockfile-->>scanTree: return package names and versions
scanTree->>npmRegistry: request license metadata
npmRegistry-->>scanTree: return license or unresolved result
scanTree->>dependencyReport: generate Node rows and counts

Loading

Suggested reviewers:balajinvda

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title uses valid Conventional Commits syntax and accurately describes the pnpm dependency collection feature.
Linked Issues check	✅ Passed	The changes implement pnpm lockfile parsing and Node.js license reporting required by issue #581.
Out of Scope Changes check	✅ Passed	The code, tests, and documentation changes directly support pnpm dependency collection and related license reporting.

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch kpathak/deps-nvcf-ui

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies"

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🧹 Nitpick comments (2)

tools/collect-dependencies/node_deps.go (1)

16-16: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Rename node_deps.go to follow the tools filename rule.

Rename this new file to node-deps.go. Update any filename-based tooling in the same change.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tools/collect-dependencies/node_deps.go` at line 16, Rename the Go source
file currently named node_deps.go to node-deps.go to comply with the tools
filename convention, and update any filename-based tooling or references that
depend on the old name.

Source: Coding guidelines

tools/collect-dependencies/main_test.go (1)

176-184: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add coverage for link: dependency exclusion.

The fixture verifies file: exclusion but not link: exclusion. Add a quoted package key such as linked@link:../workspace and assert that parsePNPMLock excludes it.

As per coding guidelines, for changed tool behavior, add or update focused tests.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tools/collect-dependencies/main_test.go` around lines 176 - 184, In the
lockfile fixture passed to writeTestFile, add a quoted package entry with a
link: dependency (such as linked@link:../workspace with its resolution block)
alongside the existing file:../workspace entry. Update the assertions that
follow the parsePNPMLock call to verify that this new link: dependency is
excluded from the parsed results, mirroring the coverage already in place for
the file: exclusion.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tools/collect-dependencies/node_deps.go`:
- Around line 39-45: Update parsePNPMLock to return the underlying read or YAML
parsing error instead of an empty dependency set, then propagate that error
through scanTree and run so failures stop report generation rather than
producing incomplete results.
- Around line 91-92: Update httpGetString and npmLicense to accept and propagate
context.Context, create requests with http.NewRequestWithContext, and inject W3C
trace context into outbound headers. On request failures, mark the active span
with error=true and otel.status_code=ERROR, and add the required OpenTelemetry
dependency while updating all affected callers.
---
Nitpick comments:
In `@tools/collect-dependencies/main_test.go`:
- Around line 176-184: In the lockfile fixture passed to writeTestFile, add a
quoted package entry with a link: dependency (such as linked@link:../workspace
with its resolution block) alongside the existing file:../workspace entry.
Update the assertions that follow the parsePNPMLock call to verify that this new
link: dependency is excluded from the parsed results, mirroring the coverage
already in place for the file: exclusion.
In `@tools/collect-dependencies/node_deps.go`:
- Line 16: Rename the Go source file currently named node_deps.go to
node-deps.go to comply with the tools filename convention, and update any
filename-based tooling or references that depend on the old name.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d056f9f8-f55e-43ee-93e8-d5520bb7457f

📥 Commits

Reviewing files that changed from the base of the PR and between c73b36e and f305938.

📒 Files selected for processing (7)

• dependencies.md
• tools/collect-dependencies/README.md
• tools/collect-dependencies/common.go
• tools/collect-dependencies/main.go
• tools/collect-dependencies/main_test.go
• tools/collect-dependencies/node_deps.go
• tools/collect-dependencies/scan.go

[balajinvda]

lgtm

[coderabbitai]

🧹 Nitpick comments (2)

tools/collect-dependencies/node-deps.go (2)

113-123: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Handle the array form of the npm license field.

Older npm packages publish license or licenses as an array of objects, for example [{"type":"MIT","url":"..."}]. npmLicenseValue returns "" for that shape, so those packages get the "unknown or unreachable" label even though the registry returned a license. Add an []any case.

♻️ Proposed handling for array license values

 func npmLicenseValue(value any) string {
switch license := value.(type) {
case string:
return strings.TrimSpace(license)
case map[string]any:
if name, ok := license["type"].(string); ok {
return strings.TrimSpace(name)
}
+	case []any:+ names := []string{}+ for _, item := range license {+ if name := npmLicenseValue(item); name != "" {+ names = append(names, name)+ }+ }+ return strings.Join(names, " OR ")
}
return ""
}

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tools/collect-dependencies/node-deps.go` around lines 113 - 123, Update
npmLicenseValue to handle []any license values by inspecting the array entries
for a license object and extracting its string "type" field with the same
trimming behavior as the existing map case; preserve the current string, map,
and empty-result behavior.

---

125-140: 🚀 Performance & Scalability | 🔵 Trivial | 🏗️ Heavy lift

Consider bounded concurrency for the npm lookups.

buildNodeRows performs one sequential HTTPS request per package. A pnpm lockfile in a monorepo can hold thousands of entries, so a full run can take several minutes. A worker pool with a small fixed limit (for example 8) plus a mutex around cache would reduce wall-clock time substantially. This matches the existing pattern for the other languages, so it can also be deferred to a follow-up that changes all collectors together.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tools/collect-dependencies/node-deps.go` around lines 125 - 140, Replace the
sequential npmLicense calls within the buildNodeRows loop with a bounded worker
pool pattern (e.g., 8 concurrent workers) that processes the dependency keys
concurrently. Add a mutex to protect access to the cache map since it will now
be read and written by multiple goroutines simultaneously. Preserve the existing
logic for building each dependencyRow and populating the license field, ensuring
all requests complete before the function returns the final rows slice.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@tools/collect-dependencies/node-deps.go`:
- Around line 113-123: Update npmLicenseValue to handle []any license values by
inspecting the array entries for a license object and extracting its string
"type" field with the same trimming behavior as the existing map case; preserve
the current string, map, and empty-result behavior.
- Around line 125-140: Replace the sequential npmLicense calls within the
buildNodeRows loop with a bounded worker pool pattern (e.g., 8 concurrent
workers) that processes the dependency keys concurrently. Add a mutex to protect
access to the cache map since it will now be read and written by multiple
goroutines simultaneously. Preserve the existing logic for building each
dependencyRow and populating the license field, ensuring all requests complete
before the function returns the final rows slice.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 8424fc92-7292-436a-af5b-d523f53ee14e

📥 Commits

Reviewing files that changed from the base of the PR and between b67538b and 5ccb48d.

📒 Files selected for processing (1)

• tools/collect-dependencies/node-deps.go