Jules changes and some model renaming for clarity - #14

Closed
akutuva21 wants to merge 239 commits into
RuleWorld:masterfrom
akutuva21:master
Closed

Jules changes and some model renaming for clarity#14
akutuva21 wants to merge 239 commits into
RuleWorld:masterfrom
akutuva21:master

Conversation

@akutuva21

@akutuva21akutuva21 commented Jun 3, 2026

Copy link
Copy Markdown
Member

Harden, optimize, and test RuleHub's metadata/manifest tooling

Summary

This PR is a large fork-sync covering 125 commits to RuleHub's scripts/ build tooling and the generated metadata it produces. Work spans four areas: security hardening, performance (mostly sync→async I/O), test coverage, and cleanup/refactors, plus regeneration of manifests, gallery data, and README.md/metadata.yaml files to keep CI green.

Most commits were authored by @akutuva21, with several co-authored by AI coding agents (google-labs-jules[bot]).

What changed

Security

  • Prevent prototype pollution in setNested (block __proto__ / constructor / prototype; use plain objects).
  • Fix path-traversal vulnerabilities in directory scanning / metadata extraction.
  • Prevent RegExp injection in generate-manifest.

Performance

  • Convert synchronous file I/O to async across backfill-metadata, generate-manifest, generate-gallery, apply-gallery-assignments, and metadata validation (listModelFiles, findBnglFiles, findAllMetadataFiles, extractModelIds, etc.).
  • Micro-optimizations: hoist Set construction out of loops, cache trim() and RegExp creation, optimize parseMetadataYaml string splitting.

Tests

  • Broad unit-test coverage added for the scripts layer: expectString/expectArray/expectBoolean, parseScalar, parseMetadataYaml, normalizeModelKey, generateId, inferOrigin/inferCategory, processModelLine/processActionLine, isCollectionEntry, getIgnoreDirs/isIgnoredDir, gallery parsing, migration scripts, and error/edge paths throughout.
  • Tests updated to match the new async APIs and dynamic (non-hardcoded) paths.

Refactors & cleanup

  • Extract helper functions out of parseBngl and extractCategoryMappings.
  • Remove extraneous console.log calls (converted to console.info where appropriate); drop dead code (getIgnoreDirs, redundant nesting, unused vars).
  • Fix hardcoded absolute paths.

Generated data / CI

  • Add missing README.md files and metadata.yaml updates for Published models.
  • Regenerate manifest.json, manifest-slim.json, and gallery.generated.json; exempt legacy Published models to resolve manifest-drift CI failures.

Primary files touched

scripts/backfill-metadata.js, scripts/generate-manifest.js, scripts/apply-gallery-assignments.js, scripts/validate-metadata.js, scripts/generate-gallery.js, scripts/utils.js, their .test.js counterparts, scripts/migration/*, plus regenerated manifest*.json, gallery.generated.json, and many Published/**/README.md and metadata.yaml files.

Notes for reviewers

  • The bulk of the line count is regenerated metadata/manifest output, not hand-written logic — review the scripts/ changes first.
  • Several CI-fix commits are iterative (manifest drift / missing READMEs); the net end state is consistent and passing.

Commit breakdown

Category~Count
Tests55
Fixes / security29
Refactor / chore24
Performance18

(Counts overlap where a commit spans categories; total commits = 125.)

akutuva21and others added 30 commits May 11, 2026 16:03
@akutuva21@google-labs-jules
perf: lift Set instantiation out of setNested loop (#51)
41814df
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@akutuva21@google-labs-jules
🧪 Add error path tests for expectString (#52)
54a4ebb
* Exported expectString from validate-metadata.js
* Added negative test cases to validate-metadata.test.js including:
* Non-string values
* Null values
* Empty strings
* Whitespace only strings
* Valid strings
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@akutuva21@google-labs-jules
perf: Cache \trim()\string operation in loop (#53)
221944c
Avoid redundant string allocations by caching the result of \`rawLine.trim()\` in \`parseMetadataYaml\`.
This eliminates 2 redundant \`.trim()\` operations per line, drastically reducing unnecessary allocations and processing.
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: Optimize string splitting in parseMetadataYaml
6d836ce
Replace `content.split(/\r?\n/)` with a zero-allocation `indexOf('\n')` loop in `parseMetadataYaml` inside `scripts/utils.js`. This reduces memory allocations and garbage collection overhead by avoiding the creation of an intermediate array of tokens for every line in the file.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: extract ID generation logic in backfill-metadata.js
1cfbdc4
Moved the string/path manipulation logic used to generate the `id` for metadata out of `generateMetadata` and into a new, dedicated `generateId` function. This simplifies the `generateMetadata` function and improves code readability.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add parseScalar edge case tests in utils.test.js
c27051c
Adds extensive testing to `scripts/utils.test.js` covering the `parseScalar` function's behavior with arrays, booleans, and edge cases, ensuring regex string replacements like `replace(/^"|"$/g, '')` apply as expected across nested comma elements.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add unit tests for expectString function
7c31b60
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add missing error path test for generate-gallery.js
5645f69
This adds a test to verify that `generate-gallery.js` correctly catches errors from `fs.readFileSync` or YAML parsing errors and handles them gracefully by skipping the malformed files without crashing the process.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
⚡ Optimize RegExp creation in apply-gallery-assignments.js loop
245387b
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
fix: remove extraneous console.log in backfill-metadata.js dry-run
5879cf2
Removed `console.log(yamlContent)` from the `if (dryRun)` block to
reduce noise in the standard output.
Tested by running `node --test` with all applicable unit tests.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: remove extraneous console log from apply-gallery-assignment…
01737f9
…s.js
Removed a debugging `console.log` statement from `scripts/apply-gallery-assignments.js` that was generating noise and considered technical debt.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor(scripts): remove extraneous console.log in apply-gallery-ass…
7cc6123
…ignments.js
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: remove extraneous console log from backfill-metadata.js
3783798
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
🧹 remove extraneous console log from backfill-metadata.js
fc3f117
Removed a single `console.log` line that printed out the number of bngl files found,
improving script command line output cleanliness without changing core functionality.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: remove extraneous console log from apply-gallery-assignments.js
65f5eaf
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Refactor: Remove extraneous console log in backfill-metadata.js
758c4c5
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore(scripts): remove extraneous console.log in apply-gallery-assign…
ddb07d4
…ments.js
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
🧹 Remove extraneous console log from generate-manifest.js
77c6fdf
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Remove extraneous console logs from backfill-metadata.js
19cf92d
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Add tests for scripts/apply-gallery-assignments.js
0891222
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore(scripts): remove extraneous console logs in backfill-metadata.js
6c5f1ea
Removed noisy output `console.log` statements at the start of the `main()` function in `scripts/backfill-metadata.js`.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test(utils): add comprehensive edge case tests for parseScalar
ef717c2
Added edge case assertions to scripts/utils.test.js for the `parseScalar` utility function. Test cases include handling of mismatched string quotes, mismatched brackets, irregular boolean casings, nested spaces in arrays/strings, and explicit decimal numbers.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: remove extraneous console logs from backfill-metadata.js
f2e67c2
Removed the extraneous `console.log` statements that print a summary at the end of the `main` function in `scripts/backfill-metadata.js`. This reduces noise in the output and improves code health.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: extract helper functions from parseBngl to resolve long fun…
772fac5
…ction issue
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: optimize generate-gallery metadata scanning
b3e2f94
Replaces the synchronous `fs.readdirSync` depth-first search in `listMetadataFiles` with an asynchronous `fs.promises.readdir` implementation. The refactor leverages `Promise.all` to concurrently scan directories and `try...catch` for robust `ENOENT` handling without redundant `fs.existsSync` checks. This optimizes I/O usage by preventing event loop blocking during file scanning across the repository's `SEARCH_ROOTS`.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: convert console.log to console.info in backfill-metadata.js su…
85f0709
…mmary
Replaces console.log with console.info for the summary output in scripts/backfill-metadata.js. This resolves the code health issue regarding extraneous console logs while preserving the essential summary functionality of the script.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: optimize file reading with async listModelFiles
61ac9ae
Replaced synchronous `listModelFiles` with `listModelFilesAsync` within the async context of `validateMetadataFile` to avoid blocking the Node.js event loop during heavy I/O operations.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: use async file read for gallery categories
90d3b6c
Replaced synchronous `fs.readFileSync` with asynchronous `fs.promises.readFile` in `loadGalleryCategories` inside `scripts/generate-gallery.js` to prevent blocking the Node.js event loop during initialization. Changed main to await the category load.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test(scripts): add unit tests for backfill-metadata.js
06e706d
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Remove extraneous console.log in dry run
966c455
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
akutuva21and others added 26 commits June 2, 2026 11:01
@akutuva21
Merge pull request#139from akutuva21/test-process-model-line-165761…
2bda9d0
…36943815230223
🧪 test: add tests for processModelLine in backfill-metadata.js
Bot
Merge master into PR#136
ffff48d
Bot
Merge latest master into PR#136
f477764
@akutuva21
Merge pull request#136from akutuva21/test/backfill-metadata-generat…
39796a7
…eid-12012714592641404061
🧪 [testing improvement] Add tests for generateId in backfill-metadata.js
Bot
Merge master into PR#134
8e25c5e
@akutuva21
Merge pull request#134from akutuva21/fix/expect-enum-tests-14705957…
d14e7a3
…736384652318
🧪 Add missing tests for expectEnum
Bot
Merge master into PR#130
ae46e56
@akutuva21
Merge pull request#130from akutuva21/testing-get-ignore-dirs-549341…
bcd6ad7
…965731491338
🧪 test: add unit tests for getIgnoreDirs
Bot
Merge master into PR#129
6814b85
@akutuva21
Merge pull request#129from akutuva21/test-validate-metadata-4111691…
24e3647
…123594865509
🧪 Add tests for validateMetadataFile
Bot
Merge master into PR#125
4aa0f00
@akutuva21
Merge pull request#125from akutuva21/test-iscollectionentry-1790141…
a792aa7
…5298177206964
🧪 Add tests for isCollectionEntry in generate-manifest.js
Bot
Merge master into PR#124
29cc8c9
@akutuva21
Merge pull request#124from akutuva21/fix-unused-getignore-dirs-1320…
e0abed4
…3728183734872069
🧹 Code Health: Remove unused `getIgnoreDirs` function and associated dead code
Bot
Merge master into PR#123
f097fe2
@akutuva21
Merge pull request#123from akutuva21/test-coverage-expectArray-1287…
cf13326
…0449047627298925
🧪 [test] add unit tests for expectArray
@akutuva21
Merge pull request#128from akutuva21/fix-path-traversal-input-10377…
ceb558f
…525129266671774
🔒 fix: prevent path traversal on input file read
Bot
Merge master into PR#132
e6f7d77
@akutuva21
Merge pull request#132from akutuva21/test-isignored-dir-18050093452…
cc85e1a
…0554265
🧪 Add tests for isIgnoredDir in generate-manifest
@akutuva21
Fix stale test files to match updated async APIs
9e293f0
Update test duplicates in scripts/tests/ and tests/ that were
left behind after PR merges changed functions to async and
updated normalizeModelKey to preserve hyphens instead of
stripping them.
@akutuva21
Fix CI error: define DEFAULT_IGNORE_DIRS and isIgnoredDir in generate…
4e1ee68
…-manifest.js
Add missing DEFAULT_IGNORE_DIRS constant and isIgnoredDir function
that were referenced but never defined, causing ReferenceError in CI.
Also fix syntax error in test file (extra closing braces).
@akutuva21
⚡ Optimize listModelFiles to use asynchronous I/O
7ecd47d
Replaces synchronous file I/O with async readdir/stat operations for better performance when listing model files.
@akutuva21
⚡ Extract Regex from Gallery Assignment Loop
b255844
Extracts the regex construction outside the loop to avoid recompilation on every iteration.
@akutuva21
⚡ Replace synchronous file I/O with fs.promises
9f2f3b3
Replaces sync fs calls with fs.promises.
@akutuva21
🧪 Add edge case tests for normalizeModelKey
aa18c0c
Adds edge case tests for normalizeModelKey.
@akutuva21
⚡ Optimize gallery assignments by replacing synchronous I/O
3c18cf3
Replaces synchronous I/O in gallery assignment code.
@akutuva21

akutuva21 commented Jun 10, 2026

Copy link
Copy Markdown
MemberAuthor

Closing — involves changes to Mitra2019 model files which are excluded from merges per policy.

@akutuva21

akutuva21 commented Jun 10, 2026

Copy link
Copy Markdown
MemberAuthor

Mitra2019 files excluded from merges

vcellmike pushed a commit to vcellmike/RuleHub that referenced this pull request Aug 10, 2026
@Psuedonerd
Merge pull requestRuleWorld#14from Psuedonerd/codex/inspect-model-f…
101cd51
…olders-in-rulehub-repo-ioilb1
Codex-generated pull request
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@akutuva21
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Jules changes and some model renaming for clarity - #14
Closed
akutuva21 wants to merge 239 commits into
RuleWorld:masterfrom 
akutuva21:master
Closed
Jules changes and some model renaming for clarity#14
akutuva21 wants to merge 239 commits into
RuleWorld:masterfrom 
akutuva21:master
Conversation
@akutuva21
@akutuva21akutuva21
commented
Jun 3, 2026

Copy link


Copy Markdown

Member
Harden, optimize, and test RuleHub's metadata/manifest tooling
Summary
This PR is a large fork-sync covering 125 commits to RuleHub's scripts/ build tooling and the generated metadata it produces. Work spans four areas: security hardening, performance (mostly sync→async I/O), test coverage, and cleanup/refactors, plus regeneration of manifests, gallery data, and README.md/metadata.yaml files to keep CI green.
Most commits were authored by @akutuva21, with several co-authored by AI coding agents (google-labs-jules[bot]).
What changed
Security
  • Prevent prototype pollution in setNested (block __proto__ / constructor / prototype; use plain objects).
  • Fix path-traversal vulnerabilities in directory scanning / metadata extraction.
  • Prevent RegExp injection in generate-manifest.
Performance
  • Convert synchronous file I/O to async across backfill-metadata, generate-manifest, generate-gallery, apply-gallery-assignments, and metadata validation (listModelFiles, findBnglFiles, findAllMetadataFiles, extractModelIds, etc.).
  • Micro-optimizations: hoist Set construction out of loops, cache trim() and RegExp creation, optimize parseMetadataYaml string splitting.
Tests
  • Broad unit-test coverage added for the scripts layer: expectString/expectArray/expectBoolean, parseScalar, parseMetadataYaml, normalizeModelKey, generateId, inferOrigin/inferCategory, processModelLine/processActionLine, isCollectionEntry, getIgnoreDirs/isIgnoredDir, gallery parsing, migration scripts, and error/edge paths throughout.
  • Tests updated to match the new async APIs and dynamic (non-hardcoded) paths.
Refactors & cleanup
  • Extract helper functions out of parseBngl and extractCategoryMappings.
  • Remove extraneous console.log calls (converted to console.info where appropriate); drop dead code (getIgnoreDirs, redundant nesting, unused vars).
  • Fix hardcoded absolute paths.
Generated data / CI
  • Add missing README.md files and metadata.yaml updates for Published models.
  • Regenerate manifest.json, manifest-slim.json, and gallery.generated.json; exempt legacy Published models to resolve manifest-drift CI failures.
Primary files touched
scripts/backfill-metadata.js, scripts/generate-manifest.js, scripts/apply-gallery-assignments.js, scripts/validate-metadata.js, scripts/generate-gallery.js, scripts/utils.js, their .test.js counterparts, scripts/migration/*, plus regenerated manifest*.json, gallery.generated.json, and many Published/**/README.md and metadata.yaml files.
Notes for reviewers
  • The bulk of the line count is regenerated metadata/manifest output, not hand-written logic — review the scripts/ changes first.
  • Several CI-fix commits are iterative (manifest drift / missing READMEs); the net end state is consistent and passing.
Commit breakdown
Category~Count
Tests55
Fixes / security29
Refactor / chore24
Performance18
(Counts overlap where a commit spans categories; total commits = 125.)
akutuva21and others
added 30 commits
May 11, 2026 16:03
@akutuva21@google-labs-jules
perf: lift Set instantiation out of setNested loop (#51)
41814df
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@akutuva21@google-labs-jules
🧪 Add error path tests for expectString (#52)
54a4ebb
* Exported expectString from validate-metadata.js
* Added negative test cases to validate-metadata.test.js including:
* Non-string values
* Null values
* Empty strings
* Whitespace only strings
* Valid strings
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@akutuva21@google-labs-jules
perf: Cache \trim()\string operation in loop (#53)
221944c
Avoid redundant string allocations by caching the result of \`rawLine.trim()\` in \`parseMetadataYaml\`.
This eliminates 2 redundant \`.trim()\` operations per line, drastically reducing unnecessary allocations and processing.
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: Optimize string splitting in parseMetadataYaml
6d836ce
Replace `content.split(/\r?\n/)` with a zero-allocation `indexOf('\n')` loop in `parseMetadataYaml` inside `scripts/utils.js`. This reduces memory allocations and garbage collection overhead by avoiding the creation of an intermediate array of tokens for every line in the file.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: extract ID generation logic in backfill-metadata.js
1cfbdc4
Moved the string/path manipulation logic used to generate the `id` for metadata out of `generateMetadata` and into a new, dedicated `generateId` function. This simplifies the `generateMetadata` function and improves code readability.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add parseScalar edge case tests in utils.test.js
c27051c
Adds extensive testing to `scripts/utils.test.js` covering the `parseScalar` function's behavior with arrays, booleans, and edge cases, ensuring regex string replacements like `replace(/^"|"$/g, '')` apply as expected across nested comma elements.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add unit tests for expectString function
7c31b60
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add missing error path test for generate-gallery.js
5645f69
This adds a test to verify that `generate-gallery.js` correctly catches errors from `fs.readFileSync` or YAML parsing errors and handles them gracefully by skipping the malformed files without crashing the process.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
⚡ Optimize RegExp creation in apply-gallery-assignments.js loop
245387b
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
fix: remove extraneous console.log in backfill-metadata.js dry-run
5879cf2
Removed `console.log(yamlContent)` from the `if (dryRun)` block to
reduce noise in the standard output.
Tested by running `node --test` with all applicable unit tests.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: remove extraneous console log from apply-gallery-assignment…
01737f9
…s.js
Removed a debugging `console.log` statement from `scripts/apply-gallery-assignments.js` that was generating noise and considered technical debt.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor(scripts): remove extraneous console.log in apply-gallery-ass…
7cc6123
…ignments.js
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: remove extraneous console log from backfill-metadata.js
3783798
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
🧹 remove extraneous console log from backfill-metadata.js
fc3f117
Removed a single `console.log` line that printed out the number of bngl files found,
improving script command line output cleanliness without changing core functionality.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: remove extraneous console log from apply-gallery-assignments.js
65f5eaf
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Refactor: Remove extraneous console log in backfill-metadata.js
758c4c5
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore(scripts): remove extraneous console.log in apply-gallery-assign…
ddb07d4
…ments.js
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
🧹 Remove extraneous console log from generate-manifest.js
77c6fdf
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Remove extraneous console logs from backfill-metadata.js
19cf92d
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Add tests for scripts/apply-gallery-assignments.js
0891222
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore(scripts): remove extraneous console logs in backfill-metadata.js
6c5f1ea
Removed noisy output `console.log` statements at the start of the `main()` function in `scripts/backfill-metadata.js`.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test(utils): add comprehensive edge case tests for parseScalar
ef717c2
Added edge case assertions to scripts/utils.test.js for the `parseScalar` utility function. Test cases include handling of mismatched string quotes, mismatched brackets, irregular boolean casings, nested spaces in arrays/strings, and explicit decimal numbers.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: remove extraneous console logs from backfill-metadata.js
f2e67c2
Removed the extraneous `console.log` statements that print a summary at the end of the `main` function in `scripts/backfill-metadata.js`. This reduces noise in the output and improves code health.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: extract helper functions from parseBngl to resolve long fun…
772fac5
…ction issue
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: optimize generate-gallery metadata scanning
b3e2f94
Replaces the synchronous `fs.readdirSync` depth-first search in `listMetadataFiles` with an asynchronous `fs.promises.readdir` implementation. The refactor leverages `Promise.all` to concurrently scan directories and `try...catch` for robust `ENOENT` handling without redundant `fs.existsSync` checks. This optimizes I/O usage by preventing event loop blocking during file scanning across the repository's `SEARCH_ROOTS`.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: convert console.log to console.info in backfill-metadata.js su…
85f0709
…mmary
Replaces console.log with console.info for the summary output in scripts/backfill-metadata.js. This resolves the code health issue regarding extraneous console logs while preserving the essential summary functionality of the script.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: optimize file reading with async listModelFiles
61ac9ae
Replaced synchronous `listModelFiles` with `listModelFilesAsync` within the async context of `validateMetadataFile` to avoid blocking the Node.js event loop during heavy I/O operations.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: use async file read for gallery categories
90d3b6c
Replaced synchronous `fs.readFileSync` with asynchronous `fs.promises.readFile` in `loadGalleryCategories` inside `scripts/generate-gallery.js` to prevent blocking the Node.js event loop during initialization. Changed main to await the category load.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test(scripts): add unit tests for backfill-metadata.js
06e706d
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Remove extraneous console.log in dry run
966c455
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
akutuva21and others
added 26 commits
June 2, 2026 11:01
@akutuva21
Merge pull request#139from akutuva21/test-process-model-line-165761…
2bda9d0
…36943815230223
🧪 test: add tests for processModelLine in backfill-metadata.js
Bot
Merge master into PR#136
ffff48d
Bot
Merge latest master into PR#136
f477764
@akutuva21
Merge pull request#136from akutuva21/test/backfill-metadata-generat…
39796a7
…eid-12012714592641404061
🧪 [testing improvement] Add tests for generateId in backfill-metadata.js
Bot
Merge master into PR#134
8e25c5e
@akutuva21
Merge pull request#134from akutuva21/fix/expect-enum-tests-14705957…
d14e7a3
…736384652318
🧪 Add missing tests for expectEnum
Bot
Merge master into PR#130
ae46e56
@akutuva21
Merge pull request#130from akutuva21/testing-get-ignore-dirs-549341…
bcd6ad7
…965731491338
🧪 test: add unit tests for getIgnoreDirs
Bot
Merge master into PR#129
6814b85
@akutuva21
Merge pull request#129from akutuva21/test-validate-metadata-4111691…
24e3647
…123594865509
🧪 Add tests for validateMetadataFile
Bot
Merge master into PR#125
4aa0f00
@akutuva21
Merge pull request#125from akutuva21/test-iscollectionentry-1790141…
a792aa7
…5298177206964
🧪 Add tests for isCollectionEntry in generate-manifest.js
Bot
Merge master into PR#124
29cc8c9
@akutuva21
Merge pull request#124from akutuva21/fix-unused-getignore-dirs-1320…
e0abed4
…3728183734872069
🧹 Code Health: Remove unused `getIgnoreDirs` function and associated dead code
Bot
Merge master into PR#123
f097fe2
@akutuva21
Merge pull request#123from akutuva21/test-coverage-expectArray-1287…
cf13326
…0449047627298925
🧪 [test] add unit tests for expectArray
@akutuva21
Merge pull request#128from akutuva21/fix-path-traversal-input-10377…
ceb558f
…525129266671774
🔒 fix: prevent path traversal on input file read
Bot
Merge master into PR#132
e6f7d77
@akutuva21
Merge pull request#132from akutuva21/test-isignored-dir-18050093452…
cc85e1a
…0554265
🧪 Add tests for isIgnoredDir in generate-manifest
@akutuva21
Fix stale test files to match updated async APIs
9e293f0
Update test duplicates in scripts/tests/ and tests/ that were
left behind after PR merges changed functions to async and
updated normalizeModelKey to preserve hyphens instead of
stripping them.
@akutuva21
Fix CI error: define DEFAULT_IGNORE_DIRS and isIgnoredDir in generate…
4e1ee68
…-manifest.js
Add missing DEFAULT_IGNORE_DIRS constant and isIgnoredDir function
that were referenced but never defined, causing ReferenceError in CI.
Also fix syntax error in test file (extra closing braces).
@akutuva21
⚡ Optimize listModelFiles to use asynchronous I/O
7ecd47d
Replaces synchronous file I/O with async readdir/stat operations for better performance when listing model files.
@akutuva21
⚡ Extract Regex from Gallery Assignment Loop
b255844
Extracts the regex construction outside the loop to avoid recompilation on every iteration.
@akutuva21
⚡ Replace synchronous file I/O with fs.promises
9f2f3b3
Replaces sync fs calls with fs.promises.
@akutuva21
🧪 Add edge case tests for normalizeModelKey
aa18c0c
Adds edge case tests for normalizeModelKey.
@akutuva21
⚡ Optimize gallery assignments by replacing synchronous I/O
3c18cf3
Replaces synchronous I/O in gallery assignment code.
@akutuva21
akutuva21
commented
Jun 10, 2026

Copy link


Copy Markdown

MemberAuthor
Closing — involves changes to Mitra2019 model files which are excluded from merges per policy.
@akutuva21
akutuva21
commented
Jun 10, 2026

Copy link


Copy Markdown

MemberAuthor
Mitra2019 files excluded from merges
vcellmike pushed a commit
to vcellmike/RuleHub
that referenced
this pull request
Aug 10, 2026
@Psuedonerd
Merge pull requestRuleWorld#14from Psuedonerd/codex/inspect-model-f…
101cd51
…olders-in-rulehub-repo-ioilb1
Codex-generated pull request
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@akutuva21

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Jules changes and some model renaming for clarity - #14
Closed
akutuva21 wants to merge 239 commits into
RuleWorld:masterfrom 
akutuva21:master
Closed
Jules changes and some model renaming for clarity#14
akutuva21 wants to merge 239 commits into
RuleWorld:masterfrom 
akutuva21:master
Conversation
@akutuva21
@akutuva21akutuva21
commented
Jun 3, 2026

Copy link


Copy Markdown

Member
Harden, optimize, and test RuleHub's metadata/manifest tooling
Summary
This PR is a large fork-sync covering 125 commits to RuleHub's scripts/ build tooling and the generated metadata it produces. Work spans four areas: security hardening, performance (mostly sync→async I/O), test coverage, and cleanup/refactors, plus regeneration of manifests, gallery data, and README.md/metadata.yaml files to keep CI green.
Most commits were authored by @akutuva21, with several co-authored by AI coding agents (google-labs-jules[bot]).
What changed
Security
  • Prevent prototype pollution in setNested (block __proto__ / constructor / prototype; use plain objects).
  • Fix path-traversal vulnerabilities in directory scanning / metadata extraction.
  • Prevent RegExp injection in generate-manifest.
Performance
  • Convert synchronous file I/O to async across backfill-metadata, generate-manifest, generate-gallery, apply-gallery-assignments, and metadata validation (listModelFiles, findBnglFiles, findAllMetadataFiles, extractModelIds, etc.).
  • Micro-optimizations: hoist Set construction out of loops, cache trim() and RegExp creation, optimize parseMetadataYaml string splitting.
Tests
  • Broad unit-test coverage added for the scripts layer: expectString/expectArray/expectBoolean, parseScalar, parseMetadataYaml, normalizeModelKey, generateId, inferOrigin/inferCategory, processModelLine/processActionLine, isCollectionEntry, getIgnoreDirs/isIgnoredDir, gallery parsing, migration scripts, and error/edge paths throughout.
  • Tests updated to match the new async APIs and dynamic (non-hardcoded) paths.
Refactors & cleanup
  • Extract helper functions out of parseBngl and extractCategoryMappings.
  • Remove extraneous console.log calls (converted to console.info where appropriate); drop dead code (getIgnoreDirs, redundant nesting, unused vars).
  • Fix hardcoded absolute paths.
Generated data / CI
  • Add missing README.md files and metadata.yaml updates for Published models.
  • Regenerate manifest.json, manifest-slim.json, and gallery.generated.json; exempt legacy Published models to resolve manifest-drift CI failures.
Primary files touched
scripts/backfill-metadata.js, scripts/generate-manifest.js, scripts/apply-gallery-assignments.js, scripts/validate-metadata.js, scripts/generate-gallery.js, scripts/utils.js, their .test.js counterparts, scripts/migration/*, plus regenerated manifest*.json, gallery.generated.json, and many Published/**/README.md and metadata.yaml files.
Notes for reviewers
  • The bulk of the line count is regenerated metadata/manifest output, not hand-written logic — review the scripts/ changes first.
  • Several CI-fix commits are iterative (manifest drift / missing READMEs); the net end state is consistent and passing.
Commit breakdown
Category~Count
Tests55
Fixes / security29
Refactor / chore24
Performance18
(Counts overlap where a commit spans categories; total commits = 125.)
akutuva21and others
added 30 commits
May 11, 2026 16:03
@akutuva21@google-labs-jules
perf: lift Set instantiation out of setNested loop (#51)
41814df
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@akutuva21@google-labs-jules
🧪 Add error path tests for expectString (#52)
54a4ebb
* Exported expectString from validate-metadata.js
* Added negative test cases to validate-metadata.test.js including:
* Non-string values
* Null values
* Empty strings
* Whitespace only strings
* Valid strings
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@akutuva21@google-labs-jules
perf: Cache \trim()\string operation in loop (#53)
221944c
Avoid redundant string allocations by caching the result of \`rawLine.trim()\` in \`parseMetadataYaml\`.
This eliminates 2 redundant \`.trim()\` operations per line, drastically reducing unnecessary allocations and processing.
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: Optimize string splitting in parseMetadataYaml
6d836ce
Replace `content.split(/\r?\n/)` with a zero-allocation `indexOf('\n')` loop in `parseMetadataYaml` inside `scripts/utils.js`. This reduces memory allocations and garbage collection overhead by avoiding the creation of an intermediate array of tokens for every line in the file.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: extract ID generation logic in backfill-metadata.js
1cfbdc4
Moved the string/path manipulation logic used to generate the `id` for metadata out of `generateMetadata` and into a new, dedicated `generateId` function. This simplifies the `generateMetadata` function and improves code readability.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add parseScalar edge case tests in utils.test.js
c27051c
Adds extensive testing to `scripts/utils.test.js` covering the `parseScalar` function's behavior with arrays, booleans, and edge cases, ensuring regex string replacements like `replace(/^"|"$/g, '')` apply as expected across nested comma elements.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add unit tests for expectString function
7c31b60
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add missing error path test for generate-gallery.js
5645f69
This adds a test to verify that `generate-gallery.js` correctly catches errors from `fs.readFileSync` or YAML parsing errors and handles them gracefully by skipping the malformed files without crashing the process.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
⚡ Optimize RegExp creation in apply-gallery-assignments.js loop
245387b
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
fix: remove extraneous console.log in backfill-metadata.js dry-run
5879cf2
Removed `console.log(yamlContent)` from the `if (dryRun)` block to
reduce noise in the standard output.
Tested by running `node --test` with all applicable unit tests.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: remove extraneous console log from apply-gallery-assignment…
01737f9
…s.js
Removed a debugging `console.log` statement from `scripts/apply-gallery-assignments.js` that was generating noise and considered technical debt.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor(scripts): remove extraneous console.log in apply-gallery-ass…
7cc6123
…ignments.js
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: remove extraneous console log from backfill-metadata.js
3783798
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
🧹 remove extraneous console log from backfill-metadata.js
fc3f117
Removed a single `console.log` line that printed out the number of bngl files found,
improving script command line output cleanliness without changing core functionality.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: remove extraneous console log from apply-gallery-assignments.js
65f5eaf
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Refactor: Remove extraneous console log in backfill-metadata.js
758c4c5
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore(scripts): remove extraneous console.log in apply-gallery-assign…
ddb07d4
…ments.js
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
🧹 Remove extraneous console log from generate-manifest.js
77c6fdf
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Remove extraneous console logs from backfill-metadata.js
19cf92d
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Add tests for scripts/apply-gallery-assignments.js
0891222
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore(scripts): remove extraneous console logs in backfill-metadata.js
6c5f1ea
Removed noisy output `console.log` statements at the start of the `main()` function in `scripts/backfill-metadata.js`.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test(utils): add comprehensive edge case tests for parseScalar
ef717c2
Added edge case assertions to scripts/utils.test.js for the `parseScalar` utility function. Test cases include handling of mismatched string quotes, mismatched brackets, irregular boolean casings, nested spaces in arrays/strings, and explicit decimal numbers.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: remove extraneous console logs from backfill-metadata.js
f2e67c2
Removed the extraneous `console.log` statements that print a summary at the end of the `main` function in `scripts/backfill-metadata.js`. This reduces noise in the output and improves code health.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: extract helper functions from parseBngl to resolve long fun…
772fac5
…ction issue
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: optimize generate-gallery metadata scanning
b3e2f94
Replaces the synchronous `fs.readdirSync` depth-first search in `listMetadataFiles` with an asynchronous `fs.promises.readdir` implementation. The refactor leverages `Promise.all` to concurrently scan directories and `try...catch` for robust `ENOENT` handling without redundant `fs.existsSync` checks. This optimizes I/O usage by preventing event loop blocking during file scanning across the repository's `SEARCH_ROOTS`.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: convert console.log to console.info in backfill-metadata.js su…
85f0709
…mmary
Replaces console.log with console.info for the summary output in scripts/backfill-metadata.js. This resolves the code health issue regarding extraneous console logs while preserving the essential summary functionality of the script.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: optimize file reading with async listModelFiles
61ac9ae
Replaced synchronous `listModelFiles` with `listModelFilesAsync` within the async context of `validateMetadataFile` to avoid blocking the Node.js event loop during heavy I/O operations.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: use async file read for gallery categories
90d3b6c
Replaced synchronous `fs.readFileSync` with asynchronous `fs.promises.readFile` in `loadGalleryCategories` inside `scripts/generate-gallery.js` to prevent blocking the Node.js event loop during initialization. Changed main to await the category load.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test(scripts): add unit tests for backfill-metadata.js
06e706d
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Remove extraneous console.log in dry run
966c455
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
akutuva21and others
added 26 commits
June 2, 2026 11:01
@akutuva21
Merge pull request#139from akutuva21/test-process-model-line-165761…
2bda9d0
…36943815230223
🧪 test: add tests for processModelLine in backfill-metadata.js
Bot
Merge master into PR#136
ffff48d
Bot
Merge latest master into PR#136
f477764
@akutuva21
Merge pull request#136from akutuva21/test/backfill-metadata-generat…
39796a7
…eid-12012714592641404061
🧪 [testing improvement] Add tests for generateId in backfill-metadata.js
Bot
Merge master into PR#134
8e25c5e
@akutuva21
Merge pull request#134from akutuva21/fix/expect-enum-tests-14705957…
d14e7a3
…736384652318
🧪 Add missing tests for expectEnum
Bot
Merge master into PR#130
ae46e56
@akutuva21
Merge pull request#130from akutuva21/testing-get-ignore-dirs-549341…
bcd6ad7
…965731491338
🧪 test: add unit tests for getIgnoreDirs
Bot
Merge master into PR#129
6814b85
@akutuva21
Merge pull request#129from akutuva21/test-validate-metadata-4111691…
24e3647
…123594865509
🧪 Add tests for validateMetadataFile
Bot
Merge master into PR#125
4aa0f00
@akutuva21
Merge pull request#125from akutuva21/test-iscollectionentry-1790141…
a792aa7
…5298177206964
🧪 Add tests for isCollectionEntry in generate-manifest.js
Bot
Merge master into PR#124
29cc8c9
@akutuva21
Merge pull request#124from akutuva21/fix-unused-getignore-dirs-1320…
e0abed4
…3728183734872069
🧹 Code Health: Remove unused `getIgnoreDirs` function and associated dead code
Bot
Merge master into PR#123
f097fe2
@akutuva21
Merge pull request#123from akutuva21/test-coverage-expectArray-1287…
cf13326
…0449047627298925
🧪 [test] add unit tests for expectArray
@akutuva21
Merge pull request#128from akutuva21/fix-path-traversal-input-10377…
ceb558f
…525129266671774
🔒 fix: prevent path traversal on input file read
Bot
Merge master into PR#132
e6f7d77
@akutuva21
Merge pull request#132from akutuva21/test-isignored-dir-18050093452…
cc85e1a
…0554265
🧪 Add tests for isIgnoredDir in generate-manifest
@akutuva21
Fix stale test files to match updated async APIs
9e293f0
Update test duplicates in scripts/tests/ and tests/ that were
left behind after PR merges changed functions to async and
updated normalizeModelKey to preserve hyphens instead of
stripping them.
@akutuva21
Fix CI error: define DEFAULT_IGNORE_DIRS and isIgnoredDir in generate…
4e1ee68
…-manifest.js
Add missing DEFAULT_IGNORE_DIRS constant and isIgnoredDir function
that were referenced but never defined, causing ReferenceError in CI.
Also fix syntax error in test file (extra closing braces).
@akutuva21
⚡ Optimize listModelFiles to use asynchronous I/O
7ecd47d
Replaces synchronous file I/O with async readdir/stat operations for better performance when listing model files.
@akutuva21
⚡ Extract Regex from Gallery Assignment Loop
b255844
Extracts the regex construction outside the loop to avoid recompilation on every iteration.
@akutuva21
⚡ Replace synchronous file I/O with fs.promises
9f2f3b3
Replaces sync fs calls with fs.promises.
@akutuva21
🧪 Add edge case tests for normalizeModelKey
aa18c0c
Adds edge case tests for normalizeModelKey.
@akutuva21
⚡ Optimize gallery assignments by replacing synchronous I/O
3c18cf3
Replaces synchronous I/O in gallery assignment code.
@akutuva21
akutuva21
commented
Jun 10, 2026

Copy link


Copy Markdown

MemberAuthor
Closing — involves changes to Mitra2019 model files which are excluded from merges per policy.
@akutuva21
akutuva21
commented
Jun 10, 2026

Copy link


Copy Markdown

MemberAuthor
Mitra2019 files excluded from merges
vcellmike pushed a commit
to vcellmike/RuleHub
that referenced
this pull request
Aug 10, 2026
@Psuedonerd
Merge pull requestRuleWorld#14from Psuedonerd/codex/inspect-model-f…
101cd51
…olders-in-rulehub-repo-ioilb1
Codex-generated pull request
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@akutuva21

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Jules changes and some model renaming for clarity - #14
Closed
akutuva21 wants to merge 239 commits into
RuleWorld:masterfrom 
akutuva21:master
Closed
Jules changes and some model renaming for clarity#14
akutuva21 wants to merge 239 commits into
RuleWorld:masterfrom 
akutuva21:master
Conversation
@akutuva21
@akutuva21akutuva21
commented
Jun 3, 2026

Copy link


Copy Markdown

Member
Harden, optimize, and test RuleHub's metadata/manifest tooling
Summary
This PR is a large fork-sync covering 125 commits to RuleHub's scripts/ build tooling and the generated metadata it produces. Work spans four areas: security hardening, performance (mostly sync→async I/O), test coverage, and cleanup/refactors, plus regeneration of manifests, gallery data, and README.md/metadata.yaml files to keep CI green.
Most commits were authored by @akutuva21, with several co-authored by AI coding agents (google-labs-jules[bot]).
What changed
Security
  • Prevent prototype pollution in setNested (block __proto__ / constructor / prototype; use plain objects).
  • Fix path-traversal vulnerabilities in directory scanning / metadata extraction.
  • Prevent RegExp injection in generate-manifest.
Performance
  • Convert synchronous file I/O to async across backfill-metadata, generate-manifest, generate-gallery, apply-gallery-assignments, and metadata validation (listModelFiles, findBnglFiles, findAllMetadataFiles, extractModelIds, etc.).
  • Micro-optimizations: hoist Set construction out of loops, cache trim() and RegExp creation, optimize parseMetadataYaml string splitting.
Tests
  • Broad unit-test coverage added for the scripts layer: expectString/expectArray/expectBoolean, parseScalar, parseMetadataYaml, normalizeModelKey, generateId, inferOrigin/inferCategory, processModelLine/processActionLine, isCollectionEntry, getIgnoreDirs/isIgnoredDir, gallery parsing, migration scripts, and error/edge paths throughout.
  • Tests updated to match the new async APIs and dynamic (non-hardcoded) paths.
Refactors & cleanup
  • Extract helper functions out of parseBngl and extractCategoryMappings.
  • Remove extraneous console.log calls (converted to console.info where appropriate); drop dead code (getIgnoreDirs, redundant nesting, unused vars).
  • Fix hardcoded absolute paths.
Generated data / CI
  • Add missing README.md files and metadata.yaml updates for Published models.
  • Regenerate manifest.json, manifest-slim.json, and gallery.generated.json; exempt legacy Published models to resolve manifest-drift CI failures.
Primary files touched
scripts/backfill-metadata.js, scripts/generate-manifest.js, scripts/apply-gallery-assignments.js, scripts/validate-metadata.js, scripts/generate-gallery.js, scripts/utils.js, their .test.js counterparts, scripts/migration/*, plus regenerated manifest*.json, gallery.generated.json, and many Published/**/README.md and metadata.yaml files.
Notes for reviewers
  • The bulk of the line count is regenerated metadata/manifest output, not hand-written logic — review the scripts/ changes first.
  • Several CI-fix commits are iterative (manifest drift / missing READMEs); the net end state is consistent and passing.
Commit breakdown
Category~Count
Tests55
Fixes / security29
Refactor / chore24
Performance18
(Counts overlap where a commit spans categories; total commits = 125.)
akutuva21and others
added 30 commits
May 11, 2026 16:03
@akutuva21@google-labs-jules
perf: lift Set instantiation out of setNested loop (#51)
41814df
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@akutuva21@google-labs-jules
🧪 Add error path tests for expectString (#52)
54a4ebb
* Exported expectString from validate-metadata.js
* Added negative test cases to validate-metadata.test.js including:
* Non-string values
* Null values
* Empty strings
* Whitespace only strings
* Valid strings
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@akutuva21@google-labs-jules
perf: Cache \trim()\string operation in loop (#53)
221944c
Avoid redundant string allocations by caching the result of \`rawLine.trim()\` in \`parseMetadataYaml\`.
This eliminates 2 redundant \`.trim()\` operations per line, drastically reducing unnecessary allocations and processing.
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: Optimize string splitting in parseMetadataYaml
6d836ce
Replace `content.split(/\r?\n/)` with a zero-allocation `indexOf('\n')` loop in `parseMetadataYaml` inside `scripts/utils.js`. This reduces memory allocations and garbage collection overhead by avoiding the creation of an intermediate array of tokens for every line in the file.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: extract ID generation logic in backfill-metadata.js
1cfbdc4
Moved the string/path manipulation logic used to generate the `id` for metadata out of `generateMetadata` and into a new, dedicated `generateId` function. This simplifies the `generateMetadata` function and improves code readability.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add parseScalar edge case tests in utils.test.js
c27051c
Adds extensive testing to `scripts/utils.test.js` covering the `parseScalar` function's behavior with arrays, booleans, and edge cases, ensuring regex string replacements like `replace(/^"|"$/g, '')` apply as expected across nested comma elements.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add unit tests for expectString function
7c31b60
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add missing error path test for generate-gallery.js
5645f69
This adds a test to verify that `generate-gallery.js` correctly catches errors from `fs.readFileSync` or YAML parsing errors and handles them gracefully by skipping the malformed files without crashing the process.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
⚡ Optimize RegExp creation in apply-gallery-assignments.js loop
245387b
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
fix: remove extraneous console.log in backfill-metadata.js dry-run
5879cf2
Removed `console.log(yamlContent)` from the `if (dryRun)` block to
reduce noise in the standard output.
Tested by running `node --test` with all applicable unit tests.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: remove extraneous console log from apply-gallery-assignment…
01737f9
…s.js
Removed a debugging `console.log` statement from `scripts/apply-gallery-assignments.js` that was generating noise and considered technical debt.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor(scripts): remove extraneous console.log in apply-gallery-ass…
7cc6123
…ignments.js
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: remove extraneous console log from backfill-metadata.js
3783798
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
🧹 remove extraneous console log from backfill-metadata.js
fc3f117
Removed a single `console.log` line that printed out the number of bngl files found,
improving script command line output cleanliness without changing core functionality.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: remove extraneous console log from apply-gallery-assignments.js
65f5eaf
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Refactor: Remove extraneous console log in backfill-metadata.js
758c4c5
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore(scripts): remove extraneous console.log in apply-gallery-assign…
ddb07d4
…ments.js
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
🧹 Remove extraneous console log from generate-manifest.js
77c6fdf
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Remove extraneous console logs from backfill-metadata.js
19cf92d
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Add tests for scripts/apply-gallery-assignments.js
0891222
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore(scripts): remove extraneous console logs in backfill-metadata.js
6c5f1ea
Removed noisy output `console.log` statements at the start of the `main()` function in `scripts/backfill-metadata.js`.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test(utils): add comprehensive edge case tests for parseScalar
ef717c2
Added edge case assertions to scripts/utils.test.js for the `parseScalar` utility function. Test cases include handling of mismatched string quotes, mismatched brackets, irregular boolean casings, nested spaces in arrays/strings, and explicit decimal numbers.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: remove extraneous console logs from backfill-metadata.js
f2e67c2
Removed the extraneous `console.log` statements that print a summary at the end of the `main` function in `scripts/backfill-metadata.js`. This reduces noise in the output and improves code health.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: extract helper functions from parseBngl to resolve long fun…
772fac5
…ction issue
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: optimize generate-gallery metadata scanning
b3e2f94
Replaces the synchronous `fs.readdirSync` depth-first search in `listMetadataFiles` with an asynchronous `fs.promises.readdir` implementation. The refactor leverages `Promise.all` to concurrently scan directories and `try...catch` for robust `ENOENT` handling without redundant `fs.existsSync` checks. This optimizes I/O usage by preventing event loop blocking during file scanning across the repository's `SEARCH_ROOTS`.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: convert console.log to console.info in backfill-metadata.js su…
85f0709
…mmary
Replaces console.log with console.info for the summary output in scripts/backfill-metadata.js. This resolves the code health issue regarding extraneous console logs while preserving the essential summary functionality of the script.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: optimize file reading with async listModelFiles
61ac9ae
Replaced synchronous `listModelFiles` with `listModelFilesAsync` within the async context of `validateMetadataFile` to avoid blocking the Node.js event loop during heavy I/O operations.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: use async file read for gallery categories
90d3b6c
Replaced synchronous `fs.readFileSync` with asynchronous `fs.promises.readFile` in `loadGalleryCategories` inside `scripts/generate-gallery.js` to prevent blocking the Node.js event loop during initialization. Changed main to await the category load.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test(scripts): add unit tests for backfill-metadata.js
06e706d
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Remove extraneous console.log in dry run
966c455
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
akutuva21and others
added 26 commits
June 2, 2026 11:01
@akutuva21
Merge pull request#139from akutuva21/test-process-model-line-165761…
2bda9d0
…36943815230223
🧪 test: add tests for processModelLine in backfill-metadata.js
Bot
Merge master into PR#136
ffff48d
Bot
Merge latest master into PR#136
f477764
@akutuva21
Merge pull request#136from akutuva21/test/backfill-metadata-generat…
39796a7
…eid-12012714592641404061
🧪 [testing improvement] Add tests for generateId in backfill-metadata.js
Bot
Merge master into PR#134
8e25c5e
@akutuva21
Merge pull request#134from akutuva21/fix/expect-enum-tests-14705957…
d14e7a3
…736384652318
🧪 Add missing tests for expectEnum
Bot
Merge master into PR#130
ae46e56
@akutuva21
Merge pull request#130from akutuva21/testing-get-ignore-dirs-549341…
bcd6ad7
…965731491338
🧪 test: add unit tests for getIgnoreDirs
Bot
Merge master into PR#129
6814b85
@akutuva21
Merge pull request#129from akutuva21/test-validate-metadata-4111691…
24e3647
…123594865509
🧪 Add tests for validateMetadataFile
Bot
Merge master into PR#125
4aa0f00
@akutuva21
Merge pull request#125from akutuva21/test-iscollectionentry-1790141…
a792aa7
…5298177206964
🧪 Add tests for isCollectionEntry in generate-manifest.js
Bot
Merge master into PR#124
29cc8c9
@akutuva21
Merge pull request#124from akutuva21/fix-unused-getignore-dirs-1320…
e0abed4
…3728183734872069
🧹 Code Health: Remove unused `getIgnoreDirs` function and associated dead code
Bot
Merge master into PR#123
f097fe2
@akutuva21
Merge pull request#123from akutuva21/test-coverage-expectArray-1287…
cf13326
…0449047627298925
🧪 [test] add unit tests for expectArray
@akutuva21
Merge pull request#128from akutuva21/fix-path-traversal-input-10377…
ceb558f
…525129266671774
🔒 fix: prevent path traversal on input file read
Bot
Merge master into PR#132
e6f7d77
@akutuva21
Merge pull request#132from akutuva21/test-isignored-dir-18050093452…
cc85e1a
…0554265
🧪 Add tests for isIgnoredDir in generate-manifest
@akutuva21
Fix stale test files to match updated async APIs
9e293f0
Update test duplicates in scripts/tests/ and tests/ that were
left behind after PR merges changed functions to async and
updated normalizeModelKey to preserve hyphens instead of
stripping them.
@akutuva21
Fix CI error: define DEFAULT_IGNORE_DIRS and isIgnoredDir in generate…
4e1ee68
…-manifest.js
Add missing DEFAULT_IGNORE_DIRS constant and isIgnoredDir function
that were referenced but never defined, causing ReferenceError in CI.
Also fix syntax error in test file (extra closing braces).
@akutuva21
⚡ Optimize listModelFiles to use asynchronous I/O
7ecd47d
Replaces synchronous file I/O with async readdir/stat operations for better performance when listing model files.
@akutuva21
⚡ Extract Regex from Gallery Assignment Loop
b255844
Extracts the regex construction outside the loop to avoid recompilation on every iteration.
@akutuva21
⚡ Replace synchronous file I/O with fs.promises
9f2f3b3
Replaces sync fs calls with fs.promises.
@akutuva21
🧪 Add edge case tests for normalizeModelKey
aa18c0c
Adds edge case tests for normalizeModelKey.
@akutuva21
⚡ Optimize gallery assignments by replacing synchronous I/O
3c18cf3
Replaces synchronous I/O in gallery assignment code.
@akutuva21
akutuva21
commented
Jun 10, 2026

Copy link


Copy Markdown

MemberAuthor
Closing — involves changes to Mitra2019 model files which are excluded from merges per policy.
@akutuva21
akutuva21
commented
Jun 10, 2026

Copy link


Copy Markdown

MemberAuthor
Mitra2019 files excluded from merges
vcellmike pushed a commit
to vcellmike/RuleHub
that referenced
this pull request
Aug 10, 2026
@Psuedonerd
Merge pull requestRuleWorld#14from Psuedonerd/codex/inspect-model-f…
101cd51
…olders-in-rulehub-repo-ioilb1
Codex-generated pull request
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@akutuva21

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Jules changes and some model renaming for clarity - #14
Closed
akutuva21 wants to merge 239 commits into
RuleWorld:masterfrom 
akutuva21:master
Closed
Jules changes and some model renaming for clarity#14
akutuva21 wants to merge 239 commits into
RuleWorld:masterfrom 
akutuva21:master
Conversation
@akutuva21
@akutuva21akutuva21
commented
Jun 3, 2026

Copy link


Copy Markdown

Member
Harden, optimize, and test RuleHub's metadata/manifest tooling
Summary
This PR is a large fork-sync covering 125 commits to RuleHub's scripts/ build tooling and the generated metadata it produces. Work spans four areas: security hardening, performance (mostly sync→async I/O), test coverage, and cleanup/refactors, plus regeneration of manifests, gallery data, and README.md/metadata.yaml files to keep CI green.
Most commits were authored by @akutuva21, with several co-authored by AI coding agents (google-labs-jules[bot]).
What changed
Security
  • Prevent prototype pollution in setNested (block __proto__ / constructor / prototype; use plain objects).
  • Fix path-traversal vulnerabilities in directory scanning / metadata extraction.
  • Prevent RegExp injection in generate-manifest.
Performance
  • Convert synchronous file I/O to async across backfill-metadata, generate-manifest, generate-gallery, apply-gallery-assignments, and metadata validation (listModelFiles, findBnglFiles, findAllMetadataFiles, extractModelIds, etc.).
  • Micro-optimizations: hoist Set construction out of loops, cache trim() and RegExp creation, optimize parseMetadataYaml string splitting.
Tests
  • Broad unit-test coverage added for the scripts layer: expectString/expectArray/expectBoolean, parseScalar, parseMetadataYaml, normalizeModelKey, generateId, inferOrigin/inferCategory, processModelLine/processActionLine, isCollectionEntry, getIgnoreDirs/isIgnoredDir, gallery parsing, migration scripts, and error/edge paths throughout.
  • Tests updated to match the new async APIs and dynamic (non-hardcoded) paths.
Refactors & cleanup
  • Extract helper functions out of parseBngl and extractCategoryMappings.
  • Remove extraneous console.log calls (converted to console.info where appropriate); drop dead code (getIgnoreDirs, redundant nesting, unused vars).
  • Fix hardcoded absolute paths.
Generated data / CI
  • Add missing README.md files and metadata.yaml updates for Published models.
  • Regenerate manifest.json, manifest-slim.json, and gallery.generated.json; exempt legacy Published models to resolve manifest-drift CI failures.
Primary files touched
scripts/backfill-metadata.js, scripts/generate-manifest.js, scripts/apply-gallery-assignments.js, scripts/validate-metadata.js, scripts/generate-gallery.js, scripts/utils.js, their .test.js counterparts, scripts/migration/*, plus regenerated manifest*.json, gallery.generated.json, and many Published/**/README.md and metadata.yaml files.
Notes for reviewers
  • The bulk of the line count is regenerated metadata/manifest output, not hand-written logic — review the scripts/ changes first.
  • Several CI-fix commits are iterative (manifest drift / missing READMEs); the net end state is consistent and passing.
Commit breakdown
Category~Count
Tests55
Fixes / security29
Refactor / chore24
Performance18
(Counts overlap where a commit spans categories; total commits = 125.)
akutuva21and others
added 30 commits
May 11, 2026 16:03
@akutuva21@google-labs-jules
perf: lift Set instantiation out of setNested loop (#51)
41814df
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@akutuva21@google-labs-jules
🧪 Add error path tests for expectString (#52)
54a4ebb
* Exported expectString from validate-metadata.js
* Added negative test cases to validate-metadata.test.js including:
* Non-string values
* Null values
* Empty strings
* Whitespace only strings
* Valid strings
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@akutuva21@google-labs-jules
perf: Cache \trim()\string operation in loop (#53)
221944c
Avoid redundant string allocations by caching the result of \`rawLine.trim()\` in \`parseMetadataYaml\`.
This eliminates 2 redundant \`.trim()\` operations per line, drastically reducing unnecessary allocations and processing.
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: Optimize string splitting in parseMetadataYaml
6d836ce
Replace `content.split(/\r?\n/)` with a zero-allocation `indexOf('\n')` loop in `parseMetadataYaml` inside `scripts/utils.js`. This reduces memory allocations and garbage collection overhead by avoiding the creation of an intermediate array of tokens for every line in the file.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: extract ID generation logic in backfill-metadata.js
1cfbdc4
Moved the string/path manipulation logic used to generate the `id` for metadata out of `generateMetadata` and into a new, dedicated `generateId` function. This simplifies the `generateMetadata` function and improves code readability.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add parseScalar edge case tests in utils.test.js
c27051c
Adds extensive testing to `scripts/utils.test.js` covering the `parseScalar` function's behavior with arrays, booleans, and edge cases, ensuring regex string replacements like `replace(/^"|"$/g, '')` apply as expected across nested comma elements.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add unit tests for expectString function
7c31b60
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add missing error path test for generate-gallery.js
5645f69
This adds a test to verify that `generate-gallery.js` correctly catches errors from `fs.readFileSync` or YAML parsing errors and handles them gracefully by skipping the malformed files without crashing the process.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
⚡ Optimize RegExp creation in apply-gallery-assignments.js loop
245387b
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
fix: remove extraneous console.log in backfill-metadata.js dry-run
5879cf2
Removed `console.log(yamlContent)` from the `if (dryRun)` block to
reduce noise in the standard output.
Tested by running `node --test` with all applicable unit tests.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: remove extraneous console log from apply-gallery-assignment…
01737f9
…s.js
Removed a debugging `console.log` statement from `scripts/apply-gallery-assignments.js` that was generating noise and considered technical debt.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor(scripts): remove extraneous console.log in apply-gallery-ass…
7cc6123
…ignments.js
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: remove extraneous console log from backfill-metadata.js
3783798
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
🧹 remove extraneous console log from backfill-metadata.js
fc3f117
Removed a single `console.log` line that printed out the number of bngl files found,
improving script command line output cleanliness without changing core functionality.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: remove extraneous console log from apply-gallery-assignments.js
65f5eaf
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Refactor: Remove extraneous console log in backfill-metadata.js
758c4c5
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore(scripts): remove extraneous console.log in apply-gallery-assign…
ddb07d4
…ments.js
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
🧹 Remove extraneous console log from generate-manifest.js
77c6fdf
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Remove extraneous console logs from backfill-metadata.js
19cf92d
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Add tests for scripts/apply-gallery-assignments.js
0891222
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore(scripts): remove extraneous console logs in backfill-metadata.js
6c5f1ea
Removed noisy output `console.log` statements at the start of the `main()` function in `scripts/backfill-metadata.js`.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test(utils): add comprehensive edge case tests for parseScalar
ef717c2
Added edge case assertions to scripts/utils.test.js for the `parseScalar` utility function. Test cases include handling of mismatched string quotes, mismatched brackets, irregular boolean casings, nested spaces in arrays/strings, and explicit decimal numbers.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: remove extraneous console logs from backfill-metadata.js
f2e67c2
Removed the extraneous `console.log` statements that print a summary at the end of the `main` function in `scripts/backfill-metadata.js`. This reduces noise in the output and improves code health.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: extract helper functions from parseBngl to resolve long fun…
772fac5
…ction issue
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: optimize generate-gallery metadata scanning
b3e2f94
Replaces the synchronous `fs.readdirSync` depth-first search in `listMetadataFiles` with an asynchronous `fs.promises.readdir` implementation. The refactor leverages `Promise.all` to concurrently scan directories and `try...catch` for robust `ENOENT` handling without redundant `fs.existsSync` checks. This optimizes I/O usage by preventing event loop blocking during file scanning across the repository's `SEARCH_ROOTS`.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: convert console.log to console.info in backfill-metadata.js su…
85f0709
…mmary
Replaces console.log with console.info for the summary output in scripts/backfill-metadata.js. This resolves the code health issue regarding extraneous console logs while preserving the essential summary functionality of the script.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: optimize file reading with async listModelFiles
61ac9ae
Replaced synchronous `listModelFiles` with `listModelFilesAsync` within the async context of `validateMetadataFile` to avoid blocking the Node.js event loop during heavy I/O operations.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: use async file read for gallery categories
90d3b6c
Replaced synchronous `fs.readFileSync` with asynchronous `fs.promises.readFile` in `loadGalleryCategories` inside `scripts/generate-gallery.js` to prevent blocking the Node.js event loop during initialization. Changed main to await the category load.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test(scripts): add unit tests for backfill-metadata.js
06e706d
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Remove extraneous console.log in dry run
966c455
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
akutuva21and others
added 26 commits
June 2, 2026 11:01
@akutuva21
Merge pull request#139from akutuva21/test-process-model-line-165761…
2bda9d0
…36943815230223
🧪 test: add tests for processModelLine in backfill-metadata.js
Bot
Merge master into PR#136
ffff48d
Bot
Merge latest master into PR#136
f477764
@akutuva21
Merge pull request#136from akutuva21/test/backfill-metadata-generat…
39796a7
…eid-12012714592641404061
🧪 [testing improvement] Add tests for generateId in backfill-metadata.js
Bot
Merge master into PR#134
8e25c5e
@akutuva21
Merge pull request#134from akutuva21/fix/expect-enum-tests-14705957…
d14e7a3
…736384652318
🧪 Add missing tests for expectEnum
Bot
Merge master into PR#130
ae46e56
@akutuva21
Merge pull request#130from akutuva21/testing-get-ignore-dirs-549341…
bcd6ad7
…965731491338
🧪 test: add unit tests for getIgnoreDirs
Bot
Merge master into PR#129
6814b85
@akutuva21
Merge pull request#129from akutuva21/test-validate-metadata-4111691…
24e3647
…123594865509
🧪 Add tests for validateMetadataFile
Bot
Merge master into PR#125
4aa0f00
@akutuva21
Merge pull request#125from akutuva21/test-iscollectionentry-1790141…
a792aa7
…5298177206964
🧪 Add tests for isCollectionEntry in generate-manifest.js
Bot
Merge master into PR#124
29cc8c9
@akutuva21
Merge pull request#124from akutuva21/fix-unused-getignore-dirs-1320…
e0abed4
…3728183734872069
🧹 Code Health: Remove unused `getIgnoreDirs` function and associated dead code
Bot
Merge master into PR#123
f097fe2
@akutuva21
Merge pull request#123from akutuva21/test-coverage-expectArray-1287…
cf13326
…0449047627298925
🧪 [test] add unit tests for expectArray
@akutuva21
Merge pull request#128from akutuva21/fix-path-traversal-input-10377…
ceb558f
…525129266671774
🔒 fix: prevent path traversal on input file read
Bot
Merge master into PR#132
e6f7d77
@akutuva21
Merge pull request#132from akutuva21/test-isignored-dir-18050093452…
cc85e1a
…0554265
🧪 Add tests for isIgnoredDir in generate-manifest
@akutuva21
Fix stale test files to match updated async APIs
9e293f0
Update test duplicates in scripts/tests/ and tests/ that were
left behind after PR merges changed functions to async and
updated normalizeModelKey to preserve hyphens instead of
stripping them.
@akutuva21
Fix CI error: define DEFAULT_IGNORE_DIRS and isIgnoredDir in generate…
4e1ee68
…-manifest.js
Add missing DEFAULT_IGNORE_DIRS constant and isIgnoredDir function
that were referenced but never defined, causing ReferenceError in CI.
Also fix syntax error in test file (extra closing braces).
@akutuva21
⚡ Optimize listModelFiles to use asynchronous I/O
7ecd47d
Replaces synchronous file I/O with async readdir/stat operations for better performance when listing model files.
@akutuva21
⚡ Extract Regex from Gallery Assignment Loop
b255844
Extracts the regex construction outside the loop to avoid recompilation on every iteration.
@akutuva21
⚡ Replace synchronous file I/O with fs.promises
9f2f3b3
Replaces sync fs calls with fs.promises.
@akutuva21
🧪 Add edge case tests for normalizeModelKey
aa18c0c
Adds edge case tests for normalizeModelKey.
@akutuva21
⚡ Optimize gallery assignments by replacing synchronous I/O
3c18cf3
Replaces synchronous I/O in gallery assignment code.
@akutuva21
akutuva21
commented
Jun 10, 2026

Copy link


Copy Markdown

MemberAuthor
Closing — involves changes to Mitra2019 model files which are excluded from merges per policy.
@akutuva21
akutuva21
commented
Jun 10, 2026

Copy link


Copy Markdown

MemberAuthor
Mitra2019 files excluded from merges
vcellmike pushed a commit
to vcellmike/RuleHub
that referenced
this pull request
Aug 10, 2026
@Psuedonerd
Merge pull requestRuleWorld#14from Psuedonerd/codex/inspect-model-f…
101cd51
…olders-in-rulehub-repo-ioilb1
Codex-generated pull request
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@akutuva21

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Jules changes and some model renaming for clarity - #14
Closed
akutuva21 wants to merge 239 commits into
RuleWorld:masterfrom 
akutuva21:master
Closed
Jules changes and some model renaming for clarity#14
akutuva21 wants to merge 239 commits into
RuleWorld:masterfrom 
akutuva21:master
Conversation
@akutuva21
@akutuva21akutuva21
commented
Jun 3, 2026

Copy link


Copy Markdown

Member
Harden, optimize, and test RuleHub's metadata/manifest tooling
Summary
This PR is a large fork-sync covering 125 commits to RuleHub's scripts/ build tooling and the generated metadata it produces. Work spans four areas: security hardening, performance (mostly sync→async I/O), test coverage, and cleanup/refactors, plus regeneration of manifests, gallery data, and README.md/metadata.yaml files to keep CI green.
Most commits were authored by @akutuva21, with several co-authored by AI coding agents (google-labs-jules[bot]).
What changed
Security
  • Prevent prototype pollution in setNested (block __proto__ / constructor / prototype; use plain objects).
  • Fix path-traversal vulnerabilities in directory scanning / metadata extraction.
  • Prevent RegExp injection in generate-manifest.
Performance
  • Convert synchronous file I/O to async across backfill-metadata, generate-manifest, generate-gallery, apply-gallery-assignments, and metadata validation (listModelFiles, findBnglFiles, findAllMetadataFiles, extractModelIds, etc.).
  • Micro-optimizations: hoist Set construction out of loops, cache trim() and RegExp creation, optimize parseMetadataYaml string splitting.
Tests
  • Broad unit-test coverage added for the scripts layer: expectString/expectArray/expectBoolean, parseScalar, parseMetadataYaml, normalizeModelKey, generateId, inferOrigin/inferCategory, processModelLine/processActionLine, isCollectionEntry, getIgnoreDirs/isIgnoredDir, gallery parsing, migration scripts, and error/edge paths throughout.
  • Tests updated to match the new async APIs and dynamic (non-hardcoded) paths.
Refactors & cleanup
  • Extract helper functions out of parseBngl and extractCategoryMappings.
  • Remove extraneous console.log calls (converted to console.info where appropriate); drop dead code (getIgnoreDirs, redundant nesting, unused vars).
  • Fix hardcoded absolute paths.
Generated data / CI
  • Add missing README.md files and metadata.yaml updates for Published models.
  • Regenerate manifest.json, manifest-slim.json, and gallery.generated.json; exempt legacy Published models to resolve manifest-drift CI failures.
Primary files touched
scripts/backfill-metadata.js, scripts/generate-manifest.js, scripts/apply-gallery-assignments.js, scripts/validate-metadata.js, scripts/generate-gallery.js, scripts/utils.js, their .test.js counterparts, scripts/migration/*, plus regenerated manifest*.json, gallery.generated.json, and many Published/**/README.md and metadata.yaml files.
Notes for reviewers
  • The bulk of the line count is regenerated metadata/manifest output, not hand-written logic — review the scripts/ changes first.
  • Several CI-fix commits are iterative (manifest drift / missing READMEs); the net end state is consistent and passing.
Commit breakdown
Category~Count
Tests55
Fixes / security29
Refactor / chore24
Performance18
(Counts overlap where a commit spans categories; total commits = 125.)
akutuva21and others
added 30 commits
May 11, 2026 16:03
@akutuva21@google-labs-jules
perf: lift Set instantiation out of setNested loop (#51)
41814df
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@akutuva21@google-labs-jules
🧪 Add error path tests for expectString (#52)
54a4ebb
* Exported expectString from validate-metadata.js
* Added negative test cases to validate-metadata.test.js including:
* Non-string values
* Null values
* Empty strings
* Whitespace only strings
* Valid strings
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@akutuva21@google-labs-jules
perf: Cache \trim()\string operation in loop (#53)
221944c
Avoid redundant string allocations by caching the result of \`rawLine.trim()\` in \`parseMetadataYaml\`.
This eliminates 2 redundant \`.trim()\` operations per line, drastically reducing unnecessary allocations and processing.
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: Optimize string splitting in parseMetadataYaml
6d836ce
Replace `content.split(/\r?\n/)` with a zero-allocation `indexOf('\n')` loop in `parseMetadataYaml` inside `scripts/utils.js`. This reduces memory allocations and garbage collection overhead by avoiding the creation of an intermediate array of tokens for every line in the file.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: extract ID generation logic in backfill-metadata.js
1cfbdc4
Moved the string/path manipulation logic used to generate the `id` for metadata out of `generateMetadata` and into a new, dedicated `generateId` function. This simplifies the `generateMetadata` function and improves code readability.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add parseScalar edge case tests in utils.test.js
c27051c
Adds extensive testing to `scripts/utils.test.js` covering the `parseScalar` function's behavior with arrays, booleans, and edge cases, ensuring regex string replacements like `replace(/^"|"$/g, '')` apply as expected across nested comma elements.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add unit tests for expectString function
7c31b60
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add missing error path test for generate-gallery.js
5645f69
This adds a test to verify that `generate-gallery.js` correctly catches errors from `fs.readFileSync` or YAML parsing errors and handles them gracefully by skipping the malformed files without crashing the process.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
⚡ Optimize RegExp creation in apply-gallery-assignments.js loop
245387b
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
fix: remove extraneous console.log in backfill-metadata.js dry-run
5879cf2
Removed `console.log(yamlContent)` from the `if (dryRun)` block to
reduce noise in the standard output.
Tested by running `node --test` with all applicable unit tests.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: remove extraneous console log from apply-gallery-assignment…
01737f9
…s.js
Removed a debugging `console.log` statement from `scripts/apply-gallery-assignments.js` that was generating noise and considered technical debt.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor(scripts): remove extraneous console.log in apply-gallery-ass…
7cc6123
…ignments.js
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: remove extraneous console log from backfill-metadata.js
3783798
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
🧹 remove extraneous console log from backfill-metadata.js
fc3f117
Removed a single `console.log` line that printed out the number of bngl files found,
improving script command line output cleanliness without changing core functionality.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: remove extraneous console log from apply-gallery-assignments.js
65f5eaf
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Refactor: Remove extraneous console log in backfill-metadata.js
758c4c5
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore(scripts): remove extraneous console.log in apply-gallery-assign…
ddb07d4
…ments.js
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
🧹 Remove extraneous console log from generate-manifest.js
77c6fdf
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Remove extraneous console logs from backfill-metadata.js
19cf92d
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Add tests for scripts/apply-gallery-assignments.js
0891222
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore(scripts): remove extraneous console logs in backfill-metadata.js
6c5f1ea
Removed noisy output `console.log` statements at the start of the `main()` function in `scripts/backfill-metadata.js`.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test(utils): add comprehensive edge case tests for parseScalar
ef717c2
Added edge case assertions to scripts/utils.test.js for the `parseScalar` utility function. Test cases include handling of mismatched string quotes, mismatched brackets, irregular boolean casings, nested spaces in arrays/strings, and explicit decimal numbers.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: remove extraneous console logs from backfill-metadata.js
f2e67c2
Removed the extraneous `console.log` statements that print a summary at the end of the `main` function in `scripts/backfill-metadata.js`. This reduces noise in the output and improves code health.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: extract helper functions from parseBngl to resolve long fun…
772fac5
…ction issue
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: optimize generate-gallery metadata scanning
b3e2f94
Replaces the synchronous `fs.readdirSync` depth-first search in `listMetadataFiles` with an asynchronous `fs.promises.readdir` implementation. The refactor leverages `Promise.all` to concurrently scan directories and `try...catch` for robust `ENOENT` handling without redundant `fs.existsSync` checks. This optimizes I/O usage by preventing event loop blocking during file scanning across the repository's `SEARCH_ROOTS`.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: convert console.log to console.info in backfill-metadata.js su…
85f0709
…mmary
Replaces console.log with console.info for the summary output in scripts/backfill-metadata.js. This resolves the code health issue regarding extraneous console logs while preserving the essential summary functionality of the script.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: optimize file reading with async listModelFiles
61ac9ae
Replaced synchronous `listModelFiles` with `listModelFilesAsync` within the async context of `validateMetadataFile` to avoid blocking the Node.js event loop during heavy I/O operations.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: use async file read for gallery categories
90d3b6c
Replaced synchronous `fs.readFileSync` with asynchronous `fs.promises.readFile` in `loadGalleryCategories` inside `scripts/generate-gallery.js` to prevent blocking the Node.js event loop during initialization. Changed main to await the category load.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test(scripts): add unit tests for backfill-metadata.js
06e706d
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Remove extraneous console.log in dry run
966c455
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
akutuva21and others
added 26 commits
June 2, 2026 11:01
@akutuva21
Merge pull request#139from akutuva21/test-process-model-line-165761…
2bda9d0
…36943815230223
🧪 test: add tests for processModelLine in backfill-metadata.js
Bot
Merge master into PR#136
ffff48d
Bot
Merge latest master into PR#136
f477764
@akutuva21
Merge pull request#136from akutuva21/test/backfill-metadata-generat…
39796a7
…eid-12012714592641404061
🧪 [testing improvement] Add tests for generateId in backfill-metadata.js
Bot
Merge master into PR#134
8e25c5e
@akutuva21
Merge pull request#134from akutuva21/fix/expect-enum-tests-14705957…
d14e7a3
…736384652318
🧪 Add missing tests for expectEnum
Bot
Merge master into PR#130
ae46e56
@akutuva21
Merge pull request#130from akutuva21/testing-get-ignore-dirs-549341…
bcd6ad7
…965731491338
🧪 test: add unit tests for getIgnoreDirs
Bot
Merge master into PR#129
6814b85
@akutuva21
Merge pull request#129from akutuva21/test-validate-metadata-4111691…
24e3647
…123594865509
🧪 Add tests for validateMetadataFile
Bot
Merge master into PR#125
4aa0f00
@akutuva21
Merge pull request#125from akutuva21/test-iscollectionentry-1790141…
a792aa7
…5298177206964
🧪 Add tests for isCollectionEntry in generate-manifest.js
Bot
Merge master into PR#124
29cc8c9
@akutuva21
Merge pull request#124from akutuva21/fix-unused-getignore-dirs-1320…
e0abed4
…3728183734872069
🧹 Code Health: Remove unused `getIgnoreDirs` function and associated dead code
Bot
Merge master into PR#123
f097fe2
@akutuva21
Merge pull request#123from akutuva21/test-coverage-expectArray-1287…
cf13326
…0449047627298925
🧪 [test] add unit tests for expectArray
@akutuva21
Merge pull request#128from akutuva21/fix-path-traversal-input-10377…
ceb558f
…525129266671774
🔒 fix: prevent path traversal on input file read
Bot
Merge master into PR#132
e6f7d77
@akutuva21
Merge pull request#132from akutuva21/test-isignored-dir-18050093452…
cc85e1a
…0554265
🧪 Add tests for isIgnoredDir in generate-manifest
@akutuva21
Fix stale test files to match updated async APIs
9e293f0
Update test duplicates in scripts/tests/ and tests/ that were
left behind after PR merges changed functions to async and
updated normalizeModelKey to preserve hyphens instead of
stripping them.
@akutuva21
Fix CI error: define DEFAULT_IGNORE_DIRS and isIgnoredDir in generate…
4e1ee68
…-manifest.js
Add missing DEFAULT_IGNORE_DIRS constant and isIgnoredDir function
that were referenced but never defined, causing ReferenceError in CI.
Also fix syntax error in test file (extra closing braces).
@akutuva21
⚡ Optimize listModelFiles to use asynchronous I/O
7ecd47d
Replaces synchronous file I/O with async readdir/stat operations for better performance when listing model files.
@akutuva21
⚡ Extract Regex from Gallery Assignment Loop
b255844
Extracts the regex construction outside the loop to avoid recompilation on every iteration.
@akutuva21
⚡ Replace synchronous file I/O with fs.promises
9f2f3b3
Replaces sync fs calls with fs.promises.
@akutuva21
🧪 Add edge case tests for normalizeModelKey
aa18c0c
Adds edge case tests for normalizeModelKey.
@akutuva21
⚡ Optimize gallery assignments by replacing synchronous I/O
3c18cf3
Replaces synchronous I/O in gallery assignment code.
@akutuva21
akutuva21
commented
Jun 10, 2026

Copy link


Copy Markdown

MemberAuthor
Closing — involves changes to Mitra2019 model files which are excluded from merges per policy.
@akutuva21
akutuva21
commented
Jun 10, 2026

Copy link


Copy Markdown

MemberAuthor
Mitra2019 files excluded from merges
vcellmike pushed a commit
to vcellmike/RuleHub
that referenced
this pull request
Aug 10, 2026
@Psuedonerd
Merge pull requestRuleWorld#14from Psuedonerd/codex/inspect-model-f…
101cd51
…olders-in-rulehub-repo-ioilb1
Codex-generated pull request
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@akutuva21

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Jules changes and some model renaming for clarity - #14
Closed
akutuva21 wants to merge 239 commits into
RuleWorld:masterfrom 
akutuva21:master
Closed
Jules changes and some model renaming for clarity#14
akutuva21 wants to merge 239 commits into
RuleWorld:masterfrom 
akutuva21:master
Conversation
@akutuva21
@akutuva21akutuva21
commented
Jun 3, 2026

Copy link


Copy Markdown

Member
Harden, optimize, and test RuleHub's metadata/manifest tooling
Summary
This PR is a large fork-sync covering 125 commits to RuleHub's scripts/ build tooling and the generated metadata it produces. Work spans four areas: security hardening, performance (mostly sync→async I/O), test coverage, and cleanup/refactors, plus regeneration of manifests, gallery data, and README.md/metadata.yaml files to keep CI green.
Most commits were authored by @akutuva21, with several co-authored by AI coding agents (google-labs-jules[bot]).
What changed
Security
  • Prevent prototype pollution in setNested (block __proto__ / constructor / prototype; use plain objects).
  • Fix path-traversal vulnerabilities in directory scanning / metadata extraction.
  • Prevent RegExp injection in generate-manifest.
Performance
  • Convert synchronous file I/O to async across backfill-metadata, generate-manifest, generate-gallery, apply-gallery-assignments, and metadata validation (listModelFiles, findBnglFiles, findAllMetadataFiles, extractModelIds, etc.).
  • Micro-optimizations: hoist Set construction out of loops, cache trim() and RegExp creation, optimize parseMetadataYaml string splitting.
Tests
  • Broad unit-test coverage added for the scripts layer: expectString/expectArray/expectBoolean, parseScalar, parseMetadataYaml, normalizeModelKey, generateId, inferOrigin/inferCategory, processModelLine/processActionLine, isCollectionEntry, getIgnoreDirs/isIgnoredDir, gallery parsing, migration scripts, and error/edge paths throughout.
  • Tests updated to match the new async APIs and dynamic (non-hardcoded) paths.
Refactors & cleanup
  • Extract helper functions out of parseBngl and extractCategoryMappings.
  • Remove extraneous console.log calls (converted to console.info where appropriate); drop dead code (getIgnoreDirs, redundant nesting, unused vars).
  • Fix hardcoded absolute paths.
Generated data / CI
  • Add missing README.md files and metadata.yaml updates for Published models.
  • Regenerate manifest.json, manifest-slim.json, and gallery.generated.json; exempt legacy Published models to resolve manifest-drift CI failures.
Primary files touched
scripts/backfill-metadata.js, scripts/generate-manifest.js, scripts/apply-gallery-assignments.js, scripts/validate-metadata.js, scripts/generate-gallery.js, scripts/utils.js, their .test.js counterparts, scripts/migration/*, plus regenerated manifest*.json, gallery.generated.json, and many Published/**/README.md and metadata.yaml files.
Notes for reviewers
  • The bulk of the line count is regenerated metadata/manifest output, not hand-written logic — review the scripts/ changes first.
  • Several CI-fix commits are iterative (manifest drift / missing READMEs); the net end state is consistent and passing.
Commit breakdown
Category~Count
Tests55
Fixes / security29
Refactor / chore24
Performance18
(Counts overlap where a commit spans categories; total commits = 125.)
akutuva21and others
added 30 commits
May 11, 2026 16:03
@akutuva21@google-labs-jules
perf: lift Set instantiation out of setNested loop (#51)
41814df
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@akutuva21@google-labs-jules
🧪 Add error path tests for expectString (#52)
54a4ebb
* Exported expectString from validate-metadata.js
* Added negative test cases to validate-metadata.test.js including:
* Non-string values
* Null values
* Empty strings
* Whitespace only strings
* Valid strings
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@akutuva21@google-labs-jules
perf: Cache \trim()\string operation in loop (#53)
221944c
Avoid redundant string allocations by caching the result of \`rawLine.trim()\` in \`parseMetadataYaml\`.
This eliminates 2 redundant \`.trim()\` operations per line, drastically reducing unnecessary allocations and processing.
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: Optimize string splitting in parseMetadataYaml
6d836ce
Replace `content.split(/\r?\n/)` with a zero-allocation `indexOf('\n')` loop in `parseMetadataYaml` inside `scripts/utils.js`. This reduces memory allocations and garbage collection overhead by avoiding the creation of an intermediate array of tokens for every line in the file.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: extract ID generation logic in backfill-metadata.js
1cfbdc4
Moved the string/path manipulation logic used to generate the `id` for metadata out of `generateMetadata` and into a new, dedicated `generateId` function. This simplifies the `generateMetadata` function and improves code readability.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add parseScalar edge case tests in utils.test.js
c27051c
Adds extensive testing to `scripts/utils.test.js` covering the `parseScalar` function's behavior with arrays, booleans, and edge cases, ensuring regex string replacements like `replace(/^"|"$/g, '')` apply as expected across nested comma elements.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add unit tests for expectString function
7c31b60
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add missing error path test for generate-gallery.js
5645f69
This adds a test to verify that `generate-gallery.js` correctly catches errors from `fs.readFileSync` or YAML parsing errors and handles them gracefully by skipping the malformed files without crashing the process.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
⚡ Optimize RegExp creation in apply-gallery-assignments.js loop
245387b
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
fix: remove extraneous console.log in backfill-metadata.js dry-run
5879cf2
Removed `console.log(yamlContent)` from the `if (dryRun)` block to
reduce noise in the standard output.
Tested by running `node --test` with all applicable unit tests.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: remove extraneous console log from apply-gallery-assignment…
01737f9
…s.js
Removed a debugging `console.log` statement from `scripts/apply-gallery-assignments.js` that was generating noise and considered technical debt.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor(scripts): remove extraneous console.log in apply-gallery-ass…
7cc6123
…ignments.js
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: remove extraneous console log from backfill-metadata.js
3783798
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
🧹 remove extraneous console log from backfill-metadata.js
fc3f117
Removed a single `console.log` line that printed out the number of bngl files found,
improving script command line output cleanliness without changing core functionality.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: remove extraneous console log from apply-gallery-assignments.js
65f5eaf
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Refactor: Remove extraneous console log in backfill-metadata.js
758c4c5
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore(scripts): remove extraneous console.log in apply-gallery-assign…
ddb07d4
…ments.js
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
🧹 Remove extraneous console log from generate-manifest.js
77c6fdf
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Remove extraneous console logs from backfill-metadata.js
19cf92d
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Add tests for scripts/apply-gallery-assignments.js
0891222
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore(scripts): remove extraneous console logs in backfill-metadata.js
6c5f1ea
Removed noisy output `console.log` statements at the start of the `main()` function in `scripts/backfill-metadata.js`.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test(utils): add comprehensive edge case tests for parseScalar
ef717c2
Added edge case assertions to scripts/utils.test.js for the `parseScalar` utility function. Test cases include handling of mismatched string quotes, mismatched brackets, irregular boolean casings, nested spaces in arrays/strings, and explicit decimal numbers.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: remove extraneous console logs from backfill-metadata.js
f2e67c2
Removed the extraneous `console.log` statements that print a summary at the end of the `main` function in `scripts/backfill-metadata.js`. This reduces noise in the output and improves code health.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: extract helper functions from parseBngl to resolve long fun…
772fac5
…ction issue
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: optimize generate-gallery metadata scanning
b3e2f94
Replaces the synchronous `fs.readdirSync` depth-first search in `listMetadataFiles` with an asynchronous `fs.promises.readdir` implementation. The refactor leverages `Promise.all` to concurrently scan directories and `try...catch` for robust `ENOENT` handling without redundant `fs.existsSync` checks. This optimizes I/O usage by preventing event loop blocking during file scanning across the repository's `SEARCH_ROOTS`.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: convert console.log to console.info in backfill-metadata.js su…
85f0709
…mmary
Replaces console.log with console.info for the summary output in scripts/backfill-metadata.js. This resolves the code health issue regarding extraneous console logs while preserving the essential summary functionality of the script.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: optimize file reading with async listModelFiles
61ac9ae
Replaced synchronous `listModelFiles` with `listModelFilesAsync` within the async context of `validateMetadataFile` to avoid blocking the Node.js event loop during heavy I/O operations.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: use async file read for gallery categories
90d3b6c
Replaced synchronous `fs.readFileSync` with asynchronous `fs.promises.readFile` in `loadGalleryCategories` inside `scripts/generate-gallery.js` to prevent blocking the Node.js event loop during initialization. Changed main to await the category load.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test(scripts): add unit tests for backfill-metadata.js
06e706d
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Remove extraneous console.log in dry run
966c455
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
akutuva21and others
added 26 commits
June 2, 2026 11:01
@akutuva21
Merge pull request#139from akutuva21/test-process-model-line-165761…
2bda9d0
…36943815230223
🧪 test: add tests for processModelLine in backfill-metadata.js
Bot
Merge master into PR#136
ffff48d
Bot
Merge latest master into PR#136
f477764
@akutuva21
Merge pull request#136from akutuva21/test/backfill-metadata-generat…
39796a7
…eid-12012714592641404061
🧪 [testing improvement] Add tests for generateId in backfill-metadata.js
Bot
Merge master into PR#134
8e25c5e
@akutuva21
Merge pull request#134from akutuva21/fix/expect-enum-tests-14705957…
d14e7a3
…736384652318
🧪 Add missing tests for expectEnum
Bot
Merge master into PR#130
ae46e56
@akutuva21
Merge pull request#130from akutuva21/testing-get-ignore-dirs-549341…
bcd6ad7
…965731491338
🧪 test: add unit tests for getIgnoreDirs
Bot
Merge master into PR#129
6814b85
@akutuva21
Merge pull request#129from akutuva21/test-validate-metadata-4111691…
24e3647
…123594865509
🧪 Add tests for validateMetadataFile
Bot
Merge master into PR#125
4aa0f00
@akutuva21
Merge pull request#125from akutuva21/test-iscollectionentry-1790141…
a792aa7
…5298177206964
🧪 Add tests for isCollectionEntry in generate-manifest.js
Bot
Merge master into PR#124
29cc8c9
@akutuva21
Merge pull request#124from akutuva21/fix-unused-getignore-dirs-1320…
e0abed4
…3728183734872069
🧹 Code Health: Remove unused `getIgnoreDirs` function and associated dead code
Bot
Merge master into PR#123
f097fe2
@akutuva21
Merge pull request#123from akutuva21/test-coverage-expectArray-1287…
cf13326
…0449047627298925
🧪 [test] add unit tests for expectArray
@akutuva21
Merge pull request#128from akutuva21/fix-path-traversal-input-10377…
ceb558f
…525129266671774
🔒 fix: prevent path traversal on input file read
Bot
Merge master into PR#132
e6f7d77
@akutuva21
Merge pull request#132from akutuva21/test-isignored-dir-18050093452…
cc85e1a
…0554265
🧪 Add tests for isIgnoredDir in generate-manifest
@akutuva21
Fix stale test files to match updated async APIs
9e293f0
Update test duplicates in scripts/tests/ and tests/ that were
left behind after PR merges changed functions to async and
updated normalizeModelKey to preserve hyphens instead of
stripping them.
@akutuva21
Fix CI error: define DEFAULT_IGNORE_DIRS and isIgnoredDir in generate…
4e1ee68
…-manifest.js
Add missing DEFAULT_IGNORE_DIRS constant and isIgnoredDir function
that were referenced but never defined, causing ReferenceError in CI.
Also fix syntax error in test file (extra closing braces).
@akutuva21
⚡ Optimize listModelFiles to use asynchronous I/O
7ecd47d
Replaces synchronous file I/O with async readdir/stat operations for better performance when listing model files.
@akutuva21
⚡ Extract Regex from Gallery Assignment Loop
b255844
Extracts the regex construction outside the loop to avoid recompilation on every iteration.
@akutuva21
⚡ Replace synchronous file I/O with fs.promises
9f2f3b3
Replaces sync fs calls with fs.promises.
@akutuva21
🧪 Add edge case tests for normalizeModelKey
aa18c0c
Adds edge case tests for normalizeModelKey.
@akutuva21
⚡ Optimize gallery assignments by replacing synchronous I/O
3c18cf3
Replaces synchronous I/O in gallery assignment code.
@akutuva21
akutuva21
commented
Jun 10, 2026

Copy link


Copy Markdown

MemberAuthor
Closing — involves changes to Mitra2019 model files which are excluded from merges per policy.
@akutuva21
akutuva21
commented
Jun 10, 2026

Copy link


Copy Markdown

MemberAuthor
Mitra2019 files excluded from merges
vcellmike pushed a commit
to vcellmike/RuleHub
that referenced
this pull request
Aug 10, 2026
@Psuedonerd
Merge pull requestRuleWorld#14from Psuedonerd/codex/inspect-model-f…
101cd51
…olders-in-rulehub-repo-ioilb1
Codex-generated pull request
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@akutuva21

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Jules changes and some model renaming for clarity - #14
Closed
akutuva21 wants to merge 239 commits into
RuleWorld:masterfrom 
akutuva21:master
Closed
Jules changes and some model renaming for clarity#14
akutuva21 wants to merge 239 commits into
RuleWorld:masterfrom 
akutuva21:master
Conversation
@akutuva21
@akutuva21akutuva21
commented
Jun 3, 2026

Copy link


Copy Markdown

Member
Harden, optimize, and test RuleHub's metadata/manifest tooling
Summary
This PR is a large fork-sync covering 125 commits to RuleHub's scripts/ build tooling and the generated metadata it produces. Work spans four areas: security hardening, performance (mostly sync→async I/O), test coverage, and cleanup/refactors, plus regeneration of manifests, gallery data, and README.md/metadata.yaml files to keep CI green.
Most commits were authored by @akutuva21, with several co-authored by AI coding agents (google-labs-jules[bot]).
What changed
Security
  • Prevent prototype pollution in setNested (block __proto__ / constructor / prototype; use plain objects).
  • Fix path-traversal vulnerabilities in directory scanning / metadata extraction.
  • Prevent RegExp injection in generate-manifest.
Performance
  • Convert synchronous file I/O to async across backfill-metadata, generate-manifest, generate-gallery, apply-gallery-assignments, and metadata validation (listModelFiles, findBnglFiles, findAllMetadataFiles, extractModelIds, etc.).
  • Micro-optimizations: hoist Set construction out of loops, cache trim() and RegExp creation, optimize parseMetadataYaml string splitting.
Tests
  • Broad unit-test coverage added for the scripts layer: expectString/expectArray/expectBoolean, parseScalar, parseMetadataYaml, normalizeModelKey, generateId, inferOrigin/inferCategory, processModelLine/processActionLine, isCollectionEntry, getIgnoreDirs/isIgnoredDir, gallery parsing, migration scripts, and error/edge paths throughout.
  • Tests updated to match the new async APIs and dynamic (non-hardcoded) paths.
Refactors & cleanup
  • Extract helper functions out of parseBngl and extractCategoryMappings.
  • Remove extraneous console.log calls (converted to console.info where appropriate); drop dead code (getIgnoreDirs, redundant nesting, unused vars).
  • Fix hardcoded absolute paths.
Generated data / CI
  • Add missing README.md files and metadata.yaml updates for Published models.
  • Regenerate manifest.json, manifest-slim.json, and gallery.generated.json; exempt legacy Published models to resolve manifest-drift CI failures.
Primary files touched
scripts/backfill-metadata.js, scripts/generate-manifest.js, scripts/apply-gallery-assignments.js, scripts/validate-metadata.js, scripts/generate-gallery.js, scripts/utils.js, their .test.js counterparts, scripts/migration/*, plus regenerated manifest*.json, gallery.generated.json, and many Published/**/README.md and metadata.yaml files.
Notes for reviewers
  • The bulk of the line count is regenerated metadata/manifest output, not hand-written logic — review the scripts/ changes first.
  • Several CI-fix commits are iterative (manifest drift / missing READMEs); the net end state is consistent and passing.
Commit breakdown
Category~Count
Tests55
Fixes / security29
Refactor / chore24
Performance18
(Counts overlap where a commit spans categories; total commits = 125.)
akutuva21and others
added 30 commits
May 11, 2026 16:03
@akutuva21@google-labs-jules
perf: lift Set instantiation out of setNested loop (#51)
41814df
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@akutuva21@google-labs-jules
🧪 Add error path tests for expectString (#52)
54a4ebb
* Exported expectString from validate-metadata.js
* Added negative test cases to validate-metadata.test.js including:
* Non-string values
* Null values
* Empty strings
* Whitespace only strings
* Valid strings
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@akutuva21@google-labs-jules
perf: Cache \trim()\string operation in loop (#53)
221944c
Avoid redundant string allocations by caching the result of \`rawLine.trim()\` in \`parseMetadataYaml\`.
This eliminates 2 redundant \`.trim()\` operations per line, drastically reducing unnecessary allocations and processing.
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: Optimize string splitting in parseMetadataYaml
6d836ce
Replace `content.split(/\r?\n/)` with a zero-allocation `indexOf('\n')` loop in `parseMetadataYaml` inside `scripts/utils.js`. This reduces memory allocations and garbage collection overhead by avoiding the creation of an intermediate array of tokens for every line in the file.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: extract ID generation logic in backfill-metadata.js
1cfbdc4
Moved the string/path manipulation logic used to generate the `id` for metadata out of `generateMetadata` and into a new, dedicated `generateId` function. This simplifies the `generateMetadata` function and improves code readability.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add parseScalar edge case tests in utils.test.js
c27051c
Adds extensive testing to `scripts/utils.test.js` covering the `parseScalar` function's behavior with arrays, booleans, and edge cases, ensuring regex string replacements like `replace(/^"|"$/g, '')` apply as expected across nested comma elements.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add unit tests for expectString function
7c31b60
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test: add missing error path test for generate-gallery.js
5645f69
This adds a test to verify that `generate-gallery.js` correctly catches errors from `fs.readFileSync` or YAML parsing errors and handles them gracefully by skipping the malformed files without crashing the process.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
⚡ Optimize RegExp creation in apply-gallery-assignments.js loop
245387b
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
fix: remove extraneous console.log in backfill-metadata.js dry-run
5879cf2
Removed `console.log(yamlContent)` from the `if (dryRun)` block to
reduce noise in the standard output.
Tested by running `node --test` with all applicable unit tests.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: remove extraneous console log from apply-gallery-assignment…
01737f9
…s.js
Removed a debugging `console.log` statement from `scripts/apply-gallery-assignments.js` that was generating noise and considered technical debt.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor(scripts): remove extraneous console.log in apply-gallery-ass…
7cc6123
…ignments.js
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: remove extraneous console log from backfill-metadata.js
3783798
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
🧹 remove extraneous console log from backfill-metadata.js
fc3f117
Removed a single `console.log` line that printed out the number of bngl files found,
improving script command line output cleanliness without changing core functionality.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: remove extraneous console log from apply-gallery-assignments.js
65f5eaf
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Refactor: Remove extraneous console log in backfill-metadata.js
758c4c5
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore(scripts): remove extraneous console.log in apply-gallery-assign…
ddb07d4
…ments.js
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
🧹 Remove extraneous console log from generate-manifest.js
77c6fdf
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Remove extraneous console logs from backfill-metadata.js
19cf92d
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Add tests for scripts/apply-gallery-assignments.js
0891222
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore(scripts): remove extraneous console logs in backfill-metadata.js
6c5f1ea
Removed noisy output `console.log` statements at the start of the `main()` function in `scripts/backfill-metadata.js`.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test(utils): add comprehensive edge case tests for parseScalar
ef717c2
Added edge case assertions to scripts/utils.test.js for the `parseScalar` utility function. Test cases include handling of mismatched string quotes, mismatched brackets, irregular boolean casings, nested spaces in arrays/strings, and explicit decimal numbers.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: remove extraneous console logs from backfill-metadata.js
f2e67c2
Removed the extraneous `console.log` statements that print a summary at the end of the `main` function in `scripts/backfill-metadata.js`. This reduces noise in the output and improves code health.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
refactor: extract helper functions from parseBngl to resolve long fun…
772fac5
…ction issue
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: optimize generate-gallery metadata scanning
b3e2f94
Replaces the synchronous `fs.readdirSync` depth-first search in `listMetadataFiles` with an asynchronous `fs.promises.readdir` implementation. The refactor leverages `Promise.all` to concurrently scan directories and `try...catch` for robust `ENOENT` handling without redundant `fs.existsSync` checks. This optimizes I/O usage by preventing event loop blocking during file scanning across the repository's `SEARCH_ROOTS`.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
chore: convert console.log to console.info in backfill-metadata.js su…
85f0709
…mmary
Replaces console.log with console.info for the summary output in scripts/backfill-metadata.js. This resolves the code health issue regarding extraneous console logs while preserving the essential summary functionality of the script.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: optimize file reading with async listModelFiles
61ac9ae
Replaced synchronous `listModelFiles` with `listModelFilesAsync` within the async context of `validateMetadataFile` to avoid blocking the Node.js event loop during heavy I/O operations.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
perf: use async file read for gallery categories
90d3b6c
Replaced synchronous `fs.readFileSync` with asynchronous `fs.promises.readFile` in `loadGalleryCategories` inside `scripts/generate-gallery.js` to prevent blocking the Node.js event loop during initialization. Changed main to await the category load.
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
test(scripts): add unit tests for backfill-metadata.js
06e706d
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
@google-labs-jules@akutuva21
Remove extraneous console.log in dry run
966c455
Co-authored-by: akutuva21 <44119804+akutuva21@users.noreply.github.com>
akutuva21and others
added 26 commits
June 2, 2026 11:01
@akutuva21
Merge pull request#139from akutuva21/test-process-model-line-165761…
2bda9d0
…36943815230223
🧪 test: add tests for processModelLine in backfill-metadata.js
Bot
Merge master into PR#136
ffff48d
Bot
Merge latest master into PR#136
f477764
@akutuva21
Merge pull request#136from akutuva21/test/backfill-metadata-generat…
39796a7
…eid-12012714592641404061
🧪 [testing improvement] Add tests for generateId in backfill-metadata.js
Bot
Merge master into PR#134
8e25c5e
@akutuva21
Merge pull request#134from akutuva21/fix/expect-enum-tests-14705957…
d14e7a3
…736384652318
🧪 Add missing tests for expectEnum
Bot
Merge master into PR#130
ae46e56
@akutuva21
Merge pull request#130from akutuva21/testing-get-ignore-dirs-549341…
bcd6ad7
…965731491338
🧪 test: add unit tests for getIgnoreDirs
Bot
Merge master into PR#129
6814b85
@akutuva21
Merge pull request#129from akutuva21/test-validate-metadata-4111691…
24e3647
…123594865509
🧪 Add tests for validateMetadataFile
Bot
Merge master into PR#125
4aa0f00
@akutuva21
Merge pull request#125from akutuva21/test-iscollectionentry-1790141…
a792aa7
…5298177206964
🧪 Add tests for isCollectionEntry in generate-manifest.js
Bot
Merge master into PR#124
29cc8c9
@akutuva21
Merge pull request#124from akutuva21/fix-unused-getignore-dirs-1320…
e0abed4
…3728183734872069
🧹 Code Health: Remove unused `getIgnoreDirs` function and associated dead code
Bot
Merge master into PR#123
f097fe2
@akutuva21
Merge pull request#123from akutuva21/test-coverage-expectArray-1287…
cf13326
…0449047627298925
🧪 [test] add unit tests for expectArray
@akutuva21
Merge pull request#128from akutuva21/fix-path-traversal-input-10377…
ceb558f
…525129266671774
🔒 fix: prevent path traversal on input file read
Bot
Merge master into PR#132
e6f7d77
@akutuva21
Merge pull request#132from akutuva21/test-isignored-dir-18050093452…
cc85e1a
…0554265
🧪 Add tests for isIgnoredDir in generate-manifest
@akutuva21
Fix stale test files to match updated async APIs
9e293f0
Update test duplicates in scripts/tests/ and tests/ that were
left behind after PR merges changed functions to async and
updated normalizeModelKey to preserve hyphens instead of
stripping them.
@akutuva21
Fix CI error: define DEFAULT_IGNORE_DIRS and isIgnoredDir in generate…
4e1ee68
…-manifest.js
Add missing DEFAULT_IGNORE_DIRS constant and isIgnoredDir function
that were referenced but never defined, causing ReferenceError in CI.
Also fix syntax error in test file (extra closing braces).
@akutuva21
⚡ Optimize listModelFiles to use asynchronous I/O
7ecd47d
Replaces synchronous file I/O with async readdir/stat operations for better performance when listing model files.
@akutuva21
⚡ Extract Regex from Gallery Assignment Loop
b255844
Extracts the regex construction outside the loop to avoid recompilation on every iteration.
@akutuva21
⚡ Replace synchronous file I/O with fs.promises
9f2f3b3
Replaces sync fs calls with fs.promises.
@akutuva21
🧪 Add edge case tests for normalizeModelKey
aa18c0c
Adds edge case tests for normalizeModelKey.
@akutuva21
⚡ Optimize gallery assignments by replacing synchronous I/O
3c18cf3
Replaces synchronous I/O in gallery assignment code.
@akutuva21
akutuva21
commented
Jun 10, 2026

Copy link


Copy Markdown

MemberAuthor
Closing — involves changes to Mitra2019 model files which are excluded from merges per policy.
@akutuva21
akutuva21
commented
Jun 10, 2026

Copy link


Copy Markdown

MemberAuthor
Mitra2019 files excluded from merges
vcellmike pushed a commit
to vcellmike/RuleHub
that referenced
this pull request
Aug 10, 2026
@Psuedonerd
Merge pull requestRuleWorld#14from Psuedonerd/codex/inspect-model-f…
101cd51
…olders-in-rulehub-repo-ioilb1
Codex-generated pull request
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@akutuva21