Uh oh!
There was an error while loading. Please reload this page.
fix(js/vertexai): dedupe, improve types, fix rouge - #3592
Conversation
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
MichaelDoyle
commented
May 7, 2026
/gemini summary |
MichaelDoyle
commented
May 7, 2026
/gemini review |
Summary of ChangesThis pull request refactors the Vertex AI evaluation plugin to improve type safety and maintainability. By leveraging official protobuf definitions and centralizing shared types, the codebase is now more robust and less prone to configuration errors. Additionally, the changes address specific implementation bugs in the ROUGE evaluator and fix minor naming inconsistencies. Highlights
Activity
|
MichaelDoyle
commented
May 7, 2026
@ssbushi can you take a peek? |
There was a problem hiding this comment.
Code Review
This pull request enhances type safety within the Vertex AI evaluation plugin by migrating evaluation types to a separate file and replacing any types with specific Protobuf definitions from @google-cloud/aiplatform. The evaluator factory and specific evaluator functions were updated to utilize these stricter types. Review feedback pointed out that type-casting datapoint.reference as a string in the BLEU and ROUGE evaluators could mask runtime errors if the value is undefined, suggesting the addition of explicit runtime validation checks.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
ssbushi
left a comment
There was a problem hiding this comment.
Thank you for your contribution. I made a few trivial tweak to get this ready for merging
Uh oh!
There was an error while loading. Please reload this page.
While investigating the appropriate types for vertex evaluators, I'm adding them here too, and discovered an incorrect type used for the ROUGE score evaluator. Also there was some code duplication with the types.
Checklist (if applicable):
Docs updated (updated docs or a docs bug required)