Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion CHANGELOG.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -7,6 +7,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Changed
- **Unified `list_objects` / `describe_object` tools (`service-ai`)** — Merged the duplicate
`list_metadata_objects` → `list_objects` and `describe_metadata_object` → `describe_object`
tool pairs. Both `data_chat` and `metadata_assistant` agents now share the same unified tools
with full `filter`, `includeFields`, snake_case validation, and `enableFeatures` support.
`DATA_TOOL_DEFINITIONS` is reduced from 5 to 3 (query-only tools), while
`METADATA_TOOL_DEFINITIONS` retains all 6 tools under the unified names. The duplicate
`ObjectDef`/`FieldDef` type definitions in `data-tools.ts` are removed.

### Fixed
- **Agent Chat: Vercel SSE Data Stream support** — The agent chat endpoint
(`/api/v1/ai/agents/:agentName/chat`) now returns Vercel AI SDK v6 UI Message Stream Protocol
Expand DownExpand Up@@ -157,7 +166,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
- **Metadata Assistant Agent (`service-ai`)** — New `metadata_assistant` agent definition that
binds all 6 metadata management tools (`create_object`, `add_field`, `modify_field`,
`delete_field`, `list_metadata_objects`, `describe_metadata_object`). Includes a tailored
`delete_field`, `list_objects`, `describe_object`). Includes a tailored
system prompt that guides the AI to use snake_case naming, verify existing schemas before
modifications, and warn about destructive operations. Configured with `react` planning
strategy (10 iterations, replan enabled) for multi-step schema design conversations.
Expand Down
2 changes: 1 addition & 1 deletion apps/studio/test/ai-chat-panel.test.tsx
Original file line numberDiff line numberDiff line change
Expand Up@@ -164,7 +164,7 @@ describe('Messages with tool invocation parts', () => {
role: 'assistant',
toolParts: [
{
toolName: 'list_metadata_objects',
toolName: 'list_objects',
toolCallId: 'tc_2',
state: 'output-available',
input: {},
Expand Down
Original file line numberDiff line numberDiff line change
Expand Up@@ -304,15 +304,13 @@ describe('AIService.chatWithTools', () => {

describe('Data Tools', () => {
describe('DATA_TOOL_DEFINITIONS', () => {
it('should define exactly 5 tools', () => {
expect(DATA_TOOL_DEFINITIONS).toHaveLength(5);
it('should define exactly 3 tools', () => {
expect(DATA_TOOL_DEFINITIONS).toHaveLength(3);
});

it('should include all expected tool names', () => {
const names = DATA_TOOL_DEFINITIONS.map(t => t.name);
expect(names).toEqual([
'list_objects',
'describe_object',
'query_records',
'get_record',
'aggregate_data',
Expand All@@ -336,22 +334,24 @@ describe('Data Tools', () => {
registry = new ToolRegistry();
dataEngine = createMockDataEngine();
metadataService = createMockMetadataService();
registerDataTools(registry, { dataEngine, metadataService });
registerDataTools(registry, { dataEngine });
});

it('should register all 5 tools', () => {
expect(registry.size).toBe(5);
expect(registry.has('list_objects')).toBe(true);
expect(registry.has('describe_object')).toBe(true);
it('should register all 3 tools', () => {
expect(registry.size).toBe(3);
expect(registry.has('query_records')).toBe(true);
expect(registry.has('get_record')).toBe(true);
expect(registry.has('aggregate_data')).toBe(true);
});

it('list_objects should return object names and labels', async () => {
it('list_objects should return object names and labels (via metadata tools)', async () => {
// list_objects is now part of metadata tools — register them
const { registerMetadataTools } = await import('../tools/metadata-tools.js');
registerMetadataTools(registry, { metadataService });

(metadataService.listObjects as any).mockResolvedValue([
{ name: 'account', label: 'Account' },
{ name: 'contact', label: 'Contact' },
{ name: 'account', label: 'Account', fields: { name: { type: 'text' } } },
{ name: 'contact', label: 'Contact', fields: {} },
]);

const result = await registry.execute({
Expand All@@ -362,11 +362,15 @@ describe('Data Tools', () => {
});

const parsed = JSON.parse((result.output as any).value);
expect(parsed).toHaveLength(2);
expect(parsed[0]).toEqual({ name: 'account', label: 'Account' });
expect(parsed.objects).toHaveLength(2);
expect(parsed.objects[0]).toEqual(expect.objectContaining({ name: 'account', label: 'Account' }));
});

it('describe_object should return field schema', async () => {
it('describe_object should return field schema (via metadata tools)', async () => {
// describe_object is now part of metadata tools — register them
const { registerMetadataTools } = await import('../tools/metadata-tools.js');
registerMetadataTools(registry, { metadataService });

(metadataService.getObject as any).mockResolvedValue({
name: 'account',
label: 'Account',
Expand All@@ -385,12 +389,19 @@ describe('Data Tools', () => {

const parsed = JSON.parse((result.output as any).value);
expect(parsed.name).toBe('account');
expect(parsed.fields.name.type).toBe('text');
expect(parsed.fields.name.required).toBe(true);
expect(parsed.fields.revenue.type).toBe('number');
// Unified handler returns fields as array (not object)
const nameField = parsed.fields.find((f: any) => f.name === 'name');
expect(nameField.type).toBe('text');
expect(nameField.required).toBe(true);
const revenueField = parsed.fields.find((f: any) => f.name === 'revenue');
expect(revenueField.type).toBe('number');
});

it('describe_object should return error for unknown object', async () => {
it('describe_object should return error for unknown object (via metadata tools)', async () => {
// describe_object is now part of metadata tools — register them
const { registerMetadataTools } = await import('../tools/metadata-tools.js');
registerMetadataTools(registry, { metadataService });

const result = await registry.execute({
type: 'tool-call' as const,
toolCallId: 'c1',
Expand DownExpand Up@@ -1067,8 +1078,8 @@ describe('METADATA_ASSISTANT_AGENT', () => {
expect(toolNames).toContain('add_field');
expect(toolNames).toContain('modify_field');
expect(toolNames).toContain('delete_field');
expect(toolNames).toContain('list_metadata_objects');
expect(toolNames).toContain('describe_metadata_object');
expect(toolNames).toContain('list_objects');
expect(toolNames).toContain('describe_object');
});

it('should use action type for mutation tools and query type for read tools', () => {
Expand DownExpand Up@@ -1099,7 +1110,7 @@ describe('METADATA_ASSISTANT_AGENT', () => {
it('should have instructions mentioning metadata management capabilities', () => {
const instructions = METADATA_ASSISTANT_AGENT.instructions;
expect(instructions).toContain('snake_case');
expect(instructions).toContain('list_metadata_objects');
expect(instructions).toContain('describe_metadata_object');
expect(instructions).toContain('list_objects');
expect(instructions).toContain('describe_object');
});
});
64 changes: 35 additions & 29 deletions packages/services/service-ai/src/__tests__/metadata-tools.test.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -17,8 +17,8 @@ import { createObjectTool } from '../tools/create-object.tool.js';
import { addFieldTool } from '../tools/add-field.tool.js';
import { modifyFieldTool } from '../tools/modify-field.tool.js';
import { deleteFieldTool } from '../tools/delete-field.tool.js';
import { listMetadataObjectsTool } from '../tools/list-metadata-objects.tool.js';
import { describeMetadataObjectTool } from '../tools/describe-metadata-object.tool.js';
import { listObjectsTool } from '../tools/list-objects.tool.js';
import { describeObjectTool } from '../tools/describe-object.tool.js';

// ── Helpers ────────────────────────────────────────────────────────

Expand DownExpand Up@@ -63,8 +63,8 @@ describe('Metadata Tool Definitions', () => {
'add_field',
'modify_field',
'delete_field',
'list_metadata_objects',
'describe_metadata_object',
'list_objects',
'describe_object',
]);
});

Expand All@@ -86,8 +86,8 @@ describe('Individual Tool Metadata (.tool.ts)', () => {
{ tool: addFieldTool, expectedName: 'add_field', expectedLabel: 'Add Field' },
{ tool: modifyFieldTool, expectedName: 'modify_field', expectedLabel: 'Modify Field' },
{ tool: deleteFieldTool, expectedName: 'delete_field', expectedLabel: 'Delete Field' },
{ tool: listMetadataObjectsTool, expectedName: 'list_metadata_objects', expectedLabel: 'List Metadata Objects' },
{ tool: describeMetadataObjectTool, expectedName: 'describe_metadata_object', expectedLabel: 'Describe Metadata Object' },
{ tool: listObjectsTool, expectedName: 'list_objects', expectedLabel: 'List Objects' },
{ tool: describeObjectTool, expectedName: 'describe_object', expectedLabel: 'Describe Object' },
];

for (const { tool, expectedName, expectedLabel } of tools) {
Expand DownExpand Up@@ -132,8 +132,8 @@ describe('Individual Tool Metadata (.tool.ts)', () => {
});

it('should not mark read-only tools as requiresConfirmation', () => {
expect(listMetadataObjectsTool.requiresConfirmation).toBe(false);
expect(describeMetadataObjectTool.requiresConfirmation).toBe(false);
expect(listObjectsTool.requiresConfirmation).toBe(false);
expect(describeObjectTool.requiresConfirmation).toBe(false);
});

it('should not mark add_field and modify_field as requiresConfirmation', () => {
Expand DownExpand Up@@ -162,17 +162,17 @@ describe('registerMetadataTools', () => {
expect(registry.has('add_field')).toBe(true);
expect(registry.has('modify_field')).toBe(true);
expect(registry.has('delete_field')).toBe(true);
expect(registry.has('list_metadata_objects')).toBe(true);
expect(registry.has('describe_metadata_object')).toBe(true);
expect(registry.has('list_objects')).toBe(true);
expect(registry.has('describe_object')).toBe(true);
});
});

// ═══════════════════════════════════════════════════════════════════
// Dual registration (data tools + metadata tools)
// ═══════════════════════════════════════════════════════════════════

describe('registerDataTools + registerMetadataTools — no collision', () => {
it('should register both tool sets on the same registry without overwriting', () => {
describe('registerDataTools + registerMetadataTools — unified list/describe', () => {
it('should register both tool sets on the same registry with shared list_objects and describe_object', () => {
const registry = new ToolRegistry();
const metadataService = createMockMetadataService();
const dataEngine = {
Expand All@@ -181,26 +181,32 @@ describe('registerDataTools + registerMetadataTools — no collision', () => {
aggregate: vi.fn(),
} as any;

registerDataTools(registry, { dataEngine, metadataService });
registerDataTools(registry, { dataEngine });
const sizeAfterData = registry.size;

registerMetadataTools(registry, { metadataService });
const sizeAfterBoth = registry.size;

// Data tools define: list_objects, describe_object, query_records, get_record, aggregate_data
// Metadata tools define: create_object, add_field, modify_field, delete_field, list_metadata_objects, describe_metadata_object
// No overlap — total should be sum of both
// Data tools define: query_records, get_record, aggregate_data (3)
// Metadata tools define: create_object, add_field, modify_field, delete_field, list_objects, describe_object (6)
// Total should be 3 + 6 = 9
expect(sizeAfterData).toBe(3);
expect(sizeAfterBoth).toBe(sizeAfterData + 6);

// Data tools should still be present
// Unified list/describe should be present (from metadata tools)
expect(registry.has('list_objects')).toBe(true);
expect(registry.has('describe_object')).toBe(true);

// Data-only tools should be present
expect(registry.has('query_records')).toBe(true);
expect(registry.has('get_record')).toBe(true);
expect(registry.has('aggregate_data')).toBe(true);

// Metadata tools should also be present with distinct names
expect(registry.has('list_metadata_objects')).toBe(true);
expect(registry.has('describe_metadata_object')).toBe(true);
// Metadata-only tools should be present
expect(registry.has('create_object')).toBe(true);
expect(registry.has('add_field')).toBe(true);
expect(registry.has('modify_field')).toBe(true);
expect(registry.has('delete_field')).toBe(true);
});
});

Expand DownExpand Up@@ -752,7 +758,7 @@ describe('list_metadata_objects handler', () => {
const result = await registry.execute({
type: 'tool-call' as const,
toolCallId: 'c1',
toolName: 'list_metadata_objects',
toolName: 'list_objects',
input: {},
});

Expand All@@ -767,7 +773,7 @@ describe('list_metadata_objects handler', () => {
const result = await registry.execute({
type: 'tool-call' as const,
toolCallId: 'c2',
toolName: 'list_metadata_objects',
toolName: 'list_objects',
input: { filter: 'account' },
});

Expand All@@ -780,7 +786,7 @@ describe('list_metadata_objects handler', () => {
const result = await registry.execute({
type: 'tool-call' as const,
toolCallId: 'c3',
toolName: 'list_metadata_objects',
toolName: 'list_objects',
input: { includeFields: true },
});

Expand All@@ -797,7 +803,7 @@ describe('list_metadata_objects handler', () => {
const result = await registry.execute({
type: 'tool-call' as const,
toolCallId: 'c4',
toolName: 'list_metadata_objects',
toolName: 'list_objects',
input: {},
});

Expand DownExpand Up@@ -836,7 +842,7 @@ describe('describe_metadata_object handler', () => {
const result = await registry.execute({
type: 'tool-call' as const,
toolCallId: 'c1',
toolName: 'describe_metadata_object',
toolName: 'describe_object',
input: { objectName: 'account' },
});

Expand All@@ -858,7 +864,7 @@ describe('describe_metadata_object handler', () => {
const result = await registry.execute({
type: 'tool-call' as const,
toolCallId: 'c2',
toolName: 'describe_metadata_object',
toolName: 'describe_object',
input: { objectName: 'nonexistent' },
});

Expand DownExpand Up@@ -909,7 +915,7 @@ describe('Metadata Tools — full lifecycle', () => {
const descResult = await registry.execute({
type: 'tool-call' as const,
toolCallId: 's4',
toolName: 'describe_metadata_object',
toolName: 'describe_object',
input: { objectName: 'invoice' },
});
const desc = JSON.parse((descResult.output as any).value);
Expand DownExpand Up@@ -941,7 +947,7 @@ describe('Metadata Tools — full lifecycle', () => {
const descResult2 = await registry.execute({
type: 'tool-call' as const,
toolCallId: 's7',
toolName: 'describe_metadata_object',
toolName: 'describe_object',
input: { objectName: 'invoice' },
});
const desc2 = JSON.parse((descResult2.output as any).value);
Expand All@@ -954,7 +960,7 @@ describe('Metadata Tools — full lifecycle', () => {
const listResult = await registry.execute({
type: 'tool-call' as const,
toolCallId: 's8',
toolName: 'list_metadata_objects',
toolName: 'list_objects',
input: {},
});
const list = JSON.parse((listResult.output as any).value);
Expand Down
Original file line numberDiff line numberDiff line change
Expand Up@@ -36,8 +36,8 @@ Capabilities:
- Describe the full schema of a specific object

Guidelines:
1. Before creating a new object, use list_metadata_objects to check if a similar one already exists.
2. Before modifying or deleting fields, use describe_metadata_object to understand the current schema.
1. Before creating a new object, use list_objects to check if a similar one already exists.
2. Before modifying or deleting fields, use describe_object to understand the current schema.
3. Always use snake_case for object names and field names (e.g. project_task, due_date).
4. Suggest meaningful field types based on the user's description (e.g. "deadline" → date, "active" → boolean).
5. When creating objects, propose a reasonable set of initial fields based on the entity type.
Expand All@@ -59,8 +59,8 @@ Guidelines:
{ type: 'action', name: 'add_field', description: 'Add a field to an existing object' },
{ type: 'action', name: 'modify_field', description: 'Modify an existing field definition' },
{ type: 'action', name: 'delete_field', description: 'Delete a field from an object' },
{ type: 'query', name: 'list_metadata_objects', description: 'List all metadata objects' },
{ type: 'query', name: 'describe_metadata_object', description: 'Describe an object schema' },
{ type: 'query', name: 'list_objects', description: 'List all data objects' },
{ type: 'query', name: 'describe_object', description: 'Describe an object schema' },
],

active: true,
Expand Down
4 changes: 2 additions & 2 deletions packages/services/service-ai/src/index.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -39,8 +39,8 @@ export {
addFieldTool,
modifyFieldTool,
deleteFieldTool,
listMetadataObjectsTool,
describeMetadataObjectTool,
listObjectsTool,
describeObjectTool,
} from './tools/metadata-tools.js';

// Agent runtime
Expand Down
Loading