UUICS (Universal UI Context System)

A performance-optimized, framework-agnostic system for AI agents to understand and interact with web interfaces.

UUICS bridges the gap between AI models and web UIs by providing structured context about page elements and enabling AI-driven interactions through a simple action execution system.

🎯 What is UUICS?

UUICS scans your web page, extracts all interactive elements (buttons, inputs, dropdowns, etc.), and provides this information in AI-friendly formats. When an AI decides to take action, UUICS executes it safely on the page.

┌────────────────────────────────────────────────────────────────────────────────────┐
│ HOW UUICS WORKS │
├────────────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────┐ ┌─────────────────────┐ ┌──────────────┐ │
│ │ YOUR WEB PAGE │ │ UUICS ENGINE │ │ AI MODEL │ │
│ │ │ │ │ │ │ │
│ │ • Buttons │ scan │ ┌───────────────┐ │ send │ Claude, │ │
│ │ • Inputs │ ───────▶│ │ DOM Scanner │ │ ───────▶│ GPT-4, │ │
│ │ • Forms │ │ │ │ │ │ or any LLM │ │
│ │ • Dropdowns │ │ │ • Elements │ │ │ │ │
│ │ • Radio buttons │ │ │ • States │ │ │ Analyzes │ │
│ │ • Checkboxes │ │ │ • Attributes │ │ │ context & │ │
│ │ • Links │ │ │ • Labels │ │ │ decides │ │
│ │ • Custom elements │ │ └───────────────┘ │ │ action │ │
│ │ │ │ │ │ │ │ │
│ │ Framework Support: │ │ ▼ │ └──────┬───────┘ │
│ │ ✓ Radix UI │ │ ┌───────────────┐ │ │ │
│ │ ✓ Shadcn/ui │ │ │ Serializer │ │ │ │
│ │ ✓ MUI/Chakra │ │ │ │ │ │ │
│ │ ✓ Headless UI │ │ │ JSON/Natural/ │ │ │ │
│ │ ✓ Vanilla HTML │ │ │ OpenAPI/MCP │ │ │ │
│ └─────────────────────┘ │ └───────────────┘ │ │ │
│ ▲ │ │ │ │
│ │ │ ┌───────────────┐ │ │ │
│ │ │ │Action Executor│ │ │ │
│ │ execute │ │ │◀─┼────────────────┘ │
│ └──────────────────────┼──│ click, type, │ │ command │
│ │ │ select, check │ │ │
│ Element Detection: │ │ hover, submit │ │ │
│ ───────────────── │ └───────────────┘ │ │
│ • data-uuics-* attributes │ │ │
│ • aria-label/aria-checked └─────────────────────┘ │
│ • data-state (Radix UI) │
│ • role="button/radio/..." │
│ • Standard HTML attributes │
│ │
└────────────────────────────────────────────────────────────────────────────────────┘

✨ Features

Core Capabilities

🔍 DOM Scanning: Automatically detects all interactive elements
📝 Context Serialization: JSON, Natural Language, or OpenAPI formats
⚡ Action Execution: Click, type, select, check, hover, and more
🔄 State Tracking: Track JavaScript variables alongside DOM state
🛡️ Sensitive Data Protection: Automatically exclude passwords and tokens
🔌 MCP Support: Native Model Context Protocol integration for Claude

Framework Support

Vanilla JavaScript: Direct engine usage
React: Provider pattern with hooks (see examples)
Vue, Angular, etc.: Engine works with any framework

AI Model Support

Claude: Full support with MCP tool calling + natural language context
GPT-4/OpenAI: Function calling with OpenAPI format
Any LLM: JSON format works universally

📦 Installation

npm install @angelerator/uuics-core

🚀 Quick Start

Basic Usage

import{UUICSEngine}from'@angelerator/uuics-core';// Create engineconstuuics=newUUICSEngine({scan: {interval: 0},// Manual scanningtrack: {mutations: true,clicks: true}});// Initializeawaituuics.initialize();// Scan the pageconstcontext=awaituuics.scan();// Get AI-friendly contextconstnaturalLanguage=uuics.serialize('natural');console.log(naturalLanguage);// Output:// # Page Context// ## Interactive Elements// ### Buttons (3)// - **Submit** → `#submit-btn`// - **Cancel** → `#cancel-btn`// ...// Execute an action from AI responseawaituuics.execute({action: 'setValue',target: '#email',parameters: {value: 'user@example.com'}});

With Claude AI

import{UUICSEngine}from'@angelerator/uuics-core';constuuics=newUUICSEngine();awaituuics.initialize();// Get page contextawaituuics.scan();constcontext=uuics.serialize('natural');// Send to Claude (using your preferred method)constresponse=awaitclaude.messages.create({model: 'claude-sonnet-4-20250514',system: `You are a web automation assistant. Here's the page context:\n${context}`,messages: [{role: 'user',content: 'Fill in the email field with test@example.com'}]});// Parse and execute Claude's actionconstaction=parseActionFromResponse(response);awaituuics.execute(action);

With MCP (Model Context Protocol) 🆕

UUICS includes native MCP support for Claude's tool calling capabilities:

import{UUICSEngine,MCPToolsGenerator,MCPToolHandler}from'@angelerator/uuics-core';// Initialize UUICSconstengine=newUUICSEngine();awaitengine.initialize();awaitengine.scan();// Generate MCP tools from current page contextconstgenerator=newMCPToolsGenerator();consttools=generator.generateTools(engine.getContext());// Send tools to Claudeconstresponse=awaitanthropic.messages.create({model: 'claude-sonnet-4-20250514',max_tokens: 1024,tools: tools.map(t=>({name: t.name,description: t.description,input_schema: t.input_schema,})),messages: [{role: 'user',content: 'Click the submit button'}]});// Handle tool calls from Claudeconsthandler=newMCPToolHandler(engine);for(constblockofresponse.content){if(block.type==='tool_use'){constresult=awaithandler.handleToolCall({name: block.name,input: block.input,id: block.id,});console.log('Tool result:',result);}}

📖 API Reference

UUICSEngine

The main class for interacting with UUICS.

Constructor

newUUICSEngine(config?: UUICSConfig)

Core Methods

Method	Description
`initialize()`	Initialize the engine
`scan(root?, config?)`	Scan DOM and update context
`getContext()`	Get current page context
`serialize(format?)`	Serialize context (json/natural/openapi)
`subscribe(callback)`	Subscribe to context updates

Action Methods

Method	Description
`execute(command)`	Execute a single action
`executeBatch(commands)`	Execute multiple actions sequentially

State Tracking

Method	Description
`trackState(name, obj)`	Track object with auto-updates
`registerState(name, getter)`	Register computed state
`untrackState(name)`	Stop tracking an object

Action Types

typeActionType=|'click'// Click an element|'setValue'// Set input/textarea value|'select'// Select dropdown option(s)|'check'// Check a checkbox|'uncheck'// Uncheck a checkbox|'submit'// Submit a form|'focus'// Focus an element|'scroll'// Scroll to element|'hover'// Hover over element|'custom';// Execute custom script

🔌 MCP (Model Context Protocol) Support

UUICS provides native MCP integration for Claude and other MCP-compatible AI models.

MCP Classes

Class	Description
`MCPToolsGenerator`	Generates MCP tool definitions from page context
`MCPToolHandler`	Handles MCP tool calls and executes UI actions

MCP Core Tools (16 Total)

Tool	Category	Description
`ui_scan`	context	Scan page to discover interactive elements
`ui_get_context`	context	Get current context without re-scanning
`ui_get_element`	context	Get details about a specific element
`ui_get_state`	context	Query tracked JavaScript application state
`ui_click`	interaction	Click on an element
`ui_type`	interaction	Type text into an input field
`ui_select`	interaction	Select an option from dropdown
`ui_check`	interaction	Check a checkbox
`ui_uncheck`	interaction	Uncheck a checkbox
`ui_submit`	interaction	Submit a form
`ui_scroll`	interaction	Scroll to bring element into view
`ui_focus`	interaction	Set focus on an element
`ui_hover`	interaction	Hover over element (dropdowns/tooltips)
`ui_wait_for`	utility	Wait for element or condition
`ui_screenshot`	debug	Capture element/page for visual debugging
`ui_execute_batch`	interaction	Execute multiple actions in sequence

MCP Configuration

import{MCPToolsGenerator,MCPToolHandler}from'@angelerator/uuics-core';constgenerator=newMCPToolsGenerator({includeCoreTools: true,// Include all 16 core toolsgenerateDynamicTools: true,// Generate element-specific toolsmaxDynamicTools: 50,// Limit dynamic toolstoolPrefix: 'ui_',// Tool name prefixelementTypes: ['button','input','select','checkbox','link'],customTools: [],// Add your own tools});// Generate tools for current page stateconsttools=generator.generateTools(engine.getContext());// Create handler to execute tool callsconsthandler=newMCPToolHandler(engine);

Dynamic Tools

MCP automatically generates element-specific tools based on the current page:

# For a page with:
# - Submit button (#submit-btn)
# - Email input (#email)
# - Country dropdown (#country)
Generated dynamic tools:
- ui_click_submit → Clicks the Submit button
- ui_set_email → Sets the Email input value
- ui_select_country → Selects a Country option

ui_wait_for Conditions

Condition	Description
`visible`	Element is visible in viewport
`hidden`	Element is hidden or removed
`exists`	Element exists in DOM
`not_exists`	Element does not exist
`enabled`	Form element is enabled
`disabled`	Form element is disabled

// Wait for loading spinner to disappearawaithandler.handleToolCall({name: 'ui_wait_for',input: {selector: '.loading-spinner',condition: 'hidden',timeout: 10000,}});

ui_screenshot Usage

// Capture element boundsconstresult=awaithandler.handleToolCall({name: 'ui_screenshot',input: {selector: '#main-form',format: 'png',scale: 2,}});// Returns: { dataUrl: 'data:image/png;base64,...', width, height }

ui_get_state Usage

// Query tracked application stateconstresult=awaithandler.handleToolCall({name: 'ui_get_state',input: {key: 'user',// Optional: specific keyinclude_metadata: true,// Include timestamp}});// Returns: { key: 'user', value: {...}, timestamp: 1234567890 }

Configuration

constconfig: UUICSConfig={scan: {interval: 0,// Auto-scan interval (0 = manual)depth: 10,// Max DOM depthincludeHidden: false,// Include hidden elementsrootSelectors: ['#app'],// Scan specific areas onlyexcludeSelectors: ['.ads']// Skip certain elements},track: {mutations: true,// Track DOM mutationsclicks: true,// Track click eventschanges: true,// Track input changesdebounceDelay: 100// Debounce delay (ms)},state: {enabled: true,// Enable state trackingexclude: ['*password*']// Exclude sensitive fields},performance: {enableCache: true,// Cache scanned elementsmaxElements: 1000// Max elements to scan}};

🎨 Serialization Formats

Natural Language (AI-Friendly)

# Page Context
Page: My Application
URL: https://example.com
## Interactive Elements
### Inputs (3)
- **Email** → `#email` - **Password** → `#password`
- **Name** → `#name`
### Buttons (2)
- **Submit** → `#submit-btn`
- **Cancel** → `#cancel-btn`
### Selects (1)
- **Country** [OPTIONS: "USA" (value: us), "UK" (value: uk)] → `#country`
## Application State
- user: { name: "John", loggedIn: true }

JSON (Structured)

{
"url": "https://example.com",
"title": "My Application",
"elements": [
{
"type": "input",
"selector": "#email",
"label": "Email",
"value": ""
}
],
"actions": [
{ "type": "setValue", "target": "#email" },
{ "type": "click", "target": "#submit-btn" }
],
"state": {
"user": { "name": "John" }
}
}

OpenAPI (Function Calling)

{
"tools": [
{
"type": "function",
"function": {
"name": "ui_click",
"parameters": {
"properties": {
"target": { "enum": ["#submit-btn", "#cancel-btn"] }
}
}
}
}
]
}

📂 Project Structure

uuics/
├── packages/
│ └── core/ # @angelerator/uuics-core
│ ├── src/
│ │ ├── scanner/ # DOM scanning
│ │ ├── tracker/ # Mutation & state tracking
│ │ ├── aggregator/ # Context aggregation
│ │ ├── serializer/ # Output formatting
│ │ ├── executor/ # Action execution
│ │ ├── mcp/ # MCP (Model Context Protocol) support
│ │ │ ├── types.ts
│ │ │ ├── MCPToolsGenerator.ts
│ │ │ ├── MCPToolHandler.ts
│ │ │ └── index.ts
│ │ └── UUICSEngine.ts
│ └── package.json
│
└── examples/
├── react-app/ # React + Claude integration
│ ├── src/
│ │ ├── UUICSProvider.tsx # React context
│ │ ├── ClaudeAdapter.ts # Claude integration
│ │ └── ...
│ └── claude-cli-proxy.cjs # Proxy for Claude Code
│
└── vanilla/ # Vanilla JavaScript example
└── index.html

🏃 Running Examples

Prerequisites

# Clone the repository
git clone https://github.com/Angelerator/UUICS.git
cd UUICS
# Install dependencies
pnpm install
# Build packages
pnpm build

React Example with Claude

# Terminal 1: Start the React appcd examples/react-app
pnpm dev
# Opens at http://localhost:5173# Terminal 2: Start Claude proxy (uses Claude Code subscription)cd examples/react-app
node claude-cli-proxy.cjs
# Proxy runs at http://localhost:3100

Then open the app and click the 🤖 button to chat with Claude!

Vanilla Example

cd examples/vanilla
pnpm dev
# Opens at http://localhost:5173

🔧 Advanced Usage

State Tracking

// Track objects with automatic change detectionconstuser=uuics.trackState('user',{name: 'John',preferences: {theme: 'dark'}});user.name='Jane';// Automatically tracked!// Register computed valuesuuics.registerState('metrics',()=>({pageViews: analytics.getPageViews(),sessionDuration: Date.now()-startTime}));

Scope Control

// Scan only specific areasawaituuics.scan(null,{rootSelectors: ['#main-form','#sidebar'],excludeSelectors: ['.advertisement','footer']});

Sensitive Data Protection

constuuics=newUUICSEngine({state: {enabled: true,exclude: ['*password*','*token*','*secret*','*key*']}});// Sensitive fields automatically excluded from contextconstauth=uuics.trackState('auth',{username: 'john',// ✅ Includedpassword: 'secret123',// ❌ Excluded as '[EXCLUDED]'apiKey: 'sk-abc123'// ❌ Excluded as '[EXCLUDED]'});

Action Chaining

// Execute multiple actions in sequenceconstresults=awaituuics.executeBatch([{action: 'click',target: '#menu-button'},{action: 'click',target: '#settings'},{action: 'setValue',target: '#theme',parameters: {value: 'dark'}},{action: 'click',target: '#save'}]);results.forEach((r,i)=>{console.log(`Step ${i+1}: ${r.success ? '✓' : '✗'}${r.message}`);});

🤖 Use Cases

AI-Powered Testing: Automated testing with natural language
Intelligent Automation: Smart bots that understand page context
Accessibility Tools: AI assistants for users with disabilities
Form Auto-Fill: Context-aware form completion
Browser Extensions: AI-powered browser tools
RPA: Intelligent workflow automation

🔄 Migrating from Previous Versions

If you were using the separate packages (@angelerator/uuics-react, @angelerator/uuics-models-claude, @angelerator/uuics-models-openai), these have been deprecated. The React integration and model adapters are now provided as examples that you can copy into your project.

See the examples/react-app/ directory for:

UUICSProvider.tsx - React context and hooks
ClaudeAdapter.ts - Claude integration

🤝 Contributing

Contributions are welcome!

Fork the repository
Create a feature branch: git checkout -b feature/amazing-feature
Commit changes: git commit -m 'feat: add amazing feature'
Push to branch: git push origin feature/amazing-feature
Open a Pull Request

📄 License

MIT - see LICENSE for details.

Name		Name	Last commit message	Last commit date
examples		examples
packages/core		packages/core
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
package-lock.json		package-lock.json
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
pnpm-workspace.yaml		pnpm-workspace.yaml
tsconfig.json		tsconfig.json

Uh oh!

Latest commit

History

Folders and files

Repository files navigation

UUICS (Universal UI Context System)

🎯 What is UUICS?

✨ Features

Core Capabilities

Framework Support

AI Model Support

📦 Installation

🚀 Quick Start

Basic Usage

With Claude AI

With MCP (Model Context Protocol) 🆕

📖 API Reference

UUICSEngine

Constructor

Core Methods

Action Methods

State Tracking

Action Types

🔌 MCP (Model Context Protocol) Support

MCP Classes

MCP Core Tools (16 Total)

MCP Configuration

Dynamic Tools

ui_wait_for Conditions

ui_screenshot Usage

ui_get_state Usage

Configuration

🎨 Serialization Formats

Natural Language (AI-Friendly)

JSON (Structured)

OpenAPI (Function Calling)

📂 Project Structure

🏃 Running Examples

Prerequisites

React Example with Claude

Vanilla Example

🔧 Advanced Usage

State Tracking

Scope Control

Sensitive Data Protection

Action Chaining

🤖 Use Cases

🔄 Migrating from Previous Versions

🤝 Contributing

📄 License

🔗 Links

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages