Skip to content

Repository files navigation

E2B Surf Preview LightE2B Surf Preview Dark

🏄 Surf - OpenAI's Computer Use Agent + E2B Desktop

A Next.js application that allows AI to interact with a virtual desktop environment. This project integrates E2B's desktop sandbox with OpenAI's API to create an AI agent that can perform tasks on a virtual computer through natural language instructions.

E2B is an open source isolated virtual computer in the cloud made for AI use cases.

Overview

The Computer Use App provides a web interface where users can:

  1. Start a virtual desktop sandbox environment
  2. Send natural language instructions to an AI agent
  3. Watch as the AI agent performs actions on the virtual desktop
  4. Interact with the AI through a chat interface

The application uses Server-Sent Events (SSE) to stream AI responses and actions in real-time, providing a seamless experience.

How It Works

Architecture

The application consists of several key components:

  1. Frontend UI (Next.js): Provides the user interface with a virtual desktop view and chat interface
  2. E2B Desktop Sandbox: Creates and manages virtual desktop environments
  3. OpenAI Computer Use: Processes user instructions and generates actions for the AI agent
  4. Streaming API: Handles real-time communication between the frontend and backend

Core Flow

  1. User starts a new sandbox instance
  2. E2B creates a virtual desktop and provides a URL for streaming
  3. User sends instructions via the chat interface
  4. Backend processes the instructions using OpenAI's API
  5. AI generates actions (clicks, typing, etc.) to perform on the virtual desktop
  6. Actions are executed on the sandbox and streamed back to the frontend
  7. The process repeats as the user continues to provide instructions

Prerequisites

Before starting, you'll need:

  1. Node.js (version specified in package.json)
  2. npm (comes with Node.js)
  3. An E2B API key
  4. An OpenAI API key

Setup Instructions

  1. Clone the repository
git clone https://github.com/e2b-dev/surf
cd surf
  1. Install dependencies
npm install
  1. Set up environment variables

Create a .env.local file in the root directory based on the provided .env.example:

E2B_API_KEY=your_e2b_api_keyOPENAI_API_KEY=your_openai_api_key
  1. Start the development server
npm run dev
  1. Open the application

Navigate to http://localhost:3000 in your browser.

Usage

  1. Start a Sandbox Instance

    • Click the "Start new Sandbox" button to initialize a virtual desktop environment
    • Wait for the sandbox to start (this may take a few seconds)
  2. Send Instructions

    • Type your instructions in the chat input (e.g., "Open Firefox and go to google.com")
    • Press Enter or click the send button
    • You can also select from example prompts if available
  3. Watch AI Actions

    • The AI will process your instructions and perform actions on the virtual desktop
    • You can see the AI's reasoning and actions in the chat interface
    • The virtual desktop will update in real-time as actions are performed
  4. Manage the Sandbox

    • The timer shows the remaining time for your sandbox instance
    • You can stop the sandbox at any time by clicking the "Stop" button
    • The sandbox will automatically extend its time when it's about to expire

Features

  • Virtual Desktop Environment: Runs a Linux-based desktop in a sandbox
  • AI-Powered Interaction: Uses OpenAI's API to understand and execute user instructions
  • Real-Time Streaming: Shows AI actions and responses as they happen
  • Chat Interface: Provides a conversational interface for interacting with the AI
  • Example Prompts: Offers pre-defined instructions to help users get started
  • Dark/Light Mode: Supports both dark and light themes

Technical Details

Dependencies

The application uses several key dependencies:

  • Next.js: React framework for the frontend
  • @e2b/desktop: SDK for creating and managing desktop sandbox environments
  • OpenAI: SDK for interacting with OpenAI's API
  • Tailwind CSS: Utility-first CSS framework for styling
  • Framer Motion: Library for animations

See package.json for a complete list of dependencies.

API Endpoints

  • /api/chat: Handles chat messages and streams AI responses and actions

Server Actions

  • createSandbox: Creates a new sandbox instance
  • increaseTimeout: Extends the sandbox timeout
  • stopSandboxAction: Stops a running sandbox instance

Troubleshooting

  • Sandbox not starting: Verify your E2B API key is correct in .env.local
  • AI not responding: Check that your OpenAI API key is valid and has access to the required models
  • Actions not working: Ensure the sandbox is running and the AI has proper instructions

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Support

If you encounter any issues or have questions:

About

Surf is a computer use AI agent powered by OpenAI that interacts with a E2B's virtual desktop environment through natural language instructions

Topics

Resources

Stars

855 stars

Watchers

6 watching

Forks

Used by

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
GitHub - e2b-dev/surf: Surf is a computer use AI agent powered by OpenAI that interacts with a E2B's virtual desktop environment through natural language instructions · GitHub
Skip to content

Repository files navigation

E2B Surf Preview LightE2B Surf Preview Dark

🏄 Surf - OpenAI's Computer Use Agent + E2B Desktop

A Next.js application that allows AI to interact with a virtual desktop environment. This project integrates E2B's desktop sandbox with OpenAI's API to create an AI agent that can perform tasks on a virtual computer through natural language instructions.

E2B is an open source isolated virtual computer in the cloud made for AI use cases.

Overview

The Computer Use App provides a web interface where users can:

  1. Start a virtual desktop sandbox environment
  2. Send natural language instructions to an AI agent
  3. Watch as the AI agent performs actions on the virtual desktop
  4. Interact with the AI through a chat interface

The application uses Server-Sent Events (SSE) to stream AI responses and actions in real-time, providing a seamless experience.

How It Works

Architecture

The application consists of several key components:

  1. Frontend UI (Next.js): Provides the user interface with a virtual desktop view and chat interface
  2. E2B Desktop Sandbox: Creates and manages virtual desktop environments
  3. OpenAI Computer Use: Processes user instructions and generates actions for the AI agent
  4. Streaming API: Handles real-time communication between the frontend and backend

Core Flow

  1. User starts a new sandbox instance
  2. E2B creates a virtual desktop and provides a URL for streaming
  3. User sends instructions via the chat interface
  4. Backend processes the instructions using OpenAI's API
  5. AI generates actions (clicks, typing, etc.) to perform on the virtual desktop
  6. Actions are executed on the sandbox and streamed back to the frontend
  7. The process repeats as the user continues to provide instructions

Prerequisites

Before starting, you'll need:

  1. Node.js (version specified in package.json)
  2. npm (comes with Node.js)
  3. An E2B API key
  4. An OpenAI API key

Setup Instructions

  1. Clone the repository
git clone https://github.com/e2b-dev/surf
cd surf
  1. Install dependencies
npm install
  1. Set up environment variables

Create a .env.local file in the root directory based on the provided .env.example:

E2B_API_KEY=your_e2b_api_keyOPENAI_API_KEY=your_openai_api_key
  1. Start the development server
npm run dev
  1. Open the application

Navigate to http://localhost:3000 in your browser.

Usage

  1. Start a Sandbox Instance

    • Click the "Start new Sandbox" button to initialize a virtual desktop environment
    • Wait for the sandbox to start (this may take a few seconds)
  2. Send Instructions

    • Type your instructions in the chat input (e.g., "Open Firefox and go to google.com")
    • Press Enter or click the send button
    • You can also select from example prompts if available
  3. Watch AI Actions

    • The AI will process your instructions and perform actions on the virtual desktop
    • You can see the AI's reasoning and actions in the chat interface
    • The virtual desktop will update in real-time as actions are performed
  4. Manage the Sandbox

    • The timer shows the remaining time for your sandbox instance
    • You can stop the sandbox at any time by clicking the "Stop" button
    • The sandbox will automatically extend its time when it's about to expire

Features

  • Virtual Desktop Environment: Runs a Linux-based desktop in a sandbox
  • AI-Powered Interaction: Uses OpenAI's API to understand and execute user instructions
  • Real-Time Streaming: Shows AI actions and responses as they happen
  • Chat Interface: Provides a conversational interface for interacting with the AI
  • Example Prompts: Offers pre-defined instructions to help users get started
  • Dark/Light Mode: Supports both dark and light themes

Technical Details

Dependencies

The application uses several key dependencies:

  • Next.js: React framework for the frontend
  • @e2b/desktop: SDK for creating and managing desktop sandbox environments
  • OpenAI: SDK for interacting with OpenAI's API
  • Tailwind CSS: Utility-first CSS framework for styling
  • Framer Motion: Library for animations

See package.json for a complete list of dependencies.

API Endpoints

  • /api/chat: Handles chat messages and streams AI responses and actions

Server Actions

  • createSandbox: Creates a new sandbox instance
  • increaseTimeout: Extends the sandbox timeout
  • stopSandboxAction: Stops a running sandbox instance

Troubleshooting

  • Sandbox not starting: Verify your E2B API key is correct in .env.local
  • AI not responding: Check that your OpenAI API key is valid and has access to the required models
  • Actions not working: Ensure the sandbox is running and the AI has proper instructions

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Support

If you encounter any issues or have questions:

About

Surf is a computer use AI agent powered by OpenAI that interacts with a E2B's virtual desktop environment through natural language instructions

Topics

Resources

Stars

855 stars

Watchers

6 watching

Forks

Used by

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' GitHub - e2b-dev/surf: Surf is a computer use AI agent powered by OpenAI that interacts with a E2B's virtual desktop environment through natural language instructions · GitHub
Skip to content

Repository files navigation

E2B Surf Preview LightE2B Surf Preview Dark

🏄 Surf - OpenAI's Computer Use Agent + E2B Desktop

A Next.js application that allows AI to interact with a virtual desktop environment. This project integrates E2B's desktop sandbox with OpenAI's API to create an AI agent that can perform tasks on a virtual computer through natural language instructions.

E2B is an open source isolated virtual computer in the cloud made for AI use cases.

Overview

The Computer Use App provides a web interface where users can:

  1. Start a virtual desktop sandbox environment
  2. Send natural language instructions to an AI agent
  3. Watch as the AI agent performs actions on the virtual desktop
  4. Interact with the AI through a chat interface

The application uses Server-Sent Events (SSE) to stream AI responses and actions in real-time, providing a seamless experience.

How It Works

Architecture

The application consists of several key components:

  1. Frontend UI (Next.js): Provides the user interface with a virtual desktop view and chat interface
  2. E2B Desktop Sandbox: Creates and manages virtual desktop environments
  3. OpenAI Computer Use: Processes user instructions and generates actions for the AI agent
  4. Streaming API: Handles real-time communication between the frontend and backend

Core Flow

  1. User starts a new sandbox instance
  2. E2B creates a virtual desktop and provides a URL for streaming
  3. User sends instructions via the chat interface
  4. Backend processes the instructions using OpenAI's API
  5. AI generates actions (clicks, typing, etc.) to perform on the virtual desktop
  6. Actions are executed on the sandbox and streamed back to the frontend
  7. The process repeats as the user continues to provide instructions

Prerequisites

Before starting, you'll need:

  1. Node.js (version specified in package.json)
  2. npm (comes with Node.js)
  3. An E2B API key
  4. An OpenAI API key

Setup Instructions

  1. Clone the repository
git clone https://github.com/e2b-dev/surf
cd surf
  1. Install dependencies
npm install
  1. Set up environment variables

Create a .env.local file in the root directory based on the provided .env.example:

E2B_API_KEY=your_e2b_api_keyOPENAI_API_KEY=your_openai_api_key
  1. Start the development server
npm run dev
  1. Open the application

Navigate to http://localhost:3000 in your browser.

Usage

  1. Start a Sandbox Instance

    • Click the "Start new Sandbox" button to initialize a virtual desktop environment
    • Wait for the sandbox to start (this may take a few seconds)
  2. Send Instructions

    • Type your instructions in the chat input (e.g., "Open Firefox and go to google.com")
    • Press Enter or click the send button
    • You can also select from example prompts if available
  3. Watch AI Actions

    • The AI will process your instructions and perform actions on the virtual desktop
    • You can see the AI's reasoning and actions in the chat interface
    • The virtual desktop will update in real-time as actions are performed
  4. Manage the Sandbox

    • The timer shows the remaining time for your sandbox instance
    • You can stop the sandbox at any time by clicking the "Stop" button
    • The sandbox will automatically extend its time when it's about to expire

Features

  • Virtual Desktop Environment: Runs a Linux-based desktop in a sandbox
  • AI-Powered Interaction: Uses OpenAI's API to understand and execute user instructions
  • Real-Time Streaming: Shows AI actions and responses as they happen
  • Chat Interface: Provides a conversational interface for interacting with the AI
  • Example Prompts: Offers pre-defined instructions to help users get started
  • Dark/Light Mode: Supports both dark and light themes

Technical Details

Dependencies

The application uses several key dependencies:

  • Next.js: React framework for the frontend
  • @e2b/desktop: SDK for creating and managing desktop sandbox environments
  • OpenAI: SDK for interacting with OpenAI's API
  • Tailwind CSS: Utility-first CSS framework for styling
  • Framer Motion: Library for animations

See package.json for a complete list of dependencies.

API Endpoints

  • /api/chat: Handles chat messages and streams AI responses and actions

Server Actions

  • createSandbox: Creates a new sandbox instance
  • increaseTimeout: Extends the sandbox timeout
  • stopSandboxAction: Stops a running sandbox instance

Troubleshooting

  • Sandbox not starting: Verify your E2B API key is correct in .env.local
  • AI not responding: Check that your OpenAI API key is valid and has access to the required models
  • Actions not working: Ensure the sandbox is running and the AI has proper instructions

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Support

If you encounter any issues or have questions:

About

Surf is a computer use AI agent powered by OpenAI that interacts with a E2B's virtual desktop environment through natural language instructions

Topics

Resources

Stars

855 stars

Watchers

6 watching

Forks

Used by

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' GitHub - e2b-dev/surf: Surf is a computer use AI agent powered by OpenAI that interacts with a E2B's virtual desktop environment through natural language instructions · GitHub
Skip to content

Repository files navigation

E2B Surf Preview LightE2B Surf Preview Dark

🏄 Surf - OpenAI's Computer Use Agent + E2B Desktop

A Next.js application that allows AI to interact with a virtual desktop environment. This project integrates E2B's desktop sandbox with OpenAI's API to create an AI agent that can perform tasks on a virtual computer through natural language instructions.

E2B is an open source isolated virtual computer in the cloud made for AI use cases.

Overview

The Computer Use App provides a web interface where users can:

  1. Start a virtual desktop sandbox environment
  2. Send natural language instructions to an AI agent
  3. Watch as the AI agent performs actions on the virtual desktop
  4. Interact with the AI through a chat interface

The application uses Server-Sent Events (SSE) to stream AI responses and actions in real-time, providing a seamless experience.

How It Works

Architecture

The application consists of several key components:

  1. Frontend UI (Next.js): Provides the user interface with a virtual desktop view and chat interface
  2. E2B Desktop Sandbox: Creates and manages virtual desktop environments
  3. OpenAI Computer Use: Processes user instructions and generates actions for the AI agent
  4. Streaming API: Handles real-time communication between the frontend and backend

Core Flow

  1. User starts a new sandbox instance
  2. E2B creates a virtual desktop and provides a URL for streaming
  3. User sends instructions via the chat interface
  4. Backend processes the instructions using OpenAI's API
  5. AI generates actions (clicks, typing, etc.) to perform on the virtual desktop
  6. Actions are executed on the sandbox and streamed back to the frontend
  7. The process repeats as the user continues to provide instructions

Prerequisites

Before starting, you'll need:

  1. Node.js (version specified in package.json)
  2. npm (comes with Node.js)
  3. An E2B API key
  4. An OpenAI API key

Setup Instructions

  1. Clone the repository
git clone https://github.com/e2b-dev/surf
cd surf
  1. Install dependencies
npm install
  1. Set up environment variables

Create a .env.local file in the root directory based on the provided .env.example:

E2B_API_KEY=your_e2b_api_keyOPENAI_API_KEY=your_openai_api_key
  1. Start the development server
npm run dev
  1. Open the application

Navigate to http://localhost:3000 in your browser.

Usage

  1. Start a Sandbox Instance

    • Click the "Start new Sandbox" button to initialize a virtual desktop environment
    • Wait for the sandbox to start (this may take a few seconds)
  2. Send Instructions

    • Type your instructions in the chat input (e.g., "Open Firefox and go to google.com")
    • Press Enter or click the send button
    • You can also select from example prompts if available
  3. Watch AI Actions

    • The AI will process your instructions and perform actions on the virtual desktop
    • You can see the AI's reasoning and actions in the chat interface
    • The virtual desktop will update in real-time as actions are performed
  4. Manage the Sandbox

    • The timer shows the remaining time for your sandbox instance
    • You can stop the sandbox at any time by clicking the "Stop" button
    • The sandbox will automatically extend its time when it's about to expire

Features

  • Virtual Desktop Environment: Runs a Linux-based desktop in a sandbox
  • AI-Powered Interaction: Uses OpenAI's API to understand and execute user instructions
  • Real-Time Streaming: Shows AI actions and responses as they happen
  • Chat Interface: Provides a conversational interface for interacting with the AI
  • Example Prompts: Offers pre-defined instructions to help users get started
  • Dark/Light Mode: Supports both dark and light themes

Technical Details

Dependencies

The application uses several key dependencies:

  • Next.js: React framework for the frontend
  • @e2b/desktop: SDK for creating and managing desktop sandbox environments
  • OpenAI: SDK for interacting with OpenAI's API
  • Tailwind CSS: Utility-first CSS framework for styling
  • Framer Motion: Library for animations

See package.json for a complete list of dependencies.

API Endpoints

  • /api/chat: Handles chat messages and streams AI responses and actions

Server Actions

  • createSandbox: Creates a new sandbox instance
  • increaseTimeout: Extends the sandbox timeout
  • stopSandboxAction: Stops a running sandbox instance

Troubleshooting

  • Sandbox not starting: Verify your E2B API key is correct in .env.local
  • AI not responding: Check that your OpenAI API key is valid and has access to the required models
  • Actions not working: Ensure the sandbox is running and the AI has proper instructions

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Support

If you encounter any issues or have questions:

About

Surf is a computer use AI agent powered by OpenAI that interacts with a E2B's virtual desktop environment through natural language instructions

Topics

Resources

Stars

855 stars

Watchers

6 watching

Forks

Used by

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' GitHub - e2b-dev/surf: Surf is a computer use AI agent powered by OpenAI that interacts with a E2B's virtual desktop environment through natural language instructions · GitHub
Skip to content

Repository files navigation

E2B Surf Preview LightE2B Surf Preview Dark

🏄 Surf - OpenAI's Computer Use Agent + E2B Desktop

A Next.js application that allows AI to interact with a virtual desktop environment. This project integrates E2B's desktop sandbox with OpenAI's API to create an AI agent that can perform tasks on a virtual computer through natural language instructions.

E2B is an open source isolated virtual computer in the cloud made for AI use cases.

Overview

The Computer Use App provides a web interface where users can:

  1. Start a virtual desktop sandbox environment
  2. Send natural language instructions to an AI agent
  3. Watch as the AI agent performs actions on the virtual desktop
  4. Interact with the AI through a chat interface

The application uses Server-Sent Events (SSE) to stream AI responses and actions in real-time, providing a seamless experience.

How It Works

Architecture

The application consists of several key components:

  1. Frontend UI (Next.js): Provides the user interface with a virtual desktop view and chat interface
  2. E2B Desktop Sandbox: Creates and manages virtual desktop environments
  3. OpenAI Computer Use: Processes user instructions and generates actions for the AI agent
  4. Streaming API: Handles real-time communication between the frontend and backend

Core Flow

  1. User starts a new sandbox instance
  2. E2B creates a virtual desktop and provides a URL for streaming
  3. User sends instructions via the chat interface
  4. Backend processes the instructions using OpenAI's API
  5. AI generates actions (clicks, typing, etc.) to perform on the virtual desktop
  6. Actions are executed on the sandbox and streamed back to the frontend
  7. The process repeats as the user continues to provide instructions

Prerequisites

Before starting, you'll need:

  1. Node.js (version specified in package.json)
  2. npm (comes with Node.js)
  3. An E2B API key
  4. An OpenAI API key

Setup Instructions

  1. Clone the repository
git clone https://github.com/e2b-dev/surf
cd surf
  1. Install dependencies
npm install
  1. Set up environment variables

Create a .env.local file in the root directory based on the provided .env.example:

E2B_API_KEY=your_e2b_api_keyOPENAI_API_KEY=your_openai_api_key
  1. Start the development server
npm run dev
  1. Open the application

Navigate to http://localhost:3000 in your browser.

Usage

  1. Start a Sandbox Instance

    • Click the "Start new Sandbox" button to initialize a virtual desktop environment
    • Wait for the sandbox to start (this may take a few seconds)
  2. Send Instructions

    • Type your instructions in the chat input (e.g., "Open Firefox and go to google.com")
    • Press Enter or click the send button
    • You can also select from example prompts if available
  3. Watch AI Actions

    • The AI will process your instructions and perform actions on the virtual desktop
    • You can see the AI's reasoning and actions in the chat interface
    • The virtual desktop will update in real-time as actions are performed
  4. Manage the Sandbox

    • The timer shows the remaining time for your sandbox instance
    • You can stop the sandbox at any time by clicking the "Stop" button
    • The sandbox will automatically extend its time when it's about to expire

Features

  • Virtual Desktop Environment: Runs a Linux-based desktop in a sandbox
  • AI-Powered Interaction: Uses OpenAI's API to understand and execute user instructions
  • Real-Time Streaming: Shows AI actions and responses as they happen
  • Chat Interface: Provides a conversational interface for interacting with the AI
  • Example Prompts: Offers pre-defined instructions to help users get started
  • Dark/Light Mode: Supports both dark and light themes

Technical Details

Dependencies

The application uses several key dependencies:

  • Next.js: React framework for the frontend
  • @e2b/desktop: SDK for creating and managing desktop sandbox environments
  • OpenAI: SDK for interacting with OpenAI's API
  • Tailwind CSS: Utility-first CSS framework for styling
  • Framer Motion: Library for animations

See package.json for a complete list of dependencies.

API Endpoints

  • /api/chat: Handles chat messages and streams AI responses and actions

Server Actions

  • createSandbox: Creates a new sandbox instance
  • increaseTimeout: Extends the sandbox timeout
  • stopSandboxAction: Stops a running sandbox instance

Troubleshooting

  • Sandbox not starting: Verify your E2B API key is correct in .env.local
  • AI not responding: Check that your OpenAI API key is valid and has access to the required models
  • Actions not working: Ensure the sandbox is running and the AI has proper instructions

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Support

If you encounter any issues or have questions:

About

Surf is a computer use AI agent powered by OpenAI that interacts with a E2B's virtual desktop environment through natural language instructions

Topics

Resources

Stars

855 stars

Watchers

6 watching

Forks

Used by

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' GitHub - e2b-dev/surf: Surf is a computer use AI agent powered by OpenAI that interacts with a E2B's virtual desktop environment through natural language instructions · GitHub
Skip to content

Repository files navigation

E2B Surf Preview LightE2B Surf Preview Dark

🏄 Surf - OpenAI's Computer Use Agent + E2B Desktop

A Next.js application that allows AI to interact with a virtual desktop environment. This project integrates E2B's desktop sandbox with OpenAI's API to create an AI agent that can perform tasks on a virtual computer through natural language instructions.

E2B is an open source isolated virtual computer in the cloud made for AI use cases.

Overview

The Computer Use App provides a web interface where users can:

  1. Start a virtual desktop sandbox environment
  2. Send natural language instructions to an AI agent
  3. Watch as the AI agent performs actions on the virtual desktop
  4. Interact with the AI through a chat interface

The application uses Server-Sent Events (SSE) to stream AI responses and actions in real-time, providing a seamless experience.

How It Works

Architecture

The application consists of several key components:

  1. Frontend UI (Next.js): Provides the user interface with a virtual desktop view and chat interface
  2. E2B Desktop Sandbox: Creates and manages virtual desktop environments
  3. OpenAI Computer Use: Processes user instructions and generates actions for the AI agent
  4. Streaming API: Handles real-time communication between the frontend and backend

Core Flow

  1. User starts a new sandbox instance
  2. E2B creates a virtual desktop and provides a URL for streaming
  3. User sends instructions via the chat interface
  4. Backend processes the instructions using OpenAI's API
  5. AI generates actions (clicks, typing, etc.) to perform on the virtual desktop
  6. Actions are executed on the sandbox and streamed back to the frontend
  7. The process repeats as the user continues to provide instructions

Prerequisites

Before starting, you'll need:

  1. Node.js (version specified in package.json)
  2. npm (comes with Node.js)
  3. An E2B API key
  4. An OpenAI API key

Setup Instructions

  1. Clone the repository
git clone https://github.com/e2b-dev/surf
cd surf
  1. Install dependencies
npm install
  1. Set up environment variables

Create a .env.local file in the root directory based on the provided .env.example:

E2B_API_KEY=your_e2b_api_keyOPENAI_API_KEY=your_openai_api_key
  1. Start the development server
npm run dev
  1. Open the application

Navigate to http://localhost:3000 in your browser.

Usage

  1. Start a Sandbox Instance

    • Click the "Start new Sandbox" button to initialize a virtual desktop environment
    • Wait for the sandbox to start (this may take a few seconds)
  2. Send Instructions

    • Type your instructions in the chat input (e.g., "Open Firefox and go to google.com")
    • Press Enter or click the send button
    • You can also select from example prompts if available
  3. Watch AI Actions

    • The AI will process your instructions and perform actions on the virtual desktop
    • You can see the AI's reasoning and actions in the chat interface
    • The virtual desktop will update in real-time as actions are performed
  4. Manage the Sandbox

    • The timer shows the remaining time for your sandbox instance
    • You can stop the sandbox at any time by clicking the "Stop" button
    • The sandbox will automatically extend its time when it's about to expire

Features

  • Virtual Desktop Environment: Runs a Linux-based desktop in a sandbox
  • AI-Powered Interaction: Uses OpenAI's API to understand and execute user instructions
  • Real-Time Streaming: Shows AI actions and responses as they happen
  • Chat Interface: Provides a conversational interface for interacting with the AI
  • Example Prompts: Offers pre-defined instructions to help users get started
  • Dark/Light Mode: Supports both dark and light themes

Technical Details

Dependencies

The application uses several key dependencies:

  • Next.js: React framework for the frontend
  • @e2b/desktop: SDK for creating and managing desktop sandbox environments
  • OpenAI: SDK for interacting with OpenAI's API
  • Tailwind CSS: Utility-first CSS framework for styling
  • Framer Motion: Library for animations

See package.json for a complete list of dependencies.

API Endpoints

  • /api/chat: Handles chat messages and streams AI responses and actions

Server Actions

  • createSandbox: Creates a new sandbox instance
  • increaseTimeout: Extends the sandbox timeout
  • stopSandboxAction: Stops a running sandbox instance

Troubleshooting

  • Sandbox not starting: Verify your E2B API key is correct in .env.local
  • AI not responding: Check that your OpenAI API key is valid and has access to the required models
  • Actions not working: Ensure the sandbox is running and the AI has proper instructions

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Support

If you encounter any issues or have questions:

About

Surf is a computer use AI agent powered by OpenAI that interacts with a E2B's virtual desktop environment through natural language instructions

Topics

Resources

Stars

855 stars

Watchers

6 watching

Forks

Used by

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' GitHub - e2b-dev/surf: Surf is a computer use AI agent powered by OpenAI that interacts with a E2B's virtual desktop environment through natural language instructions · GitHub
Skip to content

Repository files navigation

E2B Surf Preview LightE2B Surf Preview Dark

🏄 Surf - OpenAI's Computer Use Agent + E2B Desktop

A Next.js application that allows AI to interact with a virtual desktop environment. This project integrates E2B's desktop sandbox with OpenAI's API to create an AI agent that can perform tasks on a virtual computer through natural language instructions.

E2B is an open source isolated virtual computer in the cloud made for AI use cases.

Overview

The Computer Use App provides a web interface where users can:

  1. Start a virtual desktop sandbox environment
  2. Send natural language instructions to an AI agent
  3. Watch as the AI agent performs actions on the virtual desktop
  4. Interact with the AI through a chat interface

The application uses Server-Sent Events (SSE) to stream AI responses and actions in real-time, providing a seamless experience.

How It Works

Architecture

The application consists of several key components:

  1. Frontend UI (Next.js): Provides the user interface with a virtual desktop view and chat interface
  2. E2B Desktop Sandbox: Creates and manages virtual desktop environments
  3. OpenAI Computer Use: Processes user instructions and generates actions for the AI agent
  4. Streaming API: Handles real-time communication between the frontend and backend

Core Flow

  1. User starts a new sandbox instance
  2. E2B creates a virtual desktop and provides a URL for streaming
  3. User sends instructions via the chat interface
  4. Backend processes the instructions using OpenAI's API
  5. AI generates actions (clicks, typing, etc.) to perform on the virtual desktop
  6. Actions are executed on the sandbox and streamed back to the frontend
  7. The process repeats as the user continues to provide instructions

Prerequisites

Before starting, you'll need:

  1. Node.js (version specified in package.json)
  2. npm (comes with Node.js)
  3. An E2B API key
  4. An OpenAI API key

Setup Instructions

  1. Clone the repository
git clone https://github.com/e2b-dev/surf
cd surf
  1. Install dependencies
npm install
  1. Set up environment variables

Create a .env.local file in the root directory based on the provided .env.example:

E2B_API_KEY=your_e2b_api_keyOPENAI_API_KEY=your_openai_api_key
  1. Start the development server
npm run dev
  1. Open the application

Navigate to http://localhost:3000 in your browser.

Usage

  1. Start a Sandbox Instance

    • Click the "Start new Sandbox" button to initialize a virtual desktop environment
    • Wait for the sandbox to start (this may take a few seconds)
  2. Send Instructions

    • Type your instructions in the chat input (e.g., "Open Firefox and go to google.com")
    • Press Enter or click the send button
    • You can also select from example prompts if available
  3. Watch AI Actions

    • The AI will process your instructions and perform actions on the virtual desktop
    • You can see the AI's reasoning and actions in the chat interface
    • The virtual desktop will update in real-time as actions are performed
  4. Manage the Sandbox

    • The timer shows the remaining time for your sandbox instance
    • You can stop the sandbox at any time by clicking the "Stop" button
    • The sandbox will automatically extend its time when it's about to expire

Features

  • Virtual Desktop Environment: Runs a Linux-based desktop in a sandbox
  • AI-Powered Interaction: Uses OpenAI's API to understand and execute user instructions
  • Real-Time Streaming: Shows AI actions and responses as they happen
  • Chat Interface: Provides a conversational interface for interacting with the AI
  • Example Prompts: Offers pre-defined instructions to help users get started
  • Dark/Light Mode: Supports both dark and light themes

Technical Details

Dependencies

The application uses several key dependencies:

  • Next.js: React framework for the frontend
  • @e2b/desktop: SDK for creating and managing desktop sandbox environments
  • OpenAI: SDK for interacting with OpenAI's API
  • Tailwind CSS: Utility-first CSS framework for styling
  • Framer Motion: Library for animations

See package.json for a complete list of dependencies.

API Endpoints

  • /api/chat: Handles chat messages and streams AI responses and actions

Server Actions

  • createSandbox: Creates a new sandbox instance
  • increaseTimeout: Extends the sandbox timeout
  • stopSandboxAction: Stops a running sandbox instance

Troubleshooting

  • Sandbox not starting: Verify your E2B API key is correct in .env.local
  • AI not responding: Check that your OpenAI API key is valid and has access to the required models
  • Actions not working: Ensure the sandbox is running and the AI has proper instructions

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Support

If you encounter any issues or have questions:

About

Surf is a computer use AI agent powered by OpenAI that interacts with a E2B's virtual desktop environment through natural language instructions

Topics

Resources

Stars

855 stars

Watchers

6 watching

Forks

Used by

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); GitHub - e2b-dev/surf: Surf is a computer use AI agent powered by OpenAI that interacts with a E2B's virtual desktop environment through natural language instructions · GitHub
Skip to content

Repository files navigation

E2B Surf Preview LightE2B Surf Preview Dark

🏄 Surf - OpenAI's Computer Use Agent + E2B Desktop

A Next.js application that allows AI to interact with a virtual desktop environment. This project integrates E2B's desktop sandbox with OpenAI's API to create an AI agent that can perform tasks on a virtual computer through natural language instructions.

E2B is an open source isolated virtual computer in the cloud made for AI use cases.

Overview

The Computer Use App provides a web interface where users can:

  1. Start a virtual desktop sandbox environment
  2. Send natural language instructions to an AI agent
  3. Watch as the AI agent performs actions on the virtual desktop
  4. Interact with the AI through a chat interface

The application uses Server-Sent Events (SSE) to stream AI responses and actions in real-time, providing a seamless experience.

How It Works

Architecture

The application consists of several key components:

  1. Frontend UI (Next.js): Provides the user interface with a virtual desktop view and chat interface
  2. E2B Desktop Sandbox: Creates and manages virtual desktop environments
  3. OpenAI Computer Use: Processes user instructions and generates actions for the AI agent
  4. Streaming API: Handles real-time communication between the frontend and backend

Core Flow

  1. User starts a new sandbox instance
  2. E2B creates a virtual desktop and provides a URL for streaming
  3. User sends instructions via the chat interface
  4. Backend processes the instructions using OpenAI's API
  5. AI generates actions (clicks, typing, etc.) to perform on the virtual desktop
  6. Actions are executed on the sandbox and streamed back to the frontend
  7. The process repeats as the user continues to provide instructions

Prerequisites

Before starting, you'll need:

  1. Node.js (version specified in package.json)
  2. npm (comes with Node.js)
  3. An E2B API key
  4. An OpenAI API key

Setup Instructions

  1. Clone the repository
git clone https://github.com/e2b-dev/surf
cd surf
  1. Install dependencies
npm install
  1. Set up environment variables

Create a .env.local file in the root directory based on the provided .env.example:

E2B_API_KEY=your_e2b_api_keyOPENAI_API_KEY=your_openai_api_key
  1. Start the development server
npm run dev
  1. Open the application

Navigate to http://localhost:3000 in your browser.

Usage

  1. Start a Sandbox Instance

    • Click the "Start new Sandbox" button to initialize a virtual desktop environment
    • Wait for the sandbox to start (this may take a few seconds)
  2. Send Instructions

    • Type your instructions in the chat input (e.g., "Open Firefox and go to google.com")
    • Press Enter or click the send button
    • You can also select from example prompts if available
  3. Watch AI Actions

    • The AI will process your instructions and perform actions on the virtual desktop
    • You can see the AI's reasoning and actions in the chat interface
    • The virtual desktop will update in real-time as actions are performed
  4. Manage the Sandbox

    • The timer shows the remaining time for your sandbox instance
    • You can stop the sandbox at any time by clicking the "Stop" button
    • The sandbox will automatically extend its time when it's about to expire

Features

  • Virtual Desktop Environment: Runs a Linux-based desktop in a sandbox
  • AI-Powered Interaction: Uses OpenAI's API to understand and execute user instructions
  • Real-Time Streaming: Shows AI actions and responses as they happen
  • Chat Interface: Provides a conversational interface for interacting with the AI
  • Example Prompts: Offers pre-defined instructions to help users get started
  • Dark/Light Mode: Supports both dark and light themes

Technical Details

Dependencies

The application uses several key dependencies:

  • Next.js: React framework for the frontend
  • @e2b/desktop: SDK for creating and managing desktop sandbox environments
  • OpenAI: SDK for interacting with OpenAI's API
  • Tailwind CSS: Utility-first CSS framework for styling
  • Framer Motion: Library for animations

See package.json for a complete list of dependencies.

API Endpoints

  • /api/chat: Handles chat messages and streams AI responses and actions

Server Actions

  • createSandbox: Creates a new sandbox instance
  • increaseTimeout: Extends the sandbox timeout
  • stopSandboxAction: Stops a running sandbox instance

Troubleshooting

  • Sandbox not starting: Verify your E2B API key is correct in .env.local
  • AI not responding: Check that your OpenAI API key is valid and has access to the required models
  • Actions not working: Ensure the sandbox is running and the AI has proper instructions

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Support

If you encounter any issues or have questions:

About

Surf is a computer use AI agent powered by OpenAI that interacts with a E2B's virtual desktop environment through natural language instructions

Topics

Resources

Stars

855 stars

Watchers

6 watching

Forks

Used by

Contributors

Languages