BranchesTags

Repository files navigation

RATT Agent Library

Welcome to the **RATT Agent Library **! 🎧⚡ This tiny TypeScript client streams microphone (or external) audio over WebSocket, handles reconnection and heartbeats, and emits typed events for live transcription and UI state. It works in browsers (with AudioWorklets) and in Node (via external PCM).

🚀 Features

  • Low-latency audio streaming (16 kHz, 16-bit PCM) over WebSocket
  • Browser + Node: use the mic in browsers or push external PCM in any runtime
  • Typed events: ready, mic state, amplitude (VAD energy), transcription, socket messages, errors
  • Prebuffering & gating: capture early, start sending when the server says start_audio
  • Resilient WS: single-flight connect, StrictMode-safe reuse, heartbeats & auto-reconnect
  • Safe defaults: echo cancellation, AGC, noise suppression (browser)
  • Tiny API: a single class AssistantClient you can drop into your app

📦 Installation

npm i ratt-lib
# or
pnpm add ratt-lib

✨ Quick Start (Browser)

import{AssistantClient,AssistantOptions,AssistantEvent}from"ratt-lib";// Simple mutable ref for requestId (works well with React useRef) and this id will be unique for each reqconstrequestId={current: ""};constchatSessionId="test";//Unique for whole sessionconstclientId="test";//Unique for whole session// ✅ Build the REQUIRED rattAgentDetails objectconstrattAgentDetails={conciergeId: chatbotData?.id,// Your chatbot IdconciergeName: chatbotData?.name??chatbotData?.assistantDetails?.name,organizationId: chatbotData?.organization,organizationName: chatbotData?.organizationName,requestId: requestId.current,// will be overridden with a fresh value when session startsagentSettings: {voiceAgentMongoId: chatbotData?.agents?.filter((a: any)=>a.title==="RATTAgent")?.[0]?._id,},username: user?.provider?.name,// user nameuseremailId: user?.email,// user emailchatSessionId: chatSessionId,rlefVoiceTaskId: chatbotData?.audioTranscription?.modelId||CREATE_AUDIO_RELF_VOICE_TASK_ID_DEFAULT_VALUE,// rlef model idassistant_type: chatbotData?.assistant_type,// your assistant typeisAudioRequest: true,//always trueclient_id: clientId,userId: encodeParam(USER_ID_DEFAULT_VALUE),// your user id// keep below three as it is if test instant learning is not there , otherwise please send these values as well as per the requirements.testQuestion: "",testAnswer: "",testVariants: JSON.stringify({Edit: [],Add: [],Delete: []}),}asconst;constclient=newAssistantClient({url: "wss://dev-egpt.techo.camp/audioStreamingWebsocket?clientId=${clientId}&sessionId=${chatSessionId}",// Required: your WS endpoint
requestId,// Required: { current: string }
rattAgentDetails,// ✅ pass the full required objectonSend: ()=>console.log("Transcript submitted"),showToast: (type,title,msg)=>console.log(type,title,msg),// optional tunables:pingIntervalMs: 5000,maxMissedPongs: 2,workletBasePath: "/",// where recorder/vad worklets are served from});// Listen to eventsclient.on(AssistantEvent.READY,()=>{console.log("WS ready:",client.wsReady);});client.on(AssistantEvent.MIC_CONNECTING,({ detail })=>{console.log("Mic connecting:",detail.connecting);});client.on(AssistantEvent.MIC_OPEN,({ detail })=>{console.log("Mic open:",detail.open);});client.on(AssistantEvent.AMPLITUDE,({ detail })=>{// 0..~1 energy (not dB) to drive a mic meter UIconsole.log("Amplitude:",detail.value);});client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{// progressive or final textconsole.log("Transcript:",detail.text);});client.on(AssistantEvent.ERROR,({ detail })=>{console.error("Assistant error:",detail.error);});// Start a session (ask server to begin, then it will reply with start_audio)document.querySelector("#start")!.addEventListener("click",()=>{client.startSession();// toggles ON (opens mic flow) basically handleMicClick function});// Stop (ask server to disconnect and teardown locally)document.querySelector("#stop")!.addEventListener("click",()=>{client.stopAudio();// toggles OFF});

What happens under the hood?

  1. startSession() sends your rattAgentDetails + a new requestId.

  2. When your server responds with { "start_audio": true }, the client:

    • marks the mic as open,
    • starts converting to PCM16,
    • and begins streaming.

  3. As your server streams partial ASR, send either:

    • {"streaming_data":{"previous_transcription":"...", "new_transcription":"..."}} (chunked delta) or
    • {"transcription":"final text"} (full updates)

  4. End with {"stop_audio": true} and/or {"disconnect": true} when you’re done.

🧩 React one-liner (optional)

useEffect(()=>{constunsub=client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{setText(detail.text);});returnunsub;// cleanly removes listener},[]);

🖥️ Node / External Audio (no browser mic)

If you're not in a browser (or you have your own capture pipeline), set externalAudio: true and push PCM yourself.

import{AssistantClient}from"ratt-lib";importfsfrom"node:fs";constrequestId={current: ""};constclient=newAssistantClient({url: "wss://dev-egpt.techo.camp/audioStreamingWebsocket?clientId=${clientId}&sessionId=${chatSessionId}",
requestId,externalAudio: true,// ⬅️ no AudioContext, no micexternalAmplitudeRms: true,// optional: compute amplitude from PCM});// Start session -> wait for server {"start_audio": true}awaitclient.connect();awaitclient.startSession();// Now stream your PCM16 mono 16kHz dataconstpcm=fs.readFileSync("./audio.raw");// Int16 little-endian mono 16kHzclient.pushPCM16(newInt16Array(pcm.buffer,pcm.byteOffset,pcm.byteLength/2));// ...push more chunks as they arrive...// When done: or it will be auto stop by server when it detects some silence is thereawaitclient.stopAudio();

PCM format: 16 kHz, 16-bit, mono, little-endian. If you have Float32Array [-1..1], call pushFloat32() instead.

🔔 Events

All events are emitted as standard CustomEvents and strongly typed via AssistantEvents type.

  • READY — WebSocket is ready (connected & open)
  • MIC_CONNECTING — { connecting: boolean } while we prep/prompt for mic
  • MIC_OPEN — { open: boolean } mic flow is active/inactive
  • AMPLITUDE — { value: number } live energy (for a mic meter)
  • TRANSCRIPTION — { text: string, delta?: string } progressive or final
  • SOCKET_MESSAGE — { raw: MessageEvent, parsed?: any } every incoming WS message
  • ERROR — { error: unknown } any operational error
constoff=client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{console.log(detail.text);});off();// unsubscribe

⚙️ Options

typeAssistantOptions={url: string;// WS endpointrequestId: {current: string};// mutable ref; client writes new ID per sessionrattAgentDetails?: Record<string,any>;onSend?: ()=>void;// called when server requests "disconnect"showToast?: (type: "error"|"info"|"success",title: string,msg: string)=>void;// Connection / heartbeatpingIntervalMs?: number;// default 5000maxMissedPongs?: number;// default 2// Audio (browser)workletBasePath?: string;// default "/"mediaStreamProvider?: ()=>Promise<MediaStream>;// default: getUserMedia (1ch, 16k, AGC/NS/EC enabled)audioContextFactory?: ()=>AudioContext|null;// default: new AudioContext() in browser, null in NodeworkletLoader?: (base: string)=>Promise<AudioContext|null>;// default: ensureAudioContextAndWorklets// External audio (Node or custom capture)externalAudio?: boolean;// default: false in browser, true in NodeexternalAmplitudeRms?: boolean;// default: truepcmChunkSize?: number;// default: TARGET_SAMPLES (typically 16000)};

🧪 Common Methods

awaitclient.connect();// single-flight; reuses active WS if presentawaitclient.startSession();// begin mic flow or external audio sessionawaitclient.stopAudio();// stop current session (sends {disconnect:true})client.disconnect();// local teardown (no forced WS close)client.teardown();// local teardown helpersclient.closeSocket();// forcibly close WS and detach handlers// Mic helpersawaitclient.beginPrebuffering();// start capturing locally; don't send yetclient.stopPrebuffering();// stop & clear buffered audioawaitclient.startMic();// explicitly start mic captureclient.stopMic();// explicitly stop mic (and send {disconnect:true})// External audioclient.pushPCM16(int16ArrayOrBuffer);client.pushFloat32(float32Array);// State gettersclient.wsReady;// booleanclient.micOpen;// booleanclient.micConnecting;// booleanclient.amplitude;// number (0..~1)client.transcription;// latest accumulated text

🧯 Troubleshooting

  • No audio sent Ensure your server replies with {"start_audio": true}. The client buffers until the gate opens.
  • Mic blocked Browser will throw NotAllowedError. The client emits ERROR and calls showToast(...).
  • Noisy audio / echo The default constraints enable echo cancellation, AGC, and noise suppression. Override mediaStreamProvider if needed.
  • Multiple connects in React StrictMode This client reuses a global ACTIVE_WS and a single CONNECT_PROMISE—you’re safe.
  • Heartbeat timeouts Increase pingIntervalMs or maxMissedPongs if your WS hops are choppy.

🔐 Permissions & Security

  • Browsers require a user gesture to start the microphone.
  • Only minimal audio data is sent; handle it securely on your server. Use wss:// in production.

👥 Community & Support

Questions, bugs, or ideas? Open an issue in your repo or ping your team chat. Happy streaming! 🎙️💬

About

Ratt agent connection from UI

Resources

Readme
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Used by

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
BranchesTags
Repository files navigation
RATT Agent Library
Welcome to the **RATT Agent Library **! 🎧⚡
This tiny TypeScript client streams microphone (or external) audio over WebSocket, handles reconnection and heartbeats, and emits typed events for live transcription and UI state. It works in browsers (with AudioWorklets) and in Node (via external PCM).
🚀 Features
  • Low-latency audio streaming (16 kHz, 16-bit PCM) over WebSocket
  • Browser + Node: use the mic in browsers or push external PCM in any runtime
  • Typed events: ready, mic state, amplitude (VAD energy), transcription, socket messages, errors
  • Prebuffering & gating: capture early, start sending when the server says start_audio
  • Resilient WS: single-flight connect, StrictMode-safe reuse, heartbeats & auto-reconnect
  • Safe defaults: echo cancellation, AGC, noise suppression (browser)
  • Tiny API: a single class AssistantClient you can drop into your app
📦 Installation
npm i ratt-lib
# or
pnpm add ratt-lib
✨ Quick Start (Browser)
import{AssistantClient,AssistantOptions,AssistantEvent}from"ratt-lib";// Simple mutable ref for requestId (works well with React useRef) and this id will be unique for each reqconstrequestId={current: ""};constchatSessionId="test";//Unique for whole sessionconstclientId="test";//Unique for whole session// ✅ Build the REQUIRED rattAgentDetails objectconstrattAgentDetails={conciergeId: chatbotData?.id,// Your chatbot IdconciergeName: chatbotData?.name??chatbotData?.assistantDetails?.name,organizationId: chatbotData?.organization,organizationName: chatbotData?.organizationName,requestId: requestId.current,// will be overridden with a fresh value when session startsagentSettings: {voiceAgentMongoId: chatbotData?.agents?.filter((a: any)=>a.title==="RATTAgent")?.[0]?._id,},username: user?.provider?.name,// user nameuseremailId: user?.email,// user emailchatSessionId: chatSessionId,rlefVoiceTaskId: chatbotData?.audioTranscription?.modelId||CREATE_AUDIO_RELF_VOICE_TASK_ID_DEFAULT_VALUE,// rlef model idassistant_type: chatbotData?.assistant_type,// your assistant typeisAudioRequest: true,//always trueclient_id: clientId,userId: encodeParam(USER_ID_DEFAULT_VALUE),// your user id// keep below three as it is if test instant learning is not there , otherwise please send these values as well as per the requirements.testQuestion: "",testAnswer: "",testVariants: JSON.stringify({Edit: [],Add: [],Delete: []}),}asconst;constclient=newAssistantClient({url: "wss://dev-egpt.techo.camp/audioStreamingWebsocket?clientId=${clientId}&sessionId=${chatSessionId}",// Required: your WS endpoint
requestId,// Required: { current: string }
rattAgentDetails,// ✅ pass the full required objectonSend: ()=>console.log("Transcript submitted"),showToast: (type,title,msg)=>console.log(type,title,msg),// optional tunables:pingIntervalMs: 5000,maxMissedPongs: 2,workletBasePath: "/",// where recorder/vad worklets are served from});// Listen to eventsclient.on(AssistantEvent.READY,()=>{console.log("WS ready:",client.wsReady);});client.on(AssistantEvent.MIC_CONNECTING,({ detail })=>{console.log("Mic connecting:",detail.connecting);});client.on(AssistantEvent.MIC_OPEN,({ detail })=>{console.log("Mic open:",detail.open);});client.on(AssistantEvent.AMPLITUDE,({ detail })=>{// 0..~1 energy (not dB) to drive a mic meter UIconsole.log("Amplitude:",detail.value);});client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{// progressive or final textconsole.log("Transcript:",detail.text);});client.on(AssistantEvent.ERROR,({ detail })=>{console.error("Assistant error:",detail.error);});// Start a session (ask server to begin, then it will reply with start_audio)document.querySelector("#start")!.addEventListener("click",()=>{client.startSession();// toggles ON (opens mic flow) basically handleMicClick function});// Stop (ask server to disconnect and teardown locally)document.querySelector("#stop")!.addEventListener("click",()=>{client.stopAudio();// toggles OFF});
What happens under the hood?
  1. startSession() sends your rattAgentDetails + a new requestId.
  2. When your server responds with { "start_audio": true }, the client:
    • marks the mic as open,
    • starts converting to PCM16,
    • and begins streaming.
  3. As your server streams partial ASR, send either:
    • {"streaming_data":{"previous_transcription":"...", "new_transcription":"..."}} (chunked delta)
or
    • {"transcription":"final text"} (full updates)
  4. End with {"stop_audio": true} and/or {"disconnect": true} when you’re done.
🧩 React one-liner (optional)
useEffect(()=>{constunsub=client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{setText(detail.text);});returnunsub;// cleanly removes listener},[]);
🖥️ Node / External Audio (no browser mic)
If you're not in a browser (or you have your own capture pipeline), set externalAudio: true and push PCM yourself.
import{AssistantClient}from"ratt-lib";importfsfrom"node:fs";constrequestId={current: ""};constclient=newAssistantClient({url: "wss://dev-egpt.techo.camp/audioStreamingWebsocket?clientId=${clientId}&sessionId=${chatSessionId}",
requestId,externalAudio: true,// ⬅️ no AudioContext, no micexternalAmplitudeRms: true,// optional: compute amplitude from PCM});// Start session -> wait for server {"start_audio": true}awaitclient.connect();awaitclient.startSession();// Now stream your PCM16 mono 16kHz dataconstpcm=fs.readFileSync("./audio.raw");// Int16 little-endian mono 16kHzclient.pushPCM16(newInt16Array(pcm.buffer,pcm.byteOffset,pcm.byteLength/2));// ...push more chunks as they arrive...// When done: or it will be auto stop by server when it detects some silence is thereawaitclient.stopAudio();
PCM format: 16 kHz, 16-bit, mono, little-endian.
If you have Float32Array [-1..1], call pushFloat32() instead.
🔔 Events
All events are emitted as standard CustomEvents and strongly typed via AssistantEvents type.
  • READY — WebSocket is ready (connected & open)
  • MIC_CONNECTING — { connecting: boolean } while we prep/prompt for mic
  • MIC_OPEN — { open: boolean } mic flow is active/inactive
  • AMPLITUDE — { value: number } live energy (for a mic meter)
  • TRANSCRIPTION — { text: string, delta?: string } progressive or final
  • SOCKET_MESSAGE — { raw: MessageEvent, parsed?: any } every incoming WS message
  • ERROR — { error: unknown } any operational error
constoff=client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{console.log(detail.text);});off();// unsubscribe
⚙️ Options
typeAssistantOptions={url: string;// WS endpointrequestId: {current: string};// mutable ref; client writes new ID per sessionrattAgentDetails?: Record<string,any>;onSend?: ()=>void;// called when server requests "disconnect"showToast?: (type: "error"|"info"|"success",title: string,msg: string)=>void;// Connection / heartbeatpingIntervalMs?: number;// default 5000maxMissedPongs?: number;// default 2// Audio (browser)workletBasePath?: string;// default "/"mediaStreamProvider?: ()=>Promise<MediaStream>;// default: getUserMedia (1ch, 16k, AGC/NS/EC enabled)audioContextFactory?: ()=>AudioContext|null;// default: new AudioContext() in browser, null in NodeworkletLoader?: (base: string)=>Promise<AudioContext|null>;// default: ensureAudioContextAndWorklets// External audio (Node or custom capture)externalAudio?: boolean;// default: false in browser, true in NodeexternalAmplitudeRms?: boolean;// default: truepcmChunkSize?: number;// default: TARGET_SAMPLES (typically 16000)};
🧪 Common Methods
awaitclient.connect();// single-flight; reuses active WS if presentawaitclient.startSession();// begin mic flow or external audio sessionawaitclient.stopAudio();// stop current session (sends {disconnect:true})client.disconnect();// local teardown (no forced WS close)client.teardown();// local teardown helpersclient.closeSocket();// forcibly close WS and detach handlers// Mic helpersawaitclient.beginPrebuffering();// start capturing locally; don't send yetclient.stopPrebuffering();// stop & clear buffered audioawaitclient.startMic();// explicitly start mic captureclient.stopMic();// explicitly stop mic (and send {disconnect:true})// External audioclient.pushPCM16(int16ArrayOrBuffer);client.pushFloat32(float32Array);// State gettersclient.wsReady;// booleanclient.micOpen;// booleanclient.micConnecting;// booleanclient.amplitude;// number (0..~1)client.transcription;// latest accumulated text
🧯 Troubleshooting
  • No audio sent
Ensure your server replies with {"start_audio": true}. The client buffers until the gate opens.
  • Mic blocked
Browser will throw NotAllowedError. The client emits ERROR and calls showToast(...).
  • Noisy audio / echo
The default constraints enable echo cancellation, AGC, and noise suppression. Override mediaStreamProvider if needed.
  • Multiple connects in React StrictMode
This client reuses a global ACTIVE_WS and a single CONNECT_PROMISE—you’re safe.
  • Heartbeat timeouts
Increase pingIntervalMs or maxMissedPongs if your WS hops are choppy.
🔐 Permissions & Security
  • Browsers require a user gesture to start the microphone.
  • Only minimal audio data is sent; handle it securely on your server. Use wss:// in production.
👥 Community & Support
Questions, bugs, or ideas? Open an issue in your repo or ping your team chat. Happy streaming! 🎙️💬
About
Ratt agent connection from UI 
Resources
Readme
Activity
Custom properties
Stars
0 stars
Watchers
0 watching
Forks
0 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
RATT Agent Library
Welcome to the **RATT Agent Library **! 🎧⚡
This tiny TypeScript client streams microphone (or external) audio over WebSocket, handles reconnection and heartbeats, and emits typed events for live transcription and UI state. It works in browsers (with AudioWorklets) and in Node (via external PCM).
🚀 Features
  • Low-latency audio streaming (16 kHz, 16-bit PCM) over WebSocket
  • Browser + Node: use the mic in browsers or push external PCM in any runtime
  • Typed events: ready, mic state, amplitude (VAD energy), transcription, socket messages, errors
  • Prebuffering & gating: capture early, start sending when the server says start_audio
  • Resilient WS: single-flight connect, StrictMode-safe reuse, heartbeats & auto-reconnect
  • Safe defaults: echo cancellation, AGC, noise suppression (browser)
  • Tiny API: a single class AssistantClient you can drop into your app
📦 Installation
npm i ratt-lib
# or
pnpm add ratt-lib
✨ Quick Start (Browser)
import{AssistantClient,AssistantOptions,AssistantEvent}from"ratt-lib";// Simple mutable ref for requestId (works well with React useRef) and this id will be unique for each reqconstrequestId={current: ""};constchatSessionId="test";//Unique for whole sessionconstclientId="test";//Unique for whole session// ✅ Build the REQUIRED rattAgentDetails objectconstrattAgentDetails={conciergeId: chatbotData?.id,// Your chatbot IdconciergeName: chatbotData?.name??chatbotData?.assistantDetails?.name,organizationId: chatbotData?.organization,organizationName: chatbotData?.organizationName,requestId: requestId.current,// will be overridden with a fresh value when session startsagentSettings: {voiceAgentMongoId: chatbotData?.agents?.filter((a: any)=>a.title==="RATTAgent")?.[0]?._id,},username: user?.provider?.name,// user nameuseremailId: user?.email,// user emailchatSessionId: chatSessionId,rlefVoiceTaskId: chatbotData?.audioTranscription?.modelId||CREATE_AUDIO_RELF_VOICE_TASK_ID_DEFAULT_VALUE,// rlef model idassistant_type: chatbotData?.assistant_type,// your assistant typeisAudioRequest: true,//always trueclient_id: clientId,userId: encodeParam(USER_ID_DEFAULT_VALUE),// your user id// keep below three as it is if test instant learning is not there , otherwise please send these values as well as per the requirements.testQuestion: "",testAnswer: "",testVariants: JSON.stringify({Edit: [],Add: [],Delete: []}),}asconst;constclient=newAssistantClient({url: "wss://dev-egpt.techo.camp/audioStreamingWebsocket?clientId=${clientId}&sessionId=${chatSessionId}",// Required: your WS endpoint
requestId,// Required: { current: string }
rattAgentDetails,// ✅ pass the full required objectonSend: ()=>console.log("Transcript submitted"),showToast: (type,title,msg)=>console.log(type,title,msg),// optional tunables:pingIntervalMs: 5000,maxMissedPongs: 2,workletBasePath: "/",// where recorder/vad worklets are served from});// Listen to eventsclient.on(AssistantEvent.READY,()=>{console.log("WS ready:",client.wsReady);});client.on(AssistantEvent.MIC_CONNECTING,({ detail })=>{console.log("Mic connecting:",detail.connecting);});client.on(AssistantEvent.MIC_OPEN,({ detail })=>{console.log("Mic open:",detail.open);});client.on(AssistantEvent.AMPLITUDE,({ detail })=>{// 0..~1 energy (not dB) to drive a mic meter UIconsole.log("Amplitude:",detail.value);});client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{// progressive or final textconsole.log("Transcript:",detail.text);});client.on(AssistantEvent.ERROR,({ detail })=>{console.error("Assistant error:",detail.error);});// Start a session (ask server to begin, then it will reply with start_audio)document.querySelector("#start")!.addEventListener("click",()=>{client.startSession();// toggles ON (opens mic flow) basically handleMicClick function});// Stop (ask server to disconnect and teardown locally)document.querySelector("#stop")!.addEventListener("click",()=>{client.stopAudio();// toggles OFF});
What happens under the hood?
  1. startSession() sends your rattAgentDetails + a new requestId.
  2. When your server responds with { "start_audio": true }, the client:
    • marks the mic as open,
    • starts converting to PCM16,
    • and begins streaming.
  3. As your server streams partial ASR, send either:
    • {"streaming_data":{"previous_transcription":"...", "new_transcription":"..."}} (chunked delta)
or
    • {"transcription":"final text"} (full updates)
  4. End with {"stop_audio": true} and/or {"disconnect": true} when you’re done.
🧩 React one-liner (optional)
useEffect(()=>{constunsub=client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{setText(detail.text);});returnunsub;// cleanly removes listener},[]);
🖥️ Node / External Audio (no browser mic)
If you're not in a browser (or you have your own capture pipeline), set externalAudio: true and push PCM yourself.
import{AssistantClient}from"ratt-lib";importfsfrom"node:fs";constrequestId={current: ""};constclient=newAssistantClient({url: "wss://dev-egpt.techo.camp/audioStreamingWebsocket?clientId=${clientId}&sessionId=${chatSessionId}",
requestId,externalAudio: true,// ⬅️ no AudioContext, no micexternalAmplitudeRms: true,// optional: compute amplitude from PCM});// Start session -> wait for server {"start_audio": true}awaitclient.connect();awaitclient.startSession();// Now stream your PCM16 mono 16kHz dataconstpcm=fs.readFileSync("./audio.raw");// Int16 little-endian mono 16kHzclient.pushPCM16(newInt16Array(pcm.buffer,pcm.byteOffset,pcm.byteLength/2));// ...push more chunks as they arrive...// When done: or it will be auto stop by server when it detects some silence is thereawaitclient.stopAudio();
PCM format: 16 kHz, 16-bit, mono, little-endian.
If you have Float32Array [-1..1], call pushFloat32() instead.
🔔 Events
All events are emitted as standard CustomEvents and strongly typed via AssistantEvents type.
  • READY — WebSocket is ready (connected & open)
  • MIC_CONNECTING — { connecting: boolean } while we prep/prompt for mic
  • MIC_OPEN — { open: boolean } mic flow is active/inactive
  • AMPLITUDE — { value: number } live energy (for a mic meter)
  • TRANSCRIPTION — { text: string, delta?: string } progressive or final
  • SOCKET_MESSAGE — { raw: MessageEvent, parsed?: any } every incoming WS message
  • ERROR — { error: unknown } any operational error
constoff=client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{console.log(detail.text);});off();// unsubscribe
⚙️ Options
typeAssistantOptions={url: string;// WS endpointrequestId: {current: string};// mutable ref; client writes new ID per sessionrattAgentDetails?: Record<string,any>;onSend?: ()=>void;// called when server requests "disconnect"showToast?: (type: "error"|"info"|"success",title: string,msg: string)=>void;// Connection / heartbeatpingIntervalMs?: number;// default 5000maxMissedPongs?: number;// default 2// Audio (browser)workletBasePath?: string;// default "/"mediaStreamProvider?: ()=>Promise<MediaStream>;// default: getUserMedia (1ch, 16k, AGC/NS/EC enabled)audioContextFactory?: ()=>AudioContext|null;// default: new AudioContext() in browser, null in NodeworkletLoader?: (base: string)=>Promise<AudioContext|null>;// default: ensureAudioContextAndWorklets// External audio (Node or custom capture)externalAudio?: boolean;// default: false in browser, true in NodeexternalAmplitudeRms?: boolean;// default: truepcmChunkSize?: number;// default: TARGET_SAMPLES (typically 16000)};
🧪 Common Methods
awaitclient.connect();// single-flight; reuses active WS if presentawaitclient.startSession();// begin mic flow or external audio sessionawaitclient.stopAudio();// stop current session (sends {disconnect:true})client.disconnect();// local teardown (no forced WS close)client.teardown();// local teardown helpersclient.closeSocket();// forcibly close WS and detach handlers// Mic helpersawaitclient.beginPrebuffering();// start capturing locally; don't send yetclient.stopPrebuffering();// stop & clear buffered audioawaitclient.startMic();// explicitly start mic captureclient.stopMic();// explicitly stop mic (and send {disconnect:true})// External audioclient.pushPCM16(int16ArrayOrBuffer);client.pushFloat32(float32Array);// State gettersclient.wsReady;// booleanclient.micOpen;// booleanclient.micConnecting;// booleanclient.amplitude;// number (0..~1)client.transcription;// latest accumulated text
🧯 Troubleshooting
  • No audio sent
Ensure your server replies with {"start_audio": true}. The client buffers until the gate opens.
  • Mic blocked
Browser will throw NotAllowedError. The client emits ERROR and calls showToast(...).
  • Noisy audio / echo
The default constraints enable echo cancellation, AGC, and noise suppression. Override mediaStreamProvider if needed.
  • Multiple connects in React StrictMode
This client reuses a global ACTIVE_WS and a single CONNECT_PROMISE—you’re safe.
  • Heartbeat timeouts
Increase pingIntervalMs or maxMissedPongs if your WS hops are choppy.
🔐 Permissions & Security
  • Browsers require a user gesture to start the microphone.
  • Only minimal audio data is sent; handle it securely on your server. Use wss:// in production.
👥 Community & Support
Questions, bugs, or ideas? Open an issue in your repo or ping your team chat. Happy streaming! 🎙️💬
About
Ratt agent connection from UI 
Resources
Readme
Activity
Custom properties
Stars
0 stars
Watchers
0 watching
Forks
0 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
RATT Agent Library
Welcome to the **RATT Agent Library **! 🎧⚡
This tiny TypeScript client streams microphone (or external) audio over WebSocket, handles reconnection and heartbeats, and emits typed events for live transcription and UI state. It works in browsers (with AudioWorklets) and in Node (via external PCM).
🚀 Features
  • Low-latency audio streaming (16 kHz, 16-bit PCM) over WebSocket
  • Browser + Node: use the mic in browsers or push external PCM in any runtime
  • Typed events: ready, mic state, amplitude (VAD energy), transcription, socket messages, errors
  • Prebuffering & gating: capture early, start sending when the server says start_audio
  • Resilient WS: single-flight connect, StrictMode-safe reuse, heartbeats & auto-reconnect
  • Safe defaults: echo cancellation, AGC, noise suppression (browser)
  • Tiny API: a single class AssistantClient you can drop into your app
📦 Installation
npm i ratt-lib
# or
pnpm add ratt-lib
✨ Quick Start (Browser)
import{AssistantClient,AssistantOptions,AssistantEvent}from"ratt-lib";// Simple mutable ref for requestId (works well with React useRef) and this id will be unique for each reqconstrequestId={current: ""};constchatSessionId="test";//Unique for whole sessionconstclientId="test";//Unique for whole session// ✅ Build the REQUIRED rattAgentDetails objectconstrattAgentDetails={conciergeId: chatbotData?.id,// Your chatbot IdconciergeName: chatbotData?.name??chatbotData?.assistantDetails?.name,organizationId: chatbotData?.organization,organizationName: chatbotData?.organizationName,requestId: requestId.current,// will be overridden with a fresh value when session startsagentSettings: {voiceAgentMongoId: chatbotData?.agents?.filter((a: any)=>a.title==="RATTAgent")?.[0]?._id,},username: user?.provider?.name,// user nameuseremailId: user?.email,// user emailchatSessionId: chatSessionId,rlefVoiceTaskId: chatbotData?.audioTranscription?.modelId||CREATE_AUDIO_RELF_VOICE_TASK_ID_DEFAULT_VALUE,// rlef model idassistant_type: chatbotData?.assistant_type,// your assistant typeisAudioRequest: true,//always trueclient_id: clientId,userId: encodeParam(USER_ID_DEFAULT_VALUE),// your user id// keep below three as it is if test instant learning is not there , otherwise please send these values as well as per the requirements.testQuestion: "",testAnswer: "",testVariants: JSON.stringify({Edit: [],Add: [],Delete: []}),}asconst;constclient=newAssistantClient({url: "wss://dev-egpt.techo.camp/audioStreamingWebsocket?clientId=${clientId}&sessionId=${chatSessionId}",// Required: your WS endpoint
requestId,// Required: { current: string }
rattAgentDetails,// ✅ pass the full required objectonSend: ()=>console.log("Transcript submitted"),showToast: (type,title,msg)=>console.log(type,title,msg),// optional tunables:pingIntervalMs: 5000,maxMissedPongs: 2,workletBasePath: "/",// where recorder/vad worklets are served from});// Listen to eventsclient.on(AssistantEvent.READY,()=>{console.log("WS ready:",client.wsReady);});client.on(AssistantEvent.MIC_CONNECTING,({ detail })=>{console.log("Mic connecting:",detail.connecting);});client.on(AssistantEvent.MIC_OPEN,({ detail })=>{console.log("Mic open:",detail.open);});client.on(AssistantEvent.AMPLITUDE,({ detail })=>{// 0..~1 energy (not dB) to drive a mic meter UIconsole.log("Amplitude:",detail.value);});client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{// progressive or final textconsole.log("Transcript:",detail.text);});client.on(AssistantEvent.ERROR,({ detail })=>{console.error("Assistant error:",detail.error);});// Start a session (ask server to begin, then it will reply with start_audio)document.querySelector("#start")!.addEventListener("click",()=>{client.startSession();// toggles ON (opens mic flow) basically handleMicClick function});// Stop (ask server to disconnect and teardown locally)document.querySelector("#stop")!.addEventListener("click",()=>{client.stopAudio();// toggles OFF});
What happens under the hood?
  1. startSession() sends your rattAgentDetails + a new requestId.
  2. When your server responds with { "start_audio": true }, the client:
    • marks the mic as open,
    • starts converting to PCM16,
    • and begins streaming.
  3. As your server streams partial ASR, send either:
    • {"streaming_data":{"previous_transcription":"...", "new_transcription":"..."}} (chunked delta)
or
    • {"transcription":"final text"} (full updates)
  4. End with {"stop_audio": true} and/or {"disconnect": true} when you’re done.
🧩 React one-liner (optional)
useEffect(()=>{constunsub=client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{setText(detail.text);});returnunsub;// cleanly removes listener},[]);
🖥️ Node / External Audio (no browser mic)
If you're not in a browser (or you have your own capture pipeline), set externalAudio: true and push PCM yourself.
import{AssistantClient}from"ratt-lib";importfsfrom"node:fs";constrequestId={current: ""};constclient=newAssistantClient({url: "wss://dev-egpt.techo.camp/audioStreamingWebsocket?clientId=${clientId}&sessionId=${chatSessionId}",
requestId,externalAudio: true,// ⬅️ no AudioContext, no micexternalAmplitudeRms: true,// optional: compute amplitude from PCM});// Start session -> wait for server {"start_audio": true}awaitclient.connect();awaitclient.startSession();// Now stream your PCM16 mono 16kHz dataconstpcm=fs.readFileSync("./audio.raw");// Int16 little-endian mono 16kHzclient.pushPCM16(newInt16Array(pcm.buffer,pcm.byteOffset,pcm.byteLength/2));// ...push more chunks as they arrive...// When done: or it will be auto stop by server when it detects some silence is thereawaitclient.stopAudio();
PCM format: 16 kHz, 16-bit, mono, little-endian.
If you have Float32Array [-1..1], call pushFloat32() instead.
🔔 Events
All events are emitted as standard CustomEvents and strongly typed via AssistantEvents type.
  • READY — WebSocket is ready (connected & open)
  • MIC_CONNECTING — { connecting: boolean } while we prep/prompt for mic
  • MIC_OPEN — { open: boolean } mic flow is active/inactive
  • AMPLITUDE — { value: number } live energy (for a mic meter)
  • TRANSCRIPTION — { text: string, delta?: string } progressive or final
  • SOCKET_MESSAGE — { raw: MessageEvent, parsed?: any } every incoming WS message
  • ERROR — { error: unknown } any operational error
constoff=client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{console.log(detail.text);});off();// unsubscribe
⚙️ Options
typeAssistantOptions={url: string;// WS endpointrequestId: {current: string};// mutable ref; client writes new ID per sessionrattAgentDetails?: Record<string,any>;onSend?: ()=>void;// called when server requests "disconnect"showToast?: (type: "error"|"info"|"success",title: string,msg: string)=>void;// Connection / heartbeatpingIntervalMs?: number;// default 5000maxMissedPongs?: number;// default 2// Audio (browser)workletBasePath?: string;// default "/"mediaStreamProvider?: ()=>Promise<MediaStream>;// default: getUserMedia (1ch, 16k, AGC/NS/EC enabled)audioContextFactory?: ()=>AudioContext|null;// default: new AudioContext() in browser, null in NodeworkletLoader?: (base: string)=>Promise<AudioContext|null>;// default: ensureAudioContextAndWorklets// External audio (Node or custom capture)externalAudio?: boolean;// default: false in browser, true in NodeexternalAmplitudeRms?: boolean;// default: truepcmChunkSize?: number;// default: TARGET_SAMPLES (typically 16000)};
🧪 Common Methods
awaitclient.connect();// single-flight; reuses active WS if presentawaitclient.startSession();// begin mic flow or external audio sessionawaitclient.stopAudio();// stop current session (sends {disconnect:true})client.disconnect();// local teardown (no forced WS close)client.teardown();// local teardown helpersclient.closeSocket();// forcibly close WS and detach handlers// Mic helpersawaitclient.beginPrebuffering();// start capturing locally; don't send yetclient.stopPrebuffering();// stop & clear buffered audioawaitclient.startMic();// explicitly start mic captureclient.stopMic();// explicitly stop mic (and send {disconnect:true})// External audioclient.pushPCM16(int16ArrayOrBuffer);client.pushFloat32(float32Array);// State gettersclient.wsReady;// booleanclient.micOpen;// booleanclient.micConnecting;// booleanclient.amplitude;// number (0..~1)client.transcription;// latest accumulated text
🧯 Troubleshooting
  • No audio sent
Ensure your server replies with {"start_audio": true}. The client buffers until the gate opens.
  • Mic blocked
Browser will throw NotAllowedError. The client emits ERROR and calls showToast(...).
  • Noisy audio / echo
The default constraints enable echo cancellation, AGC, and noise suppression. Override mediaStreamProvider if needed.
  • Multiple connects in React StrictMode
This client reuses a global ACTIVE_WS and a single CONNECT_PROMISE—you’re safe.
  • Heartbeat timeouts
Increase pingIntervalMs or maxMissedPongs if your WS hops are choppy.
🔐 Permissions & Security
  • Browsers require a user gesture to start the microphone.
  • Only minimal audio data is sent; handle it securely on your server. Use wss:// in production.
👥 Community & Support
Questions, bugs, or ideas? Open an issue in your repo or ping your team chat. Happy streaming! 🎙️💬
About
Ratt agent connection from UI 
Resources
Readme
Activity
Custom properties
Stars
0 stars
Watchers
0 watching
Forks
0 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
BranchesTags
Repository files navigation
RATT Agent Library
Welcome to the **RATT Agent Library **! 🎧⚡
This tiny TypeScript client streams microphone (or external) audio over WebSocket, handles reconnection and heartbeats, and emits typed events for live transcription and UI state. It works in browsers (with AudioWorklets) and in Node (via external PCM).
🚀 Features
  • Low-latency audio streaming (16 kHz, 16-bit PCM) over WebSocket
  • Browser + Node: use the mic in browsers or push external PCM in any runtime
  • Typed events: ready, mic state, amplitude (VAD energy), transcription, socket messages, errors
  • Prebuffering & gating: capture early, start sending when the server says start_audio
  • Resilient WS: single-flight connect, StrictMode-safe reuse, heartbeats & auto-reconnect
  • Safe defaults: echo cancellation, AGC, noise suppression (browser)
  • Tiny API: a single class AssistantClient you can drop into your app
📦 Installation
npm i ratt-lib
# or
pnpm add ratt-lib
✨ Quick Start (Browser)
import{AssistantClient,AssistantOptions,AssistantEvent}from"ratt-lib";// Simple mutable ref for requestId (works well with React useRef) and this id will be unique for each reqconstrequestId={current: ""};constchatSessionId="test";//Unique for whole sessionconstclientId="test";//Unique for whole session// ✅ Build the REQUIRED rattAgentDetails objectconstrattAgentDetails={conciergeId: chatbotData?.id,// Your chatbot IdconciergeName: chatbotData?.name??chatbotData?.assistantDetails?.name,organizationId: chatbotData?.organization,organizationName: chatbotData?.organizationName,requestId: requestId.current,// will be overridden with a fresh value when session startsagentSettings: {voiceAgentMongoId: chatbotData?.agents?.filter((a: any)=>a.title==="RATTAgent")?.[0]?._id,},username: user?.provider?.name,// user nameuseremailId: user?.email,// user emailchatSessionId: chatSessionId,rlefVoiceTaskId: chatbotData?.audioTranscription?.modelId||CREATE_AUDIO_RELF_VOICE_TASK_ID_DEFAULT_VALUE,// rlef model idassistant_type: chatbotData?.assistant_type,// your assistant typeisAudioRequest: true,//always trueclient_id: clientId,userId: encodeParam(USER_ID_DEFAULT_VALUE),// your user id// keep below three as it is if test instant learning is not there , otherwise please send these values as well as per the requirements.testQuestion: "",testAnswer: "",testVariants: JSON.stringify({Edit: [],Add: [],Delete: []}),}asconst;constclient=newAssistantClient({url: "wss://dev-egpt.techo.camp/audioStreamingWebsocket?clientId=${clientId}&sessionId=${chatSessionId}",// Required: your WS endpoint
requestId,// Required: { current: string }
rattAgentDetails,// ✅ pass the full required objectonSend: ()=>console.log("Transcript submitted"),showToast: (type,title,msg)=>console.log(type,title,msg),// optional tunables:pingIntervalMs: 5000,maxMissedPongs: 2,workletBasePath: "/",// where recorder/vad worklets are served from});// Listen to eventsclient.on(AssistantEvent.READY,()=>{console.log("WS ready:",client.wsReady);});client.on(AssistantEvent.MIC_CONNECTING,({ detail })=>{console.log("Mic connecting:",detail.connecting);});client.on(AssistantEvent.MIC_OPEN,({ detail })=>{console.log("Mic open:",detail.open);});client.on(AssistantEvent.AMPLITUDE,({ detail })=>{// 0..~1 energy (not dB) to drive a mic meter UIconsole.log("Amplitude:",detail.value);});client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{// progressive or final textconsole.log("Transcript:",detail.text);});client.on(AssistantEvent.ERROR,({ detail })=>{console.error("Assistant error:",detail.error);});// Start a session (ask server to begin, then it will reply with start_audio)document.querySelector("#start")!.addEventListener("click",()=>{client.startSession();// toggles ON (opens mic flow) basically handleMicClick function});// Stop (ask server to disconnect and teardown locally)document.querySelector("#stop")!.addEventListener("click",()=>{client.stopAudio();// toggles OFF});
What happens under the hood?
  1. startSession() sends your rattAgentDetails + a new requestId.
  2. When your server responds with { "start_audio": true }, the client:
    • marks the mic as open,
    • starts converting to PCM16,
    • and begins streaming.
  3. As your server streams partial ASR, send either:
    • {"streaming_data":{"previous_transcription":"...", "new_transcription":"..."}} (chunked delta)
or
    • {"transcription":"final text"} (full updates)
  4. End with {"stop_audio": true} and/or {"disconnect": true} when you’re done.
🧩 React one-liner (optional)
useEffect(()=>{constunsub=client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{setText(detail.text);});returnunsub;// cleanly removes listener},[]);
🖥️ Node / External Audio (no browser mic)
If you're not in a browser (or you have your own capture pipeline), set externalAudio: true and push PCM yourself.
import{AssistantClient}from"ratt-lib";importfsfrom"node:fs";constrequestId={current: ""};constclient=newAssistantClient({url: "wss://dev-egpt.techo.camp/audioStreamingWebsocket?clientId=${clientId}&sessionId=${chatSessionId}",
requestId,externalAudio: true,// ⬅️ no AudioContext, no micexternalAmplitudeRms: true,// optional: compute amplitude from PCM});// Start session -> wait for server {"start_audio": true}awaitclient.connect();awaitclient.startSession();// Now stream your PCM16 mono 16kHz dataconstpcm=fs.readFileSync("./audio.raw");// Int16 little-endian mono 16kHzclient.pushPCM16(newInt16Array(pcm.buffer,pcm.byteOffset,pcm.byteLength/2));// ...push more chunks as they arrive...// When done: or it will be auto stop by server when it detects some silence is thereawaitclient.stopAudio();
PCM format: 16 kHz, 16-bit, mono, little-endian.
If you have Float32Array [-1..1], call pushFloat32() instead.
🔔 Events
All events are emitted as standard CustomEvents and strongly typed via AssistantEvents type.
  • READY — WebSocket is ready (connected & open)
  • MIC_CONNECTING — { connecting: boolean } while we prep/prompt for mic
  • MIC_OPEN — { open: boolean } mic flow is active/inactive
  • AMPLITUDE — { value: number } live energy (for a mic meter)
  • TRANSCRIPTION — { text: string, delta?: string } progressive or final
  • SOCKET_MESSAGE — { raw: MessageEvent, parsed?: any } every incoming WS message
  • ERROR — { error: unknown } any operational error
constoff=client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{console.log(detail.text);});off();// unsubscribe
⚙️ Options
typeAssistantOptions={url: string;// WS endpointrequestId: {current: string};// mutable ref; client writes new ID per sessionrattAgentDetails?: Record<string,any>;onSend?: ()=>void;// called when server requests "disconnect"showToast?: (type: "error"|"info"|"success",title: string,msg: string)=>void;// Connection / heartbeatpingIntervalMs?: number;// default 5000maxMissedPongs?: number;// default 2// Audio (browser)workletBasePath?: string;// default "/"mediaStreamProvider?: ()=>Promise<MediaStream>;// default: getUserMedia (1ch, 16k, AGC/NS/EC enabled)audioContextFactory?: ()=>AudioContext|null;// default: new AudioContext() in browser, null in NodeworkletLoader?: (base: string)=>Promise<AudioContext|null>;// default: ensureAudioContextAndWorklets// External audio (Node or custom capture)externalAudio?: boolean;// default: false in browser, true in NodeexternalAmplitudeRms?: boolean;// default: truepcmChunkSize?: number;// default: TARGET_SAMPLES (typically 16000)};
🧪 Common Methods
awaitclient.connect();// single-flight; reuses active WS if presentawaitclient.startSession();// begin mic flow or external audio sessionawaitclient.stopAudio();// stop current session (sends {disconnect:true})client.disconnect();// local teardown (no forced WS close)client.teardown();// local teardown helpersclient.closeSocket();// forcibly close WS and detach handlers// Mic helpersawaitclient.beginPrebuffering();// start capturing locally; don't send yetclient.stopPrebuffering();// stop & clear buffered audioawaitclient.startMic();// explicitly start mic captureclient.stopMic();// explicitly stop mic (and send {disconnect:true})// External audioclient.pushPCM16(int16ArrayOrBuffer);client.pushFloat32(float32Array);// State gettersclient.wsReady;// booleanclient.micOpen;// booleanclient.micConnecting;// booleanclient.amplitude;// number (0..~1)client.transcription;// latest accumulated text
🧯 Troubleshooting
  • No audio sent
Ensure your server replies with {"start_audio": true}. The client buffers until the gate opens.
  • Mic blocked
Browser will throw NotAllowedError. The client emits ERROR and calls showToast(...).
  • Noisy audio / echo
The default constraints enable echo cancellation, AGC, and noise suppression. Override mediaStreamProvider if needed.
  • Multiple connects in React StrictMode
This client reuses a global ACTIVE_WS and a single CONNECT_PROMISE—you’re safe.
  • Heartbeat timeouts
Increase pingIntervalMs or maxMissedPongs if your WS hops are choppy.
🔐 Permissions & Security
  • Browsers require a user gesture to start the microphone.
  • Only minimal audio data is sent; handle it securely on your server. Use wss:// in production.
👥 Community & Support
Questions, bugs, or ideas? Open an issue in your repo or ping your team chat. Happy streaming! 🎙️💬
About
Ratt agent connection from UI 
Resources
Readme
Activity
Custom properties
Stars
0 stars
Watchers
0 watching
Forks
0 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
RATT Agent Library
Welcome to the **RATT Agent Library **! 🎧⚡
This tiny TypeScript client streams microphone (or external) audio over WebSocket, handles reconnection and heartbeats, and emits typed events for live transcription and UI state. It works in browsers (with AudioWorklets) and in Node (via external PCM).
🚀 Features
  • Low-latency audio streaming (16 kHz, 16-bit PCM) over WebSocket
  • Browser + Node: use the mic in browsers or push external PCM in any runtime
  • Typed events: ready, mic state, amplitude (VAD energy), transcription, socket messages, errors
  • Prebuffering & gating: capture early, start sending when the server says start_audio
  • Resilient WS: single-flight connect, StrictMode-safe reuse, heartbeats & auto-reconnect
  • Safe defaults: echo cancellation, AGC, noise suppression (browser)
  • Tiny API: a single class AssistantClient you can drop into your app
📦 Installation
npm i ratt-lib
# or
pnpm add ratt-lib
✨ Quick Start (Browser)
import{AssistantClient,AssistantOptions,AssistantEvent}from"ratt-lib";// Simple mutable ref for requestId (works well with React useRef) and this id will be unique for each reqconstrequestId={current: ""};constchatSessionId="test";//Unique for whole sessionconstclientId="test";//Unique for whole session// ✅ Build the REQUIRED rattAgentDetails objectconstrattAgentDetails={conciergeId: chatbotData?.id,// Your chatbot IdconciergeName: chatbotData?.name??chatbotData?.assistantDetails?.name,organizationId: chatbotData?.organization,organizationName: chatbotData?.organizationName,requestId: requestId.current,// will be overridden with a fresh value when session startsagentSettings: {voiceAgentMongoId: chatbotData?.agents?.filter((a: any)=>a.title==="RATTAgent")?.[0]?._id,},username: user?.provider?.name,// user nameuseremailId: user?.email,// user emailchatSessionId: chatSessionId,rlefVoiceTaskId: chatbotData?.audioTranscription?.modelId||CREATE_AUDIO_RELF_VOICE_TASK_ID_DEFAULT_VALUE,// rlef model idassistant_type: chatbotData?.assistant_type,// your assistant typeisAudioRequest: true,//always trueclient_id: clientId,userId: encodeParam(USER_ID_DEFAULT_VALUE),// your user id// keep below three as it is if test instant learning is not there , otherwise please send these values as well as per the requirements.testQuestion: "",testAnswer: "",testVariants: JSON.stringify({Edit: [],Add: [],Delete: []}),}asconst;constclient=newAssistantClient({url: "wss://dev-egpt.techo.camp/audioStreamingWebsocket?clientId=${clientId}&sessionId=${chatSessionId}",// Required: your WS endpoint
requestId,// Required: { current: string }
rattAgentDetails,// ✅ pass the full required objectonSend: ()=>console.log("Transcript submitted"),showToast: (type,title,msg)=>console.log(type,title,msg),// optional tunables:pingIntervalMs: 5000,maxMissedPongs: 2,workletBasePath: "/",// where recorder/vad worklets are served from});// Listen to eventsclient.on(AssistantEvent.READY,()=>{console.log("WS ready:",client.wsReady);});client.on(AssistantEvent.MIC_CONNECTING,({ detail })=>{console.log("Mic connecting:",detail.connecting);});client.on(AssistantEvent.MIC_OPEN,({ detail })=>{console.log("Mic open:",detail.open);});client.on(AssistantEvent.AMPLITUDE,({ detail })=>{// 0..~1 energy (not dB) to drive a mic meter UIconsole.log("Amplitude:",detail.value);});client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{// progressive or final textconsole.log("Transcript:",detail.text);});client.on(AssistantEvent.ERROR,({ detail })=>{console.error("Assistant error:",detail.error);});// Start a session (ask server to begin, then it will reply with start_audio)document.querySelector("#start")!.addEventListener("click",()=>{client.startSession();// toggles ON (opens mic flow) basically handleMicClick function});// Stop (ask server to disconnect and teardown locally)document.querySelector("#stop")!.addEventListener("click",()=>{client.stopAudio();// toggles OFF});
What happens under the hood?
  1. startSession() sends your rattAgentDetails + a new requestId.
  2. When your server responds with { "start_audio": true }, the client:
    • marks the mic as open,
    • starts converting to PCM16,
    • and begins streaming.
  3. As your server streams partial ASR, send either:
    • {"streaming_data":{"previous_transcription":"...", "new_transcription":"..."}} (chunked delta)
or
    • {"transcription":"final text"} (full updates)
  4. End with {"stop_audio": true} and/or {"disconnect": true} when you’re done.
🧩 React one-liner (optional)
useEffect(()=>{constunsub=client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{setText(detail.text);});returnunsub;// cleanly removes listener},[]);
🖥️ Node / External Audio (no browser mic)
If you're not in a browser (or you have your own capture pipeline), set externalAudio: true and push PCM yourself.
import{AssistantClient}from"ratt-lib";importfsfrom"node:fs";constrequestId={current: ""};constclient=newAssistantClient({url: "wss://dev-egpt.techo.camp/audioStreamingWebsocket?clientId=${clientId}&sessionId=${chatSessionId}",
requestId,externalAudio: true,// ⬅️ no AudioContext, no micexternalAmplitudeRms: true,// optional: compute amplitude from PCM});// Start session -> wait for server {"start_audio": true}awaitclient.connect();awaitclient.startSession();// Now stream your PCM16 mono 16kHz dataconstpcm=fs.readFileSync("./audio.raw");// Int16 little-endian mono 16kHzclient.pushPCM16(newInt16Array(pcm.buffer,pcm.byteOffset,pcm.byteLength/2));// ...push more chunks as they arrive...// When done: or it will be auto stop by server when it detects some silence is thereawaitclient.stopAudio();
PCM format: 16 kHz, 16-bit, mono, little-endian.
If you have Float32Array [-1..1], call pushFloat32() instead.
🔔 Events
All events are emitted as standard CustomEvents and strongly typed via AssistantEvents type.
  • READY — WebSocket is ready (connected & open)
  • MIC_CONNECTING — { connecting: boolean } while we prep/prompt for mic
  • MIC_OPEN — { open: boolean } mic flow is active/inactive
  • AMPLITUDE — { value: number } live energy (for a mic meter)
  • TRANSCRIPTION — { text: string, delta?: string } progressive or final
  • SOCKET_MESSAGE — { raw: MessageEvent, parsed?: any } every incoming WS message
  • ERROR — { error: unknown } any operational error
constoff=client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{console.log(detail.text);});off();// unsubscribe
⚙️ Options
typeAssistantOptions={url: string;// WS endpointrequestId: {current: string};// mutable ref; client writes new ID per sessionrattAgentDetails?: Record<string,any>;onSend?: ()=>void;// called when server requests "disconnect"showToast?: (type: "error"|"info"|"success",title: string,msg: string)=>void;// Connection / heartbeatpingIntervalMs?: number;// default 5000maxMissedPongs?: number;// default 2// Audio (browser)workletBasePath?: string;// default "/"mediaStreamProvider?: ()=>Promise<MediaStream>;// default: getUserMedia (1ch, 16k, AGC/NS/EC enabled)audioContextFactory?: ()=>AudioContext|null;// default: new AudioContext() in browser, null in NodeworkletLoader?: (base: string)=>Promise<AudioContext|null>;// default: ensureAudioContextAndWorklets// External audio (Node or custom capture)externalAudio?: boolean;// default: false in browser, true in NodeexternalAmplitudeRms?: boolean;// default: truepcmChunkSize?: number;// default: TARGET_SAMPLES (typically 16000)};
🧪 Common Methods
awaitclient.connect();// single-flight; reuses active WS if presentawaitclient.startSession();// begin mic flow or external audio sessionawaitclient.stopAudio();// stop current session (sends {disconnect:true})client.disconnect();// local teardown (no forced WS close)client.teardown();// local teardown helpersclient.closeSocket();// forcibly close WS and detach handlers// Mic helpersawaitclient.beginPrebuffering();// start capturing locally; don't send yetclient.stopPrebuffering();// stop & clear buffered audioawaitclient.startMic();// explicitly start mic captureclient.stopMic();// explicitly stop mic (and send {disconnect:true})// External audioclient.pushPCM16(int16ArrayOrBuffer);client.pushFloat32(float32Array);// State gettersclient.wsReady;// booleanclient.micOpen;// booleanclient.micConnecting;// booleanclient.amplitude;// number (0..~1)client.transcription;// latest accumulated text
🧯 Troubleshooting
  • No audio sent
Ensure your server replies with {"start_audio": true}. The client buffers until the gate opens.
  • Mic blocked
Browser will throw NotAllowedError. The client emits ERROR and calls showToast(...).
  • Noisy audio / echo
The default constraints enable echo cancellation, AGC, and noise suppression. Override mediaStreamProvider if needed.
  • Multiple connects in React StrictMode
This client reuses a global ACTIVE_WS and a single CONNECT_PROMISE—you’re safe.
  • Heartbeat timeouts
Increase pingIntervalMs or maxMissedPongs if your WS hops are choppy.
🔐 Permissions & Security
  • Browsers require a user gesture to start the microphone.
  • Only minimal audio data is sent; handle it securely on your server. Use wss:// in production.
👥 Community & Support
Questions, bugs, or ideas? Open an issue in your repo or ping your team chat. Happy streaming! 🎙️💬
About
Ratt agent connection from UI 
Resources
Readme
Activity
Custom properties
Stars
0 stars
Watchers
0 watching
Forks
0 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
BranchesTags
Repository files navigation
RATT Agent Library
Welcome to the **RATT Agent Library **! 🎧⚡
This tiny TypeScript client streams microphone (or external) audio over WebSocket, handles reconnection and heartbeats, and emits typed events for live transcription and UI state. It works in browsers (with AudioWorklets) and in Node (via external PCM).
🚀 Features
  • Low-latency audio streaming (16 kHz, 16-bit PCM) over WebSocket
  • Browser + Node: use the mic in browsers or push external PCM in any runtime
  • Typed events: ready, mic state, amplitude (VAD energy), transcription, socket messages, errors
  • Prebuffering & gating: capture early, start sending when the server says start_audio
  • Resilient WS: single-flight connect, StrictMode-safe reuse, heartbeats & auto-reconnect
  • Safe defaults: echo cancellation, AGC, noise suppression (browser)
  • Tiny API: a single class AssistantClient you can drop into your app
📦 Installation
npm i ratt-lib
# or
pnpm add ratt-lib
✨ Quick Start (Browser)
import{AssistantClient,AssistantOptions,AssistantEvent}from"ratt-lib";// Simple mutable ref for requestId (works well with React useRef) and this id will be unique for each reqconstrequestId={current: ""};constchatSessionId="test";//Unique for whole sessionconstclientId="test";//Unique for whole session// ✅ Build the REQUIRED rattAgentDetails objectconstrattAgentDetails={conciergeId: chatbotData?.id,// Your chatbot IdconciergeName: chatbotData?.name??chatbotData?.assistantDetails?.name,organizationId: chatbotData?.organization,organizationName: chatbotData?.organizationName,requestId: requestId.current,// will be overridden with a fresh value when session startsagentSettings: {voiceAgentMongoId: chatbotData?.agents?.filter((a: any)=>a.title==="RATTAgent")?.[0]?._id,},username: user?.provider?.name,// user nameuseremailId: user?.email,// user emailchatSessionId: chatSessionId,rlefVoiceTaskId: chatbotData?.audioTranscription?.modelId||CREATE_AUDIO_RELF_VOICE_TASK_ID_DEFAULT_VALUE,// rlef model idassistant_type: chatbotData?.assistant_type,// your assistant typeisAudioRequest: true,//always trueclient_id: clientId,userId: encodeParam(USER_ID_DEFAULT_VALUE),// your user id// keep below three as it is if test instant learning is not there , otherwise please send these values as well as per the requirements.testQuestion: "",testAnswer: "",testVariants: JSON.stringify({Edit: [],Add: [],Delete: []}),}asconst;constclient=newAssistantClient({url: "wss://dev-egpt.techo.camp/audioStreamingWebsocket?clientId=${clientId}&sessionId=${chatSessionId}",// Required: your WS endpoint
requestId,// Required: { current: string }
rattAgentDetails,// ✅ pass the full required objectonSend: ()=>console.log("Transcript submitted"),showToast: (type,title,msg)=>console.log(type,title,msg),// optional tunables:pingIntervalMs: 5000,maxMissedPongs: 2,workletBasePath: "/",// where recorder/vad worklets are served from});// Listen to eventsclient.on(AssistantEvent.READY,()=>{console.log("WS ready:",client.wsReady);});client.on(AssistantEvent.MIC_CONNECTING,({ detail })=>{console.log("Mic connecting:",detail.connecting);});client.on(AssistantEvent.MIC_OPEN,({ detail })=>{console.log("Mic open:",detail.open);});client.on(AssistantEvent.AMPLITUDE,({ detail })=>{// 0..~1 energy (not dB) to drive a mic meter UIconsole.log("Amplitude:",detail.value);});client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{// progressive or final textconsole.log("Transcript:",detail.text);});client.on(AssistantEvent.ERROR,({ detail })=>{console.error("Assistant error:",detail.error);});// Start a session (ask server to begin, then it will reply with start_audio)document.querySelector("#start")!.addEventListener("click",()=>{client.startSession();// toggles ON (opens mic flow) basically handleMicClick function});// Stop (ask server to disconnect and teardown locally)document.querySelector("#stop")!.addEventListener("click",()=>{client.stopAudio();// toggles OFF});
What happens under the hood?
  1. startSession() sends your rattAgentDetails + a new requestId.
  2. When your server responds with { "start_audio": true }, the client:
    • marks the mic as open,
    • starts converting to PCM16,
    • and begins streaming.
  3. As your server streams partial ASR, send either:
    • {"streaming_data":{"previous_transcription":"...", "new_transcription":"..."}} (chunked delta)
or
    • {"transcription":"final text"} (full updates)
  4. End with {"stop_audio": true} and/or {"disconnect": true} when you’re done.
🧩 React one-liner (optional)
useEffect(()=>{constunsub=client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{setText(detail.text);});returnunsub;// cleanly removes listener},[]);
🖥️ Node / External Audio (no browser mic)
If you're not in a browser (or you have your own capture pipeline), set externalAudio: true and push PCM yourself.
import{AssistantClient}from"ratt-lib";importfsfrom"node:fs";constrequestId={current: ""};constclient=newAssistantClient({url: "wss://dev-egpt.techo.camp/audioStreamingWebsocket?clientId=${clientId}&sessionId=${chatSessionId}",
requestId,externalAudio: true,// ⬅️ no AudioContext, no micexternalAmplitudeRms: true,// optional: compute amplitude from PCM});// Start session -> wait for server {"start_audio": true}awaitclient.connect();awaitclient.startSession();// Now stream your PCM16 mono 16kHz dataconstpcm=fs.readFileSync("./audio.raw");// Int16 little-endian mono 16kHzclient.pushPCM16(newInt16Array(pcm.buffer,pcm.byteOffset,pcm.byteLength/2));// ...push more chunks as they arrive...// When done: or it will be auto stop by server when it detects some silence is thereawaitclient.stopAudio();
PCM format: 16 kHz, 16-bit, mono, little-endian.
If you have Float32Array [-1..1], call pushFloat32() instead.
🔔 Events
All events are emitted as standard CustomEvents and strongly typed via AssistantEvents type.
  • READY — WebSocket is ready (connected & open)
  • MIC_CONNECTING — { connecting: boolean } while we prep/prompt for mic
  • MIC_OPEN — { open: boolean } mic flow is active/inactive
  • AMPLITUDE — { value: number } live energy (for a mic meter)
  • TRANSCRIPTION — { text: string, delta?: string } progressive or final
  • SOCKET_MESSAGE — { raw: MessageEvent, parsed?: any } every incoming WS message
  • ERROR — { error: unknown } any operational error
constoff=client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{console.log(detail.text);});off();// unsubscribe
⚙️ Options
typeAssistantOptions={url: string;// WS endpointrequestId: {current: string};// mutable ref; client writes new ID per sessionrattAgentDetails?: Record<string,any>;onSend?: ()=>void;// called when server requests "disconnect"showToast?: (type: "error"|"info"|"success",title: string,msg: string)=>void;// Connection / heartbeatpingIntervalMs?: number;// default 5000maxMissedPongs?: number;// default 2// Audio (browser)workletBasePath?: string;// default "/"mediaStreamProvider?: ()=>Promise<MediaStream>;// default: getUserMedia (1ch, 16k, AGC/NS/EC enabled)audioContextFactory?: ()=>AudioContext|null;// default: new AudioContext() in browser, null in NodeworkletLoader?: (base: string)=>Promise<AudioContext|null>;// default: ensureAudioContextAndWorklets// External audio (Node or custom capture)externalAudio?: boolean;// default: false in browser, true in NodeexternalAmplitudeRms?: boolean;// default: truepcmChunkSize?: number;// default: TARGET_SAMPLES (typically 16000)};
🧪 Common Methods
awaitclient.connect();// single-flight; reuses active WS if presentawaitclient.startSession();// begin mic flow or external audio sessionawaitclient.stopAudio();// stop current session (sends {disconnect:true})client.disconnect();// local teardown (no forced WS close)client.teardown();// local teardown helpersclient.closeSocket();// forcibly close WS and detach handlers// Mic helpersawaitclient.beginPrebuffering();// start capturing locally; don't send yetclient.stopPrebuffering();// stop & clear buffered audioawaitclient.startMic();// explicitly start mic captureclient.stopMic();// explicitly stop mic (and send {disconnect:true})// External audioclient.pushPCM16(int16ArrayOrBuffer);client.pushFloat32(float32Array);// State gettersclient.wsReady;// booleanclient.micOpen;// booleanclient.micConnecting;// booleanclient.amplitude;// number (0..~1)client.transcription;// latest accumulated text
🧯 Troubleshooting
  • No audio sent
Ensure your server replies with {"start_audio": true}. The client buffers until the gate opens.
  • Mic blocked
Browser will throw NotAllowedError. The client emits ERROR and calls showToast(...).
  • Noisy audio / echo
The default constraints enable echo cancellation, AGC, and noise suppression. Override mediaStreamProvider if needed.
  • Multiple connects in React StrictMode
This client reuses a global ACTIVE_WS and a single CONNECT_PROMISE—you’re safe.
  • Heartbeat timeouts
Increase pingIntervalMs or maxMissedPongs if your WS hops are choppy.
🔐 Permissions & Security
  • Browsers require a user gesture to start the microphone.
  • Only minimal audio data is sent; handle it securely on your server. Use wss:// in production.
👥 Community & Support
Questions, bugs, or ideas? Open an issue in your repo or ping your team chat. Happy streaming! 🎙️💬
About
Ratt agent connection from UI 
Resources
Readme
Activity
Custom properties
Stars
0 stars
Watchers
0 watching
Forks
0 forks
Report repository
Releases
Packages
Used by
Contributors
Languages

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
BranchesTags
Repository files navigation
RATT Agent Library
Welcome to the **RATT Agent Library **! 🎧⚡
This tiny TypeScript client streams microphone (or external) audio over WebSocket, handles reconnection and heartbeats, and emits typed events for live transcription and UI state. It works in browsers (with AudioWorklets) and in Node (via external PCM).
🚀 Features
  • Low-latency audio streaming (16 kHz, 16-bit PCM) over WebSocket
  • Browser + Node: use the mic in browsers or push external PCM in any runtime
  • Typed events: ready, mic state, amplitude (VAD energy), transcription, socket messages, errors
  • Prebuffering & gating: capture early, start sending when the server says start_audio
  • Resilient WS: single-flight connect, StrictMode-safe reuse, heartbeats & auto-reconnect
  • Safe defaults: echo cancellation, AGC, noise suppression (browser)
  • Tiny API: a single class AssistantClient you can drop into your app
📦 Installation
npm i ratt-lib
# or
pnpm add ratt-lib
✨ Quick Start (Browser)
import{AssistantClient,AssistantOptions,AssistantEvent}from"ratt-lib";// Simple mutable ref for requestId (works well with React useRef) and this id will be unique for each reqconstrequestId={current: ""};constchatSessionId="test";//Unique for whole sessionconstclientId="test";//Unique for whole session// ✅ Build the REQUIRED rattAgentDetails objectconstrattAgentDetails={conciergeId: chatbotData?.id,// Your chatbot IdconciergeName: chatbotData?.name??chatbotData?.assistantDetails?.name,organizationId: chatbotData?.organization,organizationName: chatbotData?.organizationName,requestId: requestId.current,// will be overridden with a fresh value when session startsagentSettings: {voiceAgentMongoId: chatbotData?.agents?.filter((a: any)=>a.title==="RATTAgent")?.[0]?._id,},username: user?.provider?.name,// user nameuseremailId: user?.email,// user emailchatSessionId: chatSessionId,rlefVoiceTaskId: chatbotData?.audioTranscription?.modelId||CREATE_AUDIO_RELF_VOICE_TASK_ID_DEFAULT_VALUE,// rlef model idassistant_type: chatbotData?.assistant_type,// your assistant typeisAudioRequest: true,//always trueclient_id: clientId,userId: encodeParam(USER_ID_DEFAULT_VALUE),// your user id// keep below three as it is if test instant learning is not there , otherwise please send these values as well as per the requirements.testQuestion: "",testAnswer: "",testVariants: JSON.stringify({Edit: [],Add: [],Delete: []}),}asconst;constclient=newAssistantClient({url: "wss://dev-egpt.techo.camp/audioStreamingWebsocket?clientId=${clientId}&sessionId=${chatSessionId}",// Required: your WS endpoint
requestId,// Required: { current: string }
rattAgentDetails,// ✅ pass the full required objectonSend: ()=>console.log("Transcript submitted"),showToast: (type,title,msg)=>console.log(type,title,msg),// optional tunables:pingIntervalMs: 5000,maxMissedPongs: 2,workletBasePath: "/",// where recorder/vad worklets are served from});// Listen to eventsclient.on(AssistantEvent.READY,()=>{console.log("WS ready:",client.wsReady);});client.on(AssistantEvent.MIC_CONNECTING,({ detail })=>{console.log("Mic connecting:",detail.connecting);});client.on(AssistantEvent.MIC_OPEN,({ detail })=>{console.log("Mic open:",detail.open);});client.on(AssistantEvent.AMPLITUDE,({ detail })=>{// 0..~1 energy (not dB) to drive a mic meter UIconsole.log("Amplitude:",detail.value);});client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{// progressive or final textconsole.log("Transcript:",detail.text);});client.on(AssistantEvent.ERROR,({ detail })=>{console.error("Assistant error:",detail.error);});// Start a session (ask server to begin, then it will reply with start_audio)document.querySelector("#start")!.addEventListener("click",()=>{client.startSession();// toggles ON (opens mic flow) basically handleMicClick function});// Stop (ask server to disconnect and teardown locally)document.querySelector("#stop")!.addEventListener("click",()=>{client.stopAudio();// toggles OFF});
What happens under the hood?
  1. startSession() sends your rattAgentDetails + a new requestId.
  2. When your server responds with { "start_audio": true }, the client:
    • marks the mic as open,
    • starts converting to PCM16,
    • and begins streaming.
  3. As your server streams partial ASR, send either:
    • {"streaming_data":{"previous_transcription":"...", "new_transcription":"..."}} (chunked delta)
or
    • {"transcription":"final text"} (full updates)
  4. End with {"stop_audio": true} and/or {"disconnect": true} when you’re done.
🧩 React one-liner (optional)
useEffect(()=>{constunsub=client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{setText(detail.text);});returnunsub;// cleanly removes listener},[]);
🖥️ Node / External Audio (no browser mic)
If you're not in a browser (or you have your own capture pipeline), set externalAudio: true and push PCM yourself.
import{AssistantClient}from"ratt-lib";importfsfrom"node:fs";constrequestId={current: ""};constclient=newAssistantClient({url: "wss://dev-egpt.techo.camp/audioStreamingWebsocket?clientId=${clientId}&sessionId=${chatSessionId}",
requestId,externalAudio: true,// ⬅️ no AudioContext, no micexternalAmplitudeRms: true,// optional: compute amplitude from PCM});// Start session -> wait for server {"start_audio": true}awaitclient.connect();awaitclient.startSession();// Now stream your PCM16 mono 16kHz dataconstpcm=fs.readFileSync("./audio.raw");// Int16 little-endian mono 16kHzclient.pushPCM16(newInt16Array(pcm.buffer,pcm.byteOffset,pcm.byteLength/2));// ...push more chunks as they arrive...// When done: or it will be auto stop by server when it detects some silence is thereawaitclient.stopAudio();
PCM format: 16 kHz, 16-bit, mono, little-endian.
If you have Float32Array [-1..1], call pushFloat32() instead.
🔔 Events
All events are emitted as standard CustomEvents and strongly typed via AssistantEvents type.
  • READY — WebSocket is ready (connected & open)
  • MIC_CONNECTING — { connecting: boolean } while we prep/prompt for mic
  • MIC_OPEN — { open: boolean } mic flow is active/inactive
  • AMPLITUDE — { value: number } live energy (for a mic meter)
  • TRANSCRIPTION — { text: string, delta?: string } progressive or final
  • SOCKET_MESSAGE — { raw: MessageEvent, parsed?: any } every incoming WS message
  • ERROR — { error: unknown } any operational error
constoff=client.on(AssistantEvent.TRANSCRIPTION,({ detail })=>{console.log(detail.text);});off();// unsubscribe
⚙️ Options
typeAssistantOptions={url: string;// WS endpointrequestId: {current: string};// mutable ref; client writes new ID per sessionrattAgentDetails?: Record<string,any>;onSend?: ()=>void;// called when server requests "disconnect"showToast?: (type: "error"|"info"|"success",title: string,msg: string)=>void;// Connection / heartbeatpingIntervalMs?: number;// default 5000maxMissedPongs?: number;// default 2// Audio (browser)workletBasePath?: string;// default "/"mediaStreamProvider?: ()=>Promise<MediaStream>;// default: getUserMedia (1ch, 16k, AGC/NS/EC enabled)audioContextFactory?: ()=>AudioContext|null;// default: new AudioContext() in browser, null in NodeworkletLoader?: (base: string)=>Promise<AudioContext|null>;// default: ensureAudioContextAndWorklets// External audio (Node or custom capture)externalAudio?: boolean;// default: false in browser, true in NodeexternalAmplitudeRms?: boolean;// default: truepcmChunkSize?: number;// default: TARGET_SAMPLES (typically 16000)};
🧪 Common Methods
awaitclient.connect();// single-flight; reuses active WS if presentawaitclient.startSession();// begin mic flow or external audio sessionawaitclient.stopAudio();// stop current session (sends {disconnect:true})client.disconnect();// local teardown (no forced WS close)client.teardown();// local teardown helpersclient.closeSocket();// forcibly close WS and detach handlers// Mic helpersawaitclient.beginPrebuffering();// start capturing locally; don't send yetclient.stopPrebuffering();// stop & clear buffered audioawaitclient.startMic();// explicitly start mic captureclient.stopMic();// explicitly stop mic (and send {disconnect:true})// External audioclient.pushPCM16(int16ArrayOrBuffer);client.pushFloat32(float32Array);// State gettersclient.wsReady;// booleanclient.micOpen;// booleanclient.micConnecting;// booleanclient.amplitude;// number (0..~1)client.transcription;// latest accumulated text
🧯 Troubleshooting
  • No audio sent
Ensure your server replies with {"start_audio": true}. The client buffers until the gate opens.
  • Mic blocked
Browser will throw NotAllowedError. The client emits ERROR and calls showToast(...).
  • Noisy audio / echo
The default constraints enable echo cancellation, AGC, and noise suppression. Override mediaStreamProvider if needed.
  • Multiple connects in React StrictMode
This client reuses a global ACTIVE_WS and a single CONNECT_PROMISE—you’re safe.
  • Heartbeat timeouts
Increase pingIntervalMs or maxMissedPongs if your WS hops are choppy.
🔐 Permissions & Security
  • Browsers require a user gesture to start the microphone.
  • Only minimal audio data is sent; handle it securely on your server. Use wss:// in production.
👥 Community & Support
Questions, bugs, or ideas? Open an issue in your repo or ping your team chat. Happy streaming! 🎙️💬
About
Ratt agent connection from UI 
Resources
Readme
Activity
Custom properties
Stars
0 stars
Watchers
0 watching
Forks
0 forks
Report repository
Releases
Packages
Used by
Contributors
Languages