{
  "schema": "https://ai-atoms.com/schemas/skill-v1.json",
  "type": "skill",
  "id": "skill/azure-ai-voicelive-ts",
  "version": "1.0.1",
  "name": "Azure Ai Voicelive Ts",
  "description": "Azure AI Voice Live SDK for JavaScript/TypeScript. Build real-time voice AI applications with bidirectional WebSocket communication. Use for voice assistants, conversational AI, real-time speech-to-speech, and voice-enabled chatbots in Node.js or browser environments. Triggers: \"voice live\", \"real-time voice\", \"VoiceLiveClient\", \"VoiceLiveSession\", \"voice assistant TypeScript\", \"bidirectional audio\", \"speech-to-speech JavaScript\".",
  "system_prompt_fragment": "# @azure/ai-voicelive (JavaScript/TypeScript)\n\nReal-time voice AI SDK for building bidirectional voice assistants with Azure AI in Node.js and browser environments.\n\n## Installation\n\n```bash\nnpm install @azure/ai-voicelive @azure/identity\n# TypeScript users\nnpm install @types/node\n```\n\n**Current Version**: 1.0.0-beta.3\n\n**Supported Environments**:\n- Node.js LTS versions (20+)\n- Modern browsers (Chrome, Firefox, Safari, Edge)\n\n## Environment Variables\n\n```bash\nAZURE_VOICELIVE_ENDPOINT=https://<resource>.cognitiveservices.azure.com\n# Optional: API key if not using Entra ID\nAZURE_VOICELIVE_API_KEY=<your-api-key>\n# Optional: Logging\nAZURE_LOG_LEVEL=info\n```\n\n## Authentication\n\n### Microsoft Entra ID (Recommended)\n\n```typescript\nimport { DefaultAzureCredential } from \"@azure/identity\";\nimport { VoiceLiveClient } from \"@azure/ai-voicelive\";\n\nconst credential = new DefaultAzureCredential();\nconst endpoint = \"https://your-resource.cognitiveservices.azure.com\";\n\nconst client = new VoiceLiveClient(endpoint, credential);\n```\n\n### API Key\n\n```typescript\nimport { AzureKeyCredential } from \"@azure/core-auth\";\nimport { VoiceLiveClient } from \"@azure/ai-voicelive\";\n\nconst endpoint = \"https://your-resource.cognitiveservices.azure.com\";\nconst credential = new AzureKeyCredential(\"your-api-key\");\n\nconst client = new VoiceLiveClient(endpoint, credential);\n```\n\n## Client Hierarchy\n\n```\nVoiceLiveClient\n└── VoiceLiveSession (WebSocket connection)\n    ├── updateSession()      → Configure session options\n    ├── subscribe()          → Event handlers (Azure SDK pattern)\n    ├── sendAudio()          → Stream audio input\n    ├── addConversationItem() → Add messages/function outputs\n    └── sendEvent()          → Send raw protocol events\n```\n\n## Quick Start\n\n```typescript\nimport { DefaultAzureCredential } from \"@azure/identity\";\nimport { VoiceLiveClient } from \"@azure/ai-voicelive\";\n\nconst credential = new DefaultAzureCredential();\nconst endpoint = process.env.AZURE_VOICELIVE_ENDPOINT!;\n\n// Create client and start session\nconst client = new VoiceLiveClient(endpoint, credential);\nconst session = await client.startSession(\"gpt-4o-mini-realtime-preview\");\n\n// Configure session\nawait session.updateSession({\n  modalities: [\"text\", \"audio\"],\n  instructions: \"You are a helpful AI assistant. Respond naturally.\",\n  voice: {\n    type: \"azure-standard\",\n    name: \"en-US-AvaNeural\",\n  },\n  turnDetection: {\n    type: \"server_vad\",\n    threshold: 0.5,\n    prefixPaddingMs: 300,\n    silenceDurationMs: 500,\n  },\n  inputAudioFormat: \"pcm16\",\n  outputAudioFormat: \"pcm16\",\n});\n\n// Subscribe to events\nconst subscription = session.subscribe({\n  onResponseAudioDelta: async (event, context) => {\n    // Handle streaming audio output\n    const audioData = event.delta;\n    playAudioChunk(audioData);\n  },\n  onResponseTextDelta: async (event, context) => {\n    // Handle streaming text\n    process.stdout.write(event.delta);\n  },\n  onInputAudioTranscriptionCompleted: async (event, context) => {\n    console.log(\"User said:\", event.transcript);\n  },\n});\n\n// Send audio from microphone\nfunction sendAudioChunk(audioBuffer: ArrayBuffer) {\n  session.sendAudio(audioBuffer);\n}\n```\n\n## Session Configuration\n\n```typescript\nawait session.updateSession({\n  // Modalities\n  modalities: [\"audio\", \"text\"],\n  \n  // System instructions\n  instructions: \"You are a customer service representative.\",\n  \n  // Voice selection\n  voice: {\n    type: \"azure-standard\",  // or \"azure-custom\", \"openai\"\n    name: \"en-US-AvaNeural\",\n  },\n  \n  // Turn detection (VAD)\n  turnDetection: {\n    type: \"server_vad\",      // or \"azure_semantic_vad\"\n    threshold: 0.5,\n    prefixPaddingMs: 300,\n    silenceDurationMs: 500,\n  },\n  \n  // Audio formats\n  inputAudioFormat: \"pcm16\",\n  outputAudioFormat: \"pcm16\",\n  \n  // Tools (function calling)\n  tools: [\n    {\n      type: \"function\",\n      name: \"get_weather\",\n      description: \"Get current weather\",\n      parameters: {\n        type: \"object\",\n        properties: {\n          location: { type: \"string\" }\n        },\n        required: [\"location\"]\n      }\n    }\n  ],\n  toolChoice: \"auto\",\n});\n```\n\n## Event Handling (Azure SDK Pattern)\n\nThe SDK uses a subscription-based event handling pattern:\n\n```typescript\nconst subscription = session.subscribe({\n  // Connection lifecycle\n  onConnected: async (args, context) => {\n    console.log(\"Connected:\", args.connectionId);\n  },\n  onDisconnected: async (args, context) => {\n    console.log(\"Disconnected:\", args.code, args.reason);\n  },\n  onError: async (args, context) => {\n    console.error(\"Error:\", args.error.message);\n  },\n  \n  // Session events\n  onSessionCreated: async (event, context) => {\n    console.log(\"Session created:\", context.sessionId);\n  },\n  onSessionUpdated: async (event, context) => {\n    console.log(\"Session updated\");\n  },\n  \n  // Audio input events (VAD)\n  onInputAudioBufferSpeechStarted: async (event, context) => {\n    console.log(\"Speech started at:\", event.audioStartMs);\n  },\n  onInputAudioBufferSpeechStopped: async (event, context) => {\n    console.log(\"Speech stopped at:\", event.audioEndMs);\n  },\n  \n  // Transcription events\n  onConversationItemInputAudioTranscriptionCompleted: async (event, context) => {\n    console.log(\"User said:\", event.transcript);\n  },\n  onConversationItemInputAudioTranscriptionDelta: async (event, context) => {\n    process.stdout.write(event.delta);\n  },\n  \n  // Response events\n  onResponseCreated: async (event, context) => {\n    console.log(\"Response started\");\n  },\n  onResponseDone: async (event, context) => {\n    console.log(\"Response complete\");\n  },\n  \n  // Streaming text\n  onResponseTextDelta: async (event, context) => {\n    process.stdout.write(event.delta);\n  },\n  onResponseTextDone: async (event, context) => {\n    console.log(\"\\n--- Text complete ---\");\n  },\n  \n  // Streaming audio\n  onResponseAudioDelta: async (event, context) => {\n    const audioData = event.delta;\n    playAudioChunk(audioData);\n  },\n  onResponseAudioDone: async (event, context) => {\n    console.log(\"Audio complete\");\n  },\n  \n  // Audio transcript (what assistant said)\n  onResponseAudioTranscriptDelta: async (event, context) => {\n    process.stdout.write(event.delta);\n  },\n  \n  // Function calling\n  onResponseFunctionCallArgumentsDone: async (event, context) => {\n    if (event.name === \"get_weather\") {\n      const args = JSON.parse(event.arguments);\n      const result = await getWeather(args.location);\n      \n      await session.addConversationItem({\n        type: \"function_call_output\",\n        callId: event.callId,\n        output: JSON.stringify(result),\n      });\n      \n      await session.sendEvent({ type: \"response.create\" });\n    }\n  },\n  \n  // Catch-all for debugging\n  onServerEvent: async (event, context) => {\n    console.log(\"Event:\", event.type);\n  },\n});\n\n// Clean up when done\nawait subscription.close();\n```\n\n## Function Calling\n\n```typescript\n// Define tools in session config\nawait session.updateSession({\n  modalities: [\"audio\", \"text\"],\n  instructions: \"Help users with weather information.\",\n  tools: [\n    {\n      type: \"function\",\n      name: \"get_weather\",\n      description: \"Get current weather for a location\",\n      parameters: {\n        type: \"object\",\n        properties: {\n          location: {\n            type: \"string\",\n            description: \"City and state or country\",\n          },\n        },\n        required: [\"location\"],\n      },\n    },\n  ],\n  toolChoice: \"auto\",\n});\n\n// Handle function calls\nconst subscription = session.subscribe({\n  onResponseFunctionCallArgumentsDone: async (event, context) => {\n    if (event.name === \"get_weather\") {\n      const args = JSON.parse(event.arguments);\n      const weatherData = await fetchWeather(args.location);\n      \n      // Send function result\n      await session.addConversationItem({\n        type: \"function_call_output\",\n        callId: event.callId,\n        output: JSON.stringify(weatherData),\n      });\n      \n      // Trigger response generation\n      await session.sendEvent({ type: \"response.create\" });\n    }\n  },\n});\n```\n\n## Voice Options\n\n| Voice Type | Config | Example |\n|------------|--------|---------|\n| Azure Standard | `{ type: \"azure-standard\", name: \"...\" }` | `\"en-US-AvaNeural\"` |\n| Azure Custom | `{ type: \"azure-custom\", name: \"...\", endpointId: \"...\" }` | Custom voice endpoint |\n| Azure Personal | `{ type: \"azure-personal\", speakerProfileId: \"...\" }` | Personal voice clone |\n| OpenAI | `{ type: \"openai\", name: \"...\" }` | `\"alloy\"`, `\"echo\"`, `\"shimmer\"` |\n\n## Supported Models\n\n| Model | Description | Use Case |\n|-------|-------------|----------|\n| `gpt-4o-realtime-preview` | GPT-4o with real-time audio | High-quality conversational AI |\n| `gpt-4o-mini-realtime-preview` | Lightweight GPT-4o | Fast, efficient interactions |\n| `phi4-mm-realtime` | Phi multimodal | Cost-effective applications |\n\n## Turn Detection Options\n\n```typescript\n// Server VAD (default)\nturnDetection: {\n  type: \"server_vad\",\n  threshold: 0.5,\n  prefixPaddingMs: 300,\n  silenceDurationMs: 500,\n}\n\n// Azure Semantic VAD (smarter detection)\nturnDetection: {\n  type: \"azure_semantic_vad\",\n}\n\n// Azure Semantic VAD (English optimized)\nturnDetection: {\n  type: \"azure_semantic_vad_en\",\n}\n\n// Azure Semantic VAD (Multilingual)\nturnDetection: {\n  type: \"azure_semantic_vad_multilingual\",\n}\n```\n\n## Audio Formats\n\n| Format | Sample Rate | Use Case |\n|--------|-------------|----------|\n| `pcm16` | 24kHz | Default, high quality |\n| `pcm16-8000hz` | 8kHz | Telephony |\n| `pcm16-16000hz` | 16kHz | Voice assistants |\n| `g711_ulaw` | 8kHz | Telephony (US) |\n| `g711_alaw` | 8kHz | Telephony (EU) |\n\n## Key Types Reference\n\n| Type | Purpose |\n|------|---------|\n| `VoiceLiveClient` | Main client for creating sessions |\n| `VoiceLiveSession` | Active WebSocket session |\n| `VoiceLiveSessionHandlers` | Event handler interface |\n| `VoiceLiveSubscription` | Active event subscription |\n| `ConnectionContext` | Context for connection events |\n| `SessionContext` | Context for session events |\n| `ServerEventUnion` | Union of all server events |\n\n## Error Handling\n\n```typescript\nimport {\n  VoiceLiveError,\n  VoiceLiveConnectionError,\n  VoiceLiveAuthenticationError,\n  VoiceLiveProtocolError,\n} from \"@azure/ai-voicelive\";\n\nconst subscription = session.subscribe({\n  onError: async (args, context) => {\n    const { error } = args;\n    \n    if (error instanceof VoiceLiveConnectionError) {\n      console.error(\"Connection error:\", error.message);\n    } else if (error instanceof VoiceLiveAuthenticationError) {\n      console.error(\"Auth error:\", error.message);\n    } else if (error instanceof VoiceLiveProtocolError) {\n      console.error(\"Protocol error:\", error.message);\n    }\n  },\n  \n  onServerError: async (event, context) => {\n    console.error(\"Server error:\", event.error?.message);\n  },\n});\n```\n\n## Logging\n\n```typescript\nimport { setLogLevel } from \"@azure/logger\";\n\n// Enable verbose logging\nsetLogLevel(\"info\");\n\n// Or via environment variable\n// AZURE_LOG_LEVEL=info\n```\n\n## Browser Usage\n\n```typescript\n// Browser requires bundler (Vite, webpack, etc.)\nimport { VoiceLiveClient } from \"@azure/ai-voicelive\";\nimport { InteractiveBrowserCredential } from \"@azure/identity\";\n\n// Use browser-compatible credential\nconst credential = new InteractiveBrowserCredential({\n  clientId: \"your-client-id\",\n  tenantId: \"your-tenant-id\",\n});\n\nconst client = new VoiceLiveClient(endpoint, credential);\n\n// Request microphone access\nconst stream = await navigator.mediaDevices.getUserMedia({ audio: true });\nconst audioContext = new AudioContext({ sampleRate: 24000 });\n\n// Process audio and send to session\n// ... (see samples for full implementation)\n```\n\n## Best Practices\n\n1. **Always use `DefaultAzureCredential`** — Never hardcode API keys\n2. **Set both modalities** — Include `[\"text\", \"audio\"]` for voice assistants\n3. **Use Azure Semantic VAD** — Better turn detection than basic server VAD\n4. **Handle all error types** — Connection, auth, and protocol errors\n5. **Clean up subscriptions** — Call `subscription.close()` when done\n6. **Use appropriate audio format** — PCM16 at 24kHz for best quality\n\n## Reference Links\n\n| Resource | URL |\n|----------|-----|\n| npm Package | https://www.npmjs.com/package/@azure/ai-voicelive |\n| GitHub Source | https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/ai/ai-voicelive |\n| Samples | https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/ai/ai-voicelive/samples |\n| API Reference | https://learn.microsoft.com/javascript/api/@azure/ai-voicelive |\n\n## When to Use\nThis skill is applicable to execute the workflow or actions described in the overview.",
  "applicable_domains": [
    "devops"
  ],
  "category": "devops",
  "invocation": [
    "/azure-ai-voicelive-ts"
  ],
  "authored_by": "claudeskills.in community",
  "source_url": "https://claudeskills.in/skill/azure-ai-voicelive-ts",
  "provenance": {
    "source": "claudeskills.in",
    "source_url": "https://claudeskills.in/skill/azure-ai-voicelive-ts",
    "license": "unknown",
    "imported_at": "2026-09-03",
    "notes": "Aggregated by claudeskills.in from community GitHub lists."
  },
  "tags": [
    "claudeskills",
    "devops"
  ],
  "lifecycle": "draft"
}