Talk to an AI Assistant over WebSocket
Real-time voice conversation with a Telnyx AI Assistant. The client streams PCM16 audio to Telnyx and receives lifecycle, transcription, and assistant-audio frames in return.
Authentication
Requires authentication via a Bearer token (Telnyx API v2 key).
{
"type": "input_audio_buffer.append",
"audio": "eW91ciBiYXNlNjQgZW5jb2RlZCBwY20xNiBhdWRpbw=="
}{
"type": "conversation.item.create",
"item": {
"type": "message",
"role": "user",
"content": [
{
"type": "input_text",
"text": "What are your business hours?"
}
]
}
}{
"type": "response.cancel",
"response_id": "resp_9c1f4a2b"
}{
"type": "session.created",
"session": {
"conversation_id": "3E6F995F-85F7-4705-9741-53B116D28237",
"assistant_id": "assistant-0f4e8b2a",
"audio": {
"input": {
"format": {
"type": "audio/pcm",
"rate": 16000
},
"turn_detection": {
"type": "server_vad"
}
},
"output": {
"format": {
"type": "audio/pcm",
"rate": 24000
}
}
}
}
}{
"type": "input_audio_buffer.speech_started"
}{
"type": "input_audio_buffer.speech_stopped"
}{
"type": "conversation.item.input_audio_transcription.completed",
"transcript": "What are your business hours?"
}{
"type": "response.created",
"response": {
"id": "resp_9c1f4a2b"
}
}{
"type": "response.output_audio.delta",
"response_id": "resp_9c1f4a2b",
"item_id": "item_5d2e7f10",
"delta": "eW91ciBiYXNlNjQgZW5jb2RlZCBwY20xNiBhdWRpbw=="
}{
"type": "response.output_audio_transcript.delta",
"response_id": "resp_9c1f4a2b",
"item_id": "item_5d2e7f10",
"delta": "We are open from 9am "
}{
"type": "response.output_audio.done",
"response_id": "resp_9c1f4a2b"
}{
"type": "response.done",
"response": {
"id": "resp_9c1f4a2b",
"status": "completed"
}
}{
"type": "conversation.item.created",
"item": {
"type": "function_call",
"call_id": "call_4b8e12a7",
"name": "get_weather",
"arguments": "{\"city\":\"Chicago\"}"
}
}{
"type": "response.tool_call.started",
"tool_call": {
"id": "call_7a3f21b8",
"name": "get_business_hours",
"arguments": "{\"location\":\"downtown\"}"
}
}{
"type": "response.tool_call.completed",
"tool_call": {
"id": "call_7a3f21b8",
"name": "get_business_hours"
},
"status": "success"
}{
"type": "error",
"error": {
"code": "invalid_audio",
"message": "Invalid base64-encoded audio payload"
}
}Identifier of the AI Assistant to converse with.
Telnyx API v2 Bearer token authentication.
Query parameters passed when opening the WebSocket connection.
Client-to-Telnyx frame carrying a chunk of microphone audio.
The audio field is a base64-encoded PCM16 payload at the sample rate requested with the input_sample_rate query parameter. Frames larger than 1 MiB are rejected with a frame_too_large error.
Client-to-Telnyx frame that adds a conversation item. Two item types are supported: a user message with input_text content (injects a completed user turn, which the assistant answers exactly as it would a spoken one), and a function_call_output that returns the result of a client-side tool function_call previously requested by Telnyx. There is no separate frame to request a response — the assistant owns turn-taking. An item of any other shape is rejected with an invalid_item error.
Client-to-Telnyx frame that cancels the in-progress assistant response (client-initiated barge-in without audio).
Include response_id to target a specific response; omit it to cancel the current one. Cancelling flushes playback and Telnyx emits a response.done frame with status cancelled. Cancelling a stale or already-finished response is a no-op.
First frame sent by Telnyx once the conversation has started. Confirms the session and the negotiated input and output audio formats.
Sent when server-side voice-activity detection determines the user has started speaking. Edge-triggered.
Sent when server-side voice-activity detection determines the user has stopped speaking. Edge-triggered, and never sent without a preceding speech-started.
Transcript of the audio the user streamed for the current turn.
Marks the start of an assistant turn. The response.id correlates the audio and transcript deltas that follow.
A chunk of the assistant''s synthesized speech, as base64-encoded PCM16 at the output rate announced in session.created.
A chunk of the text transcript of the assistant''s spoken response, streamed alongside the audio deltas.
Sent when the assistant has finished streaming audio for the turn.
Marks the end of an assistant turn, including its final status.
Telnyx-to-client frame requesting a client-side tool call.
When the assistant invokes a client_side_tool, Telnyx sends a function_call item that the client executes locally. The client returns the result with a conversation.item.create frame carrying a function_call_output item that references the same call_id. The assistant continues once the result arrives, or after the tool times out.
Informational frame sent when the assistant invokes a server-side tool (for example a webhook or MCP integration). Telnyx executes tools server-side, so these frames are observational only: the client displays tool activity but does not execute or respond to the call.
Informational frame sent when a server-side tool invocation finishes, with its outcome.
Sent when an error occurs. Some errors are non-fatal and leave the session open; others close the connection after the frame is sent.
Was this page helpful?
{
"type": "input_audio_buffer.append",
"audio": "eW91ciBiYXNlNjQgZW5jb2RlZCBwY20xNiBhdWRpbw=="
}{
"type": "conversation.item.create",
"item": {
"type": "message",
"role": "user",
"content": [
{
"type": "input_text",
"text": "What are your business hours?"
}
]
}
}{
"type": "response.cancel",
"response_id": "resp_9c1f4a2b"
}{
"type": "session.created",
"session": {
"conversation_id": "3E6F995F-85F7-4705-9741-53B116D28237",
"assistant_id": "assistant-0f4e8b2a",
"audio": {
"input": {
"format": {
"type": "audio/pcm",
"rate": 16000
},
"turn_detection": {
"type": "server_vad"
}
},
"output": {
"format": {
"type": "audio/pcm",
"rate": 24000
}
}
}
}
}{
"type": "input_audio_buffer.speech_started"
}{
"type": "input_audio_buffer.speech_stopped"
}{
"type": "conversation.item.input_audio_transcription.completed",
"transcript": "What are your business hours?"
}{
"type": "response.created",
"response": {
"id": "resp_9c1f4a2b"
}
}{
"type": "response.output_audio.delta",
"response_id": "resp_9c1f4a2b",
"item_id": "item_5d2e7f10",
"delta": "eW91ciBiYXNlNjQgZW5jb2RlZCBwY20xNiBhdWRpbw=="
}{
"type": "response.output_audio_transcript.delta",
"response_id": "resp_9c1f4a2b",
"item_id": "item_5d2e7f10",
"delta": "We are open from 9am "
}{
"type": "response.output_audio.done",
"response_id": "resp_9c1f4a2b"
}{
"type": "response.done",
"response": {
"id": "resp_9c1f4a2b",
"status": "completed"
}
}{
"type": "conversation.item.created",
"item": {
"type": "function_call",
"call_id": "call_4b8e12a7",
"name": "get_weather",
"arguments": "{\"city\":\"Chicago\"}"
}
}{
"type": "response.tool_call.started",
"tool_call": {
"id": "call_7a3f21b8",
"name": "get_business_hours",
"arguments": "{\"location\":\"downtown\"}"
}
}{
"type": "response.tool_call.completed",
"tool_call": {
"id": "call_7a3f21b8",
"name": "get_business_hours"
},
"status": "success"
}{
"type": "error",
"error": {
"code": "invalid_audio",
"message": "Invalid base64-encoded audio payload"
}
}