Speech-to-speech models

The speech-to-speech models available to Telnyx AI Assistants, GPT Live and NemotronLabs VoiceChat, what each one needs, and how to configure them with delegation for tools.

A speech-to-speech model hears the caller and speaks its answer in one model, rather than through separate transcription, language model, and speech synthesis stages. Telnyx AI Assistants support two speech-to-speech models:

• GPT Live, OpenAI’s speech-to-speech model, used with your OpenAI API key.
• NemotronLabs VoiceChat (beta), NVIDIA’s NemotronLabs VoiceChat 11B, hosted by Telnyx.

Neither model calls the assistant’s tools itself. For lookups and actions, keep delegation enabled: the speaking model handles the conversation, and a backend model or a server you host performs the work and returns the result.

Requirements at a glance

	GPT Live	NemotronLabs VoiceChat (beta)
model	openai/gpt-live-1	nvidia/NVIDIA-NemotronLabs-VoiceChat-11B
Provider API key	An OpenAI API key stored as an integration secret, set in llm_api_key_ref	None
voice_settings.voice	An OpenAILive. voice, such as OpenAILive.marin	NemotronVoiceChat.Aria
Languages	Multilingual	English only
Tools	Through delegation	Through delegation
Conversation flows	Not supported	Not supported

In both cases the model and voice must be paired: a speech-to-speech model with a voice from another provider is rejected when the assistant is saved.

GPT Live

GPT Live runs on OpenAI with an OpenAI API key you store as an integration secret. It supports OpenAILive. voices and Telnyx avatars. The vision tool works through a delegation model that accepts images; this is currently available in the development environment.

See GPT Live voice assistants for the full configuration, delegation behavior, and verification steps.

NemotronLabs VoiceChat

NemotronLabs VoiceChat is in beta.

NemotronLabs VoiceChat is NVIDIA’s NemotronLabs VoiceChat 11B speech-to-speech model, hosted by Telnyx. It needs no provider API key. It speaks English, in one voice, NemotronVoiceChat.Aria.

Configure the speaking model

Set these fields on the assistant:

Field	Configuration
model	nvidia/NVIDIA-NemotronLabs-VoiceChat-11B
voice_settings.voice	NemotronVoiceChat.Aria. Required.
enabled_features	Include "telephony" to enable calls.
greeting	The opening message. NemotronLabs VoiceChat speaks it as written when the conversation starts, then listens.
delegation_settings	The backend configuration for lookups and actions. Set enabled: true, mode: "telnyx", and an available backend model. If you omit delegation_settings when creating the assistant, a Telnyx-hosted backend model is set for you.

Send this body to Create an assistant, POST /v2/ai/assistants:

{
  "name": "VoiceChat support assistant",
  "enabled_features": ["telephony"],
  "model": "nvidia/NVIDIA-NemotronLabs-VoiceChat-11B",
  "instructions": "Help callers with order questions. Ask for the order number before a lookup. Never invent an order status or claim an action succeeded without a result.",
  "greeting": "Hi, I'm your order support assistant. How can I help you today?",
  "voice_settings": {
    "voice": "NemotronVoiceChat.Aria"
  },
  "delegation_settings": {
    "enabled": true,
    "mode": "telnyx",
    "model": "zai-org/GLM-5.3-Flash",
    "instructions": "Use the configured order lookup tool to retrieve order status. If no lookup tool is available or the lookup fails, explain that the status cannot be verified.",
    "speak_results": true
  }
}

The example selects zai-org/GLM-5.3-Flash as the delegation backend. Confirm that this model is available to your account, or replace it with another supported backend model. Attach a shared lookup tool using the tools library before testing order lookups.

Voice

NemotronLabs VoiceChat has one voice, NemotronVoiceChat.Aria, a female voice with an American English accent. List available voices returns it with provider: "nemotronvoicechat".

Tools and delegation

NemotronLabs VoiceChat hands a request that needs a lookup or action to the delegation backend, then speaks the result. When the result takes more than a moment, it first tells the caller it is checking. The backend runs the assistant’s tools. The delegation guide covers the settings, telnyx and client modes, and result handling.

Questions about what the caller shows on camera go to the backend the same way when the assistant has the vision tool and a delegation model that accepts images.

Post-conversation processing runs on the delegation backend, so it requires delegation enabled in telnyx mode with a backend model.

Limits

• Language: English only.
• Instructions: about 2,500 tokens, roughly 10,000 characters of plain English. Longer instructions are accepted when the assistant is saved, but the model then reads only their beginning and end, and leaves out the middle.
• Conversation flows and fallback models: not supported.

Pricing

NemotronLabs VoiceChat conversations cost $0.04 USD per minute.

Verify the assistant

Use the assistant ID with the voice assistant calling setup or the realtime voice conversation WebSocket, then:

1. Confirm the assistant speaks its configured greeting, then ask a conversational question and verify the response uses the NemotronVoiceChat.Aria voice.
2. Ask for information that requires a configured tool. Confirm the backend performs the lookup and the assistant speaks the returned result without inventing data.

Next steps

• GPT Live voice assistants — GPT Live configuration in full
• Delegation — backend settings, spoken results, and the client event contract
• Tools library — attach tools for delegated lookups and actions