SAI Agent
Self-hosted real-time voice AI agents
Put an AI agent into any SRTC call or phone line. Plug in any ASR / LLM / TTS stack or a speech-to-speech realtime model, get replies in about a second, run it on your own servers, and ship voice agents for support, outbound calling, interviews or telehealth.
SAI Agent is the real-time AI conversation engine built on SRTC: your agent joins an SRTC channel as a server-side participant and talks with people on web, mobile or desktop in real time. It is built on the SRTC Python SDK, publicly available on PyPI (pip install srtc) with a built-in pipecat transport, so you can plug in any ASR / LLM / TTS combination or a realtime speech model such as Gemini Live, OpenAI Realtime, Qwen Omni or Doubao, with replies in about a second. Through the Room Connector’s SIP access, the same agent can answer and place phone calls for voice support and outbound campaigns. Media servers and models can all run inside your own network.
At a glance
| Item | Details |
|---|---|
| SDK | SRTC Python SDK: pip install srtc; pipecat: pip install "srtc[pipecat]" |
| Models | Cascaded: any ASR → LLM → TTS. Speech-to-speech: Gemini Live / OpenAI Realtime / Qwen Omni / Doubao |
| How it connects | The agent joins an SRTC channel as a server-side participant |
| Audio | In: per-speaker PCM frames. Out: PCM at any sample rate, resampled and paced by the SDK |
| Barge-in | Unplayed audio is flushed as soon as the user interrupts |
| AI response | About a second, interruptible |
| Media latency | <400ms end-to-end (excluding model inference) |
| Phone calls | Inbound and outbound calls, including AI outbound, via SIP on the Room Connector |
| Deployment | Self-hosted: SRTC servers, agent processes and private models can all stay on your network |
Pluggable AI pipeline
# pip install "srtc[pipecat]"
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineTask
from srtc.pipecat_transport import SRTCParams, SRTCTransport
# token comes from your server; stt / llm / tts can be any pipecat service
transport = SRTCTransport(
token,
SRTCParams(audio_in_enabled=True, audio_out_enabled=True),
)
pipeline = Pipeline([
transport.input(),
stt,
context_aggregator.user(),
llm,
tts,
transport.output(),
context_aggregator.assistant(),
])
await PipelineRunner().run(PipelineTask(pipeline))Features
Real-time voice
Ultra-low-latency SRTC audio with replies in about a second, per-speaker frames, barge-in and multi-turn conversation.
Video input
Receive user video frames (optionally decoded to RGB) and feed them to a multimodal model.
Bring your own models
Mix and match ASR / LLM / TTS vendors, or swap in a realtime speech model, without changing the transport.
pipecat transport
Replace the Daily / LiveKit transport in any pipecat example with SRTCTransport; the rest of the pipeline stays the same.
Inbound & outbound phone calls
Connect phone lines over SIP through the Room Connector for AI phone support and outbound calling.
Conversation analytics
Call recordings, transcripts, sentiment analysis and service-quality scoring for QA.
Self-hosted
Run everything inside your network so conversation data never leaves it.
Any language your models speak
Language support follows the ASR / TTS you choose.
Use cases
Voice support agents
Phone support | Web chat | IVR
24/7 voice support that takes SIP phone calls, answers routine questions and hands off to humans.
AI interviewers
Screening | Follow-ups | Reports
Multi-turn interviews with follow-up questions, live scoring and a written report.
Telehealth triage
Triage | Health Q&A | Medication
A self-hosted intake assistant that keeps patient data on your servers.
Digital humans
Avatars | Virtual hosts | Live
Drive an avatar in real time with LLM + TTS for virtual hosts and digital staff.
Ready to build?
Get a self-hosted trial, SDK access and hands-on help from our engineers