Voice Agent 示例

构建 voice agent：聆听用户、用 LLM 思考并实时通过 WebSocket 语音回复。 Beta

本指南结束时你将拥有：

带 speech-to-text 与 text-to-speech 的服务端 voice agent
流式响应的 LLM 驱动 onTurn handler
对话期间 agent 可调用的 tool
带 push-to-talk 风格 UI 的 React client

前置条件

具备 Workers AI 访问权限的 Cloudflare 账户
Node.js 18+

1. 创建项目

用 Vite 与 React 脚手架新建 Workers 项目，然后添加 voice 依赖：

npm create cloudflare@latest voice-agent -- --template cloudflare/agents-starter
cd voice-agent
npm install @cloudflare/voice

Starter 提供可用的 Vite + React + Cloudflare Workers 设置。后续步骤将替换 server 与 client 代码。

2. 配置 wrangler

更新 wrangler.jsonc，包含 Workers AI binding 与 voice agent 的 Durable Object：

{
	"name": "voice-agent",
	// Set this to today's date
	"compatibility_date": "2026-08-17",
	"compatibility_flags": ["nodejs_compat"],
	"main": "src/server.ts",
	"ai": {
		"binding": "AI"
	},
	"durable_objects": {
		"bindings": [
			{
				"name": "MyVoiceAgent",
				"class_name": "MyVoiceAgent"
			}
		]
	},
	"migrations": [
		{
			"tag": "v1",
			"new_sqlite_classes": ["MyVoiceAgent"]
		}
	]
}

name = "voice-agent"
# Set this to today's date
compatibility_date = "2026-08-17"
compatibility_flags = [ "nodejs_compat" ]
main = "src/server.ts"

[ai]
binding = "AI"

[[durable_objects.bindings]]
name = "MyVoiceAgent"
class_name = "MyVoiceAgent"

[[migrations]]
tag = "v1"
new_sqlite_classes = [ "MyVoiceAgent" ]

3. 构建 server

将 src/server.ts 替换为以下内容。withVoice mixin 为 standard Agent 类添加完整 voice pipeline — STT、句子分块、TTS 与对话持久化。

import { Agent, routeAgentRequest } from "agents";
import { withVoice, WorkersAIFluxSTT, WorkersAITTS } from "@cloudflare/voice";
import { streamText, tool, stepCountIs } from "ai";
import { createWorkersAI } from "workers-ai-provider";
import { z } from "zod";

const VoiceAgent = withVoice(Agent);

export class MyVoiceAgent extends VoiceAgent {
	transcriber = new WorkersAIFluxSTT(this.env.AI);
	tts = new WorkersAITTS(this.env.AI);

	async onTurn(transcript, context) {
		const workersAi = createWorkersAI({ binding: this.env.AI });

		const result = streamText({
			model: workersAi("@cf/moonshotai/kimi-k2.6"),
			system:
				"You are a helpful voice assistant. Keep responses concise — you are being spoken aloud.",
			messages: [
				...context.messages.map((m) => ({
					role: m.role,
					content: m.content,
				})),
				{ role: "user", content: transcript },
			],
			tools: {
				get_current_time: tool({
					description: "Get the current date and time.",
					inputSchema: z.object({}),
					execute: async () => ({
						time: new Date().toLocaleTimeString("en-US", {
							hour: "2-digit",
							minute: "2-digit",
						}),
					}),
				}),
			},
			stopWhen: stepCountIs(3),
			abortSignal: context.signal,
		});

		return result.textStream;
	}

	async onCallStart(connection) {
		await this.speak(connection, "Hi there! How can I help you today?");
	}
}

export default {
	async fetch(request, env) {
		return (
			(await routeAgentRequest(request, env)) ??
			new Response("Not found", { status: 404 })
		);
	},
};

import { Agent, routeAgentRequest, type Connection } from "agents";
import {
	withVoice,
	WorkersAIFluxSTT,
	WorkersAITTS,
	type VoiceTurnContext,
} from "@cloudflare/voice";
import { streamText, tool, stepCountIs } from "ai";
import { createWorkersAI } from "workers-ai-provider";
import { z } from "zod";

const VoiceAgent = withVoice(Agent);

export class MyVoiceAgent extends VoiceAgent<Env> {
	transcriber = new WorkersAIFluxSTT(this.env.AI);
	tts = new WorkersAITTS(this.env.AI);

	async onTurn(transcript: string, context: VoiceTurnContext) {
		const workersAi = createWorkersAI({ binding: this.env.AI });

		const result = streamText({
			model: workersAi("@cf/moonshotai/kimi-k2.6"),
			system:
				"You are a helpful voice assistant. Keep responses concise — you are being spoken aloud.",
			messages: [
				...context.messages.map((m) => ({
					role: m.role as "user" | "assistant",
					content: m.content,
				})),
				{ role: "user" as const, content: transcript },
			],
			tools: {
				get_current_time: tool({
					description: "Get the current date and time.",
					inputSchema: z.object({}),
					execute: async () => ({
						time: new Date().toLocaleTimeString("en-US", {
							hour: "2-digit",
							minute: "2-digit",
						}),
					}),
				}),
			},
			stopWhen: stepCountIs(3),
			abortSignal: context.signal,
		});

		return result.textStream;
	}

	async onCallStart(connection: Connection) {
		await this.speak(connection, "Hi there! How can I help you today?");
	}
}

export default {
	async fetch(request: Request, env: Env) {
		return (
			(await routeAgentRequest(request, env)) ??
			new Response("Not found", { status: 404 })
		);
	},
} satisfies ExportedHandler<Env>;

要点：

WorkersAIFluxSTT 处理 continuous speech-to-text — model 检测用户何时说完。
WorkersAITTS 将 LLM 响应逐句转为 audio。
onTurn 接收 transcript 并返回 stream。Mixin 将 stream 分句并合成每一句。
onCallStart 在用户连接时发送问候。
context.messages 含 SQLite 中的完整对话历史。
用户 interrupt 或 disconnect 时 context.signal 会被 abort。

4. 构建 client

将 src/client.tsx 替换为使用 useVoiceAgent hook 的 React 组件。Hook 管理 WebSocket 连接、mic 采集、audio 播放与 interrupt 检测。

import { useVoiceAgent } from "@cloudflare/voice/react";

function App() {
	const {
		status,
		transcript,
		interimTranscript,
		metrics,
		audioLevel,
		isMuted,
		startCall,
		endCall,
		toggleMute,
	} = useVoiceAgent({ agent: "MyVoiceAgent" });

	return (
		<div>
			<h1>Voice Agent</h1>
			<p>Status: {status}</p>

			<div>
				<button onClick={status === "idle" ? startCall : endCall}>
					{status === "idle" ? "Start Call" : "End Call"}
				</button>
				{status !== "idle" && (
					<button onClick={toggleMute}>{isMuted ? "Unmute" : "Mute"}</button>
				)}
			</div>

			{interimTranscript && (
				<p>
					<em>{interimTranscript}</em>
				</p>
			)}

			{transcript.map((msg, i) => (
				<p key={i}>
					<strong>{msg.role}:</strong> {msg.text}
				</p>
			))}

			{metrics && (
				<p>
					LLM: {metrics.llm_ms}ms | TTS: {metrics.tts_ms}ms | First audio:{" "}
					{metrics.first_audio_ms}ms
				</p>
			)}
		</div>
	);
}

status 字段循环 "idle" → "listening" → "thinking" → "speaking" → "listening"，足以构建响应式 UI。

5. 运行

npm run dev

在浏览器打开应用，选择 Start Call（开始通话） 并说话。你将实时看到 transcript，agent 回复会通过扬声器播放。

添加 pipeline hook

可在 pipeline 各阶段 intercept 并 transform 数据。例如过滤短 transcript（噪声）并在 TTS 前调整发音：

export class MyVoiceAgent extends VoiceAgent {
	transcriber = new WorkersAIFluxSTT(this.env.AI);
	tts = new WorkersAITTS(this.env.AI);

	afterTranscribe(transcript, connection) {
		if (transcript.length < 3) return null;
		return transcript;
	}

	beforeSynthesize(text, connection) {
		return text.replace(/\bAI\b/g, "A.I.");
	}

	async onTurn(transcript, context) {
		return "You said: " + transcript;
	}
}

export class MyVoiceAgent extends VoiceAgent<Env> {
	transcriber = new WorkersAIFluxSTT(this.env.AI);
	tts = new WorkersAITTS(this.env.AI);

	afterTranscribe(transcript: string, connection: Connection) {
		if (transcript.length < 3) return null;
		return transcript;
	}

	beforeSynthesize(text: string, connection: Connection) {
		return text.replace(/\bAI\b/g, "A.I.");
	}

	async onTurn(transcript: string, context: VoiceTurnContext) {
		return "You said: " + transcript;
	}
}

从 afterTranscribe 返回 null 会完全丢弃该 utterance — 适用于过滤噪声或过短 transcript。

使用第三方 provider

更换第三方 STT 或 TTS provider 而无需改动 agent 逻辑：

import { ElevenLabsTTS } from "@cloudflare/voice-elevenlabs";
import { DeepgramSTT } from "@cloudflare/voice-deepgram";

export class MyVoiceAgent extends VoiceAgent {
	transcriber = new DeepgramSTT({
		apiKey: this.env.DEEPGRAM_API_KEY,
	});

	tts = new ElevenLabsTTS({
		apiKey: this.env.ELEVENLABS_API_KEY,
		voiceId: "21m00Tcm4TlvDq8ikWAM",
	});

	async onTurn(transcript, context) {
		return "You said: " + transcript;
	}
}

import { ElevenLabsTTS } from "@cloudflare/voice-elevenlabs";
import { DeepgramSTT } from "@cloudflare/voice-deepgram";

export class MyVoiceAgent extends VoiceAgent<Env> {
	transcriber = new DeepgramSTT({
		apiKey: this.env.DEEPGRAM_API_KEY,
	});

	tts = new ElevenLabsTTS({
		apiKey: this.env.ELEVENLABS_API_KEY,
		voiceId: "21m00Tcm4TlvDq8ikWAM",
	});

	async onTurn(transcript: string, context: VoiceTurnContext) {
		return "You said: " + transcript;
	}
}

后续步骤

Voice Agent API 参考

withVoice、withVoiceInput、React hook、VoiceClient 与所有 provider 的完整参考。

聊天 Agent

用 AIChatAgent 与 useAgentChat 构建文本 AI 聊天。

使用 AI 模型

在 agent 中使用 Workers AI、OpenAI、Anthropic、Gemini 或任意 provider。