本教程将构建一个你可以与之对话的语音代理,它会从你的 AI Search 知识库中大声回答。它使用 Cloudflare Agents 的 @cloudflare/voice 包实现语音管道,并将 AI Search 作为代理的知识库,以检索工具的形式暴露给代理模型调用。
你将构建: 一个语音代理,它会转录你的语音,调用 AI Search 从已索引的知识库中检索相关内容,生成有依据的回答,并语音播报。
@cloudflare/voice 包为 Cloudflare Agent 添加完整的语音管道:语音转文本 (STT)、你在其中生成回复的 “turn” 处理程序,以及文本转语音 (TTS)。该管道在由 Durable Object 支持的单个 Worker 中运行,浏览器通过 WebSocket 连接到它。
你需要编写的唯一方法是 onTurn(),它接收用户的转录文本并返回要播报的文本。这就是 AI Search 的用武之地:你运行语言模型,并将 AI Search 作为检索工具提供给它。模型决定何时搜索你的知识库,依据返回的结果生成回复,并返回回答文本,再由管道播报。
- 注册 Cloudflare 账户 ↗。
- 安装
Node.js↗。
Node.js 版本管理器
使用 Volta ↗ 或 nvm ↗ 等 Node 版本管理器,以避免权限问题并切换 Node.js 版本。本指南后续将介绍的 Wrangler 需要 Node 版本 16.17.0 或更高。
你还需要一个已包含索引内容的 AI Search 实例。这就是你将与之对话的知识库。要创建实例并添加内容,请参阅 快速入门。
使用语音入门模板搭建 Cloudflare Agents 项目,其中包含 Durable Object 接线与 React 客户端:
npm create cloudflare@latest voice-knowledge-base -- --template cloudflare/agents-starter
cd voice-knowledge-base安装语音包:
npm i @cloudflare/voiceyarn add @cloudflare/voicepnpm add @cloudflare/voicebun add @cloudflare/voice@cloudflare/voice 包提供 withVoice mixin 与 Workers AI 提供商(WorkersAIFluxSTT 与 WorkersAITTS)。有关语音代理本身(包括浏览器客户端)的完整演练,请参阅 语音代理示例。
将 AI Search 绑定 添加到你的 Wrangler 配置文件,与 Workers AI 绑定以及代理的 Durable Object 并列。将 my-instance 替换为你的实例名称。
{
"name": "voice-knowledge-base",
"main": "src/server.ts",
// Set this to today's date
"compatibility_date": "2026-08-17",
"compatibility_flags": ["nodejs_compat"],
"ai": {
"binding": "AI",
"remote": true
},
"ai_search": [
{
"binding": "AI_SEARCH",
"instance_name": "my-instance",
"remote": true
}
],
"durable_objects": {
"bindings": [
{
"name": "TalkToDocs",
"class_name": "TalkToDocs"
}
]
},
"migrations": [
{
"tag": "v1",
"new_sqlite_classes": ["TalkToDocs"]
}
]
}name = "voice-knowledge-base"
main = "src/server.ts"
# Set this to today's date
compatibility_date = "2026-08-17"
compatibility_flags = [ "nodejs_compat" ]
[ai]
binding = "AI"
remote = true
[[ai_search]]
binding = "AI_SEARCH"
instance_name = "my-instance"
remote = true
[[durable_objects.bindings]]
name = "TalkToDocs"
class_name = "TalkToDocs"
[[migrations]]
tag = "v1"
new_sqlite_classes = [ "TalkToDocs" ]重新生成绑定类型,以便 env.AI 与 env.AI_SEARCH 具有类型:
npx wrangler typesyarn wrangler typespnpm wrangler types更新 src/server.ts。使用 withVoice mixin 构建代理,设置 STT 与 TTS 提供商,并在 onTurn() 中运行将 AI Search 作为检索工具调用的 Workers AI 模型。
模型会获得一个调用 AI Search search() 进行检索的 searchKnowledgeBase 工具。当它需要来自知识库的事实时会调用该工具,依据返回的分块回答,你再返回生成的文本供管道播报。代理会自动存储对话历史,因此你可以传递 context.messages 以支持后续问题。
import { Agent, routeAgentRequest } from "agents";
import { withVoice, WorkersAIFluxSTT, WorkersAITTS } from "@cloudflare/voice";
import { generateText, tool, stepCountIs } from "ai";
import { createWorkersAI } from "workers-ai-provider";
import { z } from "zod";
const VoiceAgent = withVoice(Agent);
export class TalkToDocs extends VoiceAgent {
// Workers AI powers speech-to-text and text-to-speech (no API keys needed).
transcriber = new WorkersAIFluxSTT(this.env.AI);
tts = new WorkersAITTS(this.env.AI);
// Called each time the user finishes speaking. The agent's model generates
// the reply, calling AI Search as a retrieval tool when it needs facts from
// your knowledge base.
async onTurn(transcript, context) {
const workersai = createWorkersAI({ binding: this.env.AI });
const result = await generateText({
// Use a Workers AI model that supports function calling.
model: workersai("@cf/zai-org/glm-5.2"),
system:
"You are a helpful voice assistant that answers from a Cloudflare AI Search knowledge base. " +
"For questions about the product, call the searchKnowledgeBase tool first and answer using the results. " +
"Skip the tool for greetings and small talk. Keep replies short and conversational.",
messages: [
...context.messages.map((message) => ({
role: message.role,
content: message.content,
})),
{ role: "user", content: transcript },
],
tools: {
searchKnowledgeBase: tool({
description:
"Search the knowledge base for information to answer the user's question.",
inputSchema: z.object({
query: z.string().describe("A focused search query"),
}),
execute: async ({ query }) => {
// search() runs retrieval only and returns the matching chunks.
const res = await this.env.AI_SEARCH.search({
query,
ai_search_options: { retrieval: { max_num_results: 5 } },
});
return res.chunks.map((chunk) => chunk.text).join("\n\n");
},
}),
},
// Let the model call the tool, then answer from the results.
stopWhen: stepCountIs(4),
});
// Return the generated answer for the pipeline to speak.
return result.text;
}
}
export default {
async fetch(request, env) {
return (
(await routeAgentRequest(request, env)) ??
new Response("Not found", { status: 404 })
);
},
};import { Agent, routeAgentRequest } from "agents";
import {
withVoice,
WorkersAIFluxSTT,
WorkersAITTS,
type VoiceTurnContext,
} from "@cloudflare/voice";
import { generateText, tool, stepCountIs } from "ai";
import { createWorkersAI } from "workers-ai-provider";
import { z } from "zod";
const VoiceAgent = withVoice(Agent);
export class TalkToDocs extends VoiceAgent<Env> {
// Workers AI powers speech-to-text and text-to-speech (no API keys needed).
transcriber = new WorkersAIFluxSTT(this.env.AI);
tts = new WorkersAITTS(this.env.AI);
// Called each time the user finishes speaking. The agent's model generates
// the reply, calling AI Search as a retrieval tool when it needs facts from
// your knowledge base.
async onTurn(transcript: string, context: VoiceTurnContext) {
const workersai = createWorkersAI({ binding: this.env.AI });
const result = await generateText({
// Use a Workers AI model that supports function calling.
model: workersai("@cf/zai-org/glm-5.2"),
system:
"You are a helpful voice assistant that answers from a Cloudflare AI Search knowledge base. " +
"For questions about the product, call the searchKnowledgeBase tool first and answer using the results. " +
"Skip the tool for greetings and small talk. Keep replies short and conversational.",
messages: [
...context.messages.map((message) => ({
role: message.role as "user" | "assistant",
content: message.content,
})),
{ role: "user" as const, content: transcript },
],
tools: {
searchKnowledgeBase: tool({
description:
"Search the knowledge base for information to answer the user's question.",
inputSchema: z.object({
query: z.string().describe("A focused search query"),
}),
execute: async ({ query }) => {
// search() runs retrieval only and returns the matching chunks.
const res = await this.env.AI_SEARCH.search({
query,
ai_search_options: { retrieval: { max_num_results: 5 } },
});
return res.chunks.map((chunk) => chunk.text).join("\n\n");
},
}),
},
// Let the model call the tool, then answer from the results.
stopWhen: stepCountIs(4),
});
// Return the generated answer for the pipeline to speak.
return result.text;
}
}
export default {
async fetch(request: Request, env: Env) {
return (
(await routeAgentRequest(request, env)) ??
new Response("Not found", { status: 404 })
);
},
} satisfies ExportedHandler<Env>;search() 返回的每个分块都包含其来源条目与相关性分数,因此你可以展示引用或记录代理检索到的内容。
将 src/client.tsx 替换为使用 useVoiceAgent hook 的 React 组件。该 hook 管理麦克风、到代理的 WebSocket 连接、音频播放与打断检测,因此组件只需渲染控件。将 agent 设为你的代理类名 TalkToDocs。
import { useVoiceAgent } from "@cloudflare/voice/react";
function App() {
// useVoiceAgent connects to your agent over WebSocket, captures the
// microphone, plays the spoken response, and exposes the live call state.
// `agent` matches your Durable Object class name.
const {
// Pipeline state: "idle" | "listening" | "thinking" | "speaking".
status,
// Finalized conversation turns (your speech and the agent's replies).
transcript,
// Live partial transcription of what you are currently saying.
interimTranscript,
// Whether the WebSocket connection to the agent is open.
connected,
startCall,
endCall,
toggleMute,
isMuted,
} = useVoiceAgent({ agent: "TalkToDocs" });
const inCall = status !== "idle";
return (
<div>
<h1>Talk to your knowledge base</h1>
{/* Shows "thinking" while the agent searches the KB and generates a reply. */}
<p>Status: {status}</p>
{/* Toggle the call. Disabled until the agent connection is open. */}
<button
onClick={inCall ? endCall : startCall}
disabled={!connected && !inCall}
>
{!connected ? "Connecting…" : inCall ? "End call" : "Start call"}
</button>
{/* Mute only applies once a call is active. */}
{inCall && (
<button onClick={toggleMute}>{isMuted ? "Unmute" : "Mute"}</button>
)}
{/* Lightweight loading state while the agent works on a reply. */}
{status === "thinking" && <p>Thinking…</p>}
{/* Live partial transcript, updated as you speak. */}
{interimTranscript && (
<p>
<em>{interimTranscript}</em>
</p>
)}
{/* Finalized turns from both you and the agent. */}
{transcript.map((message, index) => (
<p key={index}>
<strong>{message.role}:</strong> {message.text}
</p>
))}
</div>
);
}
export default App;该 hook 处理麦克风与播放,因此没有按键通话按钮。模型会检测你何时说完话,运行 onTurn(),并自动播放口语回答。
启动本地开发服务器:
npm run devyarn run devpnpm run dev在浏览器中打开应用,选择 Start call(开始通话) 并允许麦克风访问,然后提出你的内容能够回答的问题。你会实时看到自己的话语被转录,代理会从知识库中语音回答。工作过程中,status 值会经过 listening、thinking 与 speaking。
部署你的代理,使其在互联网上可用:
npx wrangler deployyarn wrangler deploypnpm wrangler deploy本教程构建的是单用户语音代理。如果你需要多人在实时房间中一起与知识库对话,请使用 RealtimeKit 作为多方音视频层,并保留此语音代理作为从 AI Search 回答的组件。RealtimeKit 提供会议室与转录,但不托管回答引擎。