跳转到内容
搜索文档

与知识库对话

最后更新 查看 MarkdownAgent 设置

本教程将构建一个你可以与之对话的语音代理,它会从你的 AI Search 知识库中大声回答。它使用 Cloudflare Agents@cloudflare/voice 包实现语音管道,并将 AI Search 作为代理的知识库,以检索工具的形式暴露给代理模型调用。

你将构建: 一个语音代理,它会转录你的语音,调用 AI Search 从已索引的知识库中检索相关内容,生成有依据的回答,并语音播报。

工作原理

@cloudflare/voice 包为 Cloudflare Agent 添加完整的语音管道:语音转文本 (STT)、你在其中生成回复的 “turn” 处理程序,以及文本转语音 (TTS)。该管道在由 Durable Object 支持的单个 Worker 中运行,浏览器通过 WebSocket 连接到它。

你需要编写的唯一方法是 onTurn(),它接收用户的转录文本并返回要播报的文本。这就是 AI Search 的用武之地:你运行语言模型,并将 AI Search 作为检索工具提供给它。模型决定何时搜索你的知识库,依据返回的结果生成回复,并返回回答文本,再由管道播报。

BrowserMic
Workers AISpeech-to-text
Cloudflare AgentsonTurn(transcript)
Workers AIText-to-speech
BrowserSpeaker

前提条件

  1. 注册 Cloudflare 账户
  2. 安装 Node.js

Node.js 版本管理器

使用 Voltanvm 等 Node 版本管理器,以避免权限问题并切换 Node.js 版本。本指南后续将介绍的 Wrangler 需要 Node 版本 16.17.0 或更高。

你还需要一个已包含索引内容的 AI Search 实例。这就是你将与之对话的知识库。要创建实例并添加内容,请参阅 快速入门

1. 创建语音代理

使用语音入门模板搭建 Cloudflare Agents 项目,其中包含 Durable Object 接线与 React 客户端:

npm create cloudflare@latest voice-knowledge-base -- --template cloudflare/agents-starter
cd voice-knowledge-base

安装语音包:

npm i @cloudflare/voice

@cloudflare/voice 包提供 withVoice mixin 与 Workers AI 提供商(WorkersAIFluxSTTWorkersAITTS)。有关语音代理本身(包括浏览器客户端)的完整演练,请参阅 语音代理示例

2. 添加 AI Search 绑定

AI Search 绑定 添加到你的 Wrangler 配置文件,与 Workers AI 绑定以及代理的 Durable Object 并列。将 my-instance 替换为你的实例名称。

{
	"name": "voice-knowledge-base",
	"main": "src/server.ts",
	// Set this to today's date
	"compatibility_date": "2026-08-17",
	"compatibility_flags": ["nodejs_compat"],
	"ai": {
		"binding": "AI",
		"remote": true
	},
	"ai_search": [
		{
			"binding": "AI_SEARCH",
			"instance_name": "my-instance",
			"remote": true
		}
	],
	"durable_objects": {
		"bindings": [
			{
				"name": "TalkToDocs",
				"class_name": "TalkToDocs"
			}
		]
	},
	"migrations": [
		{
			"tag": "v1",
			"new_sqlite_classes": ["TalkToDocs"]
		}
	]
}
name = "voice-knowledge-base"
main = "src/server.ts"
# Set this to today's date
compatibility_date = "2026-08-17"
compatibility_flags = [ "nodejs_compat" ]

[ai]
binding = "AI"
remote = true

[[ai_search]]
binding = "AI_SEARCH"
instance_name = "my-instance"
remote = true

[[durable_objects.bindings]]
name = "TalkToDocs"
class_name = "TalkToDocs"

[[migrations]]
tag = "v1"
new_sqlite_classes = [ "TalkToDocs" ]

重新生成绑定类型,以便 env.AIenv.AI_SEARCH 具有类型:

npx wrangler types

3. 从知识库回答

更新 src/server.ts。使用 withVoice mixin 构建代理,设置 STT 与 TTS 提供商,并在 onTurn() 中运行将 AI Search 作为检索工具调用的 Workers AI 模型。

模型会获得一个调用 AI Search search() 进行检索的 searchKnowledgeBase 工具。当它需要来自知识库的事实时会调用该工具,依据返回的分块回答,你再返回生成的文本供管道播报。代理会自动存储对话历史,因此你可以传递 context.messages 以支持后续问题。

src/server.jsjs
import { Agent, routeAgentRequest } from "agents";
import { withVoice, WorkersAIFluxSTT, WorkersAITTS } from "@cloudflare/voice";
import { generateText, tool, stepCountIs } from "ai";
import { createWorkersAI } from "workers-ai-provider";
import { z } from "zod";

const VoiceAgent = withVoice(Agent);

export class TalkToDocs extends VoiceAgent {
	// Workers AI powers speech-to-text and text-to-speech (no API keys needed).
	transcriber = new WorkersAIFluxSTT(this.env.AI);
	tts = new WorkersAITTS(this.env.AI);

	// Called each time the user finishes speaking. The agent's model generates
	// the reply, calling AI Search as a retrieval tool when it needs facts from
	// your knowledge base.
	async onTurn(transcript, context) {
		const workersai = createWorkersAI({ binding: this.env.AI });

		const result = await generateText({
			// Use a Workers AI model that supports function calling.
			model: workersai("@cf/zai-org/glm-5.2"),
			system:
				"You are a helpful voice assistant that answers from a Cloudflare AI Search knowledge base. " +
				"For questions about the product, call the searchKnowledgeBase tool first and answer using the results. " +
				"Skip the tool for greetings and small talk. Keep replies short and conversational.",
			messages: [
				...context.messages.map((message) => ({
					role: message.role,
					content: message.content,
				})),
				{ role: "user", content: transcript },
			],
			tools: {
				searchKnowledgeBase: tool({
					description:
						"Search the knowledge base for information to answer the user's question.",
					inputSchema: z.object({
						query: z.string().describe("A focused search query"),
					}),
					execute: async ({ query }) => {
						// search() runs retrieval only and returns the matching chunks.
						const res = await this.env.AI_SEARCH.search({
							query,
							ai_search_options: { retrieval: { max_num_results: 5 } },
						});
						return res.chunks.map((chunk) => chunk.text).join("\n\n");
					},
				}),
			},
			// Let the model call the tool, then answer from the results.
			stopWhen: stepCountIs(4),
		});

		// Return the generated answer for the pipeline to speak.
		return result.text;
	}
}

export default {
	async fetch(request, env) {
		return (
			(await routeAgentRequest(request, env)) ??
			new Response("Not found", { status: 404 })
		);
	},
};
src/server.tsts
import { Agent, routeAgentRequest } from "agents";
import {
	withVoice,
	WorkersAIFluxSTT,
	WorkersAITTS,
	type VoiceTurnContext,
} from "@cloudflare/voice";
import { generateText, tool, stepCountIs } from "ai";
import { createWorkersAI } from "workers-ai-provider";
import { z } from "zod";

const VoiceAgent = withVoice(Agent);

export class TalkToDocs extends VoiceAgent<Env> {
	// Workers AI powers speech-to-text and text-to-speech (no API keys needed).
	transcriber = new WorkersAIFluxSTT(this.env.AI);
	tts = new WorkersAITTS(this.env.AI);

	// Called each time the user finishes speaking. The agent's model generates
	// the reply, calling AI Search as a retrieval tool when it needs facts from
	// your knowledge base.
	async onTurn(transcript: string, context: VoiceTurnContext) {
		const workersai = createWorkersAI({ binding: this.env.AI });

		const result = await generateText({
			// Use a Workers AI model that supports function calling.
			model: workersai("@cf/zai-org/glm-5.2"),
			system:
				"You are a helpful voice assistant that answers from a Cloudflare AI Search knowledge base. " +
				"For questions about the product, call the searchKnowledgeBase tool first and answer using the results. " +
				"Skip the tool for greetings and small talk. Keep replies short and conversational.",
			messages: [
				...context.messages.map((message) => ({
					role: message.role as "user" | "assistant",
					content: message.content,
				})),
				{ role: "user" as const, content: transcript },
			],
			tools: {
				searchKnowledgeBase: tool({
					description:
						"Search the knowledge base for information to answer the user's question.",
					inputSchema: z.object({
						query: z.string().describe("A focused search query"),
					}),
					execute: async ({ query }) => {
						// search() runs retrieval only and returns the matching chunks.
						const res = await this.env.AI_SEARCH.search({
							query,
							ai_search_options: { retrieval: { max_num_results: 5 } },
						});
						return res.chunks.map((chunk) => chunk.text).join("\n\n");
					},
				}),
			},
			// Let the model call the tool, then answer from the results.
			stopWhen: stepCountIs(4),
		});

		// Return the generated answer for the pipeline to speak.
		return result.text;
	}
}

export default {
	async fetch(request: Request, env: Env) {
		return (
			(await routeAgentRequest(request, env)) ??
			new Response("Not found", { status: 404 })
		);
	},
} satisfies ExportedHandler<Env>;

search() 返回的每个分块都包含其来源条目与相关性分数,因此你可以展示引用或记录代理检索到的内容。

4. 构建客户端

src/client.tsx 替换为使用 useVoiceAgent hook 的 React 组件。该 hook 管理麦克风、到代理的 WebSocket 连接、音频播放与打断检测,因此组件只需渲染控件。将 agent 设为你的代理类名 TalkToDocs

src/client.tsxtsx
import { useVoiceAgent } from "@cloudflare/voice/react";

function App() {
	// useVoiceAgent connects to your agent over WebSocket, captures the
	// microphone, plays the spoken response, and exposes the live call state.
	// `agent` matches your Durable Object class name.
	const {
		// Pipeline state: "idle" | "listening" | "thinking" | "speaking".
		status,
		// Finalized conversation turns (your speech and the agent's replies).
		transcript,
		// Live partial transcription of what you are currently saying.
		interimTranscript,
		// Whether the WebSocket connection to the agent is open.
		connected,
		startCall,
		endCall,
		toggleMute,
		isMuted,
	} = useVoiceAgent({ agent: "TalkToDocs" });

	const inCall = status !== "idle";

	return (
		<div>
			<h1>Talk to your knowledge base</h1>

			{/* Shows "thinking" while the agent searches the KB and generates a reply. */}
			<p>Status: {status}</p>

			{/* Toggle the call. Disabled until the agent connection is open. */}
			<button
				onClick={inCall ? endCall : startCall}
				disabled={!connected && !inCall}
			>
				{!connected ? "Connecting…" : inCall ? "End call" : "Start call"}
			</button>

			{/* Mute only applies once a call is active. */}
			{inCall && (
				<button onClick={toggleMute}>{isMuted ? "Unmute" : "Mute"}</button>
			)}

			{/* Lightweight loading state while the agent works on a reply. */}
			{status === "thinking" && <p>Thinking…</p>}

			{/* Live partial transcript, updated as you speak. */}
			{interimTranscript && (
				<p>
					<em>{interimTranscript}</em>
				</p>
			)}

			{/* Finalized turns from both you and the agent. */}
			{transcript.map((message, index) => (
				<p key={index}>
					<strong>{message.role}:</strong> {message.text}
				</p>
			))}
		</div>
	);
}

export default App;

该 hook 处理麦克风与播放,因此没有按键通话按钮。模型会检测你何时说完话,运行 onTurn(),并自动播放口语回答。

5. 本地运行

启动本地开发服务器:

npm run dev

在浏览器中打开应用,选择 Start call(开始通话) 并允许麦克风访问,然后提出你的内容能够回答的问题。你会实时看到自己的话语被转录,代理会从知识库中语音回答。工作过程中,status 值会经过 listeningthinkingspeaking

6. 部署

部署你的代理,使其在互联网上可用:

npx wrangler deploy

与多人一起与知识库对话

本教程构建的是单用户语音代理。如果你需要多人在实时房间中一起与知识库对话,请使用 RealtimeKit 作为多方音视频层,并保留此语音代理作为从 AI Search 回答的组件。RealtimeKit 提供会议室与转录,但不托管回答引擎。

后续步骤

这篇文档对您有帮助吗?