使用 Core SDK 在 RealtimeKit 会议中为参与者的视频源添加视频背景特效和模糊。
npm i @cloudflare/realtimekit-virtual-backgroundyarn add @cloudflare/realtimekit-virtual-backgroundpnpm add @cloudflare/realtimekit-virtual-backgroundbun add @cloudflare/realtimekit-virtual-background禁用视频中间件的默认每帧渲染，通过让该中间件自行控制来提高速度和质量：
await meeting.self.setVideoMiddlewareGlobalConfig({
disablePerFrameCanvasRendering: true,
});创建视频背景转换器对象：
import RealtimeKitVideoBackgroundTransformer from "@cloudflare/realtimekit-virtual-background";
const videoBackgroundTransformer =
await RealtimeKitVideoBackgroundTransformer.init({
meeting,
});videoBackgroundTransformer 提供了两种类型的中间件：
使用 createStaticBackgroundVideoMiddleware 将图片设置为背景：
const imageUrl = "https://images.unsplash.com/photo-1487088678257-3a541e6e3922";
meeting.self.addVideoMiddleware(
await videoBackgroundTransformer.createStaticBackgroundVideoMiddleware(
imageUrl,
),
);使用 createBackgroundBlurVideoMiddleware 来模糊背景。传递 blurStrength（0-100）作为参数（默认值为 50%）：
meeting.self.addVideoMiddleware(
await videoBackgroundTransformer.createBackgroundBlurVideoMiddleware(50),
);在初始化之前检查浏览器支持：
if (RealtimeKitVideoBackgroundTransformer.isSupported()) {
const videoBackgroundTransformer =
await RealtimeKitVideoBackgroundTransformer.init({
meeting: meeting,
});
meeting.self.addVideoMiddleware(
await videoBackgroundTransformer.createStaticBackgroundVideoMiddleware(
imageUrl,
),
);
}为了获得更好、更清晰的效果，请传递自定义的分段配置：
const videoBackgroundTransformer =
await RealtimeKitVideoBackgroundTransformer.init({
meeting,
segmentationConfig: {
model: "mlkit", // 'meet' | 'mlkit'
backend: "wasmSimd",
inputResolution: "256x256", // '256x144' for meet
pipeline: "webgl2", // 'webgl2' | 'canvas2dCpu'
// canvas2dCpu gives sharper blur, webgl2 is faster
targetFps: 35,
},
});npm i @cloudflare/realtimekit-virtual-backgroundyarn add @cloudflare/realtimekit-virtual-backgroundpnpm add @cloudflare/realtimekit-virtual-backgroundbun add @cloudflare/realtimekit-virtual-backgroundimport { useState, useEffect } from "react";
import { useRealtimeKitClient } from "@cloudflare/realtimekit-react";
import RealtimeKitVideoBackgroundTransformer from "@cloudflare/realtimekit-virtual-background";
function App() {
const [meeting] = useRealtimeKitClient();
const [videoBackgroundTransformer, setVideoBackgroundTransformer] =
useState(null);
useEffect(() => {
const initializeTransformer = async () => {
if (!meeting) return;
// Check browser support
if (!RealtimeKitVideoBackgroundTransformer.isSupported()) {
console.warn("Video background not supported in this browser");
return;
}
// Disable default per frame rendering
await meeting.self.setVideoMiddlewareGlobalConfig({
disablePerFrameCanvasRendering: true,
});
// Initialize transformer
const transformer = await RealtimeKitVideoBackgroundTransformer.init({
meeting,
});
setVideoBackgroundTransformer(transformer);
};
initializeTransformer();
}, [meeting]);
const applyStaticBackground = async (imageUrl) => {
if (!videoBackgroundTransformer) return;
meeting.self.addVideoMiddleware(
await videoBackgroundTransformer.createStaticBackgroundVideoMiddleware(
imageUrl,
),
);
};
const applyBlur = async (blurStrength = 50) => {
if (!videoBackgroundTransformer) return;
meeting.self.addVideoMiddleware(
await videoBackgroundTransformer.createBackgroundBlurVideoMiddleware(
blurStrength,
),
);
};
const removeBackground = () => {
// Remove all video middlewares
meeting.self.removeVideoMiddleware();
};
return (
<div>
<button
onClick={() =>
applyStaticBackground(
"https://images.unsplash.com/photo-1487088678257-3a541e6e3922",
)
}
>
Apply Background
</button>
<button onClick={() => applyBlur(50)}>Apply Blur</button>
<button onClick={removeBackground}>Remove Background</button>
</div>
);
}为了获得更好、更清晰的效果，请传递自定义的分段配置：
const transformer = await RealtimeKitVideoBackgroundTransformer.init({
meeting,
segmentationConfig: {
model: "mlkit", // 'meet' | 'mlkit'
backend: "wasmSimd",
inputResolution: "256x256", // '256x144' for meet
pipeline: "webgl2", // 'webgl2' | 'canvas2dCpu'
// canvas2dCpu gives sharper blur, webgl2 is faster
targetFps: 35,
},
});npm i @cloudflare/realtimekit-virtual-backgroundyarn add @cloudflare/realtimekit-virtual-backgroundpnpm add @cloudflare/realtimekit-virtual-backgroundbun add @cloudflare/realtimekit-virtual-background在您的组件 TypeScript 文件中：
import { Component, OnInit } from "@angular/core";
import RealtimeKitClient from "@cloudflare/realtimekit";
import RealtimeKitVideoBackgroundTransformer from "@cloudflare/realtimekit-virtual-background";
@Component({
selector: "app-meeting",
templateUrl: "./meeting.component.html",
})
export class MeetingComponent implements OnInit {
meeting: any;
videoBackgroundTransformer: any;
async ngOnInit() {
// Initialize meeting
this.meeting = await RealtimeKitClient.init({
authToken: "<participant_auth_token>",
});
await this.meeting.join();
// Check browser support
if (!RealtimeKitVideoBackgroundTransformer.isSupported()) {
console.warn("Video background not supported in this browser");
return;
}
// Disable default per frame rendering
await this.meeting.self.setVideoMiddlewareGlobalConfig({
disablePerFrameCanvasRendering: true,
});
// Initialize transformer
this.videoBackgroundTransformer =
await RealtimeKitVideoBackgroundTransformer.init({
meeting: this.meeting,
});
}
async applyStaticBackground(imageUrl: string) {
if (!this.videoBackgroundTransformer) return;
this.meeting.self.addVideoMiddleware(
await this.videoBackgroundTransformer.createStaticBackgroundVideoMiddleware(
imageUrl,
),
);
}
async applyBlur(blurStrength: number = 50) {
if (!this.videoBackgroundTransformer) return;
this.meeting.self.addVideoMiddleware(
await this.videoBackgroundTransformer.createBackgroundBlurVideoMiddleware(
blurStrength,
),
);
}
removeBackground() {
// Remove all video middlewares
this.meeting.self.removeVideoMiddleware();
}
}在您的组件模板中：
<button
(click)="applyStaticBackground('https://images.unsplash.com/photo-1487088678257-3a541e6e3922')"
>
Apply Background
</button>
<button (click)="applyBlur(50)">Apply Blur</button>
<button (click)="removeBackground()">Remove Background</button>为了获得更好、更清晰的效果，请传递自定义的分段配置：
this.videoBackgroundTransformer =
await RealtimeKitVideoBackgroundTransformer.init({
meeting: this.meeting,
segmentationConfig: {
model: "mlkit", // 'meet' | 'mlkit'
backend: "wasmSimd",
inputResolution: "256x256", // '256x144' for meet
pipeline: "webgl2", // 'webgl2' | 'canvas2dCpu'
// canvas2dCpu gives sharper blur, webgl2 is faster
targetFps: 35,
},
});您可以通过在 build.gradle 文件中添加以下依赖项，将预先打包的滤镜添加到您的项目中：
dependencies {
// (other dependencies)
implementation 'com.cloudflare.realtimekit:filters:0.1.0'
}该包目前提供了 VirtualBackgroundVideoFilter，可与 FilterVideoProcessor 配合使用：
// Create a virtual background filter with a custom background image.
val bgFilter = VirtualBackgroundVideoFilter(context, R.drawable.background)
// Initialize the video processor with the filter.
val processor = FilterVideoProcessor(eglBase, bgFilter)
// // Set the video processor on the meeting builder.
val meeting = RealtimeKitMeetingBuilder
.setVideoProcessor(eglBase, processor)
.build(activity)您还可以创建自己的自定义滤镜，以直接向实时视频流应用特效、滤镜或分析。我们的 VideoProcessor API 提供了灵活且强大的方法来操作视频帧。
我们提供三种类型的视频处理器：
NoDropVideoProcessor：允许自定义视频处理而不丢帧。ChainVideoProcessor：将多个帧处理器链接在一起，这对于向视频流应用多个特效或滤镜非常有用。FilterVideoProcessor：向视频流应用单个特效或滤镜的更简单、更高效的方法。
尽管如此，您也可以通过直接实现 VideoProcessor 接口来创建自己的自定义视频处理器：
import realtimekit.org.webrtc.VideoFrame
import realtimekit.org.webrtc.VideoProcessor
import realtimekit.org.webrtc.VideoSink
class CustomVideoProcessor : VideoProcessor {
override fun onCapturerStarted(started: Boolean) {}
override fun onCapturerStopped() {}
override fun onFrameCaptured(frame: VideoFrame?) {}
override fun setSink(sink: VideoSink?) {}
}创建并配置 VideoProcessor 后，将其传递给 RealtimeKitMeetingBuilder 对象。这将在摄像头捕获的视频帧发送给其他参与者或进行本地渲染之前对其进行处理：
// Assuming 'myCustomProcessor' is an instance of any VideoProcessor implementation
// (for example, ChainVideoProcessor, FilterVideoProcessor, and more).
val myCustomProcessor = CustomProcessor()
// Set the video processor on the meeting builder.
val meeting = RealtimeKitMeetingBuilder
.setVideoProcessor(processor = myCustomProcessor)
.build(activity)
// You can also pass an EglBase to the builder
// This is useful when using FilterVideoProcessor
val eglBase = EglBase.create()
val meeting = RealtimeKitMeetingBuilder
.setVideoProcessor(eglBase = eglBase, processor = myCustomProcessor)
.build(activity)