The Virtual Background component for Playcanvas adds AI automatic background removal with the use of Mediapipe and green screen Chromakey rendering mixed with background images.
The component takes video or image elements as a source input. Or image and video assets as textures configuring a texture directly.
The only other input required is the meshInstance supplying a material to add a canvas display texture to and a background image for mixing with the source and the mask.
The feature uses efficient background GPU rendering for mixing sources with the Mediapipe mask and background image. And applied to the main canvas texture using efficient bitmap rendering where supported.
Safari Ventura and below cannot support GPU contexts with offscreen canvas tasks so a normal canvas can be used but with efficient bitmap rendering.
Features
- Mediapipe API masking of video and image textures.
- Chromekey render support for video.
- Takes video and image element sources directly.
- Takes image assets loaded as textures using the AssetListLoader.
- Efficient WebGL GPU mixing of video, mask and images.
- Efficient render of background task to the main display canvas.
- Conveniently load the component as a script asset.