ios – Implement PIP mode for my agora video name app

Unable to Use Native PiP Mode with AVSampleBufferDisplayLayer in Agora iOS SDK
I’m engaged on an Agora Dwell Streaming characteristic in an iOS Swift app, the place I’m utilizing AVSampleBufferDisplayLayer to render Agora’s onRenderVideoFrame delegate frames. The stay video stream works properly within the foreground, however I’m unable to allow native Image-in-Image (PiP) mode when the app goes into the background.

What I’ve finished thus far:

1. Agora Package Initialization

   I’ve initialized Agora with the next configurations:

   agoraKit.setParameters("{"che.video.keep_last_frame": true}")
   agoraKit.setVideoFrameDelegate(self)
   agoraKit.setParameters("{"che.video.force_renderer_background": true}")
   agoraKit.enableVideo()
   agoraKit.setVideoFrameDelegate(self)
   agoraKit.enableLocalVideo(true)
   agoraKit.setChannelProfile(.liveBroadcasting)
   agoraKit.setClientRole(.broadcaster)
   // agoraKit.enableWebSdkInteroperability(true)
   agoraKit.setParameters("{"rtc.video.pip": true}")
   agoraKit.setParameters("{"rtc.video.background_mode": 1}")
   agoraKit.setParameters("{"rtc.video.enable_ios_background_mode": true}")
   

2. Processing Video Frames in onRenderVideoFrame
   I’m processing the incoming frames and rendering them utilizing AVSampleBufferDisplayLayer.

   non-public var videoLayerRemote: AVSampleBufferDisplayLayer?

   func onRenderVideoFrame(_ videoFrame: AgoraOutputVideoFrame, uid: UInt, channelId: String) -> Bool {
       frameCount += 1
   
       // Ignore native video frames
       if uid == localUid {
           return true
       }
   
       var sampleBuffer: CMSampleBuffer?
   
       if videoFrame.sort == 1 { // I420 Format
           guard let yBuffer = videoFrame.yBuffer,
                 let uBuffer = videoFrame.uBuffer,
                 let vBuffer = videoFrame.vBuffer else {
               return true
           }
   
           sampleBuffer = convertI420ToCMSampleBuffer(yPlane: yBuffer,
                                                      uPlane: uBuffer,
                                                      vPlane: vBuffer,
                                                      width: Int(videoFrame.width),
                                                      peak: Int(videoFrame.peak))
       } else {
           guard let pixelBuffer = videoFrame.pixelBuffer else {
               return true
           }
           sampleBuffer = createSampleBuffer(from: pixelBuffer)
       }
   
       guard let videoLayer = videoLayerRemote else {
           return true
       }
   
       if videoLayer.standing == .failed {
           videoLayer.flush()
       }
   
       if !videoLayer.isReadyForMoreMediaData {
           return true
       }
   
       videoLayer.enqueue(sampleBuffer!)
   
       return true
   }
   

3. PiP Configuration
   I’ve finished the next to allow Image-in-Image (PiP):

   Enabled Background Modes:

   • Audio, AirPlay, and Image in Image
   • Background Processing
   • Background Fetch
   • Multiprocessing
     Added Functionality for Multi-Digital camera Entry

Tried Utilizing AVPictureInPictureController with AVSampleBufferDisplayLayer, however no success
Subject
When the app goes into the background:

The video stops rendering.
PiP mode doesn’t activate.
The app doesn’t show the AVSampleBufferDisplayLayer when within the background.

I need to:

Preserve AVSampleBufferDisplayLayer lively when the app goes into the background.
Allow native iOS Image-in-Image (PiP) mode.
Robotically conceal AVSampleBufferDisplayLayer when the app involves the foreground.

What I’ve tried

Utilizing AVPictureInPictureController

Since AVSampleBufferDisplayLayer doesn’t immediately assist PiP, I attempted wrapping it in a AVPlayerLayer, however this didn’t work.
Utilizing Agora setParameters(“{“rtc.video.pip”: true}”)

No impact on enabling PiP mode.

Dealing with app state transitions

I attempted dealing with UIApplication.didEnterBackgroundNotification to manually configure PiP however could not get AVSampleBufferDisplayLayer to show.

Questions

How can I maintain rendering the Agora video stream utilizing AVSampleBufferDisplayLayer within the background?

How can I allow native Image-in-Image (PiP) mode utilizing AVSampleBufferDisplayLayer in an Agora iOS app?splay it when the app goes in background and conceal when app is available in foreground