goal c – iOS Digicam Freezes and Crashes with MLKit Barcode Scanning in Kotlin Multiplatform

I am creating an app utilizing Kotlin Multiplatform and Googles MLKit for barcode scanning. I am unable to create a VisionImage within the captureOutput technique of a AVCaptureVideoDataOutputSampleBufferDelegate. When making an attempt, I’m encountering periodic freezes each 15 frames for 10 seconds.

In my CameraViewController (iosMain)

class CameraViewController(
    personal val onQRCodeDetected: (String) -> Unit
) : UIViewController(null, null), AVCaptureVideoDataOutputSampleBufferDelegateProtocol {

I overwrite captureOutput

    override enjoyable captureOutput(
        output: AVCaptureOutput,
        didOutputSampleBuffer: CMSampleBufferRef?,
        fromConnection: AVCaptureConnection
    ) {        
        frameCounter+=1
        logging("CViewC").i { "FRAME #$frameCounter" }

up to now every little thing works good. captureOutput will get known as every body in time and logs the framecount. After I add the next line issues break.

val visionImage = MLKVisionImage(didOutputSampleBuffer)

Now the primary 15 calles of captureOutput are okay. Throughout this length the preview on the telephone works fluent. Then the preview freezes and captureOutput shouldn’t be known as. This lasts for 10 seconds. After the ten seconds I obtain once more 15 frames usually. This cicle goes on and the app by no means crashes.

After I add now the road

CFRelease(didOutputSampleBuffer)

I get a steady circulation of 288 frames till a crash. The crash report signifies that I attempted to launch an already launched buffer(Detected over-release of a CFTypeRef %p (%lu / %s)).

I attempted to repeat the Buffer:

val copiedBufferVar = nativeHeap.alloc()
CMSampleBufferCreateCopy(
    allocator = kCFAllocatorDefault,
    sbuf = didOutputSampleBuffer,
    sampleBufferOut = copiedBufferVar.ptr
)
val visionImage = MLKVisionImage(didOutputSampleBuffer.worth)
CFRelease(copiedBufferVar.worth)

Since I assumed the Buffers weren’t launched from reminiscence due to the sturdy reference of the VisionImage to the Buffer. (I do not know if that is sensible – I desperately tried to make sense of one thing). This resulted in the identical factor. A cycle of 15 frames OK and 10 Seconds freeze.

I attempted to launch the VisionImage from reminiscence by making it nullable and setting it to null after utilization. This additionally change nothing. LLM’s beneficial utilizing a autoreleasepool however that additionally did not assist.

So I assume it’s a reminiscence downside. I believed my AVCaptureSession takes care of these things routinely. I configured it like this. I eliminated some null checks to maintain it shorter right here.

@OptIn(ExperimentalForeignApi::class)
personal enjoyable setupCaptureSession() {

    captureSession = AVCaptureSession().apply {
        beginConfiguration()
        sessionPreset = AVCaptureSessionPreset1280x720
    }

    val system = AVCaptureDevice.defaultDeviceWithMediaType(AVMediaTypeVideo)
    val enter = AVCaptureDeviceInput.deviceInputWithDevice(system, null)
    if (captureSession?.canAddInput(enter) == true) {
        captureSession?.addInput(enter)
    }
    videoOutput = AVCaptureVideoDataOutput().apply {
        setSampleBufferDelegate(this@CameraViewController, processingQueue)
        videoSettings = mapOf(
            kCVPixelBufferPixelFormatTypeKey to NSNumber(kCVPixelFormatType_32BGRA) 
        )
        alwaysDiscardsLateVideoFrames = true

    }
    if (captureSession?.canAddOutput(videoOutput!!) == true) {
        captureSession?.addOutput(videoOutput!!)
    }
    captureSession?.commitConfiguration()
    previewLayer = AVCaptureVideoPreviewLayer(session = captureSession!!).apply {
        videoGravity = AVLayerVideoGravityResizeAspectFill
    }
}