4.6 C
New York
Friday, November 29, 2024

Inconsistent UI Updates with CameraPermissionManager in iOS on Compose Multiplatform


I am engaged on a Compose Multiplatform challenge the place I’ve a display screen applied in commonMain that makes use of a CameraPermissionManager class, which is platform-specific. The display screen works effective on Android, however I am working into points on iOS the place the UI is not constantly updating primarily based on state modifications.

Here is a short overview of my setup:
In commonMain:

I’ve a Composable operate App() that units up the UI. It features a ReadTextScreen composable the place the digital camera preview is began and the detected textual content is processed.

@Composable
enjoyable App() {
    SosMilkTheme {
        ReadTextScreen()
    }
}

@Composable
personal enjoyable ReadTextScreen() {
    val cameraPermissionManager = bear in mind { CameraPermissionManager() }
    val snackBarHostState = bear in mind { SnackbarHostState() }
    var milkCondition by bear in mind { mutableStateOf(null) }
    var detected by rememberSaveable { mutableStateOf("") }

    cameraPermissionManager.RequestCameraPermission(
        onPermissionGranted = {
            Scaffold(
                snackbarHost = {
                    SnackbarHost(hostState = snackBarHostState)
                },
                topBar = { /* ... */ },
                bottomBar = {
                    FooterDetection(
                        title = "Detected Phrases:",
                        desc = detected,
                    )
                }
            ) { innerPadding ->
                Field {
                    cameraPermissionManager.StartCameraPreview { detectedText ->
                        val res = processDetectedTextUseCase.invoke(detectedText)
                        detected = detectedText
                        milkCondition = res
                    }
                    // different UI components...
                }
            }
        },
        onPermissionDenied = {
            BodyMedium(textual content = "Digicam entry is required.")
        }
    )
}

In iosMain:

My CameraPermissionManager class is accountable for dealing with digital camera permissions and beginning the digital camera preview. The textual content detection makes use of Apple’s Imaginative and prescient framework.

@OptIn(ExperimentalForeignApi::class)
precise class CameraPermissionManager {

    @Composable
    precise enjoyable RequestCameraPermission(
        onPermissionGranted: @Composable () -> Unit,
        onPermissionDenied: @Composable () -> Unit
    ) {
        // Permission dealing with logic
    }

@OptIn(ExperimentalForeignApi::class)
@Composable
precise enjoyable StartCameraPreview(onTextDetected: (String) -> Unit) {
    val system = AVCaptureDevice.devicesWithMediaType(AVMediaTypeVideo)
        .firstOrNull { system -> (system as AVCaptureDevice).place == AVCaptureDevicePositionBack } as? AVCaptureDevice

    if (system == null)  No digital camera discovered")
        return
    

    val enter = AVCaptureDeviceInput.deviceInputWithDevice(system, null)
    if (enter == null) 

    val videoOutput = AVCaptureVideoDataOutput()

    videoOutput.alwaysDiscardsLateVideoFrames = true
    videoOutput.videoSettings = mapOf("PixelFormatType" to kCVPixelFormatType_32BGRA)

    val session = AVCaptureSession()

    session.sessionPreset = AVCaptureSessionPresetHigh
    session.addInput(enter)
    session.addOutput(videoOutput)

    val cameraPreviewLayer = bear in mind { AVCaptureVideoPreviewLayer(session = session) }

    UIKitView(
        modifier = Modifier.fillMaxSize(),
        background = Coloration.Black,
        manufacturing unit = {
            val container = UIView()
            container.layer.addSublayer(cameraPreviewLayer)
            cameraPreviewLayer.videoGravity = AVLayerVideoGravityResizeAspectFill
            cameraPreviewLayer.body = container.bounds
            dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT.toLong(), 0u)) {
                session.startRunning()
            }
            container
        },
        onResize = { container: UIView, rect: CValue ->
            CATransaction.start()
            CATransaction.setValue(true, kCATransactionDisableActions)
            container.layer.setFrame(rect)
            cameraPreviewLayer.setFrame(rect)
            CATransaction.commit()
        })

    val delegate = object : NSObject(), AVCaptureVideoDataOutputSampleBufferDelegateProtocol {
        override enjoyable captureOutput(
            output: AVCaptureOutput,
            didOutputSampleBuffer: CMSampleBufferRef?,
            fromConnection: AVCaptureConnection
        ) {
            didOutputSampleBuffer?.let { processSampleBuffer(it, onTextDetected) }
        }
    }

    videoOutput.setSampleBufferDelegate(delegate, dispatch_get_main_queue())
}

personal enjoyable processSampleBuffer(
    sampleBuffer: CMSampleBufferRef,
    onTextDetected: (String) -> Unit
) {
    val pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)

    if (pixelBuffer == null)  Pixel buffer is null")
        return
    

    val handler = VNImageRequestHandler(pixelBuffer, choices = mapOf())

    val request = VNRecognizeTextRequest { request, error ->
        if (error != null) 

        val observations = request?.outcomes?.filterIsInstance()

        observations?.forEach { statement ->
            val topCandidates = statement.topCandidates(1u)
            topCandidates.firstOrNull()?.let { candidate ->
                val recognizedText = candidate as? VNRecognizedText
                recognizedText?.string?.let { textual content ->
                    onTextDetected(textual content)
                } ?: run  Acknowledged textual content is null")
                
            }
        }
    }

    attempt {
        handler.performRequests(listOf(request), null)
    } catch (e: Exception) 
}

The Drawback:
On iOS, the textual content detected by the digital camera (detectedText) isn’t reliably updating the UI components just like the FooterDetection and the milkCondition. Nevertheless, I can see the proper values being logged, so the textual content is being detected correctly. The difficulty appears to be associated to how the UI state updates are dealt with, probably because of threading.

Potential Trigger:

I am conscious that on iOS, UI updates must happen on the primary thread. I’ve already wrapped the onTextDetected callback in dispatch_async(dispatch_get_main_queue()) to make sure it is working on the primary thread. Regardless of this, the UI updates are nonetheless inconsistent, and I am not sure if there’s one thing else I am lacking associated to string dealing with or Compose-specific conduct on iOS.

Query:
Has anybody else encountered comparable points with Compose Multiplatform on iOS? Is there one thing particular to Compose on iOS that I would like to contemplate for making certain constant UI updates? Any steerage or options can be significantly appreciated!

Thanks prematurely to your assist!

Related Articles

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Latest Articles