ios – Add Anchor to ARView with assist of CoreML and VisionKit

I am utilizing RealityKit's ARView to develop a prototype.
The target of the prototype is to scan totally different totally different merchandise organized in several racks in a shelf (assume that any provision retailer the place chips are positioned in an enormous shelf and organized in racks contained in the shelf).
Now we have developed a .mlpackage, which can present the predictions of merchandise and its related bounding field coordinates for the given enter picture. We used ARSessionDelegate delegate’s
func session(_ session: ARSession, didUpdate body: ARFrame) methodology to seize the stay preview.

func session(_ session: ARSession, didUpdate body: ARFrame) {
    guard let pixelBuffer = body.capturedImage as CVPixelBuffer? else { return }
    DispatchQueue.international(qos: .userInitiated).async {
        do {
            let handler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, orientation: .proper, choices: [:])
            attempt handler.carry out([self.visionRequest])
        } catch {
            print("error in classify fame error (error)")
        }
    }
}

we’re utilizing VisionKit’s VNImageRequestHandler to course of the requests like under.

lazy var visionRequest: VNCoreMLRequest = {
        let request = VNCoreMLRequest(mannequin: mlModel, completionHandler: {
            [weak self] request, error in
          DispatchQueue.primary.async {
              self?.processPredictions(predictions: request.outcomes as? [VNRecognizedObjectObservation] ?? [])
          }
        })
        return request
}()

Right here what we wished to attain is that, when the predictions given by visionRequest, we name processPredictions methodology to course of. We anticipated so as to add a anchor to every of the prediction receievd from visionRequest. Following is the code carried out to attain the identical. convertToWorldPosition is a helpful methodology which makes use of the hitTest methodology of ARView to supply the world remodel vector for given ( which is nothing however a centroid of the every prediction bounding field.)

func processPredictions(predictions: [VNRecognizedObjectObservation]) {
        print("depend (predictions.depend)")
        overlayLayer.sublayers?.forEach { $0.removeFromSuperlayer() }
        for prediction in predictions {
                let boundingBox = prediction.boundingBox
                let centroid = CGPoint(
                    x: boundingBox.origin.x + boundingBox.width / 2,
                    y: boundingBox.origin.y + boundingBox.top / 2
                )
                
            if let worldTransform = convertToWorldPosition(from: centroid, body: self.arView.session.currentFrame!) {
                    let anchor = ARAnchor(identify: "Detected Object", remodel: worldTransform)
                    arView.session.add(anchor: anchor)
                    lastAnchors[anchor.identifier] = worldTransform
                }
            }
    }
    
    non-public func convertToWorldPosition(from level: CGPoint, body: ARFrame) -> simd_float4x4? {
        let hitTestResults = body.hitTest(level, sorts: [.featurePoint, .estimatedHorizontalPlane])
        return hitTestResults.first?.worldTransform
    }

There are 3 points occurring right here:

1. The anchors are usually not being positioned within the middle of the bounding
   field.
2. The anchors are hold floating across the display.
3. The anchor doen’t draw precisely on the given centroid of the situation.

Can somebody please counsel what might be potential fallacious in implementation ? When you have any totally different strategy to deal with this use case, please counsel.