Coordinate Transforms in iOS using Swift, Part 2: Automatically Finding the Coordinate Frame

The Test Target

  • Align the barcodes, text, and shapes with the horizontal and vertical edges of the paper.
  • Print QR Codes and matching text next to each other.
  • For OCR, use entire words rather than individual letters.
  • For rectangle detection, print a thick border with sharp corners.
  • Make the QR codes about 1 inch (2.54 cm) on a side.
  • For OCR, select a larger font. I used Calibri Bold, 22 points.
  • Design the target to find coordinates — not to test the accuracy of vision results.
A test target of four QR codes surrounding a thick-lined square. Inside of the square are four texts: Ant, Bee, Can, and Dog.
Test target with QR codes a, b, c, d; a thick-bordered rectangle suitable for rectangle detection; and four three-letter words suitable for OCR (optical character recognition).

QR Code coordinates

extension CameraViewController: AVCaptureMetadataOutputObjectsDelegate {
/// Data stream for QR Codes and/or DataMatrix codes.
func metadataOutput(_ output: AVCaptureMetadataOutput, didOutput metadataObjects: [AVMetadataObject], from connection: AVCaptureConnection) {
let objects = metadataObjects.compactMap( { $0 as? AVMetadataMachineReadableCodeObject } )
for obj in objects {
if let s = obj.stringValue {
//check for texts starting with "A," "B,"...
//format according to personal taste
if let c = obj.centroid() {
print("\(s) @ (\(c.x), \(c.y))")
}
}
}
}
}

OCR result coordinates

var request = VNRecognizeTextRequest { ...   //...
request.recognitionLevel = .accurate
request.recognitionLanguages = ["en-US", "en-GB"]
request.usesLanguageCorrection = true
request.minimumTextHeight = 0.005 //pick some minimum
}
var request = VNRecognizeTextRequest {observation in observations {
guard let topCandidate = observation.topCandidates(1).first else {
return
}

ocrLoop: for observation in observations {
guard let topCandidate = observation.topCandidates(1).first else {
return
}

// exclude OCR text that falls inside a QR code (
if checkObjectOverlap {
overlapCheck: for sample in self.qrCodes {
let q = sample.value.bounds
if q.intersects(observation.boundingBox) {
if let s = sample.value.text {
print("QR code '\(s)' found as text")
}
else {
print("QR code overlap")
}

continue ocrLoop
}
}
}
let b = observation.boundingBox//...
}
}
}

Rectangle Detection

Print to Console, Create an ImageFrame

--

--

--

Founder of Echobatix, developing assistive technology for the blind. echobatix@gmail.com

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Modern modular apps with Xcode 11 and Swift Package Manager

Xcode Previews speed up app UI development and testing

Apple iOS 14- Feature all we need

iPhone Biometric (FaceID) do not work in several apps which used same code base.

Biometry protection in iOS

Bereev 2.0 is live! 🎉

How to Make iOS App Secure From Screenshot and Recording?

How to prevent screen capturing and recording in iOS App?

UIKit: Programmatically embed a UIStackView in an UIScrollView using AutoLayout

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Gary Bartos

Gary Bartos

Founder of Echobatix, developing assistive technology for the blind. echobatix@gmail.com

More from Medium

Split mirror photo effect for iOS

Check UITableViewCell currently exists before tableview reloadRows刷新Cell前檢查Cell當下元件是否已生成

Classes vs Struct — Swift

Structures, Classes, and Protocols in Swift