CMSampleBuffer and the brightness of iOS camera images

If your iPhone app uses the video feed from a front or back camera, then you may want a method to detect when the image is too dark.

Notify the user if the image is too dark for whatever image processing you want to perform. Hand detection may fail if the image is too dark, or if there’s insufficient contrast between the user’s skin tone and the background color.

What brightness would we measure for this public domain image?

Code

The code below is from user StephenFeather in a 2018 StackOverflow post:

https://stackoverflow.com/questions/22753165/detecting-if-iphone-is-in-a-dark-room

The only tweaks I made were to change the function getBrightness(sampleBuffer:) to a computed property in an extension of CMSampleBuffer.

This code worked fine for me in Xcode 15.4, Swift 5, targeting iOS 16.0 or later.

import AVFoundation

extension CMSampleBuffer
{
    /// Get the brightness of a CMSampleBuffer
    /// Code source: https://stackoverflow.com/questions/22753165/detecting-if-iphone-is-in-a-dark-room
    var brightness: Double
    {
        let rawMetadata = CMCopyDictionaryOfAttachments(
            allocator: nil,
            target: self,
            attachmentMode: CMAttachmentMode(kCMAttachmentMode_ShouldPropagate))
        
        let metadata = CFDictionaryCreateMutableCopy(nil, 0, rawMetadata) as NSMutableDictionary
        let exifData = metadata.value(forKey: "{Exif}") as? NSMutableDictionary
        let brightnessValue : Double = exifData?[kCGImagePropertyExifBrightnessValue as String] as! Double
        return brightnessValue
    }
}

Range of Brightness Values

Maybe you can help with this!

When I tested brightness values from fully black with the lens blocked to the brightest scene I could find —pointing my phone at the sun on a bright day — the range of brightness values was

-6.00 to +12.00 (iPhone 13 Mini)

Do you get a different range for your iPhone?

Some caveats:

• Covering the lens and imaging a nearly but not entirely black surface can yield the same minimum value.
• An iPhone 13 Mini was the only phone I had to test that runs iOS 16 or later.
• Negative values can correspond to indoor scenes that that you might not consider dark.
• OCR for white text on a black background worked fine for me. The reported brightness approached the minimum value, which in my testing was -6.00. A minimum value may indicate only that most pixels are black.
• Unless you programmatically control exposure and white balance, the camera will adapt to changes in scene brightness.
• Pro tip: don’t point your phone at the sun for too long.

In the StackOverflow post, comments indicate different approximate ranges than what I observed:

-8.5 to +13.5

-7 to +14

Is the range dependent on the selected camera, such as wide angle or telephoto? Does the range depend on the iPhone model?

CMSampleBuffer?

CMSampleBuffer is the data provided by the captureOutput function of a AVCaptureVideoDataOutputSampleBufferDelegate. If you capture live images from an iOS device, you’ll encounter CMSampleBuffer.

A VNImageRequestHandler for hand detection, barcode reading, or OCR can be initialized with a CMSampleBuffer.

In 2021 I wrote a post that includes a CMSampleBuffer extension for conversion to a UIImage:

CMSampleBuffer to UIImage in Swift