What is Apple Deep Fusion and how does it work? - MrLiambi's blog


My tweets


Tuesday 8 March 2022

What is Apple Deep Fusion and how does it work?

Deep Fusion is a brand term used by Apple to describe the way many of its phones process images, particularly around selfies and portrait photos. It first appeared on the iPhone 11 and has since been updated and added to most new iPhones launched since.  

What is Deep Fusion?

Deep Fusion is essentially Apple's version of neural image processing.

It's been available now to iPhone users for more than two years, and is being improved and updated with new phone models, firmware updates and new processors. 

How does Deep Fusion work?

Deep Fusion uses the iPhone's processor and neural engine for its machine learning prowess. The phone camera takes nine shots (two groups of four, prior to pressing the shutter) and one longer exposure (at the point of press, at various shutter settings). It then automatically looks through these shots and selects the best combinations and composites them for the sake of sharpness.

This is also a clever way to help negate image noise - that multi-coloured dotting that can appear in images. As noise won't appear identically in each frame, the system will be able to select the least noise-ridden parts into the image for a cleaner, sharper result. Apple is looking to use processing, rather than cramming pixels onto a sensor, to produce its best results.

Deep Fusion is not visible; there's no indicator in the camera app, photo roll, or even in the EXIF data. Which of the lenses utilise this capability depends on the model you have. For instance, the iPhone 11's main and telephoto used it, but the ultrawide didn't. 

And it all takes about a second longer than, say, an Smart HDR image, which means the feature won't work in burst mode.

Deep Fusion examples

Apple senior VP Phil Schiller described Deep Fusion as 'computational photography mad science' when he introduced the feature for indoor and medium-lighting situations. Here's a sample shot shared by the company:

Here's another Deep Fusion image from Apple:

Which devices have Deep Fusion?

Deep Fusion is exclusive to the iPhone. It's available on iPhone 11 and later, including the iPhone 12 and 12 Pro series, iPhone 13 and 13 Pro series, plus the latest iPhone SE launched in 2022.

When will Deep Fusion be available?

It's been available to iPhone users since the Apple's iOS 13.2 update. We're now up to iOS 15.1, so it was some time ago and your phone - if it's new enough - will support it. 

Source : https://www.pocket-lint.com/phones/news/apple/149594-what-is-apple-deep-fusion

No comments:

Post a Comment