smartphone camera

Smartphone cameras, pixel-binning, and the art of megapixel hype

/ spqr / Leave a comment

Many smartphones are now marketed as having at least one camera with a ridiculous amount of megapixels. The iPhone 14 Pro has 48MP, the Samsung Galaxy S23 has 200MP. Is it just too much? The answers is yes, and some would argue it’s more of a marketing hype than anything else. I mean who doesn’t want to take a 48MP or 200MP image? Well, most people may try it once, but many won’t routinely use it, and the reasons why are varied. First, let’s look at the technology.

Smartphone sensors are no different to any other sensors, they are just usually smaller than many conventional digital camera sensors. The sensors contain a bunch of photosites, so no different there. But there are limits to the size of sensor that can be used inside a smartphone, made more restrictive by the fact that many smartphones now have 2-3 rear-facing cameras. Higher resolution means that more photosites need to be crammed into the sensor’s surface area. The iPhone 14 Pro has a wide angle camera with a 48MP resolution. It uses a 1/1.28” sensor, which is 10×7.5mm in size with a photosite pitch of 1.22µm, which is extremely small. Typically smaller pixels have a harder time getting light than larger ones, leading to some issues in low-light situations. So smartphones typically get around this by creating reduced resolution images with “bigger” pixels that are created by means of photosite binning (or pixel binning if you like).

Fig.1: Pixel binning (2×2) with the iPhone 14 Pro – converting 48MP to 12MP by merging

Photosite binning artificially groups smaller pixels into larger ones, potentially boosting the amount of light that can be gathered. The example in Figure 1 shows part of the quad-Bayer sensor of the iPhone 14 Pro. It illustrates how a 12MP is generated from a 48MP sensor. Here four photosites (2×2) are binned from the sensor, producing a 12 megapixel image, i.e. 48÷4=12. The Samsung Galaxy S22 Ultra takes 108MP images, and also defaults to 12MP, but instead of using a 2×2 binning, it uses what Samsung calls “Nonacell” technology”, merging 3×3 photosites into a super pixel. So the photosites, which have a pitch of 0.8µm, are merged to form a 2.4µm super-pixel. Figure 2 shows how a true 48MP images is create via some sort of remosaicing algorithm (pixel rearrangement algorithm).

Fig.2: The iPhone 14 Pro Quad—48MP bayer sensor to 48MP image via a simple remosaicing algorithm

The 200MP Samsung ISOCELL HP-3 1/1.14” sensor takes it a step further. It uses a new “Tetra2” binning mechanism, with 0.56µm photosites. It can produce 200MP, 50MP, and 12.5MP images (shown in Figure 3). In the first stage 2×2 binning is used on the sensors 0.56µm photosites, producing a 1.12µm “super-photosite”, and a 50MP image. Then another round of 2×2 binning is performed, creating a 2.24µm “super-super-photosite”, and a 12.5MP image.

These high resolution sensors are typically only used at full-resolution in bright scenes, reducing to a lower resolution in dark conditions. The idea of binning is to allow smartphone cameras to become more intelligent, choosing the optimal resolution based on the photographic conditions. Of course the question is, does anyone need 50 or 200MP images? As with all technology, there are drawbacks.

Fig.3: Conversion between the ISOCELL HP-3 sensor and 200MP, 50MP, 12MP

Firstly, in the world of digital cameras, 100MP or thereabouts is usually found on a medium format camera with a sensor size of about 44×33mm, e.g. Fujifilm GFX 100S. These cameras are designed to take high resolution images, with sensors containing photosites of a reasonable light-gathering size (e.g. 3.76µm). Now compare the photosite area of this medium format camera, at 14µm2, with that of the Samsung Galaxy S22 at 0.64µm2 – 22 times more light gathering surface area. There is no comparison. Digital cameras also generally use high-quality lenses, with a lot more light gathering potential than those found on smartphones – when it comes to optics, smaller is always marred in compromises.

The process of binning pixels may also introduce artifacts, whether it be a small change in the overall colour of the image, or perhaps blurring artifacts – it really depends on the technology, algorithms, etc. Even remosaicing is a little more challenging, primarily because the different colours are further apart. So there isn’t really 4× more detail in 48MP mode than there is in 12MP mode. Then there is storage. 200MP image files should be large, but due to some sort of compression wizardry, the 12240×16320 images are reduced to files about 30MB in size, which is quite reasonable. Supposedly RAW (DNG) images can take up to 120MB. So storage is an issue. Also, what do you really need a 200MP camera for?

So if the mainstay is a quasi-12 megapixels why do manufacturers waste effort in creating hyper-megapixel smartphones? Perhaps for that one ideal 200MP shot? That perfect sunset while on vacation in Iceland. But what would you do with a 200MP image? Make a print to hang up on the wall? More megapixels does allow for better digital zooming, but the more likely case is that manufacturers know that megapixels matter to consumers, a situation they themselves have hyped up over the past two decades.

Further reading:

Quad Bayer sensors: what they are and what they are not (2019)

So you want to upgrade from a smartphone camera?

/ spqr / Leave a comment

Most people who use smartphones have little, if any, idea about things like aperture and shutter speed. They just use their smartphone camera to take pictures, and tend to ignore functional specifics. Settings are whatever the smartphone deems appropriate for the situation. For example clicking on ×0.5 in the Camera app on a modern iPhone will get you an image automatically taken with the ultra-wide camera. Yes you have some control over things, or more control when using a 3rd-party app, but generally these things don’t matter to most people. The future will bring more AI to smartphone cameras to produce so-called “perfect” photos – and if you like point-and-click photography, that’s fine. But sometimes that’s just not enough.

So what happens when you are intrigued enough to upgrade from a smartphone to a “real” digital camera? Should you run out and buy a full-frame (FF), or should you opt instead for a compact camera? To figure out what you really need, you have to first determine why you want to upgrade. Is it because you want to learn more about photography, or perhaps you want better control of the pictures you take? Or because you feel hamstrung using smartphone a camera and want more megapixels, better optics, or just a better way of taking pictures. Regardless of what people say, a smartphone camera will never provide the same sort of control, or image quality of a dedicated camera. There are many reasons for this, but the big ones are optics, storage space, and battery life. But this isn’t a post about that, here I want to consider options for “upgrading” from a smartphone camera (I’ll cover those in a separate post).

Upgrading from a smartphone to a compact – some specs.

Once you have figured out why, then we move onto what sort of photography you will be focusing on. Do you just want a camera for better travel photographs, or are you interested in landscapes? Or perhaps macro-photography? At this stage it is best to make a list of things you would like to achieve with a digital camera. Some of these things will help you narrow down the type of camera is best for you. For instance if you like street photography, then the best camera might be a compact camera like the Ricoh GRIII/IIIx, or the Fujifilm X100V. Compact cameras offer several advantages over smartphones – a larger sensor is the most obvious benefit, while physical controls and ergonomics offer a more tactile shooting experience. Most compact cameras now also use touchscreen interfaces, making them very accessible. These cameras generally have a fixed focal length lens, and a sensor somewhere between 20-24MP (which is more than adequate). Compact and inconspicuous cameras are perfect for street photography – the last thing you want as a street photographer is lugging around a huge hunk of a camera – it makes you stick out like a sore thumb.

Some of the benefits of digital cameras

If you want a better camera for travel, then a compact is good as well, as are crop-sensor cameras. Here cameras with mirrorless APS-C sensors have become popular, like the Fuji-series of cameras. Cameras for travel have to be versatile, compact and light – the new Fujifilm X-S20 weighs only 491g (without lens) – add a general purpose Fujifilm XF 23mm lens at 180g, and you get a total of 671g (and frankly you don’t need to travel with a cornucopia of lenses). You could also go for a smaller Micro-Four-Thirds sized sensor, which provides a camera with an even smaller form-factor. Now you could even go for a full-frame (FF) sensor, but I would not really recommend it for people upgrading from a smartphone. They are generally heavy, ostentatious (for travel anyway), and are not a good fit for novice photographers. Learn on something smaller before deciding on whether you really need a FF (or buy an inexpensive, older FF camera). Then there are those that want a more specialized set-up for landscapes, macro, sport or wildlife. As these types of photography are much more specialized, requiring specialized lenses, I would not jump straight into them. They can be expensive, and often need a good amount of experience to be used in an effective manner.

Choosing a camera is about what you are interested in photographing, budget, future expandability (if that is important), camera ergonomics (it has to feel right to use, or you will hate using it), diversity of lenses, and a myriad of other things. Decisions on choosing a camera are often made based on sensor size, or ultimately megapixels, but upgrading should not be purely about megapixels. Most good cameras have around 24-26 megapixels, which is more than adequate. You don’t need 40 megapixels – really, you don’t. Choice of sensor size, Micro-Four-Thirds (MFT)/APS-C/FF, is often a factor of the type of photography a person is interested in. Every different camera sensor has its own advantages and disadvantages.

If you want to delve into the world of real cameras, it doesn’t have to be expensive. Start with a used camera, with a single, versatile lens. You can add other lenses as required, and even add vintage lenses from 35mm film cameras. For instance you can readily purchase vintage telephoto lenses for very little $. There are an abundance of them out there. They require manual focusing (that’s a good skill to learn), but it’s a good way to find out if you like wildlife photography before going out and spending thousands of $. There is no need to run out and buy the latest and greatest. When everything is taken into consideration, upgrading from a smartphone camera to an actual digital camera allows for increased flexibility and enhanced artistic opportunities.