As I begin my journey at this year’s Mobile World Congress in Barcelona, I am thinking through the major trends and inflections that will most impact mobile technology and usage models going forward. One of those I wrote about last week was quick charging. What I would like to talk about now is 4K capabilities on mobile devices. Let me start with explaining what 4K is and how it applies to mobile devices. I also wanted to touch on some things Qualcomm is doing in this arena.
Don't Miss: The Best HDR TVs
What is 4K?
Most of today’s TVs, many tablets, and some phones today are display at what is called “1080p”. For a phone, this would mean that it displays 1,080 horizontal pixels by 1,920 vertical pixels, combining for around 2M pixels. “4K” devices display 4X the amount of pixels (8M) driven by a resolution of 3,840 vertical and 2,160 horizontal pixels. Another related and arguably more important measure of merit has become “pixels-per-inch” or PPI as this describes, how dense the pixels are in each square inch. My 5.7”, 1,920 x 1,080 Samsung Galaxy Note 3 has 386 PPI while my 9.7”, 2,048 x 1,536 iPad Air has a PPI of 264. Net-net, higher resolution does not necessarily mean higher picture quality, as you need to factor in varying display sizes and viewing distance
Why 4K-Derived PPI matters
My first reaction when I heard of 4K displays on phones and tablets was skepticism, until I researched the topic. First of all, many in the industry generally agree that humans can perceive differences in max PPI ranging roughly from 350 to 575 PPI at a ten inch distance. The better the vision, the higher the visual acuity, and my Note 3 at nearly half the maximum has a ways to go. Personally, I am painfully near-sighted and I can see pixels on a phone or tablet where others cannot. Others in the industry (Sharp and Qualcomm) and some researchers are talking about vernier acuity, or the ability to see the differences among two line segments. This suggests that human’s visual system is much more complex, indicating that humans can assess differences 5-10X the detail of visual acuity.
Regardless of what theory you buy into, I’m very safe saying that many consumers will be able to distinguish the difference between a 386 PPI device, a 441 PPI tablet (10”, 4K) and 774 PPI phone (5.7”, 4K). In the end, you have to determine if it matters to you. If you are at Mobile World Congress, you can see this in action at Qualcomm’s booth. Now let’s talk about content.
Where’s all the 4K Content?
Ironically, the biggest generator of 4K content is probably sitting in your pocket already. Most all modern smartphones take 8 mega-pixel pictures so you are taking 4K pictures already. The problem is that your phone or tablet display is lower resolution than your pictures and therefore displays them at that lower resolution. Some smartphones also take 4K videos, too like my Note 3, and LG just announced their new G2 phones will have the same capability. Watching content captured in 4K is best viewed on a 4K display because it eliminates what is called “scaling artifacts”, image defects created by downscaling content from 4K to 1080P.
4K paid video content is a much easier story to tell in North America after this year’s CES:
- Netflix: announced they would stream in 4K to devices supporting HEVC (H.265) in 2014
- Amazon: announced they would provide Prime Instant subscribers access to 4K content
- YouTube: announced they will support 4K using their VP9 video format. You can see 4K YouTube videos here.
Other benefits for 4K exist, namely improved text legibility and games.
The English alphabet is pretty easy to render on devices, but that is not the case with other alphabets. In particular, Asian characters look very blocky on a low PPI display and having more lifelike text as you are reading an e-book on a 4K tablet could be a big benefit. I urge you to try 4K mobile devices to see if you see a difference.
Let’s talk games. While even the latest game consoles only render to 1080p, mobile games would have the capability to upscale to the native resolution of a 4K tablet or phone. To me, the benefit is simple; the higher the resolution and PPI, the more detailed the game is and more enjoyable the experience.
Finally, I want to point out that everything on the 4K mobile display would be at 4K, providing the opportunity for OEMs and ISVs to create much more immersive environments. Apple already proved the power of this with their Retina Display.
But is it a Good 4K Experience?
There will be many different kinds of mobile 4K experiences delivered at MWC and I like the approach Qualcomm has taken to 4K on the Snapdragon 805, one they are calling a “full 4K experience”. It enables consumers to view 4K on their mobile device display and also connect to a 4K HDTV. Literally, all of the content is 4K and “post-processed” for quality; every video, text, overlay, UI, game, and picture.
Adding new and desired capabilities to any product is a good thing unless it is counter-balanced by something negative. Adding a rocket engine to the family mini-van is a great thing for speed to pick up groceries, but you’ll need to upgrade your brakes, fuel system, dashboard, steering wheel, air bags and you may need to add wings and a crash cage. The same thing goes for adding 4K to a phone or tablet. Just adding 4K to a mobile device without considerations for the holistic experience could result in a bad experience with limited battery life. As you are running 4X the data inside the SoC, something must change.
It’s a mobile heterogeneous computing approach, with 4K. I’ve written a lot on heterogeneous computing and it sounds complex, but think of it as using a special processing engine for specific workloads like a golfer uses a different golf club for a different golf stroke.
Qualcomm has used different custom-designed heterogeneous compute “engines” to tackle specific jobs, then integrated them together to deliver the holistic experience. I am most familiar with what they have done, so let me outline what they have done on their Snapdragon 805 to optimize for 4K:
- 4K Multimedia engines: 4K video runs (or is decoded) on a specialized, fixed function block and supports H.265 video playback at 30fps. Running this entirely on a CPU is not the right approach for mobile with limited power and thermal envelops. A newly introduced engine also runs very sophisticated video quality enhancements based on technology from the recently acquired HQV (Hollywood Quality Video) company. Finally, to actually display an image at 4K on the phone or tablet’s display, Qualcomm uses a special display engine.
- 1.2 GPixel/s camera engine: Known in the industry as an ISP (image signal processor), the 805 enables sharp, higher-quality, higher-resolution photos with pretty advanced pre and post-processing noise reduction features for low-light conditions in both snapshot and video modes. It has what is called a “dual pipeline”, which provides support for image resolution up to 55Mpixels, picture-in-picture from dual cameras, 4K video encode with high resolution snapshot.
- GPU: The Adreno 420 delivers support for up-scaled games for 4K displays as well as advanced 3D graphics capabilities like “hardware tessellation” and “displacement mapping” found in the latest game consoles. Hardware tessellation enables greater detail for more visually realistic scenes in a manner that requires less memory bandwidth and lower power consumption.
- Security Engines: With new 4K content comes an increased vigor to protect it. The 805 uses a pre-integrated security solution called “Snapdragon StudioAccess” which protects both the compressed and uncompressed premium video. Hardware-based access control is enforced on each component that touches 4K content like the video decoder and the cryptographic engine. The chipset uses the SecureMSM architecture which provides fundamental security services at boot-time and while playing content.
- Memory: 4K data density is 4X larger than 1080P, so you need a wide highway to move it back and forth. The Snapdragon 805 added a memory controller that is as wide as many notebooks on the market today, but does it in a way that maximizes battery life. Qualcomm runs the memory controller like a smart HOV Autobahn, only opening the lanes it needs and directing cars to pack closely together. This results in less power needed to run the highway.
As you can hopefully gleam from this, getting to a much better mobile 4K experience isn’t about slapping on the jet engine, but architecting it from the start.
Mobile as the Primary 4K Driver?
Getting back to the headline of this column, I ask, “could mobile be the primary 4K driver for the industry?” I have personally participated in many content transitions and a few characteristics come to mind as “drivers”:
- Content: plain and simple, no content, no transition. Mobile arguably has more content today with photos and video than PCs or TVs and the first 4K video services, OTT, are devoured on mobile devices. Yes, TVs are first out of the gate, but mobile will come shortly afterward.
- Volume: content producers want high volume to spread their expensive content across. With TVs getting upgraded every 8-10 years and selling in the tens of millions, PCs upgraded every 5 years selling in the 100s of millions, and smart phones upgraded every 2-3 years in the billions, phones have the 4K volume potential advantage.
- Demonstrable improvement: On large TV and PC displays, there is no doubt that 4K looks better, particularly at anything less than 10 feet (TV) and 2 feet (PC). For mobility, industry experts peg the max demonstrable PPI between from around 350 to 575 and some say we can perceive 5-10X this through vernier acuity. This is well within the range of 4K. I urge you to see it yourself and decide on your own.
- Marketing Ease: Anytime consumers are faced with too many “if-thens” or contingencies on that big new feature, they don’t buy. 3D TV was a great example; “if” I have the right TV, “if” I have the right player, “if” I’m the 90% who doesn’t get sick, “if” I have enough glasses for everyone in the room, etc. A 4K tablet has 4X the display resolution and every decent smartphone takes photos in 4K (8MP). How simple is that?
When I factor all these things in, I have to conclude that there is a very high probability that mobile will be the primary driver to 4K.
Don't Miss: See the first leaked Black Friday 2016 Ad