camera behind lcd screen manufacturer

For years, smartphone manufacturers have tried numerous ways to create a truly edge-to-edge display. The ultimate aim is to have a smartphone with a screen that reaches to all four edges of the frame, with no interruption.

The only issue has been the need for a selfie camera. That inevitably has be be put somewhere, and we"ve seen any number of inventive methods that aim to try to hide it, make it less of an obstruction, or at least, reduce its visual impact.

There have been pop-up camera mechanisms, tiny dewdrop notches, flip cameras, and punch-hole cameras put on the front of phones. But there is one new technology aimed at hiding it completely: the under-display camera. Also known as USC (under-screen camera) or UPC (under-panel camera).

Thankfully, the clue is very clearly in the description. The UPC/USC or under display camera is a camera that"s hidden behind the display panel of the smartphone.

In basic terms, it"s similar to in-display optical fingerprint sensors. A small portion of the display panel is transparent, and lets light through to a camera that"s sat behind the display. Or to be more technically accurate, a small portion near the top of the screen is actually a second, tiny transparent display.

If you"re wondering why they can"t do what they do with optical fingerprint sensors and just make a transparent portion of the main screen yet, it"s because standard OLED panels aren"t yet able to let enough light through to the other side to create a decent coloured image. So for now, companies like ZTE and Xiaomi have resorted to using a secondary, much smaller "invisible" display within a display.

And, if they used this display as the entire panel, that would have dire consequences for the fidelity of the image on the display. So they put it in a part of the screen where - most of the time - the quality of the image doesn"t matter: in the status bar.

While the eventual aim is surely to have it implemented in a way that makes it completely invisible, early iterations haven"t quite managed it. It"s mostly invisible on darker days, but once you shine light on the area of the display hiding the camera, you can clearly see the area that"s allowing light through. As technology develops, we expect this to improve.

The first phone to have the under-screen camera was the ZTE Axon 20 5G. So far it"s also the only commercially available product with the under-display camera.

As mentioned already, part of the reason is that it"s not possible (yet) to completely hide that secondary transparent screen. The other problem is to do with image quality from the camera that"s behind it.

By adding a layer of material that"s not completely clear in front of a camera, it makes it harder to get a really good photo. After all, cameras require light to take pictures, but crucially, also need that light to come through to the sensor without any disturbance to the signal in order produce sharp and accurate results.

This is an extreme over-simplification, but it"s almost like covering the camera with a really thin layer of tracing paper and asking it to take as good a picture as if you hadn"t. It just can"t be done. Or hasn"t been so far.

The aim undoubtedly is to make the transparent display portion more transparent, but also develop better AI/algorithms to correct the issues that arise from filtering light through the screen.

There"s not much in the way of independent footage of this phone, but YouTuber AmazTech has thoughtfully pointed a flashlight at the display, which reveals the camera.

ZTE has officially announced the world"s first commercial phone with a behind-the-screen camera: the ZTE Axon 20 5G. Shrinking phone bezels have made locating the front camera a major design point of phones for the past few years. We"ve seen big camera notches, small camera notches,round camera cutouts, and pop-up cameras. Rather than any of those compromises, the under-display camera lets you just put the camera under the display, and by peering through the pixels, you can still take a picture. It"s the holy grail of front-camera design.

As we"ve seen in explainers from Xiaomi, these under-display cameras work by thinning out the pixels above the display, either by reducing the number of pixels or by making the pixels smaller, which allows more light to reach the camera. In the area above the camera, manufacturers will have to strike a balance between a denser display with lower-quality camera results or better camera output in exchange for an uglier above-the-camera display.

ZTE"s official renders of the device claim the camera is completely invisible, which can"t be right. It"s standard practice to not make any attempt at a realistic-looking pixel display in these renders, but in this case, that"s a big deal, since the display should look slightly darker above the camera. With COVID cutting down everyone"s ability to travel, there isn"t much in the way of live footage of the phone, either. ZTE posted an official live video to Weibo that really goes out of its way to never linger on a close-up shot of the camera, which is highly suspicious given the camera is the phone"s only headline feature. The best footage we can find right now is a YouTube unboxing from AmazTech, which at least takes the time to scrutinize the sensor location. AmazTech doesn"t have the sharpest video quality on Earth, but it doesn"t seem like ZTE has a lot to hide: the camera is still hard to spot. I would still like a better look at the screen, particularly with lower brightness levels, but so far it looks amazing. Advertisement

Covering the fancy new camera tech is a 6.92-inch 2460×1080 OLED display. The base model phone has a Snapdragon 765G SoC, 6GB of RAM, 128GB of storage, a 4220mAh battery, and a bunch of other unremarkable specs. ZTE lists Chinese prices starting at ¥2198 ($321). Huawei gets all of the "banned in the USA" headlines, but ZTE isn"t welcome in the United States either. That means you shouldn"t expect much in the way of distribution.

Although Chinese manufacturers usually get the jump on new technology like this, everyone picks from the same parts bin. So you can expect to see under-display cameras from most major Android manufacturers in the next year or two. This also means we"re rocketing toward the age of the completely invisible camera, a privacy-nightmare world where any device with a screen could secretly be recording your every movement. We"ve already run into devices that can discretely include microphones, and last year Google got into hot water for shipping a device with an undisclosed microphone. Now we get to do this with cameras! Welcome to the future, I guess.

For decades, we’ve lived with an inconvenient technological truth: Cameras and other sensors cannot occupy the same space as our screens. It’s why, increasingly, smartphones rely on the dreaded “notch” as a way of maximizing screen-to-body ratios while preserving the front-facing camera and other sensors.

Some phone makers, from Oppo to OnePlus, get around this problem by using motorized pop-up cameras, while others have resorted to punching holes in displays to provide the camera with its own peephole. It’s also why even the latest high-end laptops still have pronounced bezels around their displays. The webcam needs a home and it seems no one is willing to live with a notch or hole-punch on a computer.

But it turns out that cameras and screens aren’t quite as incompatible as they seem. Thanks to improvements in manufacturing techniques, these two adversaries are about to end their long-standing territorial dispute. This isn’t a far-flung prediction; it’s happening right now.

Complaining about a phone notch, hole-punch or a large screen bezel is the very definition of a first-world problem. And judging from Apple’s stellar sales numbers, none of these side effects of forward-facing cameras are dealbreakers for buyers.

First, it lets you make phones that have true edge-to-edge screens. Videos and photos look better, and app developers can make use of every square millimeter for their designs — all while keeping the phone’s body as small as possible.

Second, from a design and manufacturing point of view, if cameras and sensors can be placed anywhere, with fewer restrictions on their size and visibility, it redraws the map for phone design. Bigger batteries, thinner phones, more sensors, and much better cameras are all potential upsides.

Cameras placed in bezels or notches create the now all-too-familiar, awkward downward gaze that happens during video calls. “Most of the time, you’re not actually looking at each other when you’re talking over video chat,” Michael Helander, CEO at Toronto-based OTI Lumionics told Digital Trends. “The current placement of videoconferencing cameras in all of these devices is really suboptimal.”

Helander has probably thought about this problem more than most. His company creates specialty materials that enable what was once impossible — making displays transparent enough that you can place a camera behind them.

Once a camera is sitting behind the display, it will finally make our video interactions look and feel like real, in-person interactions — a game changer that couldn’t come at a better time in our COVID-restricted world.

Screen technology is dominated by two kinds of displays. The most common are liquid crystal displays (LCD), which include LED TVs and QLED TVs. The second, organic light-emitting diode (OLED), dominates smartphones and tablets, and is growing in use in laptops and even desktop monitors

LCDs are actually transparent when not in use — that’s why you see a gray background on a calculator screen wherever the black digit segments aren’t active. But taking advantage of this transparency to take a photo poses big technical hurdles, especially once you factor in the need for a backlight.

One solution favored by Xiaomi and Oppo in their UDC prototypes is to rely on an OLED pixel’s inherent transparency. When an OLED pixel isn’t being used to emit light, it lets light in. So you can place a camera behind an OLED display and it will be able to gather enough light to capture images. But there’s a catch: You still need to place the camera at the top or bottom of the screen, because when the camera is active, the OLED pixels above it must be shut off, which creates a temporary black area on the screen. That approach is a solution to the notch and hole-punch problem, but it does nothing to solve the downward gaze issue.

The first commercially available phone with an under-display camera — the ZTE Axon 20 5G — uses this technique, but it also suffers from a less-than-ideal compromise. Modern smartphones have incredibly densely packed pixels. The iPhone 12 Pro has a 460ppi (pixels per inch) display, which means that there are more than 200,000 pixels in one square inch. Sony’s Xperia XZ Premium had a whopping 807ppi screen (more than 650,000 pixels per square inch).

Punching holes in between those pixels, even with a laser, is so tricky that ZTE had to remove some pixels from the area above the camera to buy some extra room. The result is a noticeably lower-resolution square on the screen.

A lower-resolution section of the screen might not bother you when it’s near the top, in an area that’s used mostly for inconsequential information. But few people would accept such an obvious reduction of resolution in the center of their phone’s display, which is what we would need to counteract the downward-gaze problem.

Helander claims the self-assembly process works on any screen size, and lets manufacturers decide how many openings are needed — from just one to 1 billion.

As exciting as it is to think that we’ll soon be able to have much more natural video calls, placing a camera under a display puts an even bigger onus on manufacturers to provide trustworthy privacy measures.

We’ll need some kind of reliable indicator of when the camera is active and an equally reliable way of disabling it. Because it’s under the screen, there’s no way to physically block the lens without blocking content on the screen as well.

Apple recently updated iOS to show a small green dot near the notch when its forward-facing camera is in use, and an orange dot to show when the mic is active. That’s a good way to inform us of what’s going on, but we need something more.

Smart speakers like the Google Nest mini ship with physical switches that can be used to disable the microphones. Assuming that there’s no way to remotely overcome the switch’s position, it provides a very good level of trust. A similar mechanism on TVs, monitors, and laptops should come standard once cameras become invisible.

OTI Lumionics already has agreements in place with several Chinese smartphone manufacturers, but due to confidentiality restrictions, these companies can’t be named just yet. “Many of them have prototype phones that have been built and everything looks great,” Helander notes, “but none of them want to disclose anything publicly until they’re ready for their actual official product announcements.” He’s confident that we’ll see these new under-display camera models sometime in 2021, although they may remain a Chinese market exclusive until 2022.

To grossly oversimplify it, there are two challenges behind making an under-display camera (UDC): to convincingly hide the camera under the screen and to produce images that look as good as traditional selfie cameras. With its third-gen UDC on the Axon 40 Ultra, ZTE has nailed the “hide the camera” part of that challenge. As for “take good photos,” ZTE has come a long way on that since its first attempt, but there’s still work to be done.

First, some basics: this is a 16-megapixel camera residing under a 6.8-inch 2480 x 1116 OLED panel. On the back of the phone, there are three 64-megapixel cameras: a 16mm-equivalent ultrawide; a stabilized 35mm standard wide; and a stabilized 91mm telephoto lens. ZTE has made some improvements to the computational imaging algorithms for its rear camera system, but I won’t be covering that here or much else about the phone, really, aside from its most intriguing feature: the selfie camera.

If you want a much more nuanced explanation of under-display camera technology — and why it’s so difficult to develop — I highly recommend you watch my former colleague Sam Byford’s video on the subject. It puts into context just how impressive this achievement is, even for a work in progress. For one, the technology concealing the camera under the display has gotten really good. ZTE’s press release says that the screen area over the camera is integrated with the rest of the screen more smoothly in this iteration, with some tweaks to its independent pixel drive technology and how those pixels’ circuits are wired. There have also been some software updates to better synchronize the UDC’s part of the screen with the rest of the display.

You don’t need to understand any of that to recognize that ZTE has worked some magic here. The selfie camera really does disappear into the rest of the screen almost all of the time. During the time I spent setting up the phone, I truly forgot about it — maybe even the entire concept of punch-holes and notches. I guess that’s the highest compliment I can pay it: most of the time I used it, it was just a screen. You can’t say that about the under-screen camera in Samsung’s Galaxy Z Fold 3, which is always obviously there.

If I really look for it, I can see the camera when I’m using the phone. The lens is visible outside in bright light, but it’s not distracting. Indoors, if I turn the screen brightness way down, I can see it a little better, but I still have to actually look for it. Even with white screens, which challenged previous versions of the display, it’s very hard to spot. Job well done, ZTE engineers.

The camera side of the UDC has seen some image processing improvements, and in some situations, the results look good enough to pass as a standard selfie camera selfie. If you look closely, you can see some artifacts signaling that something different is going on; in some bright outdoor shots, there’s a little bit of color bleeding between my skin and the black shirt I’m wearing.

Backlighting is also very challenging for the camera. You can see lots of flare and sometimes a grid pattern across the image with the sun hitting the camera directly. ZTE relies on its image processing to sharpen selfies and clean up some of the blooming you can see in the live preview, which generally works, but it just can’t contend with direct sunlight. Video is another sore spot: the phone can’t perform all of that processing in real-time, so clips look very soft, even in good lighting.

Selfies indoors look softer than selfies from a traditional front-facing camera, but in decent lighting, they’re just regular bad — not “what the hell is going on here” bad. That’s an improvement. Light sources sometimes appear with a distracting grid pattern across them, which seems like a tough one to solve in software. In lower light conditions, images get very soft, and the on-screen flash isn’t terribly flattering.

Side by side with a standard selfie camera on a Google Pixel 6 Pro, there’s still a noticeable difference in image quality. Looking at the Axon 40 Ultra’s selfies in isolation, though, I’m not sure most people would spot that drop in image quality provided there’s no light source behind the subject or sunlight directly on the lens. With the display piece of the puzzle sorted out, maybe ZTE’s UDC tech is a couple of generations away from “good enough” for most people.

But then there’s another question to answer: do most people actually hate the notches and hole-punches on their phone displays? Or have we trained our brains to edit them out of our view when we’re looking at our phones? The best-case scenario — the one ZTE has achieved in the Axon 40 Ultra — is that an under-display camera disappears. But I think that most of us are all so used to the selfie cameras that we’ve essentially made them disappear from our conscious attention, too. Like bezels and status bars, we just stop seeing them after a while. ZTE’s engineers deserve praise for what they have achieved here, but I’m not sure that the problem they’re solving is one that bothers many people.

But this Oppo solution means you don"t need notches, holes or mechanical stowaway cameras. Instead, the camera lens sits under the screen. It hides completely when you"re watching videos or doing anything else with the phone, but the camera lens will appear when you need it.

"It utilizes a customized camera module, an enhanced translucent panel material combined with advanced processing algorithms to take vivid pictures without a notch or motorized camera," Oppo said in a tweet. In a video demo from Oppo, you can see that the area around where the camera appears becomes completely black. This likely also helps prevent any outside light from ruining a picture.

Oppo said this is just a taste of its technology, and didn"t say when it might be available in a phone you can buy. But this is probably how most front-facing cameras will work in the future.

Until recently, there was only one smartphone on the market equipped with an under-screen camera: last year"s ZTE Axon 20 5G. Other players such as Vivo, Oppo and Xiaomi had also been testing this futuristic tech, but given the subpar image quality back then, it"s no wonder that phone makers largely stuck with punch-hole cameras for selfies.

Despite much criticism of its first under-screen camera, ZTE worked what it claims to be an improved version into its new Axon 30 5G, which launched in China last week. Coincidentally, today Oppo unveiled its third-gen under-screen camera which, based on a sample shot it provided, appears to be surprisingly promising — no noticeable haziness nor glare. But that was just one photo, of course, so I"ll obviously reserve my final judgement until I get to play with one. Even so, the AI tricks and display circuitry that made this possible are intriguing.

In a nutshell, nothing has changed in terms of how the under-screen camera sees through the screen. Its performance is limited by how much light can travel through the gaps between each OLED pixel. Therefore, AI compensation is still a must. For its latest under-screen camera, Oppo says it trained its own AI engine "using tens of thousands of photos" in order to achieve more accurate corrections on diffraction, white balance and HDR. Hence the surprisingly natural-looking sample shot.

Another noteworthy improvement here lies within the display panel"s consistency. The earlier designs chose to lower the pixel density in the area above the camera, in order to let sufficient light into the sensor. This resulted in a noticeable patch above the camera, which would have been a major turn-off when you watched videos or read fine text on that screen.

But now, Oppo — or the display panel maker, which could be Samsung — figured out a way to boost light transmittance by slightly shrinking each pixel"s geometry above the camera. In order words, we get to keep the same 400-ppi pixel density as the rest of the screen, thus creating a more consistent look.

Oppo added that this is further enhanced by a transparent wiring material, as well as a one-to-one pixel-circuit-to-pixel architecture (instead of two-to-one like before) in the screen area above the camera. The latter promises more precise image control and greater sharpness, with the bonus being a 50-percent longer panel lifespan due to better burn-in prevention.

Oppo didn"t say when or if consumers will get to use its next-gen under-screen camera, but given the timing, I wouldn"t be surprised if this turns out to be the same solution on the ZTE Axon 30 5G. In any case, it would be nice if the industry eventually agreed to dump punch-hole cameras in favor of invisible ones.

The idea of embedding cameras in a display is not new. From the earliest days of videoconferencing it was recognized that the separation of the camera and the display meant the system could not convey gaze awareness

A second challenge has emerged more recently. The desire to maximize screen size on small devices such as cell phones leaves little room outside the display to locate a camera.

Placing cameras behind the screen could solve these problems, but doing so tends to degrade the image. Diffraction from the screen’s pixel structure can blur the image, reduce contrast, reduce usable light

In this project we investigate how machine learning can help overcome some of the image degradation problems associated with placing cameras behind the display, and can help frame remote conversations in

Locating the camera above the display results in a vantage point that’s different from a face-to-face conversation, especially with large displays, which can create a sense of looking down on the person speaking.

Worse, the distance between the camera and the display mean that the participants will not experience a sense of eye contact. If I look directly into your eyes on the screen, you will see me apparently gazing below

your face. Conversely, if I look directly into the camera to give you a sense that I am looking into your eyes, I’m no longer in fact able to see your eyes, and I may miss subtle non-verbal feedback cues.

Relocating the camera to the point on the screen where the remote participant’s face appears would achieve a natural perspective and a sense of eye contact.

With transparent OLED displays (T-OLED), we can position a camera behind the screen, potentially solving the perspective problem. But because the screen is not fully transparent, looking through it degrades image quality by introducing diffraction and noise.

To compensate for the image degradation inherent in photographing through a T-OLED screen, we used a U-Net neural-network structure that both improves the signal-to-noise ratio and de-blurs the image.

The ability to position cameras in the display and still maintain good image quality provides an effective solution to the perennial problems of gaze awareness and perspective.

Isolating the people from the background opens up additional options. You can screen out a background that is distracting or that contains sensitive information. You can also use the background region to

Human interaction in videoconferences can be made more natural by correcting gaze, scale, and position by using convolutional neural network segmentation together with cameras embedded in a partially transparent display. The diffraction and noise resulting from placing the camera behind the screen can be effectively removed using U-net neural network. Segmentation of live video also makes it possible to combine the speaker with a choice of background content.

Under-display cameras are poised to be The Hot New Feature, following up on the similar success of the in-display fingerprint sensor. It almost seems like science fiction: You can"t see the camera, but it can see you. The technology promises to eliminate the last impediment in the all-screen phone dream. But how does it work, and when will you actually be able to buy a phone that has one?

According to Helander, there are two engineering approaches to designing under-display cameras: You either do everything you can to make the entire display as transparent as possible above the camera, or you essentially make tiny transparent holes in an otherwise opaque screen between the pixels.

In the first case, that means changing materials and rearranging things in the area above the camera. Certain metals in the various layers can be replaced by transparent conducting materials like indium tin oxide, and the structure of the display itself can be rearranged to reroute anything that might interfere with optimal transparency in that area. Anything that can"t ultimately be moved or made transparent can be made as small as possible.

There are a few limitations to this route: primarily brightness, uniformity, and resolution. Typically, OLED pixels are designed to be reflective on one side and transparent on the other, ensuring most of the light produced goes in one direction: toward you. Making the display transparent in one section interferes with that sort of design, and it can make the area the camera is in look distinctly different and less bright than the rest of the screen. Compensating for that effect by cranking brightness and calibrating differently in that tiny area can result in other long-term issues like burn-in around the transparent camera area. We"re also told that all the rerouting and transparency-increasing steps often mean accepting a lower display resolution in that particular spot — a handful of big pixels in a sea of smaller ones. This is allegedly the approach that ZTE is taking in devices like the Axon 20 5G.

The second method is a little different. Rather than making an entire stack of the display transparent across one area, you can carve out individual transparent "holes" between the pixels and rely on them to transmit light through the screen. You can do this in a few different ways, like cutting down on display resolution to carve out an area for one in every X number of pixels, or just shuffling and rerouting things to make regular patterned spaces.

As before, this means rerouting some components to ensure you have a clear line through the screen, but you don"t have to worry about the whole display stack being transparent, just specific spots at regular intervals. If your resolution is low enough, you can accommodate these extra holes without any loss, but at very high densities, it can also mean giving up some pixels and accepting a lower resolution. Importantly, though, this route means the individual pixels above the camera have the same individual brightness and performance characteristics of the pixels elsewhere on the display, so you shouldn"t have as many issues with uniformity. This second route is what we"re told Xiaomi is planning for its upcoming phones, and it sounds like it may work the best out of the possible solutions available right now.

Now, whichever route manufacturers take, under-display cameras won"t work quite the same as they did before. Either way, the camera is going to get a little less light with more stuff in the way, and there are other optical effects these designs have to fight. There are issues like reflection and diffraction from all the various materials, layers, and holes that the light travels through. These are problems that can"t be fully eliminated, but Helander tells us they can be compensated for in software and reduced by advances in material science and engineering. Some of these issues also result in "softer" looking images, mimicking some of the effects of the beauty filters so many people enjoy, so it isn"t all bad. Helander also claims that machine learning models can compensate for many of these issues pretty well.

This also opens a pretty big door for us in the future. Right now, most under-display prototypes just put the camera in the same place and extend the screen to cover it, but nothing is stopping us from using some of these solutions to make the entire display transparent and putting the camera wherever we like. Ultimately, we could move the camera down to the center of the screen, making it easier to keep the effect of eye contact when in video calls, or we could even toss several cameras under the screen in different places. Some day other optical sensors, like the infrared cameras used for face unlock systems, could also be moved under the screen. Eventually, we could do the same with desktop computer monitors, too.

Before this technology can replace the notch or the hole-punch cutout, it needs to be scaled up. And given the sort of engineering costs involved, Helander tells us that, counter-intuitively, we"ll see this technology roll out in the mid-range market first. Right now, sacrifices required when it comes to resolution and brightness mean this technology probably won"t be a good fit in the flagship space for a while, where customers expect the very best. Issues like a big gray square or circle in the screen at max brightness, a resolution drop in one corner, or an overall lower display resolution all won"t play at the thousand-dollar price point, but they"re more acceptable in a mid-range product, and the details of ZTE"s upcoming device lend further evidence to that argument.

In Helander"s estimation, it could be 2022 or 2023 before this technology becomes mainstream, engineering problems are solved, production ramps up, and the feature works its way up and down the market. In the meantime, most of us will have to make do with being able to actually see our camera in a bezel, notch, or hole-punch cutout.

"Do you want a sneak peek at the future?" the tweet says. "Here you go ... introducing you to Under-Display Camera technology!"Do you want a sneak peek at the future? Here you go...introducing you to Under-Display Camera technology!#Xiaomi #InnovationForEveryone pic.twitter.com/d2HL6FHkh1— Xiaomi #5GIsHere (@Xiaomi) June 3, 2019

Xiaomi SVP Wang Xiang tweeted more detail on the under-display camera technology, calling it "the ultimate solution for a Full Screen Display coexisting with a front camera." Recently, Vivo revealed a concept phone called the

"Xiaomi is currently exploring the possibility of hiding the front-facing camera under the display," his tweet reveals. "When the selfie camera is activated, the display area over the camera lens becomes transparent in an instant, allowing light to enter."

"The transparent display doubles as the camera lens," Xiaomi explained. "By allowing more light into the lens, the display-embedded camera combo is able to produce perfect selfies, clear and crisper than the pinhole camera solution when the camera is activated."Xiaomi"s Under-Display Camera Technology could be the ultimate solution for a Full Screen Display coexisting with a front camera! RT if you love it. #InnovationForEveryone pic.twitter.com/8e7EdEBn8J— Wang Xiang (@XiangW_) June 3, 2019

The under-display camera is a feature that is available in very few modern smartphones and this feature helps front cameras to produce images even after being behind the displays. ZTE Axon 20 5G was the first-ever smartphone to come out with an under-camera display in 2020. In 2021, Xiaomi Mi Mix 4, Samsung Galaxy Z Fold 3 and ZTE Axon 30 were released with this feature. Here we will be discussing this new technology, how it works and what has made it popular.

The part of the display that goes over the camera is transparent and these devices need to have a display-in-a-display. The primary display in these devices are made up of OLED or LED technology and the small cut-out uses a different kind of glass that allows the camera under it to capture light. These ‘holes’ are usually visible during regular use even when the camera is turned off.

The demand for an all-screen smartphone has increased in leaps and bounds in recent years. Consumers demanded the largest-possible screen real estate on their phones while being easy to hold. For users, the edge around the screens is free spaces that can be used for content. So, smartphone makers are trying hard to hide sensors and other parts that will be facing the users.

Some manufacturers have tackled the all-screen affairs by flipping, rotating or pop-up mechanisms. All of these moving parts have a damage risk and are usually expensive to repair, so none of these features was very popular among the masses.

It can pop out while binge-watching content or gaming in full-screen mode and is usually described as a ‘blurrier patch’. However, smartphone brands are aiming to make them less obvious to the user"s eyes. These holes become less noticeable as the screen gets brighter.

Poor image quality has been the biggest challenge for the under-display camera feature. It is difficult for the front camera sensor to get the best image quality as the light has to go through multiple layers of glass and is partly blocked by pixels.

Oppo, Vivo, and Realme have either announced or demonstrated their prototype devices with under-display cameras, but all three brands are yet to introduce them to the market.

The first smartphone with an under-display camera was launched in 2020 and this technology is in its infancy. We can expect this technology in more flagship smartphones in the upcoming years and gradually it will trickle down to mid-range and even entry-level devices.

A decent front-facing camera is important for any smartphone. How else are you take the perfect selfie with your adoring cat? Alternatively, you could use it for business meetings or chatting to family and friends, but you get the gist.

One thing smartphone manufacturers have long desired is an under-display smartphone camera. Instead of the front-facing camera poking out of the top of the screen, or requiring a cutout, it just exists perfectly under the screen.

Your smartphone front-facing camera is visible to you. Whether you have an iPhone, Samsung, OnePlus, Huawei, or something different, the front-facing camera requires a small screen cut out. Understandably, otherwise, your front-facing camera would only take pictures of the rear of the smartphone screen.

As smartphone bezels become smaller and smartphone design pushes toward a single sheet of glass, many believe that the front-facing camera cut out is a blemish, spoiling an otherwise perfect screen.

The answer to the problem? Put the front-facing camera under the screen. Of course, the answer seems obvious, so why has no smartphone manufacturer brought an under-screen front-facing smartphone camera to market?

Although under-screen cameras have hit the news a few times in recent years, the announcement largely focuses on prospective development. Xiaomi made waves in 2019 after releasing a series of slides detailing its under-screen camera development.

Another manufacturer, OPPO, also revealed a working under-screen front-facing camera in 2019. Its announcement at MWC Shanghai 2019 was a hit at the smartphone-focused tech show, though OPPO made it clear that the under-screen camera was a work in progress, destined for a future device.

Then, at IFA 2020, ZTE revealed the Axon 20 5G, the first smartphone to feature a front-facing under-screen camera---that you can actually buy. So, how does an under-screen camera work?

The first thing required is a new screen type. ZTE developed a new "high transparency material" that includes new organic and inorganic films. The newly developed material allows greater levels of light to pass through the screen to reach the camera.

Second, the Axon 20 5G requires special internal workings. An independent control chip with integrated driver circuits provides completely separate color synchronization for light passing through the display. The extra control chip allows the camera to filter out any light and color distortion the camera experiences from the screen, keeping images clear and free of interference.

ZTE also created a "special matrix" that optimizes pixels, providing greater display consistency across the under-screen camera area. The increase in consistency through the matrix ensures the screen looks exactly the same throughout, rather than a small, distorted area.

Finally, the Axon 20 5G"s under-screen camera uses a special algorithm to optimize performance during operation. The algorithm considers multiple inputs through the screen, adjusting lighting, contrast, and other dynamic ranges before snapping a photo.

In late August 2020, the General Manager of Axon products, Mr. Lei, sent a selfie using the Axon 20 5G under-screen camera. ZTE uploaded the selfie to Weibo, the Chinese social media site.

Of course, the introduction of under-screen cameras raises another issue: broken phone screens. With more advanced screens, easily fixing your cracked screen suddenly becomes more difficult.

A quick word on the ZTE Axon 20 5G specs. Aside from the under-screen camera and the extra circuitry wizardry, the Axon 20 5G comes with a 6.92-inch FHD+ display, a Snapdragon 765G processor, and multiple memory and storage configurations.

The rear-facing camera is 64-megapixels, flanked by three additional sensors. This alone is a great feature, but adds to the question "Why does my phone need more than one camera?" It also has a 4,220mAh battery that also supports fast charging. Here"s the ZTE reveal advert:

Unfortunately, the only picture currently available from the ZTE Axon 20 5G under-screen camera is the selfie mentioned above. Now, I"m sure Mr. Lei is a jolly nice chap, but it doesn"t give much else away.

There is another common question regarding an under-screen smartphone camera. Can you see the camera under the screen? Is it obvious that a camera lurks behind the glass?

From the images provided by ZTE, the Axon 20 5G appears to hide the under-screen camera from the user. Without a hands-on experience with the smartphone itself, it is impossible to comment accurately. Where a single top-down or face-on shot hides the under-screen camera perfectly, a side-angle in certain light might reveal a little more.

The Xiaomi solution, detailed in the Tweet in the first section of the article, indicates that the under-screen camera area on its device will reveal when selfie mode activates. The front-facing camera remains completely obscured until that moment, giving users the sheer slab of glass effect many desire.

Similarly, OPPO uses a custom multi-layered transparent material that allows light into the camera but does not interfere with the display. That said, hands-on experiences with the OPPO test model suggest the camera is visible at certain angles and in certain conditions.

Are the other major smartphone manufacturers considering under-screen cameras for their flagship devices? There is speculation that the Samsung Galaxy S21 will feature some form of under-screen front-facing camera, although, at the time of writing, nothing is confirmed.

Apple is also seemingly non-committal to an under-screen front-facing camera. There was an alleged leak of an iPhone 13 prototype in mid-2020, although it"s fair to say there was zero-confirmation of its similarities to an actual iPhone 13. The supposed prototype did not feature any front-facing camera cutout, leading to speculation that the iPhone 13 will include an under-screen camera. But that"s all speculation, and the only people who truly know are Apple employees.

Samsung, Apple, and other manufacturers will continue to develop under-screen camera solutions until the image quality isn"t affected, the cost of development and implementation is beneficial, or a combination of both.

The development in the smartphone industry over the past few years has been massive. The display segment has seen tremendous improvements all in a bid to achieve a full-screen display. The front camera and some other sensors are the major challenges in achieving a true full-screen display. However, we have seen designs such as a notch, waterdrop design, punch-hole design, pop-up camera, thin bezels, retro mechanical structure, and several other designs all seeking to improve the display. The latest design seeking to improve the smartphone display is the under-screen camera technology. As of now, the under screen cameras are not quite ready. However, many manufacturers are gearing up to use this technology.

The under-display camera design is a perfect full-screen solution. It will do a better job than the waterdrop, notch, punch-hole, and other special designs. Another interesting part of the under-display cameras is that they will not increase the thickness of the device. Taking into account the vision and feel, this technology will even achieve a lighter and thinner bezel device.

Today, popular Weibo leakster, @DCS, claims that two manufacturers are already mass-producing this technology. According to him, Visionox and Huaxing Optoelectronics are mass-producing this technology. The official launch of the under-display camera should be in Q3 this year. These companies are focusing on high quality and speed. If the technology arrives in Q3, then we will see under-screen camera phones this year. If the applicability is good, we will probably experience a rush in the use of this technology next year.

The display is entirely screen and there is no special cut on the display. OCD users no longer have to worry about the special-shaped design such as punch-hole or notch on the screen.

Many mobile phone brands have shown off-screen camera technology before, such as Xiaomi. Last year, Xiaomi showcased its third-generation under-screen camera technology, which was officially claimed to have reached the commercial standard for mass production.

Compared with the previous two-generation solutions, the third-generation under-screen camera greatly improves the display effect of the camera area through the self-developed pixel arrangement. Furthermore, through the optimization of the camera algorithm, it brings the same self-portrait performance as the conventional front camera. We can not say much about the performance of this camera until it arrives. There are reports that Xiaomi’s under-screen cameras will be official in August. Furthermore, the suppliers will be China Star Optoelectronics or Visionox. We will wait and see.

The illusory all-screen phone design is no longer just a dream. iPhones have a notch at the top, whereas Android handsets come with hole-punch selfie cameras. These are necessary compromises between form and function. We need cameras and sensors on the front side of the screen, but we also want the display to occupy as much real estate as possible. The “perfect” smartphone design is already in development. More phones with cameras under the screen are coming this year, after the world’s first such device launched in 2020. But some handset vendors came up with a novel way to place the selfie camera behind the display.

Phones with under-display cameras aren’t the same as phones with the selfie camera behind the screen, even though it might seem like it’s the same concept.

When we say under-display cameras we mean selfie cameras that are identical to current selfie cams when it comes to placement. We might also refer to them as in-display, under-screen, or under-panel cameras. They sit vertically, but they’re placed right under the OLED display. When using the selfie camera is in use, the OLED layer stops displaying images to allow light to pass through to the camera sensor.

After the ZTE Axon 20 last year, we might see more phones with under-display cameras this summer, starting with Samsung’s Galaxy Z Fold 3. Samsung already has a marketing term for it — Under Panel Camera (UPC).

Placing the selfie camera behind the display is a different concept, as seen in the image above. The camera is still located under the screen; that’s why it might all be confusing at first. But the light will not pass through the OLED display layer. The camera will be rotated in this particular case. We’re looking at a sensor sitting horizontally behind the screen.

The light reaches the sensor from a tiny camera hole opening that sits in a small bezel at the top of the handset. The light might reach the camera via optical fibers or a prism. A prism is also used for a different type of phone camera that’s rotated inside the phone. That’s the telephoto lens equipping devices with periscope cameras.

The images above come from a Xiaomi patent awarded to the Chinese company in early June in Europe and the US. Found by Dutch blog LetsGoDigital, the document offers a viable alternative to the under-display camera tech we eagerly await.

Under-display cameras sound great in theory, but the selfie quality might take a hit. That’s not the kind of compromise smartphone users want. Placing the selfie cameras behind the screen like Xiaomi proposes might offer the best of both worlds. The display won’t get a camera hole, and selfie photography won’t require corrective algorithms. There might be a different sort of compromise. The top bezel might be slightly thicker than before.

That’s not to say that Xiaomi will manufacture a phone with a selfie camera placed behind the screen anytime soon. But the Chinese vendor is certainly looking at different options. Xiaomi will launch its own phone with an under-display camera this year, other rumors claim.

Who could even think that just within 20 years, mobile phones, particularly smartphones, will change our lives completely? From 2007 to now, it has evolved rapidly within a few years. From bulky Motorola DynaTAC to iPhone 1, manufacturers are updating smartphones displays day by day. And now it’s trying to shift to a full-screen version. So, you might not even see the camera on your next smartphone.

For several years now, phone companies have been trying to eliminate or minimize the bezel around the screen and create an edge-to-edge display. In addition, they have been attempting to use hole punches and pop-up cameras to optimize screen space. It doesn’t matter if you’re on the no-notch side or if you don’t mind the barren black patches. The full-screen era is coming.

Ideally, your smartphone should have a screen that reaches all four corners without interruption. Tech enthusiasts dream of seeing an under-display camera hidden under the screen. And now, it’s happening! Several brands, including Samsung, Xiaomi, ZTE, and Apple, have progressed with this technology. Still, they need to do more research and development. However, the question is, would it be worth it to sacrifice the quality of the front camera for a notch-less phone? And why do we not see it from flagship to mid-range smartphones?

Under-display cameras are hidden behind a smartphone’s display panel. One of the most prominent devices that features under-display cameras is Samsung’s latest smartphone Galaxy Z Fold 3. There is a 4MP sensor hidden underneath its large and folding display. Though ZTE sounds so new, many tech reviewers praised their under-display camera phones.

You’re likely wondering how the cameras produce images since they’re behind the screen. In essence, these devices contain displays inside displays. The manufacturer uses OLED or LED technology for the primary production, while a different type of glass is used for the camera cut-out underneath. This camera works like a fingerprint scanner, one of the most common methods of unlocking your smartphone.

Although the camera is not in use, these “holes” can still be seen during regular use. Usually described as a “blurrier patch” on the screen, it can be noticeable when watching full-screen content or playing games. However, companies are working on reducing their appearance. Now let’s check out some advantages of an under-display camera.

An under-display camera will allow you to enjoy a genuinely bezel-less edge-to-edge full-view screen for content consumption. In addition, you will not need a notch with an under-screen camera. As a result, there will be no more obtrusive indents cutting into interfaces.

The mainly under-display camera is famous for two advantages. Firstly, there are fewer concerns about longevity, storage, and the possibility of thinner phones. And the second advantage is that this system is easier to implement with no moving parts. Like another side of the coin, an under-display camera also holds numerous problems. Let’s find out about them.

Most tech experts find out that the image quality of an under-display camera isn’t entirely satisfactory because of its lack of light. The reason is that light must pass through the display to the sensor before capturing photos. And as a result, the flow of light is not ideal. To fix this, companies are working on improving display technology. Although companies claim their cameras are hidden, this has not always been the case. There is a small pixelated area on the screens of these phones that you might notice from certain angles or if you pay close attention. It will be interesting to see how annoying this partial visibility is.

Display panels are typically more scratch-prone than camera lenses. But what if the display is worn down or scuffed? Will it affect the camera’s performance? In this way, durability makes sense.

Although impressive, this technology is yet far behind perfection. Samsung may produce decent images, but many reviewers have noted that the pictures have to go through a lot of image processing before they can be used. Under-display cameras produce inferior images compared to those taken with a standard selfie camera on a flagship smartphone. Although the Galaxy Z Fold 3 is the most high-profile smartphone to adopt this new technology, it did not opt for an under-display camera as its primary front-facing camera. But it might take just a few years until under-display cameras are fully included in displays. We’ll have to wait and see.

Selfie cameras are a vital part of any smartphone — but finding the perfect place for the front-facing camera can often be an obstacle to seamless viewing experience. However, thanks to the Under Display Camera (UDC), users can say goodbye to the little black dot on the front of their screens. Samsung Electronics has given the UDC a serious upgrade in the newly-released Galaxy Z Fold4, offering users an even more immersive and uninterrupted full-screen experience.

Read on to see what John Seok-Jun Lee, Head of Computational Camera Part at Samsung Research America (SRA), and Alok Shukla, Head of AI Vision Solutions Part at Samsung R&D Institute India-Bangalore (SRI-B), had to say about their collaboration behind bringing enhanced UDC technology.

Seok-Jun:An Under Display Camera (UDC) is an innovative camera solution that enables an uninterrupted viewing experience on a mobile device by hiding the camera under the display and dedicating the whole screen to users while applications are running. It not only requires hardware innovation by placing a camera under a display panel but also requires algorithm innovation for restoring image quality — one of the most complex image restoration problems.

Alok:In UDC, a front-facing camera sits underneath the display. The screen pixel density is reduced in this area, allowing the screen to display imagery unhindered — but at the same time, not block light from reaching the camera when it’s in use. As the camera is placed underneath the display, UDC can suffer from poor image quality caused by diffraction artifacts, which results in flare, saturation, blur and haze. Therefore, while the UDC brings a better display experience, it also affects camera image quality and other downstream vision tasks. These complex and diverse distortions make restoring UDC images extremely challenging.

Seok-Jun: Samsung’s foldables are designed to be new and different, combining smartphone and immersive viewing experiences into one mobile device via breakthrough display technology. When folded, it provides smartphone-like use, and when unfolded, the display provides a more immersive and productive experience. Our consumer research showed that Galaxy Z Fold series users prefer the Cover Screen camera for capturing photos and selfies and the immersive Main Display for more productive tasks like watching videos, playing games and working on documents. Therefore, we thought UDC technology would be the perfect fit to meet their needs for an uninterrupted screen while still being capable of tasks like opening video calls. Samsung has launched the first-ever foldable UDC solution in Galaxy Z Fold3 and enhanced its quality further in the Galaxy Z Fold4.

Alok:Galaxy Z Fold4 truly unfolds a world of immersive display. UDC technology is at the core of the pursuit of a true full-screen display. By combining the leading innovation in display technology with advanced AI techniques for image restoration, Galaxy Z Fold4 provides the perfect balance between screen quality and camera image quality.

As mentioned previously, when the camera is placed underneath the display, this can result in poor image quality caused by diffraction artifacts, which cause flare, saturation, blur and haze.

Seok-Jun: The SRA MPI Lab has been working with SRI-B throughout MPI’s entire history at Samsung, from 2014’s Galaxy Note4 camera commercialization all the way to 2022’s Galaxy Z Fold4 and Z Flip4. SRA MPI has been leading MX multi-frame processing (MFP) technology R&D for UDC, Night Mode, Expert RAW and other technology. SRI-B has contributed to all commercialized SRA MPI’s MFP by providing amazing camera software, image quality tuning and resources toward commercialization as well as AI camera R&D. Our 3-way collaboration with MX R&D group, SRA MPI and SRI-B has been a success — especially when the worldwide team used time zone differences effectively to complete milestones on time for such a complicated project. It’s always a pleasure to work with them and deliver Galaxy devices’ camera successes together to accomplish our mutual goal of being the world’s best smartphone camera experience.

SRI-B also played a key role in utilizing a Neural Processing Unit (NPU) to execute the AI model, achieving real-time performance in previews and significantly faster capture processing time than in Samsung Galaxy Z Fold3 UDC. Camera previews, video recordings and video calling scenarios need real-time performance without losing any frames. AI-based UDC restoration accelerated with NPU for preview processing was applied for the first time in Samsung Galaxy Z Fold4. Although real-time processing was not required, as camera capture requires higher image quality than a preview, achieving better image quality with less processing time was desired for user experience.

Seok-Jun: Low light transmittance and light diffraction are the main challenges for UDC camera technology. Still, one more challenge, which is different from a punch hole camera image pipeline, is that UDC capture quality is affected by not only the camera module but also the display panel. SRA MPI’s AI-based 3D ISP algorithm was designed with the needed flexibility to tune the output image quality with the different display characteristics. Having such flexibility in 3D ISP helped to overcome such challenges.

Alok: Samsung Galaxy Z Fold4’s UDC has opened up a new dimension to the smartphone camera form factor. UDCs in the future will enable placing larger camera sensors and multiple cameras at flexible positions without compromising the viewing experience. Another big challenge is towards mass adoption of UDC technology in more products as the main front camera. For this to happen, there will need to be a significant improvement in camera image quality, processing time and display experience.

Seok-Jun:Since the Galaxy Z Fold4 UDC provides a much better experience in terms of hiding the camera under the display and improving image quality, the R&D direction for the upcoming flagship devices will work towards making the UDC completely invisible. This will provide the best uninterrupted immersive mobile display experience while achieving the same image quality as a punch-hole camera.