Hello, and welcome to Android bites powered by SPER I'm David Reddick. And each week I'm joined by my co-host Michelle ramen. As we dive deep into the world of Android this week, we're talking about something that I think everyone on Android has had some kind of frustrating experience with, which is how. Third party applications be they Snapchat, Instagram, um, or any other application that has to use your camera on your device to capture content. How does that interact with the operating system? How does it interact with your device and what are the challenges associated with that? And we have a guest today who is really experienced in working with the Android camera APIs, and, uh, Michelle I'll allow you to introduce him.

Thanks David. So on today's episode, we've invited Moji. He's part of the graphine OS project. If you recall, we talked about this project on a previous episode of the show and when we invited one of the developers on the actual operating system, but today we're talking about. The Android camera API. So we wanted to invite an expert who has worked with Android camera API. So that's why I invited Mohit who's part of the development of the camera application on graph OS. So Moji, can you just give us a brief introduction? Just tell us what is the application that you work on at graph OS. Hi

everyone. So basically I've been working on the secure camera app at Graphos over the last few months. Currently, this application is being used in production by Graphos for all this devices, which is on replacement to the default USP camera app. Apart from that, you can always find the application on the play store, which so we support under devices starting off. Android

10. Yeah. Thank you. Mohit. So he's working on a camera application and of course, in order to develop a camera application that supports multiple devices, you need access to the Android camera APIs. But as David mentioned at the beginning of this show, for those of you who are using Snapchat, Instagram, you're wondering like, how do they work and why do they not. Nearly as good photos as their stock camera app counterparts. It's a really tricky and complicated question to answer. And if you look on the iOS side of things, that's not true. Snapchat, Instagram you'll take pretty much nearly identical photos. If you use the stock iOS camera app, as you do with those social media apps, but on Android, you'll get very different results depending on. Why the issue of course stems from the big F word. The one we always bring up on pretty much every episode on like every newsletter it's fragmentation, but how exactly does fragmentation affect things? There's two aspects. There's camera, hardware, fragmentation, and there's camera, software fragmentation. First of all, hardware, fragmentation. This is like the reason why everything is the way it is and why camera software for fragmentation stems from. This one's kind of easy to imagine. Hardware is fragmented because whereas on iOS, you would have one manufacturer. You have apple who makes a handful of phones and they control the software stack from top to bottom. And there's only a handful of image sensors to consider. On the other hand, Android, you have thousands of different device models from dozens of different brands, each running their own operating system, fork of a O S P with different Silicon from media tech, Qualcomm, et cetera. And thus they have their own image, signal, processor, implementations, and then they also source image sensors from different vendors like Samsung and Sony. So you have all these different combinations of hardware and software to consider in the Android. So if you're a app developer looking to implement to create a camera app that supports all these different combinations of devices, you're gonna have a bad time because there's so many different quirks and different capabilities of devices that it's, it's a staggering mess to consider. So Moji, I wanted to ask you just taking a step back and looking at the overall picture. Do you think it was inevitable? The Android camera implementations, like the situation would've become a mess. Was there anything that Google could have done or maybe mandated in the early days of Android? So he wouldn't get to this.

So before we speak about how we could have solved this problem back then and diving deeper into this discussion, it is important for us to understand what the problem really is. So the messy Android camera implementations that are often found in apps that directly use the camera one or the camera to API targeting a wide range of devices are mainly attributed to the OS or vendors. Not. Implementing these standard APIs with the hardware that the device relies on this in tone leads to other apps, somehow working around those unexpected issues from time to time while trying to solve this may sound as simple and bad as short debugging. Analyzing, what could have possibly gone wrong since having access to device specific code isn't always feasible. It isn't actually the case. It's much worse than that. Workarounds that work for a specific device may or may not work for in though. Or could even worse cause existing support for another device to break this leads to having an entire mapping of device specific implementations, just for a single functionality, even for a simple camera app, that's expected to work across a variety range of standard Android devices. Now try imagining this for every possible camera functionality and more importantly, your code and project that was meant to fulfill a different set of requirements, but is now busy, heavily investing resources, working around device specific bugs. All these issues could have greatly been resolved if Google or the O HHA back then. Stricter compatibility, test suites and compatibility definition, document requirements. That's CTS and CDD requirements would have been developed, promoted and well enforced instead of the current scenario where the vendors don't really need to get their devices CTS or CD compliant certified before releasing that device into the market. The scope of these CTAs could have been expanded, further, more requirements that are described as recommendations could be made mandatory by enforcing the passing of those CTS CTAs documents for vendors. Apart from that, they could have also designed better camera APIs from the beginning that are not needlessly complex for both the app developers and the camera driver or hardware abstraction layer implementations, to handle that by making the overall process for writing compliant code. Much easier. So I

guess I, I have a question here and it's kind of. philosophical. If Google is more strict about this and had a much more rigorous CTS process around camera, API and compatibility and features, how much do we think this would reduce the amount of innovation that has happened in handsets? Because they've been able to do things. Break Android's camera framework, essentially because they won't work with third party apps the right way, but they will work with the vendor's camera application where they've developed this very special functionality, whatever it is, super zoom or some kind of special portrait mode or lighting filters even can be stuck behind that wall now. So I guess that's my question. You know, if Google is more strict, would we see. Fewer features, camera features on phones

for that. Like we could separate those implementations. And like, for example, for the basic functionalities that other applications are expected to use, we could have hard and separate implementation for them. And in order to add those additional features such as. The filters that we see on modern phones, we precipitated them into different implementation that could have been interfaced on top of the existing, basic implementations. And for the initial basic implementations, we could have ensured that all the test cases passed and we could have, as I just said before, we could span the CTS further and. Requirements that are described as recommendation in the docs could be made mandatory by enforcing the passing of the CTAs or CGD for the vendors that who want to release their devices into the market.

So I have a kind of different take on this question that you asked David. So like, Would innovation be hampered by enforcing camera compatibility with third party applications. And I think because of the way things work right now, I think the answer would be yes right now. Like there, there are ways that OEMs could implement feature parody between what their stock camera has access to. And what third party camera apps have access to. Of course, as Moit mentioned, Lee APIs themselves. They're a little frustrating to use and they're a little difficult to use and Google has made strides over the years to improve them, but there are APIs. Like they, they could expose a lot of the functionality they offer in their stock camera app. But why don't they the question then? Like what would imposing a CTS requirement actually do? And I think that would just significantly delay the launch of any devices. The way the Android market works for now is like, if you're in like a, a Chee or a shams competing in Southeast Asia, right? You don't have time to focus on making sure your latest mid-range phone, a camera app, your stock camera app, and third party camera apps have access to the same functionality. Right? You gotta get that phone from concept to design, to testing, to launch within like maybe a year or less for some of these like mass market phones. And if you're doing that with a whole range of, you know, budget mid-range and sometimes high end phones, you just don't have time and resources to invest in making sure all of the innovative features that you wanna market on your phone, on your stock applications will work the same on third party camera apps, because there's just so many different considerations to implement. Like why bother doing.

And the solution for the same could be like Google or the open handset Alliance could have probably created a set of designed a set of standard classes and tied up with certain hardware camera companies and probably given out the code for each of those standard classes in the a S P source code itself. So that would even ensure quick delivery. The devices in the market, as well as we could have an fallback class for all the other vendors that wish to have their own well research implementations. And we could over there then see the CT C D thing in a much more stricter way over there. So like tracking those issues might, would be much easier than having the mess we currently are in. So like, that's just an out of box solution. Like I haven't really worked with hardware and all, so like there's an abstract solution away.

That is a good point that you brought up though. Google should. I think that would be the most effective solution. Bypassing working with OEMs and working directly with the is P designers and working with the image sensor vendors, because OEMs, they have multiple different models, but the is P and the image sensor vendors. They're distributing just a few specific products and they're writing the drivers. So if Google could get them to standardize the way their drivers interact with Linux and Android, then that would go, that would do wonders. For how all that propagates through the market, but there are some complications to consider. First of all, is there even enough of a desire from consumers for OEMs to actually like bring feature parody between stock camera apps and third party camera apps? We do. We have seen, you know, some marketing from Samsung and Google, you know, the partnerships with Snapchat and Instagram and whatnot. So there is clearly some potential there because if they're not gonna be marketing these features, if they think people don't care about. Are they even legally allowed to expose some of the features that they offer in their stock camera app. There are a lot of vendors behind the scenes whose tech are in smartphones that you've never heard of. There are some facial recognition vendors who provide their software implementations to smartphones. We don't know the exact terms of their agreements, but maybe their terms say that you're only allowed to use this in your camera app. And we don't want our technology being used in by any arbitrary third party app, because we don't. A licensing fee from them. So like what would happen in that situation? Would the OEM even be allowed to license, maybe their BD mode drive from some third party tech, maybe, maybe not. So that could be one of the reasons why, you know, these features aren't being across the board, exposed to third party apps. So mahi, I kind of wanted to ask you, there are ways for OEMs to expose features to third party apps through camera APIs. There are also a lot of features that aren't even possible that they can't be exposed to third party apps simply because the Android camera APIs don't provide a way to expose those features. So what exists, what are some examples of, of features that say you can't implement and secure camera app? Because they're just not an API for it.

So the secure camera that web graph has mainly focused on providing the most simple features to the users initially. So we, for the timing have don't have any search fusion, mind that cannot be implemented, but we are currently sticking to only using the camera cyber to ensure that, uh, like we don't end. Spending too much time on dealing with the device specific cos. So in my experience, there are not really many examples of such cases in general. Like the camera APIs that Android provides while being complex are highly extensible nature and hence implementing any valid feature with some additional code or liability. Our support library. Isn't impossible as search, but of course there could be other limitations that could be based on the scope of the project, or maybe how practical is it to implement a certain feature in terms of its maintenance or maybe the hardware limitations that could. Probably make it impractical to have a certain feature and quite a lot of devices that the project targets. So like mainly depends on the, how the developers and perceive the problem and like what the scope of the project is and how much time and efforts they are willing to give for it. And of course they can be hardware limitations apart from that.

Right. So can you just give me some examples of features that you can't implement because of API limitations? So like portrait mode, That is that something you can implement beauty mode? Is that something you could implement, et cetera? What are some features that you can't because of, you know, a lack of support in the API? So

like one of the straightforward examples for that could be the vendor extensions that the camera X library provides, which aren't available for most of the device in the market, which mainly include the portrait mode and the night mode

could that include augmented reality features, for

example, Yeah. Like we could do that, but as far as I know, like the Android camera repairs don't support anything related to augmented reality, but we could surely use some external library for the same.

Yeah, that's a good one. David pixel camera used to have a AR stickers. I think they were called in the camera app directly integrated and there was just no way like that wasn't exposed at. Two third party apps via API. Like of course there's a separate AR core API. That's part of Google play services, but the actual AR sticker feature in the Google camera app, they're just it. I wasn't exposed to third party apps as far as I'm aware. So speaking of the camera API, we've kind of talked about the hardware side device fragmentation side. Now I wanted to actually talk about the API itself. I think that's probably the most interesting part. The thing people are actually wanting to hear about clearly they've evolved over the years. Right? Mohi mentioned we had camera one and we have camera two and I'm sure people have also heard of camera Hal three. So like what do these numbers mean? And we'll get to that in a bit, but I wanted to talk briefly about the evolution of the camera API and how they work. It's only recently that the camera APIs have become not a nightmare to use. As Malik mentioned, the secure camera. App uses the camera X API, which is part of the reason why camera app development has become simpler. But in the past most camera app makers had to use the framework camera and they were kind of a nightmare to implement from what I've heard, reading on, uh, developer forums and people speculate the reason Snapchat for many years used to just take a screenshot of the view finder instead of actually using the camera API directly to take a photo is because they'd rather. Deal with using a, a low quality screenshot of the view. Find to actually implement the on every device they wanted to support which remember this is Snapchat. So that's like millions, tens of millions of users on multiple different devices. Then rather infamously, a few years ago, moment who sells camera, hardware, accessories, and they also make an app for iOS called pro camera. They actually tried to port their pro camera app to Android. But after two years they just gave up. They just said we quit. And they gave the reason to nine to five Google. They said that the reason we quit is because of fragmentation. They had this chart that showed here are all the features we support on pixels on Samsung devices. And like there's is like a whole bunch of green, whole bunch of yellow, whole bunch of reds. It's just so inconsistent with what they had to support. And there's so many different models. It just became not really feasible to support without investing. significant man hours. So why is this such a nightmare? Like what exactly left moment to quit? So let's talk about the Android camera architecture, a. So at the low level, you have the drivers for the actual camera slash image sensor on the device. Those are written image, sensor vendors, I Samsungs and Sony. Those are distributed to OEMs for integration into their builds. And then next in the pipeline, all the raw Bayer data from those image sensors are processed by the image signal processor. That's part of the SOC inside device. And of course, that image signal processor is developed by another company, the Qualcomm media tech, or Samsung. That ISB does processing of its own on the raw aired output from the image sensors. That doesn't happen if you know, you're taking off photo, of course, but that's another topic entirely. It's a feature that's not supported on every device. So you have the drivers that are closed source and provided by the image, uh, sensor vendors. And then you have the, is P architecture implementation. That's written by the Silicon. Both of those are black boxes, pretty much to camera apps. Like you have no insight into exactly what capabilities they have or their data sheets. You can instead only rely on what capabilities they expose to the framework, which is determined by the hardware abstraction layers that are written by the OEM slash the vendors. So apps on the Android side, they. With the camera hardware using the camera two API, which is the framework API that interacts with the underlying camera service and Android, which then interacts with the camera, hardware, abstraction, layers, hardware, abstraction, layers to those. You don't know. They define the standard interface between the higher level Android framework and the lower level camera driver. And as I mentioned, that implementation is what defines what capabilities are exposed to. So there are multiple camera. How interfaces that OEMs have to implement. There's a camera provider, how, and there's a camera device. How, but the problem is that OEMs aren't required to implement a recent version of each. How nor are they required to implement every capability introduced with each how version. So as Mohi mentioned, Google could update the CD and the CT. To test for more recent versions of these Hals and see if OEMs have actually implemented them and defined certain capabilities, but they don't right now in order to pass certification, if you launch a device with Android 10 or later, you only have to implement camera device, Hal version 3.2 or later, Hal 3.2 was actually introduced all the way back with Android 5.0 lolli. Even Android 13 still has backward compatibility with how 3.2, even though the latest, how is 3.8, which added support for a really basic feature, the flashlight brightness control. So as you can see, a lot of features are being added along the way. Some even quite basic, but because there's no specific requirement to implement a specific it's, it's all gonna depend on the OEM and the Silicon vendor. Like what exactly are they willing to implement and expose to third party camera apps?

Michelle, you know, before we drop into the next part, the reason this exists is basically the Google requirements freeze, right? Because Google has helped letting the phone vendors lag behind, essentially buy up to what, like three years effectively or more, or is this a different situation?

That's actually something that's going to make this situation even worse because in the past, the hardware abstraction layers could be updated. They were basically required to be updated whenever a vendor like Qualcomm would have to update their implementation to support a newer Android version. But with GRF, as you mentioned, say, a device launches with Android 11, then whatever hardware abstraction layers that device ship with will never be updated until that device reaches. Once they're updated to Android 15. Because Google's guaranteeing backward compatibility with vendor implementations that are re letter versions behind. So device running Android, like to Android, 12 Android, 13 Android 14, and keep the same hardware abstraction layers, and the kernel interface that was introduced with Android 11, for example, the camera Hal 3.8, which introduces support for flashlight whiteness. Device that launches with Android 12 is not gonna have 3.8 because that version wasn't even introduced until now until this release. So device upgrading the Android 13, probably won't even get support for this basic flashlight brightness control feature, because vendors aren't gonna go back and, and update how so? Yeah, it's, uh, it's, it's a mess because there's no real requirement on. OEMs are required to expose and actually implement in their house. But of course, Google chugs along. They keep updating the underlying hardware abstraction layer interface. They keep defining new capabilities in each Hal version. For example, with camera device, Hal 3.5, they introduced the ability for OEMs to define zoom ratio, which actually provides support for optical camera, zoom capabilities to third party. Then, of course, as I mentioned, 3.8, introduce a flashlight, brightness control. And then also in Android 11, with 3.5, they introduced the ability for OEMs to expose bouquet support. So like they have to define these certain constants in their hardware abstraction layer to expose optical camera, zoom capabilities, and bouquet. It's basically up to the Goodwill and the willingness of the OEM and the vendor they provided the house from, in order to implement these features. So I wanted to ask you now, Mohit, if you are aware of any way your work has been affected by the, how versions and implementations across different devices, like, are you able to access certain features that are supported by Android technically, but because the device doesn't have the right how version, you know, you can't actually use it and secure camera. So

the secure camera app that we are working on graph know is at least for most of the time hasn't directly relied on the hardware abstraction layers or the camera one or the camera to API. The main reason why we push for such a design is the device specific issues that we would otherwise have to spend a lot of our time to deal with that could unexpectedly blow up as more non camera supported features accumulate. And as a app gets used by more kinds of devices, which is quite closely related to the fragmentation issue that we were discussing earlier on this podcast. This valuable time could be spent by contributors on other places that require more help and attention and graph. However we did recently make a few exceptions rule by supporting an AI story that relies on the camera to interrupt API and introducing experimental support for Zeel that has not been supported by a lot of devices for the time being. So coming back to the question, no, like we haven't really faced any issues or troubles while dealing with different versions of file, primarily because we haven't really dealt with them. In the code of a camera application.

Yeah. At least with project travel introduction and the vendor test requirements surrounding how releases you can be assured that at least the main rear facing and front facing camera will be operable on any given device. So for example, if you were to take an device that supports project trouble, such as the Lenovo tab, K 10, and you were to flash a generic system image of Android, 11, 12, or 13 onto it. Very very likely you could just open the, a SB camera app and the rear facing camera and the front facing camera would work. And the reason is because that's something that Android actually mandates testing for part of the vendor test requirements is that the camera has to be at least the main rear facing and the front facing have to be operable. But of course, nothing else is guaranteed. You're not guaranteed to have the image processing models. And the add on camera features that the stock camera app has most devices are at least most smartphones. These days have multiple rear cameras. If you try to use those on a GSI, you probably won't be able to actually access the secondary cameras. Android actually does provide support for third party camera apps. Use those cameras. But the issue is that it's through an API that Google introduced an Android nine called the multi-camera API. So what OEMs have to do is they have to define logical camera devices, logical being like they're not physical. These logical cameras are composed of two or more physical cameras that point in the same direction. So for example, you can have a main camera and a telephoto camera, and you can create a logical camera. Is composed of the main and the telephoto and the benefit of doing that is that for the third party camera app, they see it as one camera and it can change between the two basically seamlessly. So if you're like zooming out from one X to five X, the underlying camera, how would basically handle the switching between the lenses seamlessly? The app itself wouldn't have to manually detect, oh, I'm supposed to change lenses here. The logical camera interface would define that change. The only problem of course is this is yet another thing that OEMs don't have to implement. Google has tried to make support for multi-camera implementation mandatory, but they actually reneged because that conflicts with GF, which is something David mentioned earlier, they wanted to make it mandatory for all. And Android 12 launch devices to support multiple cameras. They want to make it so that if a device ships with. Or more rear cameras on the back. The OEM has to define at least one logical camera for those rear cameras. But the problem is that because devices can launch with Android 12, but running Android 11 vendor software because of GF, Google can't mandate that because that would make those devices not capable of supporting this feature and thus not capable of launching with Android. so, uh, Mohi, I wanted to ask you a bit about your thoughts on the multi-camera support on Android. What are your general thoughts on Android support for exposing multiple cameras and actually using them in apps? Do you think a requirement for OEMs to support the API would actually make a difference? The

multiple cameras that we often find on modern Android phones for the most of the time have had separate IDs assigned to each of them, such that we could choose a camera by their vendor specified IDs. A while ago, Android came up with some support for a multi-camera API. That can be used to merge multiple cameras into a single logical camera instance, having its own ID that is often used to enhance these zoom levels provided by a single logical camera. Although the API again, can be certainly used for several of the purposes. Zooming is just one of them for the secure camera application. Yes, it does make a difference in the range of zooming levels that we provide to our users. The multi camera zoom. Expected to be implemented by the vendor in our case. So like for example, we've come across many instances, especially in case of non pixel devices where the device did have the hardware required to enhance the zoom range, such as the ultraviolet links to support zooming out a bit further. But the devices themselves didn't support zooming below the default Onex, mainly because the vendors didn't actually implement them for their own reasons, such as there was in case where the device actually predated the feature itself. So the vendor themselves could not no longer roll out updates since they no longer supported that device. So now having a gen solution by maintaining a database of physical camera IDs for. All the also devices on end wouldn't really be feasible. And perhaps it isn't even a viable solution for the camera X team, as there's a possibility that these IDs may just change in between updates, which could again, drastically increase the complexity and overall efforts required for maintaining this entire work around. Hence like according to us, it might be better to mandate multi camera zoom in the test suite for all. Upcoming devices that otherwise have the hardware to support it.

Okay. So yeah, you just mentioned something. I actually wanted to talk about a little bit next it's about the camera IDs and how actually apps are actually supposed to be able to control the camera hardware using the API. So I've already mentioned the camera two API, which is the actual framework API. That enables enumerating IE listing the camera devices that are available on the device, or that are exposed to Android. That API also lets apps connect those devices, configure the outputs, send capture requests, and then read resulting metadata and image data. So this API it's, uh, a little difficult to use. Because there's a lot of legwork, a lot of preparation work that apps have to do before they can actually start doing capture requests. First of all, since OEMs have to actually expose the capabilities for each individual camera to the camera, too apps then actually have to probe. What features are supported by this specific camera device on this ID. If you're ever used one of those camera, two API probing apps before you've probably gotten like a high level summary of what they. You see strings that say limited or level three, those strings basically tell you, okay, this specific camera supports this list of features, but then it's up to the actual camera app to decide, okay, because this specific sensor supports this feature. This is what I'll enable inside the app for when the user is using it. Whenever you are trying to use a app that uses the camera two API on a specific device, you may notice different features than are what are available on other devices used by other users. So Mo I'm sure you've at least heard or seen bad reviews or complaints from users who report a feature or two is missing from their device. They blame you. Or, you know, the graph OS project, secure camera developers. One, in fact, it's because the OEM didn't expose the feature when writing their interface for camera two. So how do you deal with this? Like, what exactly can you do about this? So

while most of the features that our app provides are available across all under devices that we support. There are a few features that we only support when the OEM implements and exposes them for other apps to use. Just as discussed earlier, we have recently very often heard of complaints regarding our app, not supporting zooming. The default range, despite the device physically having support for it on a lot of non pixel devices. Apart from that, the major complaint that we have received is regarding the support for vendor specific extensions or modes that the user thinks are missing while they actually need to be implement either exposed or, or UN implement. By the vendor themselves for a app to show up those modes on a given device, based on where the issues reported, we inform the user about it and explain them, why does a camera app not support the given set of features on the device? And either try to look for some timeline for the scene that is if it's easily available or request the user to inquire regarding the same with their respective vendors, if by any chance it is something that the camera X team needs to fix on their. Or is related to pixel devices, not implementing something correctly. We reported to the camera X team and discuss how the issue needs to be resolved or could be further addressed. They're quite friendly and supportive. So reporting issues to the camera X team has never been on hassle.

We've actually at least mentioned camera X a lot, but we've never actually explained what it is for those of you who don't know camera X is something that Google introduced back in IO of 2019. And the secure camera app that Mohi works on actually uses camera X. It's one of the few apps that I'm aware of that proudly states that it uses the camera X API. I wanted to ask you Mo this is a very basic question about camera X, just so our readers, our listeners can understand what exactly it does. What benefits does camera X offer over camera to? And like, why does the secure camera use it?

so essentially camera X is a better designed, simpler, higher level API, which is packed codes compatible with the camera and API. But it also as well takes advantage of the camera too. And the advanced camera to features whenever they're available. So essentially. This ensures that the camera features that you develop are of high quality work across a wide range of devices while allowing quick and steady development of your application. Thanks to the use case based modeling of the classes that, and the high level simplicity that the library has to offer. Apart from that, it has a very high quality and highly performing cameras of face stack. That's far better than the vast majority of. Camera applications that are currently available as one short example in the recent experimental Edison support that they recently added to the library, they included a ring buffer and some pre-processing, which was mainly to improve the overall performance of that feature. And hence the quality of the ZL feature that they provide. ZL basically essentially stands for zero shorter lag, which can be used on the latency mode to take faster shots from your camera application. Apart from that, the overall code quality of the implementation of the camera features tends to improve a lot while using the camera X library in terms of readability, which in turn drastically reduces the overall maintenance cost. The reason being that camera X works around a lot of device specific. Cos impacting a specific or a certain range of devices. So which in turn it, it makes the code a lot more readable. So that way, uh, by making any changes or perhaps trying to add any custom implementation over gets a lot more easier. Apart from that one can always expect good support from the camera X team for the device specific issues that the users face. With the officially supported camera X features. So basically this in tone has also encouraged us to investigate such issues and bug reports further and report them to the camera X team, which could potentially help us help a lot of organizations for your to come while using the camera X library. One can always fall back to using the camera to APIs using. Camera to interrupt API, which offers classes that are compatible with the main classes that the camera library itself offers. So, but however, one must note that, um, the support is limited to an extent that is not all features of the camera to APIs can be accessed via camera.

Thank you Mohi for the rundown of camera X versus camera two. So for those of you who listened to our previous episode on modern Android app development, we talked a bit about jet pack, support libraries and how they simplify app development. Well, camera X is actually one of the support libraries that's under jet pack and like the other libraries, you know, it simplifies developing across specific Android OS versions because camera two is an API updated with the OS itself. So, of course there's gonna be OS specific API methods. There's gonna be different behaviors depending on the OS version and the way the code is written, what each method accepts and uses. So like it's there and it's updated with each OS version and it's little complicated to use. So what camera X does is simplifies all of that by wrapping around it and basically just letting developers not worry. The underlying implementation in the OS and just use camera X, but under the hood camera X just passes those calls to camera two and simplifies that interface for app developers. Camera X, of course, because it's a newer API and it's actually just wrapping what's available in camera two. It's not fully interoperable with all the features that are available with camera too. As Moit mentioned, that's why. The camera X offers the camera to interop API, which lets some apps use camera two APIs when there's not a camera X equivalent, but of course that has some of its own limitations. And then one of the other things that, uh, we've been talking about a bit or MOS mentioned several times are vendor extensions. So that's actually something that was introduced alongside camera X, actually a little bit later in camera. X's, uh, life cycle that came about. So vendor extensions, for those of you don't know, basically it's a library that OEMs create. That exposes specific features to third party apps. So for example, an OEM can write a camera X vendor extension for HDR face free touch and night mode. And that would allow apps that are using the camera X API to actually use those OEM provided effects in their own apps. So like if an OEM has its own HDR implementation, a third party app could see that this device has a vendor extension for HDR, and then they could use that in their own camera. I wanted to ask you Mohe, the secure camera app uses vendor extensions. Can you tell us about the implementation in the secure camera app? Like what vendor extensions do you use if they're available and what are some of the challenges with vendor extensions as they are right now?

Secure camera currently supports all the five standard vendor extensions that the camera lab degree has described in its talk. However, the availability of these vendor specific extensions, such as bouquet, HDR, face research and night. Site depends upon whether the vendor or OEM of the device decides to expose them to other apps by these standards defined by the official documentation of the camera X lab degree. Currently, from what we have known from our community of active users, not a lot of devices, support camera, X's vendor specific extensions, all pixels, the devices that mainly target. At the time of this recording don't support any camera extension. Although in a recent pixel feature drop support for night site was added to Excel six as in camera to extension, which is unfortunately different from camera ex extensions, very few flagship Samsung devices support all the camera, X vendor specific extensions. However. In one of the events that was recently conducted under Google IO, 2022 Google said that they'll be launching their new extensions through camera versus camera two, where they'll be providing their own fully software extensions for when. Entry level devices that don't have them yet starting from the bouquet portrait extension mode. We're quite optimistic about it as this could help us giving a more consistent experience for our application across all the supported devices, including our place to users. And honestly, if this gets implemented as expected, it might shortly be both the long weight of camera extensions. Finally arriving for pixel devices.

Honestly, when I first heard about vendor extensions in camera X, I believe back when I was at XDA, our developer author, Zachary wrote an article about vendor extensions and there was a lot of hype around them. Maybe this will finally solve the feature parody issue between stock camera apps, and third party camera apps. And as the more I've learned, it seems like there's been major issues with the implementation as you brought up, even Google's own pixel. Don't support camera X, vendor extensions, which is hugely disappointing because they're the ones who are pushing developers like use camera X, use camera X. Well, they don't even support it properly for their own devices. So that's a huge disappointment. And then with the pixel feature drop, as you brought up that enabled night site in Snapchat. But the way they did that is they released a camera two vendor extension rather than a vendor extension through camera X. And you think they'd be the same thing, right? If there's a vendor extension, there's a vendor extension, but it's not only apps that implement the camera two vendor extension API, which is something that only introduce an Android 12 can use it. Whereas camera X is vendor. Extension is available across more Android versions and is something that more apps are expected to support. But then Google goes ahead and does. A camera two extension instead of camera X, which they've been pushing on app developers. So it's kinda like a, it's a, it's a really weird situation where we're in, where one side of Google is telling developers use camera X, but then the other side is like, uh, yeah, we will continue to support camera two. And here is our flagship pixel phone with an extension that's only available through this API. We're telling you not to. But fortunately, at least Google IO, that was a pretty significant announcement. As you brought up that Google itself will start providing some software vendor extensions for low end devices, starting with like a bouquet portrait mode. So all low end devices will have a vendor extension for portrait mode that apps using camera X can hook into. It remains to be seen if they're able to extend that to the other extensions that are possible, including like night. But, um, I'm not really sure if Google plans to expose its brilliant night site feature to all low end devices that are on its. So we we'll see what happens with that. Yeah.

And I think that it's a, it's a question of the economics of a feature. And so Boca and portrait mode, for example, is basically democratized, um, even very low end media tech chip sets. I'm sure at this point in the camera stack support for some kind of portrait mode. So it's no longer one of those features that you would want to. To a high end device and not giving people access to that, especially in a consistent way is probably worse for the platform than just coming up with a standard implementation that works for everybody. And Google can keep adding value on their pixel phones by introducing things like IOS's portrait, lighting style mode. Which is much more advanced and uses a lot more algorithms to get the output. So, I mean, a lot more algorithms. I sound real smart when I say that. Uh, but you, you understand what I mean? So these initial extensions are probably a good sign of what Google thinks is most important to largest number of camera users, portrait mode. Obviously being a big one. And I think that over time. Yeah. Well, you will almost certainly see more of these features. They used to be gated to the high end devices. Probably just start to become a standard smartphone feature, like having tool cameras, all of this stuff sort of democratizes over time. Right.

Yeah. And Mo said that he was optimistic about the feature of camera. And honestly, I can, I can understand why they have been slow to implement some basic features. Like version 1.1 of the library is what brought video capture support. And I think that just came out earlier this year and stable. And then one of the features that they've been working on in a O S P support for CSL or zero shutter lag, but we'd mention that as well. So like, these are features that you've already had access to with camera. But they're not in camera X yet. And then Android 13, for example, introduces HDR video capture support in camera two, but I don't think there's a, there's an equivalent API through camera X, but if you're looking to implement a very basic app that uses some very basic camera capture functionality, it's easier than ever thanks to camera X and other libraries. So for example, if you wanted to do QR code scanning or barcode scanning, Google play services has a drop in solution for that. But there are also, you know, open source solutions that you could implement with camera X. The ability for app developers to add camera functionality into their applications is easier than ever. And it's not like the realm of professional camera, app developers anymore.

And with that, this is where we do our S per plug, because if you've been listening this entire episode, you've understandably come to appreciate just how complex the situation is with cameras on Android and how the OS interacts with the camera and how applications then interact with the operating system, which interacts with the camera. do you want to build for camera X? Do you want to just keep building for camera two? Do you even need direct access to the camera to accomplish what you're doing? It depends. And if you find yourself in a situation where you're trying to build the device that needs to do something like Michelle said, capture a QR code or scan a barcode or recognize a face. those are obviously very different scenarios. The camera needs to be able to do different things to accomplish those tasks. So if you're wondering, okay, is the camera on this device going to be suitable for my work purpose, whatever it is, I'm capturing faces or barcodes or taking pictures of cats. I don't really know what you're doing with the camera, but if there's something specific and you're wondering, okay. Is there an expert out there who can tell me, what can I actually do with this camera? And is it extensible? Is it scalable? Can I do this with a bunch of different cameras running Android, come talk to us at Esper. This is the kind of thing we deal with regularly differences in implementations, across vendors and hardware and software. That is our bread and butter and understanding how to make that experience consistent for you and your devices is what Esther's all about. So if you want to learn more about ESER check us out at SPER dot, I.

Thanks David. And with that, I wanted to give Mohi a brief chance to tell us about where people can follow him as well. So Mohi, tell us about where people can follow your work. So yeah.

Could follow me at my official getup page. So it's get.com/. MH she, which is M H S H E T T Y or could follow me on LinkedIn as very, as a student at the shin engineering college, you could follow me there as well. So that's where I'm mainly active.

And if people want to follow, if people want to try out your work, they can go to the Google play store and download the secure camera app, or they can try installing graph OS on a compatible device. If you have one, go listen to our previous episode with the graph OS developer, Gabe, if you wanna learn more about the project and, uh, thank you for listening