Using Infrared Camera for Face Recognition on Xiaomi 845 Series with Custom ROMs Background The Xiaomi 845 series, specifically the Mi 8 and Mi 8 Pro (UDFPS), feature a dedicated front-facing infrared camera for facial recognition. This enables facial unlock functionality even in complete darkness. However, when using a custom ROM like LineageOS compiled from its standard device tree, the infrared camera remains inaccessible, forcing the system to rely on the standard front camera. Interestingly, PixelExperience ROMs successfully utilize this hardware. This article explores the reasons behind this discrepancy and outlines the solution. The Cause CameraID The first step is directing the face recognition module to use the infrared camera. Examining the PixelExperience device tree source code reveals an overlay that configures the face unlock service to use the camera with ID 5. https://github.com/PixelExperience-Devices/device_xiaomi_dipper/blob/fourteen/overlay/packages/apps/FaceUnlockService/app/src/main/res/values/config.xml <?xml version="1.0" encoding="utf-8"?> <resources> <integer name="override_front_cam_id">5</integer> <bool name="use_alternative_vendor_impl">true</bool> </resources> PixelExperience uses Motorola's face unlock solution. Many other custom ROMs, however, utilize AOSPA's ParanoidSense. Therefore, the PixelExperience overlay method isn't directly applicable. Inspecting the ParanoidSense source code shows that it uses a system property, ro.face.sense_service.camera_id, to identify which camera to use. https://github.com/AOSPA/android_packages_apps_ParanoidSense/blob/uvite/src/co/aospa/sense/camera/CameraUtil.kt#L32 val cameraIdProp = SystemProperties.get("ro.face.sense_service.camera_id") Thus, in theory, simply setting the ro.face.sense_service.camera_id property to 5 should direct it to the infrared camera. The vendor.camera.aux.packagelist Property Merely setting the CameraID property is insufficient; the face unlock module would simply fail to access the camera. Delving into the Android framework code reveals that the system, by default, hides auxiliary cameras (AuxCameras) beyond the primary front and rear cameras. The infrared camera falls into this AuxCamera category. Using LineageOS framework code as an example: https://github.com/LineageOS/android_frameworks_base/blob/lineage-21.0/core/java/android/hardware/Camera.java#L295-L301 /** * Returns the number of physical cameras available on this device. * The return value of this method might change dynamically if the device * supports external cameras and an external camera is connected or * disconnected. * * If there is a * {@link android.hardware.camera2.CameraCharacteristics#REQUEST_AVAILABLE_CAPABILITIES_LOGICAL_MULTI_CAMERA * logical multi-camera} in the system, to maintain app backward compatibility, this method will * only expose one camera per facing for all logical camera and physical camera groups. * Use camera2 API to see all cameras. * * @return total number of accessible camera devices, or 0 if there are no * cameras or an error was encountered enumerating them. */ public static int getNumberOfCameras() { int numberOfCameras = _getNumberOfCameras(); if (!shouldExposeAuxCamera() && numberOfCameras > 2) { numberOfCameras = 2; } return numberOfCameras; } As shown, even if a device has more than two cameras, if the shouldExposeAuxCamera() function returns False, the system reports only two cameras, preventing access to any camera with an ID greater than 1. Let's examine this function: https://github.com/LineageOS/android_frameworks_base/blob/lineage-21.0/core/java/android/hardware/Camera.java#L264-L278 /** * @hide */ public static boolean shouldExposeAuxCamera() { /** * Force to expose only two cameras * if the package name does not falls in this bucket */ String packageName = ActivityThread.currentOpPackageName(); if (packageName == null) return true; List<String> packageList = Arrays.asList( SystemProperties.get("vendor.camera.aux.packagelist", packageName).split(",")); List<String> packageExcludelist = Arrays.asList( SystemProperties.get("vendor.camera.aux.packageexcludelist", "").split(",")); return packageList.contains(packageName) && !packageExcludelist.contains(packageName); } The logic is clear: if the current application's package name is listed in the vendor.camera.aux.packagelist system property and is not in the vendor.camera.aux.packageexcludelist property, the function returns True, allowing access to auxiliary cameras. The solution becomes evident: add the ParanoidSense package name (co.aospa.sense) to the vendor.camera.aux.packagelist property. The Solution Add the ParanoidSense package name co.aospa.sense to the vendor.camera.aux.packagelist property: https://github.com/YoriInstitute/crDroidAndroid_device_xiaomi_sdm845-common/commit/28fe5bc41d49b31b4acb840bf167b70d70a40c61 Set the ro.face.sense_service.camera_id property to specify the correct camera ID: https://github.com/YoriInstitute/crDroidAndroid_device_xiaomi_equuleus/commit/67518f130650e4592b5f4c7210248072058d48cc https://github.com/YoriInstitute/AOSPA_device_xiaomi_equuleus/blob/topaz/device.mk#L397-L399 In the crDroid-modified version of ParanoidSense, which is placed in the system_ext partition (refer: https://gitlab.com/crdroidandroid/android_packages_apps_FaceUnlock/-/commit/545688260eb32ba19f348e84e3cae89ba29f20d1), this property should be added to the corresponding system_ext prop file.

Brief Understanding of Some Files and Concepts in AOSP Device Trees {alert type="warning"} The following content largely represents my personal understanding of some concepts within device trees and is not official definitions. Corrections are welcome if any errors are found. {/alert} The common contents of a device tree can be broadly categorized as follows: Android.bp Various Makefiles, with the suffix .mk prop files, with the suffix .prop overlay, placed in the overlay folder sepolicy, i.e., SELinux rules, placed in the sepolicy folder manifest.xml and FCM (framework_compatibility_matrix.xml) extract-files.py, setup-makefiles.py, and proprietary-files.txt Dependency files ending with .dependencies Various other configurations Android.bp This is the configuration file for the AOSP build system, Ninja. The device tree doesn't rely heavily on it (main content is usually in Android.mk and other Makefiles). It's generally used to introduce other soong_namespaces, allowing references to Modules from other directories. Let's take the LineageOS socrates device tree as an example: // // Copyright (C) 2024 The LineageOS Project // // SPDX-License-Identifier: Apache-2.0 // soong_namespace { imports: [ "hardware/xiaomi", ], } Here, it declares the import of the namespace from hardware/xiaomi, enabling PRODUCT_PACKAGES to reference items from hardware/xiaomi. Makefile Android.mk This file defines the entry point for the entire device tree (it should be). Again, using socrates as an example: # # Copyright (C) 2024 The LineageOS Project # # SPDX-License-Identifier: Apache-2.0 # LOCAL_PATH := $(call my-dir) ifeq ($(TARGET_DEVICE),socrates) include $(LOCAL_PATH)/vendor-symlinks.mk include $(call all-subdir-makefiles,$(LOCAL_PATH)) include $(CLEAR_VARS) # A/B builds require us to create the mount points at compile time. # Just creating it for all cases since it does not hurt. FIRMWARE_MOUNT_POINT := $(TARGET_OUT_VENDOR)/firmware_mnt $(FIRMWARE_MOUNT_POINT): $(LOCAL_INSTALLED_MODULE) @echo "Creating $(FIRMWARE_MOUNT_POINT)" @mkdir -p $(TARGET_OUT_VENDOR)/firmware_mnt BT_FIRMWARE_MOUNT_POINT := $(TARGET_OUT_VENDOR)/bt_firmware $(BT_FIRMWARE_MOUNT_POINT): $(LOCAL_INSTALLED_MODULE) @echo "Creating $(BT_FIRMWARE_MOUNT_POINT)" @mkdir -p $(TARGET_OUT_VENDOR)/bt_firmware DSP_MOUNT_POINT := $(TARGET_OUT_VENDOR)/dsp $(DSP_MOUNT_POINT): $(LOCAL_INSTALLED_MODULE) @echo "Creating $(DSP_MOUNT_POINT)" @mkdir -p $(TARGET_OUT_VENDOR)/dsp ALL_DEFAULT_INSTALLED_MODULES += $(FIRMWARE_MOUNT_POINT) $(BT_FIRMWARE_MOUNT_POINT) $(DSP_MOUNT_POINT) endif It shows that if TARGET_DEVICE equals socrates, the build system will proceed to include all Makefiles in this directory and define some mount points here. AndroidProduct.mk This file is typically used to include the os_codename.mk file below and define COMMON_LUNCH_CHOICES. It defines the types supported by the device during lunch (generally distinguishing between user, userdebug, and eng builds). Note: COMMON_LUNCH_CHOICES seems to have been removed in Android 14 and above builds. BoardConfig.mk {alert type="info"} In common trees (if they exist), this is often named BoardConfigCommon.mk {/alert} As the name suggests, this is the board configuration file for the phone. It generally defines things like CPU architecture, partition sizes, kernel parameters, etc. It usually includes the SEPolicy folder, various prop files, manifest.xml, FCM, and other hardware-related parameters. Most configurations here are generally reusable across different ROMs. device.mk {alert type="info"} In common trees (if they exist), this is often named soc.mk or common.mk. For example, it's named sdm845.mk in Xiaomi's sdm845-common and lito.mk in Xiaomi's sm7250-common. {/alert} This file generally defines services within the Android operating system, includes various libraries, copies permission files (e.g., LineageOS/android_device_xiaomi_sdm845-common/blob/lineage-22.1/sdm845.mk#L26-L62). Some of the content here is reusable across different ROMs; please decide based on the specific situation. os_codename.mk This file typically defines content highly specific to the current ROM. For example, using socrates: # # Copyright (C) 2024 The LineageOS Project # # SPDX-License-Identifier: Apache-2.0 # # Inherit common AOSP configurations $(call inherit-product, build/make/target/product/full_base_telephony.mk) $(call inherit-product, build/make/target/product/core_64_bit_only.mk) # Inherit device-specific configurations $(call inherit-product, device/xiaomi/socrates/device.mk) # Inherit LineageOS configurations $(call inherit-product, vendor/lineage/config/common_full_phone.mk) PRODUCT_NAME := lineage_socrates PRODUCT_DEVICE := socrates PRODUCT_MODEL := Redmi K60 Pro PRODUCT_BRAND := Redmi PRODUCT_MANUFACTURER := Xiaomi BUILD_FINGERPRINT := Redmi/socrates/socrates:14/UKQ1.230804.001/V816.0.11.0.UMKCNXM:user/release-keys It includes necessary configurations for the build and LineageOS configurations. Referencing different configuration files here can set the operating system bitness and distinguish between tablet and phone. For instance, including build/make/target/product/core_64_bit_only.mk means the ROM built from this device tree only supports 64-bit apps. If core_64_bit.mk were included instead, it would also support 32-bit apps. Including vendor/lineage/config/common_full_phone.mk indicates adaptation for a phone-type device. For a tablet-type device, common_full_tablet.mk or common_full_tablet_wifionly.mk would be included. The section below defines device-related information like manufacturer, brand, device codename, and model. Prop Files These define Property attributes within the system, functioning similarly to prop files inside the system. They define entries required by software that relies on props for settings. It's recommended to refer to online introductions for details, as we won't elaborate much here. Overlay Official introduction: Android Overlay is a resource replacement mechanism that allows replacing resource files without repackaging the APK (res directory, not assets directory). Personal understanding: Used to replace and override specific content in the res XML of specified software within the system, allowing modifications to system settings, toggles, etc. For example: https://github.com/LineageOS/android_device_xiaomi_socrates/blob/lineage-22.1/overlay/SystemUIOverlaySocrates/res/values/config.xml#L24 {collapse} {collapse-item label="Expand Code"} <?xml version="1.0" encoding="utf-8"?> <!-- /* ** Copyright 2009, The Android Open Source Project ** ** Licensed under the Apache License, Version 2.0 (the "License"); ** you may not use this file except in compliance with the License. ** You may obtain a copy of the License at ** ** http://www.apache.org/licenses/LICENSE-2.0 ** ** Unless required by applicable law or agreed to in writing, software ** distributed under the License is distributed on an "AS IS" BASIS, ** WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ** See the License for the specific language governing permissions and ** limitations under the License. */ --> <!-- These resources are around just to allow their values to be customized for different hardware and product builds. --> <resources> <!-- Doze: does this device support STATE_DOZE? --> <bool name="doze_display_state_supported">true</bool><!--这一行--> <!-- Color of the UDFPS pressed view --> <color name="config_udfpsColor">#00FFFFFF</color> <!-- paddings for container with status icons and battery --> <dimen name="status_bar_icons_padding_start">0dp</dimen> <dimen name="status_bar_icons_padding_end">0dp</dimen> <!-- the padding on the start of the statusbar --> <dimen name="status_bar_padding_start">10dp</dimen> <!-- the padding on the end of the statusbar --> <dimen name="status_bar_padding_end">4dp</dimen> </resources> {/collapse-item} {/collapse} This modifies the variable of the same name here: https://github.com/LineageOS/android_frameworks_base/blob/lineage-22.1/packages/SystemUI/res/values/config.xml#L175 and thereby indicates whether the device supports Doze display:https://github.com/LineageOS/android_frameworks_base/blob/472a48741ac22537b3a0d1c0b87dbff2c1c2af8f/packages/SystemUI/src/com/android/systemui/statusbar/phone/DozeParameters.java#L185-L187 When writing overlays, one generally needs to reference the definitions of items in the res within the system and replace their values as needed. .dependencies Files As the name implies, dependencies. This file defines which dependent repositories are needed to use this device tree, encoded in JSON. The basic format is as follows: [ { "repository": "", "target_path": "", "branch": "", "remote": "" }, { "repository": "", "target_path": "", "branch": "", "remote": "" } ] Parameter details: remote: The remote address, corresponding to the remote address in the main source tree's manifest. This parameter is optional. If not specified, it defaults to cloning from the default remote address in the main source tree's manifest. repository: The repository address, i.e., this repository under the remote address. Required. branch: The branch corresponding to the branch used when downloading from the specified repository. Optional. If not specified, it defaults to the same branch as the main source tree. target_path: The clone target path. Required. clone-path: The clone depth, similar to the --depth parameter in the git clone command. Optional. If not specified, it defaults to a full download (same logic as the git command). revision: Can be understood as the branch. Optional. Therefore, a simplest dependencies file without the optional parameters can look like this: https://github.com/LineageOS/android_device_xiaomi_equuleus/blob/lineage-22.1/lineage.dependencies [ { "repository": "android_device_xiaomi_sdm845-common", "target_path": "device/xiaomi/sdm845-common" } ] When using commands like breakfast that automatically download supported device trees, this file is used to integrate the required repositories into the main source tree's manifest and download them together. extract-files.py, setup-makefiles.py, and proprietary-files.txt These files are used when generating the Vendor tree. Generally, the required blobs are listed in proprietary-files.txt, then extract-files.py is run to extract blobs from the system or a dump and create the vendor tree. That's all for now, more may be added later.

Enabling EROFS Support for an Android ROM with Kernel 4.9 Foreword As everyone knows, I'm a long-standing Mi 8 Pro (Under-Screen Fingerprint Edition) holdout I've always wanted to port erofs to this device (please search online for what erofs is). But after some research, I found out that erofs was merged into the Linux kernel in version 5.10, which means I have to manually integrate it into our 4.9 kernel. Materials Needed Android ROM Source Code: Using LineageOS 22.0 here. Android Kernel Source Code: Using the official LineageOS kernel for Xiaomi SDM845 devices: LineageOS/android_kernel_xiaomi_sdm845 Device Tree: Using the official LineageOS device trees, which include the sdm845-common part: LineageOS/android_device_xiaomi_sdm845-common and the device-specific part: LineageOS/android_device_xiaomi_equuleus A build server {alert type="info"} Note: Since I'm primarily focused on ROM building, this article mainly documents the process within that context. {/alert} Let's Start Setting up the Android ROM build environment won't be covered here, as there are plenty of guides online. Download the ROM source, device tree, and kernel source, and place them in the correct locations. Adding EROFS Support to the Kernel The code changes were picked from ReallySnow's msm-4.9 kernel (Thanks to ReallySnow for the support!). You can refer to these branches: https://github.com/ReallySnow/msm-4.9/tree/erofs https://github.com/YoriInstitute/android_kernel_xiaomi_sdm845/tree/lineage-22.0-erofs Before picking the commits, you need to resolve conflicts in the lz4 compression code. For the detailed relation chain, refer to: https://review.lineageos.org/c/LineageOS/android_kernel_xiaomi_sdm845/+/412614 I tried to submit these changes to LineageOS official, but obviously, they weren't merged x After picking the changes, enable erofs support in the kernel's defconfig file: Open $KERNEL_SOURCE/arch/arm64/configs/vendor/xiaomi/mi845_defconfig and add: CONFIG_EROFS_FS=y: commit 7ae5caab2c15982b3d3821c97e2f2e6126ca1281 (HEAD -> erofs, refs/published/erofs) Author: KaguraiYoRoy <[email protected]> Date: Tue Dec 24 22:49:07 2024 +0800 arch: arm64: configs: mi845: Enable CONFIG_EROFS_FS Change-Id: Ie387b03bde54700d73ccf19f0baf4db62526cc8f diff --git a/arch/arm64/configs/vendor/xiaomi/mi845_defconfig b/arch/arm64/configs/vendor/xiaomi/mi845_defconfig index 7684be360d98..69ea580fdb11 100644 --- a/arch/arm64/configs/vendor/xiaomi/mi845_defconfig +++ b/arch/arm64/configs/vendor/xiaomi/mi845_defconfig @@ -651,3 +651,4 @@ CONFIG_SOUNDWIRE_WCD_CTRL=y CONFIG_WCD9XXX_CODEC_CORE=y CONFIG_WCD_DSP_GLINK=y CONFIG_WCD_SPI_AC=y +CONFIG_EROFS_FS=y \ No newline at end of file Modifying Partition Filesystems and Fstab Open the board configuration file in the device tree and change the filesystem type for the partitions you want to convert to erofs. The example configuration file is located at $WORKING_DIR/device/xiaomi/sdm845-common/BoardConfigCommon.mk: commit 0fce86fb40ca637a8563bef6192afaea1658c881 (HEAD -> erofs, refs/published/erofs) Author: KaguraiYoRoy <[email protected]> Date: Tue Dec 24 23:37:50 2024 +0800 sdm845-common: Switch to erofs Change-Id: I9a8d6de4dcb79cd6bc177451ce3cbc4682748501 diff --git a/BoardConfigCommon.mk b/BoardConfigCommon.mk index cf73d39..700592f 100644 --- a/BoardConfigCommon.mk +++ b/BoardConfigCommon.mk @@ -78,11 +78,11 @@ BOARD_CACHEIMAGE_PARTITION_SIZE := 268435456 BOARD_USERDATAIMAGE_PARTITION_SIZE := 57453555712 BOARD_CACHEIMAGE_FILE_SYSTEM_TYPE := ext4 -BOARD_ODMIMAGE_FILE_SYSTEM_TYPE := ext4 +BOARD_ODMIMAGE_FILE_SYSTEM_TYPE := erofs BOARD_PRODUCTIMAGE_FILE_SYSTEM_TYPE := ext4 BOARD_SYSTEMIMAGE_FILE_SYSTEM_TYPE := ext4 BOARD_SYSTEM_EXTIMAGE_FILE_SYSTEM_TYPE := ext4 -BOARD_VENDORIMAGE_FILE_SYSTEM_TYPE := ext4 +BOARD_VENDORIMAGE_FILE_SYSTEM_TYPE := erofs BOARD_FLASH_BLOCK_SIZE := 262144 # (BOARD_KERNEL_PAGESIZE * 64) Modify the device's fstab file, located here at $WORKING_DIR/device/xiaomi/equuleus/rootdir/fstab.qcom. Change the filesystem type for the relevant partitions to erofs, and reduce the mount parameters to just ro: commit d66bfc8fe722f7368ef56835d35fb88b904e10fb (HEAD -> erofs, refs/published/erofs) Author: KaguraiYoRoy <[email protected]> Date: Tue Dec 24 23:40:08 2024 +0800 equuleus: Update fstab for erofs Change-Id: I814eab1dd77cf2c184ebd398cc4914785bc37ad6 diff --git a/rootdir/etc/fstab.qcom b/rootdir/etc/fstab.qcom index 7aca90a..685ec84 100644 --- a/rootdir/etc/fstab.qcom +++ b/rootdir/etc/fstab.qcom @@ -6,8 +6,8 @@ system /system ext4 ro,barrier=1,discard wait,logical,first_stage_mount system_ext /system_ext ext4 ro,barrier=1,discard wait,logical,first_stage_mount product /product ext4 ro,barrier=1,discard wait,logical,first_stage_mount -vendor /vendor ext4 ro,barrier=1,discard wait,logical,first_stage_mount -odm /odm ext4 ro,barrier=1,discard wait,logical,first_stage_mount +vendor /vendor erofs ro wait,logical,first_stage_mount +odm /odm erofs ro wait,logical,first_stage_mount /dev/block/bootdevice/by-name/userdata /data ext4 noatime,nosuid,nodev,barrier=0,noauto_da_alloc latemount,wait,check,fileencryption=ice,quota /dev/block/bootdevice/by-name/modem /vendor/firmware_mnt vfat ro,shortname=lower,uid=0,gid=1000,dmask=227,fmask=337,context=u:object_r:firmware_file:s0 wait /dev/block/bootdevice/by-name/dsp /vendor/dsp ext4 ro,nosuid,nodev,barrier=1 wait Finally, build and flash the ROM. If all goes well, it should boot into the system. Important Notes LineageOS official has implemented Dynamic Partition for my device model. The device tree modifications described above might not be suitable for your device. erofs is a read-only filesystem. If you plan to flash Gapps later, DO NOT convert the system, system_ext, and product partitions. Frequently Asked Questions Booting into Fastboot mode: Check if the fstab modifications are correct.