{"id":15186,"url":"https://patchwork.libcamera.org/api/patches/15186/?format=json","web_url":"https://patchwork.libcamera.org/patch/15186/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20211221043610.2512334-9-paul.elder@ideasonboard.com>","date":"2021-12-21T04:36:10","name":"[libcamera-devel,v3,8/8] android: Plumb all sensitivity-related controls","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"256ea9e736fa12eef5eb657786697cfcf9c5d532","submitter":{"id":17,"url":"https://patchwork.libcamera.org/api/people/17/?format=json","name":"Paul Elder","email":"paul.elder@ideasonboard.com"},"delegate":{"id":17,"url":"https://patchwork.libcamera.org/api/users/17/?format=json","username":"epaul","first_name":"Paul","last_name":"Elder","email":"paul.elder@ideasonboard.com"},"mbox":"https://patchwork.libcamera.org/patch/15186/mbox/","series":[{"id":2849,"url":"https://patchwork.libcamera.org/api/series/2849/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=2849","date":"2021-12-21T04:36:02","name":"The Great AE Changes","version":3,"mbox":"https://patchwork.libcamera.org/series/2849/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/15186/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/15186/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 2C9DBC3258\n\tfor ;\n\tTue, 21 Dec 2021 04:36:32 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id B6472608E9;\n\tTue, 21 Dec 2021 05:36:31 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 446486090B\n\tfor ;\n\tTue, 21 Dec 2021 05:36:29 +0100 (CET)","from pyrite.mediacom.info (unknown\n\t[IPv6:2604:2d80:ad90:fb00:96fd:8874:873:6c16])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 47B35881;\n\tTue, 21 Dec 2021 05:36:28 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"XxZ9pcks\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1640061389;\n\tbh=RzfdXJCi0ta34mXKRkskLbybgflNzHpIYdSBBBO21m4=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=XxZ9pcksw451/He3ed+e3IESNyDjlJCGzCJNbrim/JBknTF1QkEv6zRlVoTPDBxGE\n\tGFPI5T1bCYhdgoHIEdOQhCrkBis0i5yV6xf6cab9xfQEv4YO7pPFuh7mSWN1r5iDdu\n\t4XM0T6VdmP8l/i3Y3avftgoACIldzvfpRpz5RH20=","From":"Paul Elder ","To":"libcamera-devel@lists.libcamera.org","Date":"Mon, 20 Dec 2021 22:36:10 -0600","Message-Id":"<20211221043610.2512334-9-paul.elder@ideasonboard.com>","X-Mailer":"git-send-email 2.27.0","In-Reply-To":"<20211221043610.2512334-1-paul.elder@ideasonboard.com>","References":"<20211221043610.2512334-1-paul.elder@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v3 8/8] android: Plumb all\n\tsensitivity-related controls","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Plumb through the HAL the following three controls:\n- ANDROID_SENSOR_INFO_SENSITIVITY_RANGE (static)\n- ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY (static)\n- ANDROID_SENSOR_SENSITIVITY (request, result)\n\nAlso add a minISO to the HAL config, and add appropriate capability\ndetection.\n\nThe sensitivity range comes from:\n- min: hardcode to 100, or take minISO from the HAL config\n- max: set to minISO * max(AnalogueGain) * max(DigitalGain)\n\nThe max analog sensitivity comes from:\n- hardcode to minISO * max(AnalogueGain)\n\nThe request sensitivity is mapped to requested ISO / minISO, and split\nbetween analog and digital gain.\n\nThe result sensitivity is mapped from minISO * analog gain * digital\ngain.\n\nAlso set the sensor sensitivity in the preview template to the minimum\nISO.\n\nBug: https://bugs.libcamera.org/show_bug.cgi?id=47\nSigned-off-by: Paul Elder \n\n---\nChanges in v3:\n- fix setting metadata entries\n - temp holder variable is not optional\n- fix gain calculation (used to be integer division, fix to float\n division)\n- fix splitting the gain into analog and digital\n- set sensor sensitivity in preview template\n\nNo changes in v2\n- I'm moving this into the AE-related series, because this is AE-related\n---\n src/android/camera_capabilities.cpp | 65 +++++++++++++++++++++++++----\n src/android/camera_capabilities.h | 3 +-\n src/android/camera_device.cpp | 64 +++++++++++++++++++++++++++-\n src/android/camera_device.h | 1 +\n src/android/camera_hal_config.cpp | 10 ++++-\n src/android/camera_hal_config.h | 1 +\n 6 files changed, 134 insertions(+), 10 deletions(-)","diff":"diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp\nindex 1bea15ca..b9a1f6e5 100644\n--- a/src/android/camera_capabilities.cpp\n+++ b/src/android/camera_capabilities.cpp\n@@ -233,6 +233,15 @@ bool CameraCapabilities::validateManualSensorCapability()\n \t\treturn false;\n \t}\n \n+\t/*\n+\t * Checking the sensitivity range is sufficient, as it also covers the\n+\t * max analog sensitivity and the sensor sensitivity request/result key\n+\t */\n+\tif (!staticMetadata_->hasEntry(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE)) {\n+\t\tLOG(HAL, Info) << noMode << \"missing sensitivity range\";\n+\t\treturn false;\n+\t}\n+\n \tif (!staticMetadata_->hasEntry(ANDROID_SENSOR_INFO_EXPOSURE_TIME_RANGE)) {\n \t\tLOG(HAL, Info) << noMode << \"missing exposure time range\";\n \t\treturn false;\n@@ -402,11 +411,12 @@ void CameraCapabilities::computeHwLevel(\n }\n \n int CameraCapabilities::initialize(std::shared_ptr camera,\n-\t\t\t\t int orientation, int facing)\n+\t\t\t\t int orientation, int facing, int minISO)\n {\n \tcamera_ = camera;\n \torientation_ = orientation;\n \tfacing_ = facing;\n+\tminISO_ = minISO;\n \trawStreamAvailable_ = false;\n \tmaxFrameDuration_ = 0;\n \n@@ -826,7 +836,6 @@ int CameraCapabilities::initializeStaticMetadata()\n \t\tANDROID_SENSOR_INFO_MAX_FRAME_DURATION,\n \t\tANDROID_SENSOR_INFO_PHYSICAL_SIZE,\n \t\tANDROID_SENSOR_INFO_PIXEL_ARRAY_SIZE,\n-\t\tANDROID_SENSOR_INFO_SENSITIVITY_RANGE,\n \t\tANDROID_SENSOR_INFO_TIMESTAMP_SOURCE,\n \t\tANDROID_SENSOR_ORIENTATION,\n \t\tANDROID_STATISTICS_INFO_AVAILABLE_FACE_DETECT_MODES,\n@@ -1142,11 +1151,50 @@ int CameraCapabilities::initializeStaticMetadata()\n \t\t\t\t\t data);\n \t}\n \n-\tint32_t sensitivityRange[] = {\n-\t\t32, 2400,\n-\t};\n-\tstaticMetadata_->addEntry(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE,\n-\t\t\t\t sensitivityRange);\n+\t/*\n+\t * The following three controls are only necessary for FULL\n+\t * (specifically the manual sensor capability):\n+\t * - ANDROID_SENSOR_INFO_SENSITIVITY_RANGE (static)\n+\t * - ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY (static)\n+\t * - ANDROID_SENSOR_SENSITIVITY (request, result)\n+\t *\n+\t * The first and second depend on analog gain, and the third depend on\n+\t * both, so if analog gain is not available then we can cross out all\n+\t * of them. We'll default to 100 minimum sensitivity if it isn't\n+\t * specified in the HAL configuration file.\n+\t *\n+\t * Digital gain is optional; if it's unavailable then the max analog\n+\t * sensitivity will be equal to the max sensitivity range.\n+\t *\n+\t * The minimum sensitivity must be at most 100, while the maximum must\n+\t * be at least 800.\n+\t */\n+\tconst auto &analogGainInfo = controlsInfo.find(&controls::AnalogueGain);\n+\tif (analogGainInfo != controlsInfo.end()) {\n+\t\tfloat maxAnalogGain = analogGainInfo->second.max().get();\n+\n+\t\tconst auto &digitalGainInfo = controlsInfo.find(&controls::DigitalGain);\n+\t\tfloat maxDigitalGain = digitalGainInfo == controlsInfo.end() ?\n+\t\t\t1.0f :\n+\t\t\tdigitalGainInfo->second.max().get();\n+\n+\t\tint32_t maxISO = minISO_ * maxAnalogGain * maxDigitalGain;\n+\t\tint32_t sensitivityRange[] = {\n+\t\t\tminISO_,\n+\t\t\tmaxISO < 800 ? 800 : maxISO,\n+\t\t};\n+\t\tstaticMetadata_->addEntry(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE,\n+\t\t\t\t\t sensitivityRange);\n+\n+\t\tint32_t maxAnalogISO = minISO_ * maxAnalogGain;\n+\t\tstaticMetadata_->addEntry(ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY,\n+\t\t\t\t\t maxAnalogISO);\n+\n+\t\tavailableCharacteristicsKeys_.insert(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE);\n+\t\tavailableCharacteristicsKeys_.insert(ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY);\n+\t\tavailableRequestKeys_.insert(ANDROID_SENSOR_SENSITIVITY);\n+\t\tavailableResultKeys_.insert(ANDROID_SENSOR_SENSITIVITY);\n+\t}\n \n \t/* Report the color filter arrangement if the camera reports it. */\n \tif (properties.contains(properties::draft::ColorFilterArrangement)) {\n@@ -1624,6 +1672,9 @@ std::unique_ptr CameraCapabilities::requestTemplatePreview() con\n \t\trequestTemplate->addEntry(ANDROID_SENSOR_EXPOSURE_TIME, entry.data.i64[0]);\n \t}\n \n+\tif (availableRequestKeys_.count(ANDROID_SENSOR_SENSITIVITY))\n+\t\trequestTemplate->addEntry(ANDROID_SENSOR_SENSITIVITY, minISO_);\n+\n \tuint8_t flashMode = ANDROID_FLASH_MODE_OFF;\n \trequestTemplate->addEntry(ANDROID_FLASH_MODE, flashMode);\n \ndiff --git a/src/android/camera_capabilities.h b/src/android/camera_capabilities.h\nindex 6f66f221..7f554c63 100644\n--- a/src/android/camera_capabilities.h\n+++ b/src/android/camera_capabilities.h\n@@ -26,7 +26,7 @@ public:\n \tCameraCapabilities() = default;\n \n \tint initialize(std::shared_ptr camera,\n-\t\t int orientation, int facing);\n+\t\t int orientation, int facing, int minISO);\n \n \tCameraMetadata *staticMetadata() const { return staticMetadata_.get(); }\n \tlibcamera::PixelFormat toPixelFormat(int format) const;\n@@ -70,6 +70,7 @@ private:\n \n \tint facing_;\n \tint orientation_;\n+\tint minISO_;\n \tbool rawStreamAvailable_;\n \tint64_t maxFrameDuration_;\n \tcamera_metadata_enum_android_info_supported_hardware_level hwLevel_;\ndiff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp\nindex 3ade44c4..1a508923 100644\n--- a/src/android/camera_device.cpp\n+++ b/src/android/camera_device.cpp\n@@ -376,7 +376,15 @@ int CameraDevice::initialize(const CameraConfigData *cameraConfigData)\n \t\torientation_ = 0;\n \t}\n \n-\treturn capabilities_.initialize(camera_, orientation_, facing_);\n+\tif (cameraConfigData && cameraConfigData->minISO != -1) {\n+\t\tminISO_ = cameraConfigData->minISO;\n+\t} else {\n+\t\tLOG(HAL, Error)\n+\t\t\t<< \"Minimum ISO not in configuration file. Using 100.\";\n+\t\tminISO_ = 100;\n+\t}\n+\n+\treturn capabilities_.initialize(camera_, orientation_, facing_, minISO_);\n }\n \n /*\n@@ -860,6 +868,40 @@ int CameraDevice::processControls(Camera3RequestDescriptor *descriptor)\n \t\t}\n \t}\n \n+\tif (settings.getEntry(ANDROID_SENSOR_SENSITIVITY, &entry)) {\n+\t\tconst auto &info = camera_->controls().find(&controls::AnalogueGain);\n+\t\tif (info != camera_->controls().end()) {\n+\t\t\tconst auto &dInfo =\n+\t\t\t\tcamera_->controls().find(&controls::DigitalGain);\n+\t\t\tbool digitalGainAvailable =\n+\t\t\t\tdInfo != camera_->controls().end();\n+\n+\t\t\tfloat maxAnalogGain = info->second.max().get();\n+\t\t\tfloat maxDigitalGain = digitalGainAvailable ?\n+\t\t\t\t\t dInfo->second.max().get() : 1.0f;\n+\n+\t\t\t/* target ISO / min ISO = gain */\n+\t\t\tfloat gain = static_cast(*entry.data.i32) /\n+\t\t\t\t static_cast(minISO_);\n+\n+\t\t\t/*\n+\t\t\t * Max out analog gain before applying digital gain, if\n+\t\t\t * digital gain is available.\n+\t\t\t */\n+\t\t\tif (gain <= maxAnalogGain) {\n+\t\t\t\tlastAnalogueGain_ = gain;\n+\t\t\t} else if (gain <= maxAnalogGain * maxDigitalGain) {\n+\t\t\t\tlastAnalogueGain_ = maxAnalogGain;\n+\t\t\t\tif (digitalGainAvailable)\n+\t\t\t\t\tlastDigitalGain_ = gain / maxAnalogGain;\n+\t\t\t} else {\n+\t\t\t\tlastAnalogueGain_ = maxAnalogGain;\n+\t\t\t\tif (digitalGainAvailable)\n+\t\t\t\t\tlastDigitalGain_ = maxDigitalGain;\n+\t\t\t}\n+\t\t}\n+\t}\n+\n \t/* Trigger libcamera's locked -> manual state change */\n \tif (aeMode == Camera3RequestDescriptor::AutoMode::Manual) {\n \t\tconst auto &info = camera_->controls().find(&controls::ExposureTime);\n@@ -867,6 +909,17 @@ int CameraDevice::processControls(Camera3RequestDescriptor *descriptor)\n \t\t\tcontrols.set(controls::ExposureTime, lastExposureTime_);\n \t}\n \n+\t/* Trigger libcamera's locked -> manual state change */\n+\tif (aeMode == Camera3RequestDescriptor::AutoMode::Manual) {\n+\t\tconst auto &aInfo = camera_->controls().find(&controls::AnalogueGain);\n+\t\tif (aInfo != camera_->controls().end())\n+\t\t\tcontrols.set(controls::AnalogueGain, lastAnalogueGain_);\n+\n+\t\tconst auto &dInfo = camera_->controls().find(&controls::DigitalGain);\n+\t\tif (dInfo != camera_->controls().end())\n+\t\t\tcontrols.set(controls::DigitalGain, lastDigitalGain_);\n+\t}\n+\n \treturn 0;\n }\n \n@@ -1534,6 +1587,15 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) cons\n \t\tresultMetadata->addEntry(ANDROID_SENSOR_EXPOSURE_TIME, exposure);\n \t}\n \n+\tif (metadata.contains(controls::AnalogueGain)) {\n+\t\tbool hasDigitalGain = metadata.contains(controls::DigitalGain);\n+\t\tfloat analogGain = metadata.get(controls::AnalogueGain);\n+\t\tfloat digitalGain = hasDigitalGain ? metadata.get(controls::DigitalGain) : 1.0f;\n+\n+\t\tint32_t iso = minISO_ * analogGain * digitalGain;\n+\t\tresultMetadata->addEntry(ANDROID_SENSOR_SENSITIVITY, iso);\n+\t}\n+\n \tif (metadata.contains(controls::FrameDuration)) {\n \t\tint64_t duration = metadata.get(controls::FrameDuration) * 1000;\n \t\tresultMetadata->addEntry(ANDROID_SENSOR_FRAME_DURATION,\ndiff --git a/src/android/camera_device.h b/src/android/camera_device.h\nindex a65f1670..01c269d3 100644\n--- a/src/android/camera_device.h\n+++ b/src/android/camera_device.h\n@@ -125,6 +125,7 @@ private:\n \n \tint facing_;\n \tint orientation_;\n+\tint minISO_;\n \n \t/*\n \t * Normally resubmission of controls would be handled on libcamera's\ndiff --git a/src/android/camera_hal_config.cpp b/src/android/camera_hal_config.cpp\nindex aa90dac7..863cdff0 100644\n--- a/src/android/camera_hal_config.cpp\n+++ b/src/android/camera_hal_config.cpp\n@@ -180,6 +180,13 @@ int CameraHalConfig::Private::parseCameraConfigData(const std::string &cameraId)\n \t\t\t\t\treturn -EINVAL;\n \t\t\t\t}\n \t\t\t\tcameraConfigData.rotation = ret;\n+\t\t\t} else if (key == \"minISO\") {\n+\t\t\t\tret = std::stoi(value);\n+\t\t\t\tif (ret < 0 || ret > 100) {\n+\t\t\t\t\tLOG(HALConfig, Error)\n+\t\t\t\t\t\t<< \"Invalid minimum ISO: \" << value;\n+\t\t\t\t\treturn -EINVAL;\n+\t\t\t\t}\n \t\t\t} else {\n \t\t\t\tLOG(HALConfig, Error)\n \t\t\t\t\t<< \"Unknown key: \" << key;\n@@ -384,7 +391,8 @@ int CameraHalConfig::parseConfigurationFile()\n \t\tconst CameraConfigData &camera = c.second;\n \t\tLOG(HALConfig, Debug) << \"'\" << cameraId << \"' \"\n \t\t\t\t << \"(\" << camera.facing << \")[\"\n-\t\t\t\t << camera.rotation << \"]\";\n+\t\t\t\t << camera.rotation << \"], \"\n+\t\t\t\t << \"minISO: \" << camera.minISO;\n \t}\n \n \treturn 0;\ndiff --git a/src/android/camera_hal_config.h b/src/android/camera_hal_config.h\nindex 9df554f9..d9ea45a4 100644\n--- a/src/android/camera_hal_config.h\n+++ b/src/android/camera_hal_config.h\n@@ -15,6 +15,7 @@\n struct CameraConfigData {\n \tint facing = -1;\n \tint rotation = -1;\n+\tint minISO = -1;\n };\n \n class CameraHalConfig final : public libcamera::Extensible\n","prefixes":["libcamera-devel","v3","8/8"]}