From patchwork Thu Sep 9 10:25:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 13784 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5EFB8BDC71 for ; Thu, 9 Sep 2021 10:25:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C8BF46916E; Thu, 9 Sep 2021 12:25:44 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="hUdZ0qn5"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A90D56916B for ; Thu, 9 Sep 2021 12:25:43 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 04363466; Thu, 9 Sep 2021 12:25:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1631183143; bh=T/S8jpBOC2E4/5FsLD7tKjXiJnVXgR+GJk9fdI3Prhs=; h=From:To:Cc:Subject:Date:From; b=hUdZ0qn5YRZBmgxTo/RXAEnlFKdPBjobdefLe1A/eEPV/QG4wgfrB2Vo57wi2MGjN BXZ/Hg9BJsg9vCdc+TF5Q+6jUWdUEuy1HtAdFXUcv/seI0X6JNddqsPoiSUvARO5s3 vqcXO1NUSajFMmm6f0SBUrvJPp//H4W+gRsC6HEo= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Thu, 9 Sep 2021 19:25:33 +0900 Message-Id: <20210909102533.690353-1-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH] android: Plumb all sensitivity-related controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Plumb through the HAL the following three controls: - ANDROID_SENSOR_INFO_SENSITIVITY_RANGE (static) - ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY (static) - ANDROID_SENSOR_SENSITIVITY (request, result) Also add a minISO to the HAL config, and add appropriate capability detection. The sensitivity range comes from: - min: hardcode to 100, or take minISO from the HAL config - max: set to minISO * max(AnalogueGain) * max(DigitalGain) The max analog sensitivity comes from: - hardcode to minISO * max(AnalogueGain) The request sensitivity is mapped to requested ISO / minISO, and split between analog and digital gain. The result sensitivity is mapped from minISO * analog gain * digital gain. Signed-off-by: Paul Elder --- This is on top of my ctsw branch, so it won't apply on master, but I would like comments to confirm the direction and design. --- src/android/camera_capabilities.cpp | 61 +++++++++++++++++++++++++---- src/android/camera_capabilities.h | 3 +- src/android/camera_device.cpp | 53 ++++++++++++++++++++++++- src/android/camera_device.h | 1 + src/android/camera_hal_config.cpp | 10 ++++- src/android/camera_hal_config.h | 1 + 6 files changed, 119 insertions(+), 10 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index af972ca5..461c208b 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -233,6 +233,15 @@ bool CameraCapabilities::validateManualSensorCapability() return false; } + /* + * Checking the sensitivity range is sufficient, as it also covers the + * max analog sensitivity and the sensor sensitivity request/result key + */ + if (!staticMetadata_->hasEntry(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE)) { + LOG(HAL, Info) << noMode << "missing sensitivity range"; + return false; + } + if (!staticMetadata_->hasEntry(ANDROID_EDGE_AVAILABLE_EDGE_MODES)) { LOG(HAL, Info) << noMode << "missing edge modes"; return false; @@ -406,11 +415,12 @@ void CameraCapabilities::computeHwLevel( } int CameraCapabilities::initialize(std::shared_ptr camera, - int orientation, int facing) + int orientation, int facing, int minISO) { camera_ = camera; orientation_ = orientation; facing_ = facing; + minISO_ = minISO; rawStreamAvailable_ = false; /* Acquire the camera and initialize available stream configurations. */ @@ -727,7 +737,6 @@ int CameraCapabilities::initializeStaticMetadata() ANDROID_SENSOR_INFO_MAX_FRAME_DURATION, ANDROID_SENSOR_INFO_PHYSICAL_SIZE, ANDROID_SENSOR_INFO_PIXEL_ARRAY_SIZE, - ANDROID_SENSOR_INFO_SENSITIVITY_RANGE, ANDROID_SENSOR_INFO_TIMESTAMP_SOURCE, ANDROID_SENSOR_ORIENTATION, ANDROID_STATISTICS_INFO_AVAILABLE_FACE_DETECT_MODES, @@ -1048,11 +1057,49 @@ int CameraCapabilities::initializeStaticMetadata() data); } - int32_t sensitivityRange[] = { - 32, 2400, - }; - staticMetadata_->addEntry(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE, - sensitivityRange); + /* + * The following three controls are only necessary for FULL + * (specifically the manual sensor capability): + * - ANDROID_SENSOR_INFO_SENSITIVITY_RANGE (static) + * - ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY (static) + * - ANDROID_SENSOR_SENSITIVITY (request, result) + * + * The first and second depend on analog gain, and the third depend on + * both, so if analog gain is not available then we can cross out all + * of them. We'll default to 100 minimum sensitivity if it isn't + * specified in the HAL configuration file. + * + * Digital gain is optional; if it's unavailable then the max analog + * sensitivity will be equal to the max sensitivity range. + * + * The minimum sensitivity must be at most 100, while the maximum must + * be at least 800. + */ + const auto &analogGainInfo = controlsInfo.find(&controls::AnalogueGain); + if (analogGainInfo != controlsInfo.end()) { + float maxAnalogGain = analogGainInfo->second.max().get(); + + const auto &digitalGainInfo = controlsInfo.find(&controls::DigitalGain); + float maxDigitalGain = digitalGainInfo == controlsInfo.end() ? + 1.0f : + digitalGainInfo->second.max().get(); + + int32_t maxISO = minISO_ * maxAnalogGain * maxDigitalGain; + int32_t sensitivityRange[] = { + minISO_, + maxISO < 800 ? 800 : maxISO, + }; + staticMetadata_->addEntry(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE, + sensitivityRange); + + staticMetadata_->addEntry(ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY, + minISO_ * maxAnalogGain); + + availableCharacteristicsKeys_.insert(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE); + availableCharacteristicsKeys_.insert(ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY); + availableRequestKeys_.insert(ANDROID_SENSOR_SENSITIVITY); + availableResultKeys_.insert(ANDROID_SENSOR_SENSITIVITY); + } /* Report the color filter arrangement if the camera reports it. */ if (properties.contains(properties::draft::ColorFilterArrangement)) { diff --git a/src/android/camera_capabilities.h b/src/android/camera_capabilities.h index a1259699..b1eee37f 100644 --- a/src/android/camera_capabilities.h +++ b/src/android/camera_capabilities.h @@ -26,7 +26,7 @@ public: CameraCapabilities() = default; int initialize(std::shared_ptr camera, - int orientation, int facing); + int orientation, int facing, int minISO); CameraMetadata *staticMetadata() const { return staticMetadata_.get(); } libcamera::PixelFormat toPixelFormat(int format) const; @@ -68,6 +68,7 @@ private: int facing_; int orientation_; + int minISO_; bool rawStreamAvailable_; camera_metadata_enum_android_info_supported_hardware_level hwLevel_; std::set capabilities_; diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index e88cc0e9..373e0f98 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -390,7 +390,15 @@ int CameraDevice::initialize(const CameraConfigData *cameraConfigData) orientation_ = 0; } - return capabilities_.initialize(camera_, orientation_, facing_); + if (cameraConfigData && cameraConfigData->minISO != -1) { + minISO_ = cameraConfigData->minISO; + } else { + LOG(HAL, Error) + << "Minimum ISO not in configuration file. Using 100."; + minISO_ = 100; + } + + return capabilities_.initialize(camera_, orientation_, facing_, minISO_); } /* @@ -838,6 +846,39 @@ int CameraDevice::processControls(Camera3RequestDescriptor *descriptor) if (settings.getEntry(ANDROID_CONTROL_AWB_LOCK, &entry)) controls.set(controls::AwbLock, *entry.data.u8); + if (settings.getEntry(ANDROID_SENSOR_SENSITIVITY, &entry)) { + const auto &info = camera_->controls().find(&controls::AnalogueGain); + if (info != camera_->controls().end()) { + const auto &dInfo = + camera_->controls().find(&controls::DigitalGain); + bool digitalGainAvailable = + dInfo == camera_->controls().end(); + + float maxAnalogGain = info->second.max().get(); + float maxDigitalGain = digitalGainAvailable ? + dInfo->second.max().get() : 1.0f; + + /* target ISO / min ISO = gain */ + float gain = *entry.data.i32 / minISO_; + + /* + * Max out analog gain before applying digital gain, if + * digital gain is available. + */ + if (gain <= maxAnalogGain) { + controls.set(controls::AnalogueGain, gain); + } else if (gain <= maxAnalogGain * maxDigitalGain) { + controls.set(controls::AnalogueGain, maxAnalogGain); + if (digitalGainAvailable) + controls.set(controls::DigitalGain, gain / maxAnalogGain); + } else { + controls.set(controls::AnalogueGain, maxAnalogGain); + if (digitalGainAvailable) + controls.set(controls::DigitalGain, maxDigitalGain); + } + } + } + if (settings.getEntry(ANDROID_EDGE_MODE, &entry)) { const auto &info = camera_->controls().find(&controls::Sharpness); if (info != camera_->controls().end()) { @@ -1408,6 +1449,16 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) cons duration); } + if (metadata.contains(controls::AnalogueGain) && + settings.hasEntry(ANDROID_SENSOR_SENSITIVITY)) { + bool hasDigitalGain = metadata.contains(controls::DigitalGain); + float analogGain = metadata.get(controls::AnalogueGain); + float digitalGain = hasDigitalGain ? metadata.get(controls::DigitalGain) : 1.0f; + + resultMetadata->addEntry(ANDROID_SENSOR_SENSITIVITY, + minISO_ * analogGain * digitalGain); + } + if (metadata.contains(controls::Sharpness) && settings.getEntry(ANDROID_EDGE_MODE, &entry)) { const auto &info = camera_->controls().find(&controls::Sharpness); diff --git a/src/android/camera_device.h b/src/android/camera_device.h index a5576927..36e0174c 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -130,6 +130,7 @@ private: int facing_; int orientation_; + int minISO_; CameraMetadata lastSettings_; }; diff --git a/src/android/camera_hal_config.cpp b/src/android/camera_hal_config.cpp index aa90dac7..863cdff0 100644 --- a/src/android/camera_hal_config.cpp +++ b/src/android/camera_hal_config.cpp @@ -180,6 +180,13 @@ int CameraHalConfig::Private::parseCameraConfigData(const std::string &cameraId) return -EINVAL; } cameraConfigData.rotation = ret; + } else if (key == "minISO") { + ret = std::stoi(value); + if (ret < 0 || ret > 100) { + LOG(HALConfig, Error) + << "Invalid minimum ISO: " << value; + return -EINVAL; + } } else { LOG(HALConfig, Error) << "Unknown key: " << key; @@ -384,7 +391,8 @@ int CameraHalConfig::parseConfigurationFile() const CameraConfigData &camera = c.second; LOG(HALConfig, Debug) << "'" << cameraId << "' " << "(" << camera.facing << ")[" - << camera.rotation << "]"; + << camera.rotation << "], " + << "minISO: " << camera.minISO; } return 0; diff --git a/src/android/camera_hal_config.h b/src/android/camera_hal_config.h index a79d5d6c..afa02a9b 100644 --- a/src/android/camera_hal_config.h +++ b/src/android/camera_hal_config.h @@ -15,6 +15,7 @@ struct CameraConfigData { int facing = -1; int rotation = -1; + int minISO = -1; }; class CameraHalConfig final : public libcamera::Extensible