From patchwork Fri Oct 1 10:33:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 14022 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DD7B3BDC71 for ; Fri, 1 Oct 2021 10:33:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AB748691BB; Fri, 1 Oct 2021 12:33:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="jkvv5wqh"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1CD65691AC for ; Fri, 1 Oct 2021 12:33:50 +0200 (CEST) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A74F1596; Fri, 1 Oct 2021 12:33:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1633084429; bh=K9auD2iBnOpNG4ZMLIaruFtpf06EYn/2zB6+cFeCFGs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jkvv5wqh03b48sSkc6TyQECEo0V5/+QZniIsUHS4ZP9SjQ0qUucOZTQQG57IdNRvR 1hg+iLj4iZUPkH+8Jx5vhdHYsX6bZvCa9NDM+eM7GFzFO6ORn+7IHwbOAmB9s7MagJ cKhBs4Vy+hwFCflQDwnMQFStGe6p6tE5wSJzAS4A= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Fri, 1 Oct 2021 19:33:25 +0900 Message-Id: <20211001103325.1077590-8-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211001103325.1077590-1-paul.elder@ideasonboard.com> References: <20211001103325.1077590-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 7/7] android: Plumb all sensitivity-related controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Plumb through the HAL the following three controls: - ANDROID_SENSOR_INFO_SENSITIVITY_RANGE (static) - ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY (static) - ANDROID_SENSOR_SENSITIVITY (request, result) Also add a minISO to the HAL config, and add appropriate capability detection. The sensitivity range comes from: - min: hardcode to 100, or take minISO from the HAL config - max: set to minISO * max(AnalogueGain) * max(DigitalGain) The max analog sensitivity comes from: - hardcode to minISO * max(AnalogueGain) The request sensitivity is mapped to requested ISO / minISO, and split between analog and digital gain. The result sensitivity is mapped from minISO * analog gain * digital gain. Bug: https://bugs.libcamera.org/show_bug.cgi?id=47 Signed-off-by: Paul Elder --- No changes in v2 - I'm moving this into the AE-related series, because this is AE-related --- src/android/camera_capabilities.cpp | 61 ++++++++++++++++++++--- src/android/camera_capabilities.h | 3 +- src/android/camera_device.cpp | 76 ++++++++++++++++++++++++++++- src/android/camera_device.h | 1 + src/android/camera_hal_config.cpp | 10 +++- src/android/camera_hal_config.h | 1 + 6 files changed, 142 insertions(+), 10 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index 3fed3f83..8e84a736 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -233,6 +233,15 @@ bool CameraCapabilities::validateManualSensorCapability() return false; } + /* + * Checking the sensitivity range is sufficient, as it also covers the + * max analog sensitivity and the sensor sensitivity request/result key + */ + if (!staticMetadata_->hasEntry(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE)) { + LOG(HAL, Info) << noMode << "missing sensitivity range"; + return false; + } + if (!staticMetadata_->hasEntry(ANDROID_SENSOR_INFO_EXPOSURE_TIME_RANGE)) { LOG(HAL, Info) << noMode << "missing exposure time range"; return false; @@ -401,11 +410,12 @@ void CameraCapabilities::computeHwLevel( } int CameraCapabilities::initialize(std::shared_ptr camera, - int orientation, int facing) + int orientation, int facing, int minISO) { camera_ = camera; orientation_ = orientation; facing_ = facing; + minISO_ = minISO; rawStreamAvailable_ = false; /* Acquire the camera and initialize available stream configurations. */ @@ -721,7 +731,6 @@ int CameraCapabilities::initializeStaticMetadata() ANDROID_SENSOR_INFO_MAX_FRAME_DURATION, ANDROID_SENSOR_INFO_PHYSICAL_SIZE, ANDROID_SENSOR_INFO_PIXEL_ARRAY_SIZE, - ANDROID_SENSOR_INFO_SENSITIVITY_RANGE, ANDROID_SENSOR_INFO_TIMESTAMP_SOURCE, ANDROID_SENSOR_ORIENTATION, ANDROID_STATISTICS_INFO_AVAILABLE_FACE_DETECT_MODES, @@ -1077,11 +1086,49 @@ int CameraCapabilities::initializeStaticMetadata() data); } - int32_t sensitivityRange[] = { - 32, 2400, - }; - staticMetadata_->addEntry(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE, - sensitivityRange); + /* + * The following three controls are only necessary for FULL + * (specifically the manual sensor capability): + * - ANDROID_SENSOR_INFO_SENSITIVITY_RANGE (static) + * - ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY (static) + * - ANDROID_SENSOR_SENSITIVITY (request, result) + * + * The first and second depend on analog gain, and the third depend on + * both, so if analog gain is not available then we can cross out all + * of them. We'll default to 100 minimum sensitivity if it isn't + * specified in the HAL configuration file. + * + * Digital gain is optional; if it's unavailable then the max analog + * sensitivity will be equal to the max sensitivity range. + * + * The minimum sensitivity must be at most 100, while the maximum must + * be at least 800. + */ + const auto &analogGainInfo = controlsInfo.find(&controls::AnalogueGain); + if (analogGainInfo != controlsInfo.end()) { + float maxAnalogGain = analogGainInfo->second.max().get(); + + const auto &digitalGainInfo = controlsInfo.find(&controls::DigitalGain); + float maxDigitalGain = digitalGainInfo == controlsInfo.end() ? + 1.0f : + digitalGainInfo->second.max().get(); + + int32_t maxISO = minISO_ * maxAnalogGain * maxDigitalGain; + int32_t sensitivityRange[] = { + minISO_, + maxISO < 800 ? 800 : maxISO, + }; + staticMetadata_->addEntry(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE, + sensitivityRange); + + staticMetadata_->addEntry(ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY, + minISO_ * maxAnalogGain); + + availableCharacteristicsKeys_.insert(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE); + availableCharacteristicsKeys_.insert(ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY); + availableRequestKeys_.insert(ANDROID_SENSOR_SENSITIVITY); + availableResultKeys_.insert(ANDROID_SENSOR_SENSITIVITY); + } /* Report the color filter arrangement if the camera reports it. */ if (properties.contains(properties::draft::ColorFilterArrangement)) { diff --git a/src/android/camera_capabilities.h b/src/android/camera_capabilities.h index a1259699..b1eee37f 100644 --- a/src/android/camera_capabilities.h +++ b/src/android/camera_capabilities.h @@ -26,7 +26,7 @@ public: CameraCapabilities() = default; int initialize(std::shared_ptr camera, - int orientation, int facing); + int orientation, int facing, int minISO); CameraMetadata *staticMetadata() const { return staticMetadata_.get(); } libcamera::PixelFormat toPixelFormat(int format) const; @@ -68,6 +68,7 @@ private: int facing_; int orientation_; + int minISO_; bool rawStreamAvailable_; camera_metadata_enum_android_info_supported_hardware_level hwLevel_; std::set capabilities_; diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index d5027ec5..8854a960 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -393,7 +393,15 @@ int CameraDevice::initialize(const CameraConfigData *cameraConfigData) orientation_ = 0; } - return capabilities_.initialize(camera_, orientation_, facing_); + if (cameraConfigData && cameraConfigData->minISO != -1) { + minISO_ = cameraConfigData->minISO; + } else { + LOG(HAL, Error) + << "Minimum ISO not in configuration file. Using 100."; + minISO_ = 100; + } + + return capabilities_.initialize(camera_, orientation_, facing_, minISO_); } /* @@ -908,6 +916,51 @@ int CameraDevice::processControls(Camera3RequestDescriptor *descriptor) } } + if (settings.getEntry(ANDROID_SENSOR_SENSITIVITY, &entry)) { + const auto &info = camera_->controls().find(&controls::AnalogueGain); + if (info != camera_->controls().end()) { + const auto &dInfo = + camera_->controls().find(&controls::DigitalGain); + bool digitalGainAvailable = + dInfo == camera_->controls().end(); + + float maxAnalogGain = info->second.max().get(); + float maxDigitalGain = digitalGainAvailable ? + dInfo->second.max().get() : 1.0f; + + /* target ISO / min ISO = gain */ + float gain = *entry.data.i32 / minISO_; + + /* + * Max out analog gain before applying digital gain, if + * digital gain is available. + */ + bool setDigital = false; + if (gain <= maxAnalogGain) { + lastAnalogueGain_ = gain; + } else if (gain <= maxAnalogGain * maxDigitalGain) { + lastAnalogueGain_ = maxAnalogGain; + if (digitalGainAvailable) { + setDigital = true; + lastDigitalGain_ = gain / maxAnalogGain; + } + } else { + lastAnalogueGain_ = maxAnalogGain; + if (digitalGainAvailable) { + setDigital = true; + lastDigitalGain_ = maxDigitalGain; + } + } + + /* Don't disable libcamera's internal AeMode::Lock */ + if (aeMode != AeMode::Lock) + controls.set(controls::AnalogueGain, lastAnalogueGain_); + + if (setDigital && (aeMode != AeMode::Lock)) + controls.set(controls::DigitalGain, lastDigitalGain_); + } + } + /* Trigger libcamera's locked -> manual state change */ if (aeMode == AeMode::Manual && !settings.hasEntry(ANDROID_SENSOR_EXPOSURE_TIME)) { const auto &info = camera_->controls().find(&controls::ExposureTime); @@ -915,6 +968,17 @@ int CameraDevice::processControls(Camera3RequestDescriptor *descriptor) controls.set(controls::ExposureTime, lastExposureTime_); } + /* Trigger libcamera's locked -> manual state change */ + if (aeMode == AeMode::Manual && !settings.hasEntry(ANDROID_SENSOR_SENSITIVITY)) { + const auto &aInfo = camera_->controls().find(&controls::AnalogueGain); + if (aInfo != camera_->controls().end()) + controls.set(controls::AnalogueGain, lastAnalogueGain_); + + const auto &dInfo = camera_->controls().find(&controls::DigitalGain); + if (dInfo != camera_->controls().end()) + controls.set(controls::DigitalGain, lastDigitalGain_); + } + return 0; } @@ -1548,6 +1612,16 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) cons resultMetadata->addEntry(ANDROID_SENSOR_EXPOSURE_TIME, exposure); } + if (metadata.contains(controls::AnalogueGain) && + settings.hasEntry(ANDROID_SENSOR_SENSITIVITY)) { + bool hasDigitalGain = metadata.contains(controls::DigitalGain); + float analogGain = metadata.get(controls::AnalogueGain); + float digitalGain = hasDigitalGain ? metadata.get(controls::DigitalGain) : 1.0f; + + resultMetadata->addEntry(ANDROID_SENSOR_SENSITIVITY, + minISO_ * analogGain * digitalGain); + } + if (metadata.contains(controls::FrameDuration)) { int64_t duration = metadata.get(controls::FrameDuration) * 1000; resultMetadata->addEntry(ANDROID_SENSOR_FRAME_DURATION, diff --git a/src/android/camera_device.h b/src/android/camera_device.h index f693cdbc..e119dfd9 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -154,6 +154,7 @@ private: int facing_; int orientation_; + int minISO_; /* Track the last-set android AE controls */ bool aeOn_; diff --git a/src/android/camera_hal_config.cpp b/src/android/camera_hal_config.cpp index aa90dac7..863cdff0 100644 --- a/src/android/camera_hal_config.cpp +++ b/src/android/camera_hal_config.cpp @@ -180,6 +180,13 @@ int CameraHalConfig::Private::parseCameraConfigData(const std::string &cameraId) return -EINVAL; } cameraConfigData.rotation = ret; + } else if (key == "minISO") { + ret = std::stoi(value); + if (ret < 0 || ret > 100) { + LOG(HALConfig, Error) + << "Invalid minimum ISO: " << value; + return -EINVAL; + } } else { LOG(HALConfig, Error) << "Unknown key: " << key; @@ -384,7 +391,8 @@ int CameraHalConfig::parseConfigurationFile() const CameraConfigData &camera = c.second; LOG(HALConfig, Debug) << "'" << cameraId << "' " << "(" << camera.facing << ")[" - << camera.rotation << "]"; + << camera.rotation << "], " + << "minISO: " << camera.minISO; } return 0; diff --git a/src/android/camera_hal_config.h b/src/android/camera_hal_config.h index a79d5d6c..afa02a9b 100644 --- a/src/android/camera_hal_config.h +++ b/src/android/camera_hal_config.h @@ -15,6 +15,7 @@ struct CameraConfigData { int facing = -1; int rotation = -1; + int minISO = -1; }; class CameraHalConfig final : public libcamera::Extensible