From patchwork Tue Dec 21 04:36:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 15186 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2C9DBC3258 for ; Tue, 21 Dec 2021 04:36:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B6472608E9; Tue, 21 Dec 2021 05:36:31 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="XxZ9pcks"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 446486090B for ; Tue, 21 Dec 2021 05:36:29 +0100 (CET) Received: from pyrite.mediacom.info (unknown [IPv6:2604:2d80:ad90:fb00:96fd:8874:873:6c16]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 47B35881; Tue, 21 Dec 2021 05:36:28 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1640061389; bh=RzfdXJCi0ta34mXKRkskLbybgflNzHpIYdSBBBO21m4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XxZ9pcksw451/He3ed+e3IESNyDjlJCGzCJNbrim/JBknTF1QkEv6zRlVoTPDBxGE GFPI5T1bCYhdgoHIEdOQhCrkBis0i5yV6xf6cab9xfQEv4YO7pPFuh7mSWN1r5iDdu 4XM0T6VdmP8l/i3Y3avftgoACIldzvfpRpz5RH20= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Mon, 20 Dec 2021 22:36:10 -0600 Message-Id: <20211221043610.2512334-9-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211221043610.2512334-1-paul.elder@ideasonboard.com> References: <20211221043610.2512334-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 8/8] android: Plumb all sensitivity-related controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Plumb through the HAL the following three controls: - ANDROID_SENSOR_INFO_SENSITIVITY_RANGE (static) - ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY (static) - ANDROID_SENSOR_SENSITIVITY (request, result) Also add a minISO to the HAL config, and add appropriate capability detection. The sensitivity range comes from: - min: hardcode to 100, or take minISO from the HAL config - max: set to minISO * max(AnalogueGain) * max(DigitalGain) The max analog sensitivity comes from: - hardcode to minISO * max(AnalogueGain) The request sensitivity is mapped to requested ISO / minISO, and split between analog and digital gain. The result sensitivity is mapped from minISO * analog gain * digital gain. Also set the sensor sensitivity in the preview template to the minimum ISO. Bug: https://bugs.libcamera.org/show_bug.cgi?id=47 Signed-off-by: Paul Elder --- Changes in v3: - fix setting metadata entries - temp holder variable is not optional - fix gain calculation (used to be integer division, fix to float division) - fix splitting the gain into analog and digital - set sensor sensitivity in preview template No changes in v2 - I'm moving this into the AE-related series, because this is AE-related --- src/android/camera_capabilities.cpp | 65 +++++++++++++++++++++++++---- src/android/camera_capabilities.h | 3 +- src/android/camera_device.cpp | 64 +++++++++++++++++++++++++++- src/android/camera_device.h | 1 + src/android/camera_hal_config.cpp | 10 ++++- src/android/camera_hal_config.h | 1 + 6 files changed, 134 insertions(+), 10 deletions(-) diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp index 1bea15ca..b9a1f6e5 100644 --- a/src/android/camera_capabilities.cpp +++ b/src/android/camera_capabilities.cpp @@ -233,6 +233,15 @@ bool CameraCapabilities::validateManualSensorCapability() return false; } + /* + * Checking the sensitivity range is sufficient, as it also covers the + * max analog sensitivity and the sensor sensitivity request/result key + */ + if (!staticMetadata_->hasEntry(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE)) { + LOG(HAL, Info) << noMode << "missing sensitivity range"; + return false; + } + if (!staticMetadata_->hasEntry(ANDROID_SENSOR_INFO_EXPOSURE_TIME_RANGE)) { LOG(HAL, Info) << noMode << "missing exposure time range"; return false; @@ -402,11 +411,12 @@ void CameraCapabilities::computeHwLevel( } int CameraCapabilities::initialize(std::shared_ptr camera, - int orientation, int facing) + int orientation, int facing, int minISO) { camera_ = camera; orientation_ = orientation; facing_ = facing; + minISO_ = minISO; rawStreamAvailable_ = false; maxFrameDuration_ = 0; @@ -826,7 +836,6 @@ int CameraCapabilities::initializeStaticMetadata() ANDROID_SENSOR_INFO_MAX_FRAME_DURATION, ANDROID_SENSOR_INFO_PHYSICAL_SIZE, ANDROID_SENSOR_INFO_PIXEL_ARRAY_SIZE, - ANDROID_SENSOR_INFO_SENSITIVITY_RANGE, ANDROID_SENSOR_INFO_TIMESTAMP_SOURCE, ANDROID_SENSOR_ORIENTATION, ANDROID_STATISTICS_INFO_AVAILABLE_FACE_DETECT_MODES, @@ -1142,11 +1151,50 @@ int CameraCapabilities::initializeStaticMetadata() data); } - int32_t sensitivityRange[] = { - 32, 2400, - }; - staticMetadata_->addEntry(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE, - sensitivityRange); + /* + * The following three controls are only necessary for FULL + * (specifically the manual sensor capability): + * - ANDROID_SENSOR_INFO_SENSITIVITY_RANGE (static) + * - ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY (static) + * - ANDROID_SENSOR_SENSITIVITY (request, result) + * + * The first and second depend on analog gain, and the third depend on + * both, so if analog gain is not available then we can cross out all + * of them. We'll default to 100 minimum sensitivity if it isn't + * specified in the HAL configuration file. + * + * Digital gain is optional; if it's unavailable then the max analog + * sensitivity will be equal to the max sensitivity range. + * + * The minimum sensitivity must be at most 100, while the maximum must + * be at least 800. + */ + const auto &analogGainInfo = controlsInfo.find(&controls::AnalogueGain); + if (analogGainInfo != controlsInfo.end()) { + float maxAnalogGain = analogGainInfo->second.max().get(); + + const auto &digitalGainInfo = controlsInfo.find(&controls::DigitalGain); + float maxDigitalGain = digitalGainInfo == controlsInfo.end() ? + 1.0f : + digitalGainInfo->second.max().get(); + + int32_t maxISO = minISO_ * maxAnalogGain * maxDigitalGain; + int32_t sensitivityRange[] = { + minISO_, + maxISO < 800 ? 800 : maxISO, + }; + staticMetadata_->addEntry(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE, + sensitivityRange); + + int32_t maxAnalogISO = minISO_ * maxAnalogGain; + staticMetadata_->addEntry(ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY, + maxAnalogISO); + + availableCharacteristicsKeys_.insert(ANDROID_SENSOR_INFO_SENSITIVITY_RANGE); + availableCharacteristicsKeys_.insert(ANDROID_SENSOR_MAX_ANALOG_SENSITIVITY); + availableRequestKeys_.insert(ANDROID_SENSOR_SENSITIVITY); + availableResultKeys_.insert(ANDROID_SENSOR_SENSITIVITY); + } /* Report the color filter arrangement if the camera reports it. */ if (properties.contains(properties::draft::ColorFilterArrangement)) { @@ -1624,6 +1672,9 @@ std::unique_ptr CameraCapabilities::requestTemplatePreview() con requestTemplate->addEntry(ANDROID_SENSOR_EXPOSURE_TIME, entry.data.i64[0]); } + if (availableRequestKeys_.count(ANDROID_SENSOR_SENSITIVITY)) + requestTemplate->addEntry(ANDROID_SENSOR_SENSITIVITY, minISO_); + uint8_t flashMode = ANDROID_FLASH_MODE_OFF; requestTemplate->addEntry(ANDROID_FLASH_MODE, flashMode); diff --git a/src/android/camera_capabilities.h b/src/android/camera_capabilities.h index 6f66f221..7f554c63 100644 --- a/src/android/camera_capabilities.h +++ b/src/android/camera_capabilities.h @@ -26,7 +26,7 @@ public: CameraCapabilities() = default; int initialize(std::shared_ptr camera, - int orientation, int facing); + int orientation, int facing, int minISO); CameraMetadata *staticMetadata() const { return staticMetadata_.get(); } libcamera::PixelFormat toPixelFormat(int format) const; @@ -70,6 +70,7 @@ private: int facing_; int orientation_; + int minISO_; bool rawStreamAvailable_; int64_t maxFrameDuration_; camera_metadata_enum_android_info_supported_hardware_level hwLevel_; diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index 3ade44c4..1a508923 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -376,7 +376,15 @@ int CameraDevice::initialize(const CameraConfigData *cameraConfigData) orientation_ = 0; } - return capabilities_.initialize(camera_, orientation_, facing_); + if (cameraConfigData && cameraConfigData->minISO != -1) { + minISO_ = cameraConfigData->minISO; + } else { + LOG(HAL, Error) + << "Minimum ISO not in configuration file. Using 100."; + minISO_ = 100; + } + + return capabilities_.initialize(camera_, orientation_, facing_, minISO_); } /* @@ -860,6 +868,40 @@ int CameraDevice::processControls(Camera3RequestDescriptor *descriptor) } } + if (settings.getEntry(ANDROID_SENSOR_SENSITIVITY, &entry)) { + const auto &info = camera_->controls().find(&controls::AnalogueGain); + if (info != camera_->controls().end()) { + const auto &dInfo = + camera_->controls().find(&controls::DigitalGain); + bool digitalGainAvailable = + dInfo != camera_->controls().end(); + + float maxAnalogGain = info->second.max().get(); + float maxDigitalGain = digitalGainAvailable ? + dInfo->second.max().get() : 1.0f; + + /* target ISO / min ISO = gain */ + float gain = static_cast(*entry.data.i32) / + static_cast(minISO_); + + /* + * Max out analog gain before applying digital gain, if + * digital gain is available. + */ + if (gain <= maxAnalogGain) { + lastAnalogueGain_ = gain; + } else if (gain <= maxAnalogGain * maxDigitalGain) { + lastAnalogueGain_ = maxAnalogGain; + if (digitalGainAvailable) + lastDigitalGain_ = gain / maxAnalogGain; + } else { + lastAnalogueGain_ = maxAnalogGain; + if (digitalGainAvailable) + lastDigitalGain_ = maxDigitalGain; + } + } + } + /* Trigger libcamera's locked -> manual state change */ if (aeMode == Camera3RequestDescriptor::AutoMode::Manual) { const auto &info = camera_->controls().find(&controls::ExposureTime); @@ -867,6 +909,17 @@ int CameraDevice::processControls(Camera3RequestDescriptor *descriptor) controls.set(controls::ExposureTime, lastExposureTime_); } + /* Trigger libcamera's locked -> manual state change */ + if (aeMode == Camera3RequestDescriptor::AutoMode::Manual) { + const auto &aInfo = camera_->controls().find(&controls::AnalogueGain); + if (aInfo != camera_->controls().end()) + controls.set(controls::AnalogueGain, lastAnalogueGain_); + + const auto &dInfo = camera_->controls().find(&controls::DigitalGain); + if (dInfo != camera_->controls().end()) + controls.set(controls::DigitalGain, lastDigitalGain_); + } + return 0; } @@ -1534,6 +1587,15 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) cons resultMetadata->addEntry(ANDROID_SENSOR_EXPOSURE_TIME, exposure); } + if (metadata.contains(controls::AnalogueGain)) { + bool hasDigitalGain = metadata.contains(controls::DigitalGain); + float analogGain = metadata.get(controls::AnalogueGain); + float digitalGain = hasDigitalGain ? metadata.get(controls::DigitalGain) : 1.0f; + + int32_t iso = minISO_ * analogGain * digitalGain; + resultMetadata->addEntry(ANDROID_SENSOR_SENSITIVITY, iso); + } + if (metadata.contains(controls::FrameDuration)) { int64_t duration = metadata.get(controls::FrameDuration) * 1000; resultMetadata->addEntry(ANDROID_SENSOR_FRAME_DURATION, diff --git a/src/android/camera_device.h b/src/android/camera_device.h index a65f1670..01c269d3 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -125,6 +125,7 @@ private: int facing_; int orientation_; + int minISO_; /* * Normally resubmission of controls would be handled on libcamera's diff --git a/src/android/camera_hal_config.cpp b/src/android/camera_hal_config.cpp index aa90dac7..863cdff0 100644 --- a/src/android/camera_hal_config.cpp +++ b/src/android/camera_hal_config.cpp @@ -180,6 +180,13 @@ int CameraHalConfig::Private::parseCameraConfigData(const std::string &cameraId) return -EINVAL; } cameraConfigData.rotation = ret; + } else if (key == "minISO") { + ret = std::stoi(value); + if (ret < 0 || ret > 100) { + LOG(HALConfig, Error) + << "Invalid minimum ISO: " << value; + return -EINVAL; + } } else { LOG(HALConfig, Error) << "Unknown key: " << key; @@ -384,7 +391,8 @@ int CameraHalConfig::parseConfigurationFile() const CameraConfigData &camera = c.second; LOG(HALConfig, Debug) << "'" << cameraId << "' " << "(" << camera.facing << ")[" - << camera.rotation << "]"; + << camera.rotation << "], " + << "minISO: " << camera.minISO; } return 0; diff --git a/src/android/camera_hal_config.h b/src/android/camera_hal_config.h index 9df554f9..d9ea45a4 100644 --- a/src/android/camera_hal_config.h +++ b/src/android/camera_hal_config.h @@ -15,6 +15,7 @@ struct CameraConfigData { int facing = -1; int rotation = -1; + int minISO = -1; }; class CameraHalConfig final : public libcamera::Extensible