From patchwork Mon Jun 5 08:26:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18702 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AA0D8C31E9 for ; Mon, 5 Jun 2023 08:26:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 216C36288C; Mon, 5 Jun 2023 10:26:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1685953574; bh=rLfQ7RAKOhmYuA3lXFrxyksvQqSiaXEF1+w4XeTiCw0=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=oWUmphUfK7SqoJFE/kOxY0FKN5ALflkcqHT9RnKsab+QLFlkdp4RwRXaQXqDp3RVh BjtvpgJG+MO7pCIx7fsKnDCijqF0a/lDJY7QdD4l1fd5KPXGrgMuo3k+Zu67zHBcPc RfI/EDJ6P/U0Onr6IyBEXzybrCqngYJLvaC4CMFXG+jXaHVPjpwJvJCacn9IFjOMlz VobxKnQMiyKBcWmQ1V18l3V98W51e28u4caGEEfd8re1VJEQ5Kb3YliPTa4x4RUWdD vThio/laG2nFFWPLgaaQHPpmQusTZFSgN84qgvJFa0L2oYunEaPTaKd5ZfGSYuGAO1 Q970bsgJoTIYA== Received: from mail-lf1-x12c.google.com (mail-lf1-x12c.google.com [IPv6:2a00:1450:4864:20::12c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AB83D62887 for ; Mon, 5 Jun 2023 10:26:12 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="DPZNk+5e"; dkim-atps=neutral Received: by mail-lf1-x12c.google.com with SMTP id 2adb3069b0e04-4f611ac39c5so3702588e87.2 for ; Mon, 05 Jun 2023 01:26:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1685953571; x=1688545571; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TYgdHBr0Gu7BZvR7BXYzmgmJeMThU6C/kNNpwP8vUB4=; b=DPZNk+5edPrsvG2PEQI6MUNAoX5uSaGm8E+0YcERMp6lKgbLnboYmbkAV1rJomqxs8 JZNSsv7SOq3HAVivsDRXecjR+mf58sL2Mqe45DMEl9090BVA99Doicya41DJul0IhiTT K2r50rcBViny4QsZaFhojsZqL2T878+fT4TxvhrZXAqCJUu0ENhxqIAbddKkDV2fZQCp 70AqBPZdp1MmPwEe1iSelh93jFm3EKvZuWyJajBHRBG8TZBrxuqdW/3VccdIWQ+SGdM0 urbeFK7496h+vXhqno7ImZW0s2ShAwGNgnMgBMjL7zE6WOLlmG4QFmTrkaMPNQBgY9vr 09Rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685953571; x=1688545571; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TYgdHBr0Gu7BZvR7BXYzmgmJeMThU6C/kNNpwP8vUB4=; b=Lw072GHyovkvdeKbq5AYXGfw3wR3U6JPGxyTdtkJL3JNmYi64IAzQY3Uc77g8ih+lX hhfeRDZFL7mOA7E1NjLcpxiJHuhNd4U4MizQt9YYU63cShhwfh+oCG3KDjH+OdiNBHoH 99h+cNVbMc2FylIDQCaRj2EP8BztyA9rkFRykAdb6axYmhVd+U/6c4evioesjt7DXnBq urlHnvcqgp+uqfyBa76ugiprNQvqHfUiOVm2hQt+/vK2p2kL7AYbRX83HRmhOkheSSUc ziEd6Lq8l9d8rYKe9znsVaq8I/CBKshP3EQtVZWMSPCbfFzas1e4zwYBJLsC0H+Os5IH OiuA== X-Gm-Message-State: AC+VfDw92hopG3lbOynLFLq103BmNZwoMT8/lif9W7Aq32sKvSmqSAwO EUzS2wVKc2omMDwIP7Jp1tcTIGHeb/uHBgr+OAIjhw== X-Google-Smtp-Source: ACHHUZ5e7tRMnSdTxWEi+mOatv3XWBVMFNSmhIk1ibYoh5KRC9Om++ctygiG8g/zdwZ3Y0FnNCrOWA== X-Received: by 2002:a05:6512:40a:b0:4f3:8143:765 with SMTP id u10-20020a056512040a00b004f381430765mr4048582lfk.27.1685953571452; Mon, 05 Jun 2023 01:26:11 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id 12-20020a05600c020c00b003f080b2f9f4sm13520686wmi.27.2023.06.05.01.26.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Jun 2023 01:26:10 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Mon, 5 Jun 2023 09:26:05 +0100 Message-Id: <20230605082605.9993-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230605082605.9993-1-naush@raspberrypi.com> References: <20230605082605.9993-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 2/2] ipa: rpi: Handle controls for mono variant sensors X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Do not advertise colour related controls (i.e. [A]WB, colour saturation) in the ControlInfoMap of available controls returned out to the application. Silently ignore these controls in the control handler in case applications don't use the advertised ControlInfoMap to validate controls. As a drive-by, don't advertise controls::ColourCorrectionMatrix in the ControlInfoMap as it is not handled by the IPA. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Laurent Pinchart --- include/libcamera/ipa/raspberrypi.mojom | 1 + src/ipa/rpi/common/ipa_base.cpp | 39 ++++++++++++++++--- src/ipa/rpi/common/ipa_base.h | 1 + .../pipeline/rpi/common/pipeline_base.cpp | 10 ++++- 4 files changed, 45 insertions(+), 6 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index ba786e647ca1..1ea51f99e2bb 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -21,6 +21,7 @@ struct SensorConfig { struct InitParams { bool lensPresent; + libcamera.IPACameraSensorInfo sensorInfo; }; struct InitResult { diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index db7a0eb3a1ca..7fe8ee738e01 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -12,6 +12,7 @@ #include #include #include +#include #include "controller/af_algorithm.h" #include "controller/af_status.h" @@ -60,19 +61,22 @@ const ControlInfoMap::Map ipaControls{ { &controls::AeConstraintMode, ControlInfo(controls::AeConstraintModeValues) }, { &controls::AeExposureMode, ControlInfo(controls::AeExposureModeValues) }, { &controls::ExposureValue, ControlInfo(-8.0f, 8.0f, 0.0f) }, - { &controls::AwbEnable, ControlInfo(false, true) }, - { &controls::ColourGains, ControlInfo(0.0f, 32.0f) }, - { &controls::AwbMode, ControlInfo(controls::AwbModeValues) }, { &controls::Brightness, ControlInfo(-1.0f, 1.0f, 0.0f) }, { &controls::Contrast, ControlInfo(0.0f, 32.0f, 1.0f) }, - { &controls::Saturation, ControlInfo(0.0f, 32.0f, 1.0f) }, { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, - { &controls::ColourCorrectionMatrix, ControlInfo(-16.0f, 16.0f) }, { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, { &controls::FrameDurationLimits, ControlInfo(INT64_C(33333), INT64_C(120000)) }, { &controls::draft::NoiseReductionMode, ControlInfo(controls::draft::NoiseReductionModeValues) } }; +/* IPA controls handled conditionally, if the sensor is not mono */ +const ControlInfoMap::Map ipaColourControls{ + { &controls::AwbEnable, ControlInfo(false, true) }, + { &controls::AwbMode, ControlInfo(controls::AwbModeValues) }, + { &controls::ColourGains, ControlInfo(0.0f, 32.0f) }, + { &controls::Saturation, ControlInfo(0.0f, 32.0f, 1.0f) }, +}; + /* IPA controls handled conditionally, if the lens has a focus control */ const ControlInfoMap::Map ipaAfControls{ { &controls::AfMode, ControlInfo(controls::AfModeValues) }, @@ -146,6 +150,11 @@ int32_t IpaBase::init(const IPASettings &settings, const InitParams ¶ms, Ini ControlInfoMap::Map ctrlMap = ipaControls; if (lensPresent_) ctrlMap.merge(ControlInfoMap::Map(ipaAfControls)); + + monoSensor_ = params.sensorInfo.cfaPattern == properties::draft::ColorFilterArrangementEnum::MONO; + if (!monoSensor_) + ctrlMap.merge(ControlInfoMap::Map(ipaColourControls)); + result->controlInfo = ControlInfoMap(std::move(ctrlMap), controls::controls); return platformInit(params, result); @@ -220,6 +229,10 @@ int32_t IpaBase::configure(const IPACameraSensorInfo &sensorInfo, const ConfigPa ControlInfo(static_cast(mode_.minShutter.get()), static_cast(mode_.maxShutter.get())); + /* Declare colour processing related controls for non-mono sensors. */ + if (!monoSensor_) + ctrlMap.merge(ControlInfoMap::Map(ipaColourControls)); + /* Declare Autofocus controls, only if we have a controllable lens */ if (lensPresent_) ctrlMap.merge(ControlInfoMap::Map(ipaAfControls)); @@ -780,6 +793,10 @@ void IpaBase::applyControls(const ControlList &controls) } case controls::AWB_ENABLE: { + /* Silently ignore this control for a mono sensor. */ + if (monoSensor_) + break; + RPiController::AwbAlgorithm *awb = dynamic_cast( controller_.getAlgorithm("awb")); if (!awb) { @@ -799,6 +816,10 @@ void IpaBase::applyControls(const ControlList &controls) } case controls::AWB_MODE: { + /* Silently ignore this control for a mono sensor. */ + if (monoSensor_) + break; + RPiController::AwbAlgorithm *awb = dynamic_cast( controller_.getAlgorithm("awb")); if (!awb) { @@ -819,6 +840,10 @@ void IpaBase::applyControls(const ControlList &controls) } case controls::COLOUR_GAINS: { + /* Silently ignore this control for a mono sensor. */ + if (monoSensor_) + break; + auto gains = ctrl.second.get>(); RPiController::AwbAlgorithm *awb = dynamic_cast( controller_.getAlgorithm("awb")); @@ -867,6 +892,10 @@ void IpaBase::applyControls(const ControlList &controls) } case controls::SATURATION: { + /* Silently ignore this control for a mono sensor. */ + if (monoSensor_) + break; + RPiController::CcmAlgorithm *ccm = dynamic_cast( controller_.getAlgorithm("ccm")); if (!ccm) { diff --git a/src/ipa/rpi/common/ipa_base.h b/src/ipa/rpi/common/ipa_base.h index 6f9c46bb16b1..39d00760d012 100644 --- a/src/ipa/rpi/common/ipa_base.h +++ b/src/ipa/rpi/common/ipa_base.h @@ -87,6 +87,7 @@ private: std::map buffers_; bool lensPresent_; + bool monoSensor_; ControlList libcameraMetadata_; std::array rpiMetadata_; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 3bb5ec531e4f..3aef1f43d697 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -1130,6 +1130,8 @@ int CameraData::loadPipelineConfiguration() int CameraData::loadIPA(ipa::RPi::InitResult *result) { + int ret; + ipa_ = IPAManager::createIPA(pipe(), 1, 1); if (!ipa_) @@ -1153,8 +1155,14 @@ int CameraData::loadIPA(ipa::RPi::InitResult *result) IPASettings settings(configurationFile, sensor_->model()); ipa::RPi::InitParams params; + ret = sensor_->sensorInfo(¶ms.sensorInfo); + if (ret) { + LOG(RPI, Error) << "Failed to retrieve camera sensor info"; + return ret; + } + params.lensPresent = !!sensor_->focusLens(); - int ret = platformInitIpa(params); + ret = platformInitIpa(params); if (ret) return ret;