From patchwork Wed May 31 14:39:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18672 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1C80DC32A2 for ; Wed, 31 May 2023 14:39:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9F93A6278F; Wed, 31 May 2023 16:39:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1685543988; bh=xIPrrDCNBeF433N6Y5+8Zw/9QHYqun04ahmKZgclQzs=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=nd8uT9Ul6ChBFlMacQMtlMNyErUtwIyTOrOcyWeuGB7rxuz6ifk08/ObCA95sdv8z B9og8QK3St400tMblHmOxrGXDwDlYCBX1HCadnqK76S3S2HlQYmE2XvWQ12xcs95JH j6YqASFWVnGvG1cAynrQHj2KeFw1xmcMb1TqM9fFy6D4jAlRQ+meVqthEpLAFz4JX7 LAqHYZw5r5Wfru1TEuh+9mNEGZPqnciiRweOzHYShk+6119My/6zp5vUpbjN8cVejN eQQ5EmInLZI88WB8YziiLuBF/83QanIeH5cmrqVDCYmCdg9A+uQrPcJZMvpqMYreaM pX9W5HYfoNvlw== Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4A27F626FA for ; Wed, 31 May 2023 16:39:47 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="mhvI7Hpu"; dkim-atps=neutral Received: by mail-wr1-x42a.google.com with SMTP id ffacd0b85a97d-30aef0499b6so2653482f8f.1 for ; Wed, 31 May 2023 07:39:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1685543986; x=1688135986; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tyXUDVu1LsN3DHz4TMrGFcmEle0ZihpV+mgI1mhlvL0=; b=mhvI7HpuCNfr7Drn96mVuc/Ul4HzPR9pSB586f85ePLa5gBdtHRXrClr3pEP7NDQ+B 7lx6cjmAg1h/zUNAvRsTFwCF27Sd6qoxzxmCi3ccBkv65BsQUADEfnrLJ7Z26wtTwzKo YKBgdGbQercv1+YRv2AAAb0IhCNA9L84j6j+WGWotgexOszgEfEi9GmqbqiptVnCVKNg AP8phXJuGne0bymdPCvd41Cn/GAn4FyuTp65edMsDBVXFOTR8tJTAUQ2QBnmhl3VFmgg Q7oDpD04ykKKVOEpVtZ5kOTEcyaf4aLYuXiSyDy/XMgdCrMoBy3Y1T/AOM6JY+EDjv2g NVJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685543986; x=1688135986; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tyXUDVu1LsN3DHz4TMrGFcmEle0ZihpV+mgI1mhlvL0=; b=KVQETYWodMkrOICUcZrriZir22NR/se9axHY6+9MCi0IiXwGzSrifKPJpp/L727WQ2 bHxd5cbg7EDcl56Lx9CIkGP7sMhh6nsTjodO+wJJiQJOgJ5lQq7vGIlj52EXDSunP3de zdP1U+I4RsBm4B7ECNHSkTP++j0wfrAU3yvIFylzOTtjvnW6w0BqCw27xO0RKYIRMBRB cFE9yqbfH+m7rGeMtoV/HcQVq5x2ZH+rdSa29WSlBvMH/BBuiHzc6en4LVHR9DesnTa6 XBhOTVIobs7Rwlszd2cLSo2fs/wP/N/bnoHdho1f832BMf4ej/qmzXx00b9FzQjR8879 CasQ== X-Gm-Message-State: AC+VfDxmh1Z37t2jIue4kh5KFKBfQKj8f0v22NR7fY0j4B1XqBTndKWg dPicB8MGAXCP3DteE2Ffr1v9EHD/CY/jh6YKrVAYPw== X-Google-Smtp-Source: ACHHUZ7KqKghn1zA3Rte2pC1D0uXxJAut/a7G3RDD/uJ6fWoFdRZcNtUuh/lHycXDZrWsofWrJt1LQ== X-Received: by 2002:a5d:5448:0:b0:30a:93ad:ab9f with SMTP id w8-20020a5d5448000000b0030a93adab9fmr3778456wrv.11.1685543986743; Wed, 31 May 2023 07:39:46 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id bf3-20020a0560001cc300b00307b5376b2csm6950706wrb.90.2023.05.31.07.39.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 May 2023 07:39:46 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 31 May 2023 15:39:45 +0100 Message-Id: <20230531143946.23571-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230531143946.23571-1-naush@raspberrypi.com> References: <20230531143946.23571-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 1/2] pipeline: ipa: rpi: Flag if the sensor is a mono variant X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a flag to the ipa->init() interface to indicate a mono sensor variant. This flag will be used in a future commit to handle controls that are invalid for mono sensors. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Laurent Pinchart --- include/libcamera/ipa/raspberrypi.mojom | 1 + src/ipa/rpi/common/ipa_base.cpp | 1 + src/ipa/rpi/common/ipa_base.h | 1 + src/libcamera/pipeline/rpi/common/pipeline_base.cpp | 4 +++- 4 files changed, 6 insertions(+), 1 deletion(-) diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index ba786e647ca1..d35289ee6229 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -21,6 +21,7 @@ struct SensorConfig { struct InitParams { bool lensPresent; + bool monoSensor; }; struct InitResult { diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index db7a0eb3a1ca..150fe433d0df 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -139,6 +139,7 @@ int32_t IpaBase::init(const IPASettings &settings, const InitParams ¶ms, Ini } lensPresent_ = params.lensPresent; + monoSensor_ = params.monoSensor; controller_.initialise(); diff --git a/src/ipa/rpi/common/ipa_base.h b/src/ipa/rpi/common/ipa_base.h index 6f9c46bb16b1..39d00760d012 100644 --- a/src/ipa/rpi/common/ipa_base.h +++ b/src/ipa/rpi/common/ipa_base.h @@ -87,6 +87,7 @@ private: std::map buffers_; bool lensPresent_; + bool monoSensor_; ControlList libcameraMetadata_; std::array rpiMetadata_; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 3bb5ec531e4f..12698056cda1 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -1131,6 +1131,7 @@ int CameraData::loadPipelineConfiguration() int CameraData::loadIPA(ipa::RPi::InitResult *result) { ipa_ = IPAManager::createIPA(pipe(), 1, 1); + bool monoSensor = isMonoSensor(sensor_); if (!ipa_) return -ENOENT; @@ -1143,7 +1144,7 @@ int CameraData::loadIPA(ipa::RPi::InitResult *result) char const *configFromEnv = utils::secure_getenv("LIBCAMERA_RPI_TUNING_FILE"); if (!configFromEnv || *configFromEnv == '\0') { std::string model = sensor_->model(); - if (isMonoSensor(sensor_)) + if (monoSensor) model += "_mono"; configurationFile = ipa_->configurationFile(model + ".json"); } else { @@ -1154,6 +1155,7 @@ int CameraData::loadIPA(ipa::RPi::InitResult *result) ipa::RPi::InitParams params; params.lensPresent = !!sensor_->focusLens(); + params.monoSensor = monoSensor; int ret = platformInitIpa(params); if (ret) return ret; From patchwork Wed May 31 14:39:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18673 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5BD81C31E9 for ; Wed, 31 May 2023 14:39:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1F0AA627D1; Wed, 31 May 2023 16:39:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1685543991; bh=7BZN1AlolWFBEMGDgGFilHATZjbmFpEyNl3QkaYxdDs=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=JYutwI9qi+Z5cp6Hzgn4DbKzVzA7uMZXfsJaJEUCOehPiydAVGn/Md83emhQQIRfu OLYnxedDbsS69TTsYAHvnaIp8ArtFrw5nIN7MpSfS0bgay7Mb7hIp//ArgCBVMt3pD KJH1ktCMleQtZhrcdYXK+QQ+dFsrH+vGrM7kSEBXVjlG96NDNnD+6+Ctjibim9FtP/ DtiyldBbX2glAxREQQnO5Xbud4SD+VAB92i+5lgp+Yvzem2ucU3yBahepn3IxtsVwP pws3pfQ13nfaXiLUMJ/2jEjZDvs+wja0Zwgdq8ZNPXiEOjH8+8fymBXMV2/pRQRwy6 FYXTpdUAiFkIA== Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com [IPv6:2a00:1450:4864:20::12e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9D7AE62763 for ; Wed, 31 May 2023 16:39:48 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="FWNEhpex"; dkim-atps=neutral Received: by mail-lf1-x12e.google.com with SMTP id 2adb3069b0e04-4f004cc54f4so6927665e87.3 for ; Wed, 31 May 2023 07:39:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1685543987; x=1688135987; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tcqo6e3wIxKk44o7NyMRaUqwC8ijo2NlEHwDdfflmN8=; b=FWNEhpexzIrlSC6ViOIo3AnsRjui3NvExjMP2T8Br5ngqKvrRWRiGkGqoJ/vuEzpRg Aiw28O7Obxj7je/AwVrMeyZo88GAh94VSHEryYQh7b9ngRShWbOQ/aKyESTOrhhxVcWm RqfZt2dtTJxQ+KHkSsCC7LnnD6vI2ioCbo8aIjkH1mVE5lDlZrCt1dniN3rvUkRI7DSZ 9I0HIU8cC22c1v5L6CB0CNNpBKcYLWYexU69aYFaM/UwOilNqVWC2f3c0AZZpDo9DgYf 6/cFihqNjKOy/602Xd5aHPyYK2xVigIRsHzmNEG8Kkfw8TSb7dpZMYv9s4oO6IEod0/a pS8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685543987; x=1688135987; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tcqo6e3wIxKk44o7NyMRaUqwC8ijo2NlEHwDdfflmN8=; b=JOUoYRskdG0D4ACraN/kLR7KxhEp1zaeWqkZx6IsN61xKOxh0n/A6WvEjCO0l55QJg /rJxt0/DK+whu0gp0OBH85AtT1bcTu+7qHuIOkxx7k8y7WRPlWhGw4lFVofU8MZy79cM pBWFNulbWnh1eD8m84oN+PoAGOZ9NuTgaa+Ktjd8tXbjycZZA0Ed7Ch5UY/m3LAlforq QQKWUnXbY3BCWtOUE7fbRzTp7TIqtnUYZnS00yZpZV8cRxaBJOaNsQLLXTRLdSGcQ2lY eq0W3844UyyZjDVzEycjEEcOlxP/hCfPXR7PX9gmdQSBsJrTQvu5vVQXA8TJ5eS0G+cW lgnA== X-Gm-Message-State: AC+VfDwsHWhkxpBEGF7UQZ1OyDO+QduaOcf282phP5KWzmCnCKhNZf9v NrNhKu97nH4S8kSWLw2q/cXLQ/BeBxyUF/1frXaQeQ== X-Google-Smtp-Source: ACHHUZ6+dByuE2IphRNlk9Nv2glEW9bM5H65SGVwCWFheykryr4f+9et8HZNzxIbve8T2CBjnDk/sQ== X-Received: by 2002:ac2:599e:0:b0:4f4:d83e:4134 with SMTP id w30-20020ac2599e000000b004f4d83e4134mr2626348lfn.57.1685543987420; Wed, 31 May 2023 07:39:47 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id bf3-20020a0560001cc300b00307b5376b2csm6950706wrb.90.2023.05.31.07.39.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 May 2023 07:39:47 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 31 May 2023 15:39:46 +0100 Message-Id: <20230531143946.23571-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230531143946.23571-1-naush@raspberrypi.com> References: <20230531143946.23571-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 2/2] ipa: rpi: Handle controls for mono variant sensors X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Do not advertise colour related controls (i.e. [A]WB, colour saturation) in the ControlInfoMap of available controls returned out to the application. Silently fail these controls in the control handler in case applications don't use the advertised ControlInfoMap to validate controls. As a drive-by, don't advertise controls::ColourCorrectionMatrix in the ControlInfoMap as it is not handled by the IPA. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Laurent Pinchart --- src/ipa/rpi/common/ipa_base.cpp | 34 ++++++++++++++++++++++++++++----- 1 file changed, 29 insertions(+), 5 deletions(-) diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index 150fe433d0df..8de454637cfe 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -60,19 +60,22 @@ const ControlInfoMap::Map ipaControls{ { &controls::AeConstraintMode, ControlInfo(controls::AeConstraintModeValues) }, { &controls::AeExposureMode, ControlInfo(controls::AeExposureModeValues) }, { &controls::ExposureValue, ControlInfo(-8.0f, 8.0f, 0.0f) }, - { &controls::AwbEnable, ControlInfo(false, true) }, - { &controls::ColourGains, ControlInfo(0.0f, 32.0f) }, - { &controls::AwbMode, ControlInfo(controls::AwbModeValues) }, { &controls::Brightness, ControlInfo(-1.0f, 1.0f, 0.0f) }, { &controls::Contrast, ControlInfo(0.0f, 32.0f, 1.0f) }, - { &controls::Saturation, ControlInfo(0.0f, 32.0f, 1.0f) }, { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, - { &controls::ColourCorrectionMatrix, ControlInfo(-16.0f, 16.0f) }, { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, { &controls::FrameDurationLimits, ControlInfo(INT64_C(33333), INT64_C(120000)) }, { &controls::draft::NoiseReductionMode, ControlInfo(controls::draft::NoiseReductionModeValues) } }; +/* IPA controls handled conditionally, if the sensor is not mono */ +const ControlInfoMap::Map ipaColourControls{ + { &controls::AwbEnable, ControlInfo(false, true) }, + { &controls::AwbMode, ControlInfo(controls::AwbModeValues) }, + { &controls::ColourGains, ControlInfo(0.0f, 32.0f) }, + { &controls::Saturation, ControlInfo(0.0f, 32.0f, 1.0f) }, +}; + /* IPA controls handled conditionally, if the lens has a focus control */ const ControlInfoMap::Map ipaAfControls{ { &controls::AfMode, ControlInfo(controls::AfModeValues) }, @@ -145,6 +148,8 @@ int32_t IpaBase::init(const IPASettings &settings, const InitParams ¶ms, Ini /* Return the controls handled by the IPA */ ControlInfoMap::Map ctrlMap = ipaControls; + if (!monoSensor_) + ctrlMap.merge(ControlInfoMap::Map(ipaColourControls)); if (lensPresent_) ctrlMap.merge(ControlInfoMap::Map(ipaAfControls)); result->controlInfo = ControlInfoMap(std::move(ctrlMap), controls::controls); @@ -221,6 +226,9 @@ int32_t IpaBase::configure(const IPACameraSensorInfo &sensorInfo, const ConfigPa ControlInfo(static_cast(mode_.minShutter.get()), static_cast(mode_.maxShutter.get())); + if (!monoSensor_) + ctrlMap.merge(ControlInfoMap::Map(ipaColourControls)); + /* Declare Autofocus controls, only if we have a controllable lens */ if (lensPresent_) ctrlMap.merge(ControlInfoMap::Map(ipaAfControls)); @@ -781,6 +789,10 @@ void IpaBase::applyControls(const ControlList &controls) } case controls::AWB_ENABLE: { + /* Silently ignore this control for a mono sensor. */ + if (monoSensor_) + break; + RPiController::AwbAlgorithm *awb = dynamic_cast( controller_.getAlgorithm("awb")); if (!awb) { @@ -800,6 +812,10 @@ void IpaBase::applyControls(const ControlList &controls) } case controls::AWB_MODE: { + /* Silently ignore this control for a mono sensor. */ + if (monoSensor_) + break; + RPiController::AwbAlgorithm *awb = dynamic_cast( controller_.getAlgorithm("awb")); if (!awb) { @@ -820,6 +836,10 @@ void IpaBase::applyControls(const ControlList &controls) } case controls::COLOUR_GAINS: { + /* Silently ignore this control for a mono sensor. */ + if (monoSensor_) + break; + auto gains = ctrl.second.get>(); RPiController::AwbAlgorithm *awb = dynamic_cast( controller_.getAlgorithm("awb")); @@ -868,6 +888,10 @@ void IpaBase::applyControls(const ControlList &controls) } case controls::SATURATION: { + /* Silently ignore this control for a mono sensor. */ + if (monoSensor_) + break; + RPiController::CcmAlgorithm *ccm = dynamic_cast( controller_.getAlgorithm("ccm")); if (!ccm) {