From patchwork Fri Jun 2 13:23:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18682 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 945FDC31E9 for ; Fri, 2 Jun 2023 13:24:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E037E627B7; Fri, 2 Jun 2023 15:24:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1685712245; bh=ynXcv4Qb+p5lSh7D2ADhxWzIulk4MHPF0+UdsafwDzY=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=xzQq57X4LUCu7Q98qwG+A0NbtgXnnHIWF6CpBeVY4ZXCj5ci27JNBwcO9KNJb0Fb3 gvgXwB3l9l06G9abOvrtNUpzWzXx5i2QNnJtANZq2FO3QnfZFetTR45cUEl00pmxFM WZsjCPZlNNFaaBZi7CIlVD1XTjHbL0XUOSAyADgfA4MT/ypLY/nVN+UYr1jFZVP+/P OK+lgRNvJ0jVl4JoJhBrQ5hyhEJqdpYJD6KVMHYslsT9oDQnq86jvw5v8WbAm0vNFQ sH3Ldw1nho2VuTXzgWYt2o38adIxUc79K0cJ1H+AmUwChXNh8eeN4BTv3N4v6vKZrs ZbkhYmXIcmX4g== Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AB826626FA for ; Fri, 2 Jun 2023 15:24:03 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="iHAB8qnA"; dkim-atps=neutral Received: by mail-wm1-x32d.google.com with SMTP id 5b1f17b1804b1-3f6dfc4e01fso20760855e9.0 for ; Fri, 02 Jun 2023 06:24:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1685712243; x=1688304243; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=opMyWvUtv91kRAWjXfgo6VhVCMMQIQtCgxUQE3X/rw0=; b=iHAB8qnAbWhkVeXdmYEg8yXE4KPN5GDyb1/vd7KXkqZUOOmxmJug8p7No0QUcfV7jj M7em7v7g4XJnOYC3oo1i4wzicAKFnSJlEgplkXtHpW0XZSrpZGNHHo7utx3qgF2kZNap VJqn0nDHUJmYwmw+HRF2m2DNjBJMroL7E10lCo5GpBXYM9d6m06AGUcUFhnpQk4bpzta uEJMdvx4pGFrG/8KcirsICDmNwt4FX7XnB9fFTSbcGyrerfcL2HywYoNDfKngS0xkZFf giYQQ90M/tzYBWy0XxHB73az4LFh9yLiPc3XtQ6K7erzY+PKL4m/UKl74CLhdLSsKX44 A9fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685712243; x=1688304243; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=opMyWvUtv91kRAWjXfgo6VhVCMMQIQtCgxUQE3X/rw0=; b=fZ5PphyrD8ebxApYi1RCdfPpf2PDlLA8MWdK+kpAPE8o19aIzLwbNeedoP0qdvTv/b cLCs1PuxVgYASPLllvBMMqsjJugpTZe8kYvw0rPUfPePsV3yC5K69Q6bB9XZXrtjLTTF u/5M0SbGV5HG6SANWVmVP4eP2eG/zOe0NukcBSwoETd6/TPMMkFVAWwjfnddnyUt9+ad T/psakYSeiQRyTBUSjwa6C3tPLDr+DPRzFTETAqwpn2T+gGmStdLNQAknahxyZFEuVMC F0EpR6+Zzt/rqjYo88f5ry0Wokcz8J1vcVHdojkTE5fZZPQGOXQryu5hVODr/Ju+uFnD 2pQQ== X-Gm-Message-State: AC+VfDztg+0HoV0brWDclKFeGxcr7PaBwSGht4IoPkCpuzE11jsvFNN0 DF4Eopv+UpCmfb/3VklpCZ3IuHou7LAGR8yOewEpLQ== X-Google-Smtp-Source: ACHHUZ5Di3RIaBW+xFHMf8bgzxoyDghdg+7wh4UipbAOLPCj3blyY84Gf+OMgNpu2N4PeluI3jlbeg== X-Received: by 2002:adf:ee49:0:b0:30a:e369:5aca with SMTP id w9-20020adfee49000000b0030ae3695acamr4252306wro.25.1685712243024; Fri, 02 Jun 2023 06:24:03 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id v6-20020a1cf706000000b003f4ddde398csm1962863wmh.21.2023.06.02.06.24.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Jun 2023 06:24:02 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 2 Jun 2023 14:23:57 +0100 Message-Id: <20230602132358.16314-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230602132358.16314-1-naush@raspberrypi.com> References: <20230602132358.16314-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 1/2] camera_sensor: ipa: core: Add CFA pattern to IPACameraSensorInfo X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a new cfaPattern field to the IPACameraSensorInfo to pass the CFA/Bayer pattern for the current sensor configuration to the IPA. This field takes a value from properties::draft::ColorFilterArrangementEnum. Populate cfaPattern in CameraSensor::sensorInfo(), called by the pipeline handler before it calls ipa->init() and ipa->config(). Signed-off-by: Naushir Patuck Reviewed-by: Laurent Pinchart --- include/libcamera/ipa/core.mojom | 10 ++++++++++ src/libcamera/camera_sensor.cpp | 3 +++ 2 files changed, 13 insertions(+) diff --git a/include/libcamera/ipa/core.mojom b/include/libcamera/ipa/core.mojom index 5cef340d014c..af02efacf335 100644 --- a/include/libcamera/ipa/core.mojom +++ b/include/libcamera/ipa/core.mojom @@ -140,6 +140,15 @@ module libcamera; * image sensor */ +/** + * \var IPACameraSensorInfo::cfaPattern + * \brief The arrangement of colour filters on the image sensor + * + * This takes a value defined by properties::draft::ColorFilterArrangementEnum. + * For non-Bayer colour sensors, the cfaPattern will be set to + * properties::draft::ColorFilterArrangementEnum::RGB. + */ + /** * \var IPACameraSensorInfo::activeAreaSize * \brief The size of the pixel array active area of the sensor @@ -229,6 +238,7 @@ struct IPACameraSensorInfo { string model; uint32 bitsPerPixel; + uint32 cfaPattern; Size activeAreaSize; Rectangle analogCrop; diff --git a/src/libcamera/camera_sensor.cpp b/src/libcamera/camera_sensor.cpp index 06f315210240..60bf87b49e6e 100644 --- a/src/libcamera/camera_sensor.cpp +++ b/src/libcamera/camera_sensor.cpp @@ -956,6 +956,9 @@ int CameraSensor::sensorInfo(IPACameraSensorInfo *info) const info->bitsPerPixel = format.bitsPerPixel(); info->outputSize = format.size; + std::optional cfa = properties_.get(properties::draft::ColorFilterArrangement); + info->cfaPattern = cfa ? *cfa : properties::draft::RGB; + /* * Retrieve the pixel rate, line length and minimum/maximum frame * duration through V4L2 controls. Support for the V4L2_CID_PIXEL_RATE, From patchwork Fri Jun 2 13:23:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18683 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 89DC9C328F for ; Fri, 2 Jun 2023 13:24:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7FBC262754; Fri, 2 Jun 2023 15:24:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1685712246; bh=Pqe7SWvu9jyqsq+ZGNlm5Lae9tUO6Ejdo0ToUF7E5os=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=nT/3dtvwYjfbSixH0jhRihidqE5nVnoRg+seJBA/hSx0hi+ncgg6v1tafHscLuJ3o BlT4g+M4prKzR6qgHNwC79bYRkh+bdh2n7FTTZS2g1PnsiDXc1nas9j9xZ+mu/6xjW bed2CWmhosVeaskJq0chTBZBRIw5RJ0kE6lidUyL8CJ9XTSaGoU0oA+bGPcNsp+Aox 70epShlRLTFWvGu9K6L3W3aD0iOw58zJAN0pER/TDyIMlSvILtR6ZpXkp3ENMOFElI NpP5rUBQQUaTM9SSEREOytiNOgc45C8AOVvC5lIr1b6I7YDJgzyG8LPTLylpxbzgdH SawXdTxsaqMcw== Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8532662728 for ; Fri, 2 Jun 2023 15:24:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="TxLanRYh"; dkim-atps=neutral Received: by mail-wr1-x42e.google.com with SMTP id ffacd0b85a97d-3078cc99232so1870182f8f.3 for ; Fri, 02 Jun 2023 06:24:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1685712244; x=1688304244; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JNStbG5poG/ow/UtRjFvX8TiTOJwLn53gkEsaXMgr5Y=; b=TxLanRYhktnh10K3cbzvXRzezRdTLh6QJSrm2nuqL+k9AitWZ5JooLOOwHDBUi4O/0 UPZtVQwMDVkYHCflUL3IcwlIGmbOTol7CBq1tnT+3hFLAChhwqljiB9nkQPo8+8LfYOF gya+BmbMuUvxs6yJ5JYTmQpMEFedYSTetWv7EQrYc1R0TYc2ne25XX52qq+UCr7vLpJk sPjc2Sl+DovfB73pPiB8qUkRvZMGCK77OJfszHl81x9UizTG8CuOyN4zuHqKHzABz7cK Mop7TFbERmhGAUKD2+m5FnQXQi2c5iWz0gU7oj4d+F3Ne6rnmyFXKIGG/2Jkipo3Fv9G LVLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685712244; x=1688304244; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JNStbG5poG/ow/UtRjFvX8TiTOJwLn53gkEsaXMgr5Y=; b=QOyXV+Z5GnA7OfNU2q6B4vQ2tjeLY28taGgguAAl+8X1mWvsOVDIueKs61DTTXX2x+ 5iFrP0DvnAJ8OMOajMv3UAT5/gbI7ZyY3EnB0KwOWnCTKntZpVcnMAvKgGWuk9/GNL5m plN2hQ6CgB2LHvCK8LOAjWHNqMxQXoM86s1/s3VkPabYv+bp1RPwH9bBHdSd+TF+mSgz 90dV8XEwDp6kboktvUv/wCnBwSThPw90Z94p8caDpPlGDB7wcCGV96FdD3piqzplHZqs 2knA4SdIeoQtLbjwY0UD9qAn1eRpk4lAXJtbUFLn060VFqtwuPqyDjokh5AiBXawCT/Y S9Zg== X-Gm-Message-State: AC+VfDwVSs7WzQ+ctDGAYnap7kuq3htlc+adbUWPeAtIoxzWQp0/Nk41 0TuwT/ckTWb2yqoYYNDm6LHFFzxLBmeKvC8bBxsMig== X-Google-Smtp-Source: ACHHUZ6RA/BUR6fA7xwKCumAhID75A1hn2FOj/i/ylX4W6/oT+8OGMrkLK+nkrmsZFc9OI9TETw3aw== X-Received: by 2002:adf:fa44:0:b0:306:2dd6:95d3 with SMTP id y4-20020adffa44000000b003062dd695d3mr4444097wrr.22.1685712243857; Fri, 02 Jun 2023 06:24:03 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id v6-20020a1cf706000000b003f4ddde398csm1962863wmh.21.2023.06.02.06.24.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Jun 2023 06:24:03 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 2 Jun 2023 14:23:58 +0100 Message-Id: <20230602132358.16314-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230602132358.16314-1-naush@raspberrypi.com> References: <20230602132358.16314-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 2/2] ipa: rpi: Handle controls for mono variant sensors X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Do not advertise colour related controls (i.e. [A]WB, colour saturation) in the ControlInfoMap of available controls returned out to the application. Silently ignore these controls in the control handler in case applications don't use the advertised ControlInfoMap to validate controls. As a drive-by, don't advertise controls::ColourCorrectionMatrix in the ControlInfoMap as it is not handled by the IPA. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Laurent Pinchart --- src/ipa/rpi/common/ipa_base.cpp | 35 ++++++++++++++++++++++++++++----- src/ipa/rpi/common/ipa_base.h | 1 + 2 files changed, 31 insertions(+), 5 deletions(-) diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index db7a0eb3a1ca..813e01fa5cfd 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -12,6 +12,7 @@ #include #include #include +#include #include "controller/af_algorithm.h" #include "controller/af_status.h" @@ -60,19 +61,22 @@ const ControlInfoMap::Map ipaControls{ { &controls::AeConstraintMode, ControlInfo(controls::AeConstraintModeValues) }, { &controls::AeExposureMode, ControlInfo(controls::AeExposureModeValues) }, { &controls::ExposureValue, ControlInfo(-8.0f, 8.0f, 0.0f) }, - { &controls::AwbEnable, ControlInfo(false, true) }, - { &controls::ColourGains, ControlInfo(0.0f, 32.0f) }, - { &controls::AwbMode, ControlInfo(controls::AwbModeValues) }, { &controls::Brightness, ControlInfo(-1.0f, 1.0f, 0.0f) }, { &controls::Contrast, ControlInfo(0.0f, 32.0f, 1.0f) }, - { &controls::Saturation, ControlInfo(0.0f, 32.0f, 1.0f) }, { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, - { &controls::ColourCorrectionMatrix, ControlInfo(-16.0f, 16.0f) }, { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, { &controls::FrameDurationLimits, ControlInfo(INT64_C(33333), INT64_C(120000)) }, { &controls::draft::NoiseReductionMode, ControlInfo(controls::draft::NoiseReductionModeValues) } }; +/* IPA controls handled conditionally, if the sensor is not mono */ +const ControlInfoMap::Map ipaColourControls{ + { &controls::AwbEnable, ControlInfo(false, true) }, + { &controls::AwbMode, ControlInfo(controls::AwbModeValues) }, + { &controls::ColourGains, ControlInfo(0.0f, 32.0f) }, + { &controls::Saturation, ControlInfo(0.0f, 32.0f, 1.0f) }, +}; + /* IPA controls handled conditionally, if the lens has a focus control */ const ControlInfoMap::Map ipaAfControls{ { &controls::AfMode, ControlInfo(controls::AfModeValues) }, @@ -220,6 +224,11 @@ int32_t IpaBase::configure(const IPACameraSensorInfo &sensorInfo, const ConfigPa ControlInfo(static_cast(mode_.minShutter.get()), static_cast(mode_.maxShutter.get())); + /* Declare colour processing related controls for non-mono sensors. */ + monoSensor_ = sensorInfo.cfaPattern == properties::draft::ColorFilterArrangementEnum::MONO; + if (!monoSensor_) + ctrlMap.merge(ControlInfoMap::Map(ipaColourControls)); + /* Declare Autofocus controls, only if we have a controllable lens */ if (lensPresent_) ctrlMap.merge(ControlInfoMap::Map(ipaAfControls)); @@ -780,6 +789,10 @@ void IpaBase::applyControls(const ControlList &controls) } case controls::AWB_ENABLE: { + /* Silently ignore this control for a mono sensor. */ + if (monoSensor_) + break; + RPiController::AwbAlgorithm *awb = dynamic_cast( controller_.getAlgorithm("awb")); if (!awb) { @@ -799,6 +812,10 @@ void IpaBase::applyControls(const ControlList &controls) } case controls::AWB_MODE: { + /* Silently ignore this control for a mono sensor. */ + if (monoSensor_) + break; + RPiController::AwbAlgorithm *awb = dynamic_cast( controller_.getAlgorithm("awb")); if (!awb) { @@ -819,6 +836,10 @@ void IpaBase::applyControls(const ControlList &controls) } case controls::COLOUR_GAINS: { + /* Silently ignore this control for a mono sensor. */ + if (monoSensor_) + break; + auto gains = ctrl.second.get>(); RPiController::AwbAlgorithm *awb = dynamic_cast( controller_.getAlgorithm("awb")); @@ -867,6 +888,10 @@ void IpaBase::applyControls(const ControlList &controls) } case controls::SATURATION: { + /* Silently ignore this control for a mono sensor. */ + if (monoSensor_) + break; + RPiController::CcmAlgorithm *ccm = dynamic_cast( controller_.getAlgorithm("ccm")); if (!ccm) { diff --git a/src/ipa/rpi/common/ipa_base.h b/src/ipa/rpi/common/ipa_base.h index 6f9c46bb16b1..39d00760d012 100644 --- a/src/ipa/rpi/common/ipa_base.h +++ b/src/ipa/rpi/common/ipa_base.h @@ -87,6 +87,7 @@ private: std::map buffers_; bool lensPresent_; + bool monoSensor_; ControlList libcameraMetadata_; std::array rpiMetadata_;