From patchwork Mon Jul 21 10:46:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 23888 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8EAA8BDCC1 for ; Mon, 21 Jul 2025 10:47:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CE9316900C; Mon, 21 Jul 2025 12:47:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="osp8kEnu"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 664BE68FEF for ; Mon, 21 Jul 2025 12:46:38 +0200 (CEST) Received: from pb-laptop.local (185.221.140.39.nat.pool.zt.hu [185.221.140.39]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3C2C67981 for ; Mon, 21 Jul 2025 12:46:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1753094761; bh=aCTuYdtX+pDF7S01f/sgRFXRzXwaKNfgUWI/cJ+IUXQ=; h=From:To:Subject:Date:In-Reply-To:References:From; b=osp8kEnuh20QzUeAEIARaycmy4yNvM1sfsNioSUth30deze6O3OsQZN6ulYeVaTzc zBYtqAPVRKnBWTpQGg3paPGcYCwBPpsp4L35gJVB9TGEul2BucR+wnvCOWAQhUWGZU 4YYyLndHHi8F7c7Q725QzLKNo8mhevM9nunCBn9Q= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Subject: [RFC PATCH v2 20/22] libcamera: pipeline: Fill `MetadataListPlan` of cameras Date: Mon, 21 Jul 2025 12:46:20 +0200 Message-ID: <20250721104622.1550908-21-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250721104622.1550908-1-barnabas.pocze@ideasonboard.com> References: <20250721104622.1550908-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Fill the newly introduced `MetadataListPlan` member of the camera's private data during initializations, similarly to the camera's `ControlInfoMap`. Signed-off-by: Barnabás Pőcze --- changes in v2: * add missing controls for `rkisp1` * sort alphabetically --- .../internal/software_isp/software_isp.h | 3 +- include/libcamera/ipa/ipu3.mojom | 3 +- include/libcamera/ipa/mali-c55.mojom | 3 +- include/libcamera/ipa/raspberrypi.mojom | 1 + include/libcamera/ipa/rkisp1.mojom | 3 +- include/libcamera/ipa/soft.mojom | 3 +- src/ipa/ipu3/algorithms/agc.cpp | 4 +++ src/ipa/ipu3/algorithms/awb.cpp | 12 +++++++ src/ipa/ipu3/algorithms/awb.h | 1 + src/ipa/ipu3/ipa_context.cpp | 3 ++ src/ipa/ipu3/ipa_context.h | 3 ++ src/ipa/ipu3/ipu3.cpp | 8 +++-- src/ipa/mali-c55/algorithms/agc.cpp | 5 +++ src/ipa/mali-c55/algorithms/awb.cpp | 7 ++++ src/ipa/mali-c55/algorithms/awb.h | 1 + src/ipa/mali-c55/algorithms/blc.cpp | 2 ++ src/ipa/mali-c55/ipa_context.h | 3 ++ src/ipa/mali-c55/mali-c55.cpp | 6 ++-- src/ipa/rkisp1/algorithms/agc.cpp | 10 ++++++ src/ipa/rkisp1/algorithms/awb.cpp | 4 +++ src/ipa/rkisp1/algorithms/blc.cpp | 2 ++ src/ipa/rkisp1/algorithms/ccm.cpp | 2 ++ src/ipa/rkisp1/algorithms/goc.cpp | 2 ++ src/ipa/rkisp1/algorithms/lux.cpp | 8 ++++- src/ipa/rkisp1/ipa_context.h | 1 + src/ipa/rkisp1/rkisp1.cpp | 8 +++-- src/ipa/rpi/common/ipa_base.cpp | 34 +++++++++++++++++++ src/ipa/rpi/pisp/pisp.cpp | 5 +-- src/ipa/rpi/vc4/vc4.cpp | 4 ++- src/ipa/simple/algorithms/agc.cpp | 8 +++++ src/ipa/simple/algorithms/agc.h | 1 + src/ipa/simple/algorithms/awb.cpp | 8 +++++ src/ipa/simple/algorithms/awb.h | 1 + src/ipa/simple/algorithms/blc.cpp | 3 ++ src/ipa/simple/algorithms/ccm.cpp | 3 ++ src/ipa/simple/algorithms/lut.cpp | 3 ++ src/ipa/simple/ipa_context.h | 2 ++ src/ipa/simple/soft_simple.cpp | 8 +++-- src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 2 ++ src/libcamera/pipeline/ipu3/ipu3.cpp | 7 +++- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 2 +- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 4 ++- .../pipeline/rpi/common/pipeline_base.cpp | 8 +++++ src/libcamera/pipeline/simple/simple.cpp | 2 +- src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 2 ++ src/libcamera/pipeline/vimc/vimc.cpp | 2 ++ .../pipeline/virtual/config_parser.cpp | 3 ++ src/libcamera/software_isp/software_isp.cpp | 6 ++-- 48 files changed, 203 insertions(+), 23 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index 786246592..bd85f6b4b 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -41,6 +41,7 @@ class DebayerCpu; class FrameBuffer; class PixelFormat; class Stream; +class MetadataListPlan; struct StreamConfiguration; LOG_DECLARE_CATEGORY(SoftwareIsp) @@ -49,7 +50,7 @@ class SoftwareIsp : public Object { public: SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, - ControlInfoMap *ipaControls); + ControlInfoMap *ipaControls, MetadataListPlan *metadataPlan); ~SoftwareIsp(); int loadConfiguration([[maybe_unused]] const std::string &filename) { return 0; } diff --git a/include/libcamera/ipa/ipu3.mojom b/include/libcamera/ipa/ipu3.mojom index d9a50b01d..f49b77797 100644 --- a/include/libcamera/ipa/ipu3.mojom +++ b/include/libcamera/ipa/ipu3.mojom @@ -20,7 +20,8 @@ interface IPAIPU3Interface { init(libcamera.IPASettings settings, libcamera.IPACameraSensorInfo sensorInfo, libcamera.ControlInfoMap sensorControls) - => (int32 ret, libcamera.ControlInfoMap ipaControls); + => (int32 ret, libcamera.ControlInfoMap ipaControls, + libcamera.MetadataListPlan metadata); start() => (int32 ret); stop(); diff --git a/include/libcamera/ipa/mali-c55.mojom b/include/libcamera/ipa/mali-c55.mojom index 5d7eb4eef..7229c53e1 100644 --- a/include/libcamera/ipa/mali-c55.mojom +++ b/include/libcamera/ipa/mali-c55.mojom @@ -11,7 +11,8 @@ struct IPAConfigInfo { interface IPAMaliC55Interface { init(libcamera.IPASettings settings, IPAConfigInfo configInfo) - => (int32 ret, libcamera.ControlInfoMap ipaControls); + => (int32 ret, libcamera.ControlInfoMap ipaControls, + libcamera.MetadataListPlan metadataPlan); start() => (int32 ret); stop(); diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index 12b083e9d..c32391911 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -26,6 +26,7 @@ struct InitParams { struct InitResult { SensorConfig sensorConfig; libcamera.ControlInfoMap controlInfo; + libcamera.MetadataListPlan metadataPlan; }; struct BufferIds { diff --git a/include/libcamera/ipa/rkisp1.mojom b/include/libcamera/ipa/rkisp1.mojom index 043ad27ea..2e9df1f3b 100644 --- a/include/libcamera/ipa/rkisp1.mojom +++ b/include/libcamera/ipa/rkisp1.mojom @@ -19,7 +19,8 @@ interface IPARkISP1Interface { uint32 hwRevision, libcamera.IPACameraSensorInfo sensorInfo, libcamera.ControlInfoMap sensorControls) - => (int32 ret, libcamera.ControlInfoMap ipaControls); + => (int32 ret, libcamera.ControlInfoMap ipaControls, + libcamera.MetadataListPlan metadataPlan); start() => (int32 ret); stop(); diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index 77328c5fd..360ed668b 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -18,7 +18,8 @@ interface IPASoftInterface { libcamera.SharedFD fdParams, libcamera.IPACameraSensorInfo sensorInfo, libcamera.ControlInfoMap sensorControls) - => (int32 ret, libcamera.ControlInfoMap ipaControls, bool ccmEnabled); + => (int32 ret, libcamera.ControlInfoMap ipaControls, bool ccmEnabled, + libcamera.MetadataListPlan metadataPlan); start() => (int32 ret); stop(); configure(IPAConfigInfo configInfo) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 39d0aebb0..3d0326925 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -82,6 +82,10 @@ int Agc::init(IPAContext &context, const YamlObject &tuningData) context.ctrlMap.merge(controls()); + context.metadataPlan.set(controls::AnalogueGain); + context.metadataPlan.set(controls::ExposureTime); + context.metadataPlan.set(controls::FrameDuration); + return 0; } diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index 55de05d9e..499c7790f 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -197,6 +197,18 @@ Awb::Awb() Awb::~Awb() = default; +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Awb::init(IPAContext &context, [[maybe_unused]] const YamlObject &tuningData) +{ + context.metadataPlan.set(controls::AwbEnable); + context.metadataPlan.set(controls::ColourGains); + context.metadataPlan.set(controls::ColourTemperature); + + return 0; +} + /** * \copydoc libcamera::ipa::Algorithm::configure */ diff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h index dbf69c907..1d47bffa3 100644 --- a/src/ipa/ipu3/algorithms/awb.h +++ b/src/ipa/ipu3/algorithms/awb.h @@ -40,6 +40,7 @@ public: Awb(); ~Awb(); + int init(IPAContext &context, const YamlObject &tuningData) override; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 3b22f7917..f0b8b5dbe 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -54,6 +54,9 @@ namespace libcamera::ipa::ipu3 { * * \var IPAContext::ctrlMap * \brief A ControlInfoMap::Map of controls populated by the algorithms + * + * \var IPAContext::metadataPlan + * \brief A MetadataListPlan populated by the algorithms */ /** diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 97fcf06cd..f4f45ef4d 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -14,6 +14,7 @@ #include #include +#include #include @@ -95,6 +96,8 @@ struct IPAContext { FCQueue frameContexts; ControlInfoMap::Map ctrlMap; + + MetadataListPlan metadataPlan; // TODO: only needed during init(), how could be removed? }; } /* namespace ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 1cae08bf2..8d8811f71 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -143,7 +143,8 @@ public: int init(const IPASettings &settings, const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, - ControlInfoMap *ipaControls) override; + ControlInfoMap *ipaControls, + MetadataListPlan *metadataPlan) override; int start() override; void stop() override; @@ -299,7 +300,8 @@ void IPAIPU3::updateControls(const IPACameraSensorInfo &sensorInfo, int IPAIPU3::init(const IPASettings &settings, const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, - ControlInfoMap *ipaControls) + ControlInfoMap *ipaControls, + MetadataListPlan *metadataPlan) { camHelper_ = CameraSensorHelperFactoryBase::create(settings.sensorModel); if (camHelper_ == nullptr) { @@ -348,6 +350,8 @@ int IPAIPU3::init(const IPASettings &settings, /* Initialize controls. */ updateControls(sensorInfo, sensorControls, ipaControls); + *metadataPlan = std::move(context_.metadataPlan); + return 0; } diff --git a/src/ipa/mali-c55/algorithms/agc.cpp b/src/ipa/mali-c55/algorithms/agc.cpp index 70667db34..86d3a6a26 100644 --- a/src/ipa/mali-c55/algorithms/agc.cpp +++ b/src/ipa/mali-c55/algorithms/agc.cpp @@ -145,6 +145,11 @@ int Agc::init(IPAContext &context, const YamlObject &tuningData) ); context.ctrlMap.merge(controls()); + context.metadataPlan.set(controls::AnalogueGain); + context.metadataPlan.set(controls::ColourTemperature); + context.metadataPlan.set(controls::DigitalGain); + context.metadataPlan.set(controls::ExposureTime); + return 0; } diff --git a/src/ipa/mali-c55/algorithms/awb.cpp b/src/ipa/mali-c55/algorithms/awb.cpp index 050b191b7..0c50892f0 100644 --- a/src/ipa/mali-c55/algorithms/awb.cpp +++ b/src/ipa/mali-c55/algorithms/awb.cpp @@ -29,6 +29,13 @@ Awb::Awb() { } +int Awb::init(IPAContext &context, [[maybe_unused]] const YamlObject &tuningData) +{ + context.metadataPlan.set(controls::ColourGains); + + return 0; +} + int Awb::configure([[maybe_unused]] IPAContext &context, [[maybe_unused]] const IPACameraSensorInfo &configInfo) { diff --git a/src/ipa/mali-c55/algorithms/awb.h b/src/ipa/mali-c55/algorithms/awb.h index 800c2e834..db59e8d79 100644 --- a/src/ipa/mali-c55/algorithms/awb.h +++ b/src/ipa/mali-c55/algorithms/awb.h @@ -18,6 +18,7 @@ public: Awb(); ~Awb() = default; + int init(IPAContext &context, const YamlObject &tuningData) override; int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; void prepare(IPAContext &context, const uint32_t frame, diff --git a/src/ipa/mali-c55/algorithms/blc.cpp b/src/ipa/mali-c55/algorithms/blc.cpp index 2a54c86a9..d36c7ac66 100644 --- a/src/ipa/mali-c55/algorithms/blc.cpp +++ b/src/ipa/mali-c55/algorithms/blc.cpp @@ -51,6 +51,8 @@ int BlackLevelCorrection::init([[maybe_unused]] IPAContext &context, tuningParameters_ = true; + context.metadataPlan.set(controls::SensorBlackLevels); + LOG(MaliC55Blc, Debug) << "Black levels: 00 " << offset00 << ", 01 " << offset01 << ", 10 " << offset10 << ", 11 " << offset11; diff --git a/src/ipa/mali-c55/ipa_context.h b/src/ipa/mali-c55/ipa_context.h index 5e3e2fbde..d76fdcb2b 100644 --- a/src/ipa/mali-c55/ipa_context.h +++ b/src/ipa/mali-c55/ipa_context.h @@ -9,6 +9,7 @@ #include #include +#include #include "libcamera/internal/bayer_format.h" @@ -83,6 +84,8 @@ struct IPAContext { FCQueue frameContexts; ControlInfoMap::Map ctrlMap; + + MetadataListPlan metadataPlan; // TODO: only needed during init(), how could be removed? }; } /* namespace ipa::mali_c55 */ diff --git a/src/ipa/mali-c55/mali-c55.cpp b/src/ipa/mali-c55/mali-c55.cpp index c6941a950..7006ae55c 100644 --- a/src/ipa/mali-c55/mali-c55.cpp +++ b/src/ipa/mali-c55/mali-c55.cpp @@ -46,7 +46,7 @@ public: IPAMaliC55(); int init(const IPASettings &settings, const IPAConfigInfo &ipaConfig, - ControlInfoMap *ipaControls) override; + ControlInfoMap *ipaControls, MetadataListPlan *metadataPlan) override; int start() override; void stop() override; int configure(const IPAConfigInfo &ipaConfig, uint8_t bayerOrder, @@ -96,7 +96,7 @@ std::string IPAMaliC55::logPrefix() const } int IPAMaliC55::init(const IPASettings &settings, const IPAConfigInfo &ipaConfig, - ControlInfoMap *ipaControls) + ControlInfoMap *ipaControls, MetadataListPlan *metadataPlan) { camHelper_ = CameraSensorHelperFactoryBase::create(settings.sensorModel); if (!camHelper_) { @@ -131,6 +131,8 @@ int IPAMaliC55::init(const IPASettings &settings, const IPAConfigInfo &ipaConfig updateControls(ipaConfig.sensorInfo, ipaConfig.sensorControls, ipaControls); + *metadataPlan = std::move(context_.metadataPlan); + return 0; } diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 35440b67e..d081b2ecf 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -160,6 +160,16 @@ int Agc::init(IPAContext &context, const YamlObject &tuningData) context.ctrlMap[&controls::ExposureValue] = ControlInfo(-8.0f, 8.0f, 0.0f); context.ctrlMap.merge(controls()); + context.metadataPlan.set(controls::AeConstraintMode); + context.metadataPlan.set(controls::AeExposureMode); + context.metadataPlan.set(controls::AeMeteringMode); + context.metadataPlan.set(controls::AnalogueGain); + context.metadataPlan.set(controls::AnalogueGainMode); + context.metadataPlan.set(controls::ExposureTime); + context.metadataPlan.set(controls::ExposureTimeMode); + context.metadataPlan.set(controls::ExposureValue); + context.metadataPlan.set(controls::FrameDuration); + return 0; } diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 399fb51be..8de7ea395 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -117,6 +117,10 @@ int Awb::init(IPAContext &context, const YamlObject &tuningData) const auto &src = awbAlgo_->controls(); cmap.insert(src.begin(), src.end()); + context.metadataPlan.set(controls::AwbEnable); + context.metadataPlan.set(controls::ColourGains); + context.metadataPlan.set(controls::ColourTemperature); + return 0; } diff --git a/src/ipa/rkisp1/algorithms/blc.cpp b/src/ipa/rkisp1/algorithms/blc.cpp index 98cb7145e..e3f21eb12 100644 --- a/src/ipa/rkisp1/algorithms/blc.cpp +++ b/src/ipa/rkisp1/algorithms/blc.cpp @@ -103,6 +103,8 @@ int BlackLevelCorrection::init(IPAContext &context, const YamlObject &tuningData << ", green (blue) " << blackLevelGreenB_ << ", blue " << blackLevelBlue_; + context.metadataPlan.set(controls::SensorBlackLevels); + return 0; } diff --git a/src/ipa/rkisp1/algorithms/ccm.cpp b/src/ipa/rkisp1/algorithms/ccm.cpp index de2b6fe77..3115d3eff 100644 --- a/src/ipa/rkisp1/algorithms/ccm.cpp +++ b/src/ipa/rkisp1/algorithms/ccm.cpp @@ -66,6 +66,8 @@ int Ccm::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData offsets_.setData({ { 0, Matrix({ 0, 0, 0 }) } }); } + context.metadataPlan.set(controls::ColourCorrectionMatrix); + return 0; } diff --git a/src/ipa/rkisp1/algorithms/goc.cpp b/src/ipa/rkisp1/algorithms/goc.cpp index a9493678d..2d8040bdc 100644 --- a/src/ipa/rkisp1/algorithms/goc.cpp +++ b/src/ipa/rkisp1/algorithms/goc.cpp @@ -60,6 +60,8 @@ int GammaOutCorrection::init(IPAContext &context, const YamlObject &tuningData) defaultGamma_ = tuningData["gamma"].get(kDefaultGamma); context.ctrlMap[&controls::Gamma] = ControlInfo(0.1f, 10.0f, defaultGamma_); + context.metadataPlan.set(controls::Gamma); + return 0; } diff --git a/src/ipa/rkisp1/algorithms/lux.cpp b/src/ipa/rkisp1/algorithms/lux.cpp index a467767e1..916c6bc67 100644 --- a/src/ipa/rkisp1/algorithms/lux.cpp +++ b/src/ipa/rkisp1/algorithms/lux.cpp @@ -43,7 +43,13 @@ Lux::Lux() */ int Lux::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData) { - return lux_.parseTuningData(tuningData); + int ret = lux_.parseTuningData(tuningData); + if (ret) + return ret; + + context.metadataPlan.set(controls::Lux); + + return 0; } /** diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 7ccc7b501..402491c37 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -202,6 +202,7 @@ struct IPAContext { FCQueue frameContexts; ControlInfoMap::Map ctrlMap; + MetadataListPlan metadataPlan; // TODO: only needed during init(), how could be removed? DebugMetadata debugMetadata; diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index cf66d5553..e250dd473 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -54,7 +54,8 @@ public: int init(const IPASettings &settings, unsigned int hwRevision, const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, - ControlInfoMap *ipaControls) override; + ControlInfoMap *ipaControls, + MetadataListPlan *metadataPlan) override; int start() override; void stop() override; @@ -134,7 +135,8 @@ std::string IPARkISP1::logPrefix() const int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision, const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, - ControlInfoMap *ipaControls) + ControlInfoMap *ipaControls, + MetadataListPlan *metadataPlan) { /* \todo Add support for other revisions */ switch (hwRevision) { @@ -203,6 +205,8 @@ int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision, /* Initialize controls. */ updateControls(sensorInfo, sensorControls, ipaControls); + *metadataPlan = std::move(context_.metadataPlan); + return 0; } diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index a5bdcbb58..84c0cb37c 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -181,6 +181,40 @@ int32_t IpaBase::init(const IPASettings &settings, const InitParams ¶ms, Ini result->controlInfo = ControlInfoMap(std::move(ctrlMap), controls::controls); + // TODO: only set those that can be reported by configured algorithms? + // TODO: move this somewhere else? + result->metadataPlan.set(controls::AeConstraintMode); + result->metadataPlan.set(controls::AeExposureMode); + result->metadataPlan.set(controls::AeMeteringMode); + result->metadataPlan.set(controls::AeState); + result->metadataPlan.set(controls::AfPauseState); + result->metadataPlan.set(controls::AfState); + result->metadataPlan.set(controls::AnalogueGain); + result->metadataPlan.set(controls::AnalogueGainMode); + result->metadataPlan.set(controls::AwbEnable); + result->metadataPlan.set(controls::AwbMode); + result->metadataPlan.set(controls::Brightness); + result->metadataPlan.set(controls::ColourCorrectionMatrix); + result->metadataPlan.set(controls::ColourGains); + result->metadataPlan.set(controls::ColourTemperature); + result->metadataPlan.set(controls::Contrast); + result->metadataPlan.set(controls::DigitalGain); + result->metadataPlan.set(controls::ExposureTime); + result->metadataPlan.set(controls::ExposureTimeMode); + result->metadataPlan.set(controls::ExposureValue); + result->metadataPlan.set(controls::FocusFoM); + result->metadataPlan.set(controls::FrameDuration); + result->metadataPlan.set(controls::FrameDurationLimits); + result->metadataPlan.set(controls::HdrChannel); + result->metadataPlan.set(controls::HdrMode); + result->metadataPlan.set(controls::LensPosition); + result->metadataPlan.set(controls::Lux); + result->metadataPlan.set(controls::Saturation); + result->metadataPlan.set(controls::SensorBlackLevels); + result->metadataPlan.set(controls::SensorTemperature); + result->metadataPlan.set(controls::Sharpness); + result->metadataPlan.set(controls::draft::NoiseReductionMode); + return platformInit(params, result); } diff --git a/src/ipa/rpi/pisp/pisp.cpp b/src/ipa/rpi/pisp/pisp.cpp index bb50a9e05..69585e23e 100644 --- a/src/ipa/rpi/pisp/pisp.cpp +++ b/src/ipa/rpi/pisp/pisp.cpp @@ -267,8 +267,7 @@ private: HdrStatus lastStitchHdrStatus_; }; -int32_t IpaPiSP::platformInit(const InitParams ¶ms, - [[maybe_unused]] InitResult *result) +int32_t IpaPiSP::platformInit(const InitParams ¶ms, InitResult *result) { const std::string &target = controller_.getTarget(); if (target != "pisp") { @@ -301,6 +300,8 @@ int32_t IpaPiSP::platformInit(const InitParams ¶ms, setDefaultConfig(); + result->metadataPlan.set(controls::rpi::PispStatsOutput, sizeof(pisp_statistics)); + return 0; } diff --git a/src/ipa/rpi/vc4/vc4.cpp b/src/ipa/rpi/vc4/vc4.cpp index ba43e4741..8aab066ff 100644 --- a/src/ipa/rpi/vc4/vc4.cpp +++ b/src/ipa/rpi/vc4/vc4.cpp @@ -83,7 +83,7 @@ private: void *lsTable_; }; -int32_t IpaVc4::platformInit([[maybe_unused]] const InitParams ¶ms, [[maybe_unused]] InitResult *result) +int32_t IpaVc4::platformInit([[maybe_unused]] const InitParams ¶ms, InitResult *result) { const std::string &target = controller_.getTarget(); @@ -94,6 +94,8 @@ int32_t IpaVc4::platformInit([[maybe_unused]] const InitParams ¶ms, [[maybe_ return -EINVAL; } + result->metadataPlan.set(controls::rpi::Bcm2835StatsOutput, sizeof(bcm2835_isp_stats)); + return 0; } diff --git a/src/ipa/simple/algorithms/agc.cpp b/src/ipa/simple/algorithms/agc.cpp index c46bb0ebe..2d4de92df 100644 --- a/src/ipa/simple/algorithms/agc.cpp +++ b/src/ipa/simple/algorithms/agc.cpp @@ -41,6 +41,14 @@ Agc::Agc() { } +int Agc::init(IPAContext &context, [[maybe_unused]] const YamlObject &tuningData) +{ + context.metadataPlan.set(controls::AnalogueGain); + context.metadataPlan.set(controls::ExposureTime); + + return 0; +} + void Agc::updateExposure(IPAContext &context, IPAFrameContext &frameContext, double exposureMSV) { /* diff --git a/src/ipa/simple/algorithms/agc.h b/src/ipa/simple/algorithms/agc.h index 112d9f5a1..00e70ea70 100644 --- a/src/ipa/simple/algorithms/agc.h +++ b/src/ipa/simple/algorithms/agc.h @@ -19,6 +19,7 @@ public: Agc(); ~Agc() = default; + int init(IPAContext &context, const YamlObject &tuningData) override; void process(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, const SwIspStats *stats, diff --git a/src/ipa/simple/algorithms/awb.cpp b/src/ipa/simple/algorithms/awb.cpp index cf567e894..e69605586 100644 --- a/src/ipa/simple/algorithms/awb.cpp +++ b/src/ipa/simple/algorithms/awb.cpp @@ -25,6 +25,14 @@ LOG_DEFINE_CATEGORY(IPASoftAwb) namespace ipa::soft::algorithms { +int Awb::init(IPAContext &context, [[maybe_unused]] const YamlObject &tuningData) +{ + context.metadataPlan.set(controls::ColourGains); + context.metadataPlan.set(controls::ColourTemperature); + + return 0; +} + int Awb::configure(IPAContext &context, [[maybe_unused]] const IPAConfigInfo &configInfo) { diff --git a/src/ipa/simple/algorithms/awb.h b/src/ipa/simple/algorithms/awb.h index ad993f39c..b8ae63dcb 100644 --- a/src/ipa/simple/algorithms/awb.h +++ b/src/ipa/simple/algorithms/awb.h @@ -19,6 +19,7 @@ public: Awb() = default; ~Awb() = default; + int init(IPAContext &context, const YamlObject &tuningData) override; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void prepare(IPAContext &context, const uint32_t frame, diff --git a/src/ipa/simple/algorithms/blc.cpp b/src/ipa/simple/algorithms/blc.cpp index 8c1e9ed08..9f5cb87ab 100644 --- a/src/ipa/simple/algorithms/blc.cpp +++ b/src/ipa/simple/algorithms/blc.cpp @@ -34,6 +34,9 @@ int BlackLevel::init([[maybe_unused]] IPAContext &context, */ definedLevel_ = blackLevel.value() >> 8; } + + context.metadataPlan.set(controls::SensorBlackLevels); + return 0; } diff --git a/src/ipa/simple/algorithms/ccm.cpp b/src/ipa/simple/algorithms/ccm.cpp index 0a98406c1..b1758ff39 100644 --- a/src/ipa/simple/algorithms/ccm.cpp +++ b/src/ipa/simple/algorithms/ccm.cpp @@ -39,6 +39,9 @@ int Ccm::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData context.ccmEnabled = true; context.ctrlMap[&controls::Saturation] = ControlInfo(0.0f, 2.0f, 1.0f); + context.metadataPlan.set(controls::ColourCorrectionMatrix); + context.metadataPlan.set(controls::Saturation); + return 0; } diff --git a/src/ipa/simple/algorithms/lut.cpp b/src/ipa/simple/algorithms/lut.cpp index d1d5f7271..5a7ffbd8f 100644 --- a/src/ipa/simple/algorithms/lut.cpp +++ b/src/ipa/simple/algorithms/lut.cpp @@ -28,6 +28,9 @@ int Lut::init(IPAContext &context, [[maybe_unused]] const YamlObject &tuningData) { context.ctrlMap[&controls::Contrast] = ControlInfo(0.0f, 2.0f, 1.0f); + + context.metadataPlan.set(controls::Contrast); + return 0; } diff --git a/src/ipa/simple/ipa_context.h b/src/ipa/simple/ipa_context.h index a471b80ae..f4ecf43db 100644 --- a/src/ipa/simple/ipa_context.h +++ b/src/ipa/simple/ipa_context.h @@ -12,6 +12,7 @@ #include #include +#include #include "libcamera/internal/matrix.h" #include "libcamera/internal/vector.h" @@ -97,6 +98,7 @@ struct IPAContext { IPAActiveState activeState; FCQueue frameContexts; ControlInfoMap::Map ctrlMap; + MetadataListPlan metadataPlan; // TODO: only needed during init(), how could be removed? bool ccmEnabled = false; }; diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index c94c4cd55..db721199d 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -56,7 +56,8 @@ public: const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls, - bool *ccmEnabled) override; + bool *ccmEnabled, + MetadataListPlan *metadataPlan) override; int configure(const IPAConfigInfo &configInfo) override; int start() override; @@ -96,7 +97,8 @@ int IPASoftSimple::init(const IPASettings &settings, const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls, - bool *ccmEnabled) + bool *ccmEnabled, + MetadataListPlan *metadataPlan) { camHelper_ = CameraSensorHelperFactoryBase::create(settings.sensorModel); if (!camHelper_) { @@ -190,6 +192,8 @@ int IPASoftSimple::init(const IPASettings &settings, return -EINVAL; } + *metadataPlan = std::move(context_.metadataPlan); + return 0; } diff --git a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp index f4014b95d..e452fe2f8 100644 --- a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp +++ b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp @@ -166,6 +166,8 @@ int ISICameraData::init() properties_ = sensor_->properties(); + metadataPlan_.set(controls::SensorTimestamp); + return 0; } diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index e31e3879d..e6a7f675a 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -1078,6 +1078,11 @@ int PipelineHandlerIPU3::registerCameras() if (ret) continue; + data->metadataPlan_.set(controls::draft::PipelineDepth); + data->metadataPlan_.set(controls::draft::TestPatternMode); + data->metadataPlan_.set(controls::ScalerCrop); + data->metadataPlan_.set(controls::SensorTimestamp); + const CameraSensorProperties::SensorDelays &delays = cio2->sensor()->sensorDelays(); std::unordered_map params = { { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, @@ -1187,7 +1192,7 @@ int IPU3CameraData::loadIPA() ipa_->configurationFile(sensor->model() + ".yaml", "uncalibrated.yaml"); ret = ipa_->init(IPASettings{ ipaTuningFile, sensor->model() }, - sensorInfo, sensor->controls(), &ipaControls_); + sensorInfo, sensor->controls(), &ipaControls_, &metadataPlan_); if (ret) { LOG(IPU3, Error) << "Failed to initialise the IPU3 IPA"; return ret; diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 4acc091bd..19980f6d2 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -402,7 +402,7 @@ int MaliC55CameraData::loadIPA() ControlInfoMap ipaControls; ret = ipa_->init({ ipaTuningFile, sensor_->model() }, ipaConfig, - &ipaControls); + &ipaControls, &metadataPlan_); if (ret) { LOG(MaliC55, Error) << "Failed to initialise the Mali-C55 IPA"; return ret; diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 8b434a56e..6dce1844d 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -412,7 +412,7 @@ int RkISP1CameraData::loadIPA(unsigned int hwRevision) } ret = ipa_->init({ ipaTuningFile, sensor_->model() }, hwRevision, - sensorInfo, sensor_->controls(), &ipaControls_); + sensorInfo, sensor_->controls(), &ipaControls_, &metadataPlan_); if (ret < 0) { LOG(RkISP1, Error) << "IPA initialization failure"; return ret; @@ -1354,6 +1354,8 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) updateControls(data.get()); + data->metadataPlan_.set(controls::SensorTimestamp); + std::set streams{ &data->mainPathStream_, &data->selfPathStream_, diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 563df198e..f2a9a15cd 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -589,6 +589,9 @@ int PipelineHandlerBase::configure(Camera *camera, CameraConfiguration *config) data->controlInfo_ = ControlInfoMap(std::move(ctrlMap), result.controlInfo.idmap()); + /* Update `rpi::ScalerCrops` size for the corrent configuration. */ + data->metadataPlan_.set(controls::rpi::ScalerCrops, config->size()); + /* Setup the Video Mux/Bridge entities. */ for (auto &[device, link] : data->bridgeDevices_) { /* @@ -835,6 +838,11 @@ int PipelineHandlerBase::registerCamera(std::unique_ptr &camera /* Initialize the camera properties. */ data->properties_ = data->sensor_->properties(); + data->metadataPlan_ = std::move(result.metadataPlan); + data->metadataPlan_.set(controls::SensorTimestamp); + data->metadataPlan_.set(controls::FrameWallClock); + data->metadataPlan_.set(controls::ScalerCrop); + /* * The V4L2_CID_NOTIFY_GAINS control, if present, is used to inform the * sensor of the colour gains. It is defined to be a linear gain where diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index efb07051b..05387ca7c 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -592,7 +592,7 @@ int SimpleCameraData::init() * Instantiate Soft ISP if this is enabled for the given driver and no converter is used. */ if (!converter_ && pipe->swIspEnabled()) { - swIsp_ = std::make_unique(pipe, sensor_.get(), &controlInfo_); + swIsp_ = std::make_unique(pipe, sensor_.get(), &controlInfo_, &metadataPlan_); if (!swIsp_->isValid()) { LOG(SimplePipeline, Warning) << "Failed to create software ISP, disabling software debayering"; diff --git a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp index 4b5816dfd..59fb4bd5c 100644 --- a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp +++ b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp @@ -614,6 +614,8 @@ int UVCCameraData::init(MediaDevice *media) controlInfo_ = ControlInfoMap(std::move(ctrls), controls::controls); + metadataPlan_.set(controls::SensorTimestamp); + /* * Close to allow camera to go into runtime-suspend, video_ will be * re-opened from acquireDevice() and validate(). diff --git a/src/libcamera/pipeline/vimc/vimc.cpp b/src/libcamera/pipeline/vimc/vimc.cpp index 07273bd2b..9d73e320c 100644 --- a/src/libcamera/pipeline/vimc/vimc.cpp +++ b/src/libcamera/pipeline/vimc/vimc.cpp @@ -590,6 +590,8 @@ int VimcCameraData::init() controlInfo_ = ControlInfoMap(std::move(ctrls), controls::controls); + metadataPlan_.set(controls::SensorTimestamp); + /* Initialize the camera properties. */ properties_ = sensor_->properties(); diff --git a/src/libcamera/pipeline/virtual/config_parser.cpp b/src/libcamera/pipeline/virtual/config_parser.cpp index 1d3d9ba87..608bbd469 100644 --- a/src/libcamera/pipeline/virtual/config_parser.cpp +++ b/src/libcamera/pipeline/virtual/config_parser.cpp @@ -65,6 +65,9 @@ ConfigParser::parseConfigFile(File &file, PipelineHandler *pipe) controls[&controls::draft::FaceDetectMode] = ControlInfo(supportedFaceDetectModes); data->controlInfo_ = ControlInfoMap(std::move(controls), controls::controls); + + data->metadataPlan_.set(controls::SensorTimestamp); + configurations.push_back(std::move(data)); } diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 28e2a360e..6da69daf9 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -71,10 +71,11 @@ LOG_DEFINE_CATEGORY(SoftwareIsp) * \param[in] pipe The pipeline handler in use * \param[in] sensor Pointer to the CameraSensor instance owned by the pipeline * \param[out] ipaControls The IPA controls to update + * \param[out] metadataPlan The metadata plan to update * handler */ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, - ControlInfoMap *ipaControls) + ControlInfoMap *ipaControls, MetadataListPlan *metadataPlan) : dmaHeap_(DmaBufAllocator::DmaBufAllocatorFlag::CmaHeap | DmaBufAllocator::DmaBufAllocatorFlag::SystemHeap | DmaBufAllocator::DmaBufAllocatorFlag::UDmaBuf) @@ -146,7 +147,8 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, sensorInfo, sensor->controls(), ipaControls, - &ccmEnabled_); + &ccmEnabled_, + metadataPlan); if (ret) { LOG(SoftwareIsp, Error) << "IPA init failed"; debayer_.reset();