From patchwork Tue Feb 10 08:25:48 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jai Luthra X-Patchwork-Id: 26119 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 63166BD78E for ; Tue, 10 Feb 2026 08:26:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1B8D862190; Tue, 10 Feb 2026 09:26:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="X5E6GSdv"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 666B46218C for ; Tue, 10 Feb 2026 09:26:31 +0100 (CET) Received: from mail.ideasonboard.com (unknown [IPv6:2401:4900:1c30:2edd:807a:f3c0:8d1b:28a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 80F57E47; Tue, 10 Feb 2026 09:25:44 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1770711945; bh=AhgfcfeTf14AaWAjbj0ONq8K31iAB2AR2Dr+UC64D+o=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=X5E6GSdvtcjIo+BwuLB6TPY61TbFNZtbiLzl0MB8qsZmDTxwwpAnwNUSk4hRsXFBj 29sAcWY1LI0rZ2IWBYzddIw/yB+8qV1DFOLPwyOzmbT+hKCZE9dt/H3SoHt9JfQTnQ uluOvWDU4nf9zcOajdLT48coCUgKCVCIDCLcEYcI= From: Jai Luthra Date: Tue, 10 Feb 2026 13:55:48 +0530 Subject: [PATCH 3/3] pipeline/ipa: rpi: vc4: Use extensible parameter buffers for ISP configuration MIME-Version: 1.0 Message-Id: <20260210-pi4-upstream-v1-3-279841c15fba@ideasonboard.com> References: <20260210-pi4-upstream-v1-0-279841c15fba@ideasonboard.com> In-Reply-To: <20260210-pi4-upstream-v1-0-279841c15fba@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck , David Plowman , Laurent Pinchart , Kieran Bingham , Jacopo Mondi , Daniel Scally , Jai Luthra X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=37431; i=jai.luthra@ideasonboard.com; h=from:subject:message-id; bh=AhgfcfeTf14AaWAjbj0ONq8K31iAB2AR2Dr+UC64D+o=; b=owEBbQKS/ZANAwAKAUPekfkkmnFFAcsmYgBpiuuf4iumtmro0Ie8ysOn1Jsothx++/01MDzzb DGkuJBbkXeJAjMEAAEKAB0WIQRN4NgY5dV16NRar8VD3pH5JJpxRQUCaYrrnwAKCRBD3pH5JJpx RfOqD/0aAo27p4vxWIzjhA3QqOEHRWklg0aSQo+1s+kgeYJqJRlDgvuXQ+XyLYQ1NdoywhbtqtH q7FUC+nhgCfwqMiVadHlW8bOZgwUrn0gElYKLTMAx0tr7RRCBumj4BDzybQZ+a8ZndrSLyscezl e3Hr5sXrq+vDUg4ACNEVOA/Z24SO9A1PAbJnHVW+ECDjyqkQTG7A2hcHScfP+pMhKpflIh5wjCC GPnqWRQO/+A309elBOUXYYKRn16Q4cFN7M4yDPhx1b5sPIma1vkm492VzhjHLvDMEjGxxrnQPrX ra5Kud9xGEgEurzdRv22gtFF/qIXauaa8ohEWofcFfAO25tvSLqNQ6a0ErSJ1QT7zX4/NyX219T fw+4HSSNqd/391CSioGfnFDnaZ9XHISwuwS5/VNNZFUoBTp270fB/0wxEbNI76ut6yLLhyD0Z5t h5/YDLGhJg3/IFm6zN+pg4dBFKMWi+xGk3dxzPuMvJqRvRo+SN2enKXv9eFdBKfzT/pgFUSrp/7 6TuX+gpq4DIjPnBnpyG2858mppHJlDe/O0WgphEvKmo/nULLUDy7ln9/T9jplg1Cn/AT2wByAA1 ifIydpqbRa0eDn0vPqDYdFsj6eG+yjcQYnyTZc3X1+Y4SrgwFCebBh0EIwIAoMNdFzI4BEy077D KPcYNS+XgYmeXKg== X-Developer-Key: i=jai.luthra@ideasonboard.com; a=openpgp; fpr=4DE0D818E5D575E8D45AAFC543DE91F9249A7145 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The downstream VC4 (BCM2835) ISP driver used custom V4L2 controls to pass per-frame ISP configuration from userspace, which diverged from upstream ISP drivers (rkisp1, mali-c55, PiSP) which all use parameter buffers. As the driver is being upstreamed now, switch the VC4 IPA and pipeline handler to use the extensible parameter buffer interface instead. Introduce a Bcm2835Params helper class (params.h) that wraps V4L2Params. Each algorithm's applyXYZ function now writes directly into typed parameter blocks of the params buffer using this class. On the pipeline handler side, update the ISP media entity names to match the upstream driver. Register the new bcm2835-isp-params metadata output node as an additional ISP stream. Parameter buffer handling is a bit more complicated compared to PiSP, as we use extensible buffers. So, the buffer is acquired per frame in tryRunPipeline(), and is mapped and passed to the IPA for populating. The IPA signals the completion along with the actual number of bytes used in prepareIspComplete(). Also clean up the now unused ISP control related code, and populate the lens shading DMABUF in the IPA itself instead of patching it in the pipeline handler. Signed-off-by: Jai Luthra --- include/libcamera/ipa/raspberrypi.mojom | 15 +- src/ipa/rpi/common/ipa_base.cpp | 11 +- src/ipa/rpi/common/ipa_base.h | 3 + src/ipa/rpi/vc4/params.h | 77 ++++++++ src/ipa/rpi/vc4/vc4.cpp | 232 +++++++++++-------------- src/libcamera/pipeline/rpi/common/rpi_stream.h | 1 + src/libcamera/pipeline/rpi/pisp/pisp.cpp | 6 +- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 108 +++++++----- 8 files changed, 260 insertions(+), 193 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index 12b083e9d04b14a215382ccb7e9d39cd38a8b1bc..c9ef2952773ee94523a1d9aa1a73ef4bb5539f9f 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -32,12 +32,12 @@ struct BufferIds { uint32 bayer; uint32 embedded; uint32 stats; + uint32 params; }; struct ConfigParams { uint32 transform; libcamera.ControlInfoMap sensorControls; - libcamera.ControlInfoMap ispControls; libcamera.ControlInfoMap lensControls; /* VC4 specific */ libcamera.SharedFD lsTableHandle; @@ -226,7 +226,7 @@ interface IPARPiEventInterface { * processing the frame. The embedded data buffer may be recycled after * this event. */ - prepareIspComplete(BufferIds buffers, bool stitchSwapBuffers); + prepareIspComplete(BufferIds buffers, bool stitchSwapBuffers, uint32 paramsBytesUsed); /** * \fn processStatsComplete() @@ -251,17 +251,6 @@ interface IPARPiEventInterface { */ metadataReady(libcamera.ControlList metadata); - /** - * \fn setIspControls() - * \brief Signal ISP controls to be applied. - * \param[in] controls List of controls to be applied. - * - * This asynchronous event is signalled to the pipeline handler during - * the \a prepareISP signal after all algorithms have been run and the - * IPA requires ISP controls to be applied for the frame. - */ - setIspControls(libcamera.ControlList controls); - /** * \fn setDelayedControls() * \brief Signal Sensor controls to be applied. diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index 7072b38c765ed237d8a484940ff861c33e507e36..217a9d3e8aed42c8695d638648e90d0359842412 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -427,6 +427,7 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) unsigned int ipaContext = params.ipaContext % rpiMetadata_.size(); RPiController::Metadata &rpiMetadata = rpiMetadata_[ipaContext]; Span embeddedBuffer; + Span paramsBuffer; rpiMetadata.clear(); fillDeviceStatus(params.sensorControls, ipaContext); @@ -441,6 +442,13 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) embeddedBuffer = it->second.planes()[0]; } + if (params.buffers.params) { + auto it = buffers_.find(params.buffers.params); + ASSERT(it != buffers_.end()); + paramsBuffer = it->second.planes()[0]; + platformParamsBufferInit(paramsBuffer); + } + /* * AGC wants to know the algorithm status from the time it actioned the * sensor exposure/gain changes. So fetch it from the metadata list @@ -506,7 +514,8 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) reportMetadata(ipaContext); /* Ready to push the input buffer into the ISP. */ - prepareIspComplete.emit(params.buffers, stitchSwapBuffers_); + prepareIspComplete.emit(params.buffers, stitchSwapBuffers_, + platformParamsBytesUsed()); } void IpaBase::processStats(const ProcessParams ¶ms) diff --git a/src/ipa/rpi/common/ipa_base.h b/src/ipa/rpi/common/ipa_base.h index 5348f2ea42820308badf34d29c99fa6cf78aaf28..3704b718d885a6c4f43c0aa144f927dfeaacb0a8 100644 --- a/src/ipa/rpi/common/ipa_base.h +++ b/src/ipa/rpi/common/ipa_base.h @@ -79,6 +79,8 @@ protected: /* Whether the stitch block (if available) needs to swap buffers. */ bool stitchSwapBuffers_; + virtual size_t platformParamsBytesUsed() const { return 0; } + private: /* Number of metadata objects available in the context list. */ static constexpr unsigned int numMetadataContexts = 16; @@ -87,6 +89,7 @@ private: virtual int32_t platformStart(const ControlList &controls, StartResult *result) = 0; virtual int32_t platformConfigure(const ConfigParams ¶ms, ConfigResult *result) = 0; + virtual void platformParamsBufferInit([[maybe_unused]] Span paramsBuffer) {} virtual void platformPrepareIsp(const PrepareParams ¶ms, RPiController::Metadata &rpiMetadata) = 0; virtual void platformPrepareAgc(RPiController::Metadata &rpiMetadata) = 0; diff --git a/src/ipa/rpi/vc4/params.h b/src/ipa/rpi/vc4/params.h new file mode 100644 index 0000000000000000000000000000000000000000..db6307944e78d8183637782a093ba23b5ff1e0e3 --- /dev/null +++ b/src/ipa/rpi/vc4/params.h @@ -0,0 +1,77 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * Raspberry Pi VC4/BCM2835 ISP Parameters + */ + +#pragma once + +#include + +#include + +namespace libcamera { + +namespace ipa::RPi { + +enum class BlockType { + BlackLevel, + Geq, + Gamma, + Denoise, + Sharpen, + Dpc, + Cdn, + CcMatrix, + LensShading, + AwbGains, + DGain, +}; + +namespace details { + +template +struct block_type { +}; + +#define BCM2835_DEFINE_BLOCK_TYPE(id, structName, blockId) \ + template<> \ + struct block_type { \ + using type = struct bcm2835_isp_params_##structName; \ + static constexpr bcm2835_isp_param_block_type blockType = \ + BCM2835_ISP_PARAM_BLOCK_##blockId; \ + }; + +BCM2835_DEFINE_BLOCK_TYPE(BlackLevel, black_level, BLACK_LEVEL) +BCM2835_DEFINE_BLOCK_TYPE(Geq, geq, GEQ) +BCM2835_DEFINE_BLOCK_TYPE(Gamma, gamma, GAMMA) +BCM2835_DEFINE_BLOCK_TYPE(Denoise, denoise, DENOISE) +BCM2835_DEFINE_BLOCK_TYPE(Sharpen, sharpen, SHARPEN) +BCM2835_DEFINE_BLOCK_TYPE(Dpc, dpc, DPC) +BCM2835_DEFINE_BLOCK_TYPE(Cdn, cdn, CDN) +BCM2835_DEFINE_BLOCK_TYPE(CcMatrix, cc_matrix, CC_MATRIX) +BCM2835_DEFINE_BLOCK_TYPE(LensShading, lens_shading, LENS_SHADING) +BCM2835_DEFINE_BLOCK_TYPE(AwbGains, awb_gains, AWB_GAINS) +BCM2835_DEFINE_BLOCK_TYPE(DGain, digital_gain, DIGITAL_GAIN) + +struct params_traits { + using id_type = BlockType; + template + using id_to_details = block_type; +}; + +} /* namespace details */ + +class Bcm2835Params : public V4L2Params +{ +public: + Bcm2835Params(Span data) : V4L2Params(data, + BCM2835_ISP_PARAM_BUFFER_V1) + { + } +}; + +} /* namespace ipa::RPi */ + +} /* namespace libcamera */ diff --git a/src/ipa/rpi/vc4/vc4.cpp b/src/ipa/rpi/vc4/vc4.cpp index 2b205b2861bdb5bf81abd61fc538e9d39215293e..b35ece7185136f720e4b8a3afefc1c78f92d42b9 100644 --- a/src/ipa/rpi/vc4/vc4.cpp +++ b/src/ipa/rpi/vc4/vc4.cpp @@ -30,6 +30,7 @@ #include "controller/lux_status.h" #include "controller/noise_status.h" #include "controller/sharpen_status.h" +#include "params.h" namespace libcamera { @@ -59,34 +60,39 @@ private: int32_t platformStart(const ControlList &controls, StartResult *result) override; int32_t platformConfigure(const ConfigParams ¶ms, ConfigResult *result) override; - void platformPrepareIsp(const PrepareParams ¶ms, RPiController::Metadata &rpiMetadata) override; - void platformPrepareAgc([[maybe_unused]] RPiController::Metadata &rpiMetadata) override; + void platformParamsBufferInit(Span paramsBuffer) override; + void platformPrepareIsp([[maybe_unused]] const PrepareParams ¶ms, + RPiController::Metadata &rpiMetadata) override; + void platformPrepareAgc(RPiController::Metadata &rpiMetadata) override; RPiController::StatisticsPtr platformProcessStats(Span mem) override; void handleControls(const ControlList &controls) override; - bool validateIspControls(); - - void applyAWB(const struct AwbStatus *awbStatus, ControlList &ctrls); - void applyDG(double digitalGain, const struct AwbStatus *awbStatus, ControlList &ctrls); - void applyCCM(const struct CcmStatus *ccmStatus, ControlList &ctrls); - void applyBlackLevel(const struct BlackLevelStatus *blackLevelStatus, ControlList &ctrls); - void applyGamma(const struct ContrastStatus *contrastStatus, ControlList &ctrls); - void applyGEQ(const struct GeqStatus *geqStatus, ControlList &ctrls); - void applyDenoise(const struct DenoiseStatus *denoiseStatus, ControlList &ctrls); - void applySharpen(const struct SharpenStatus *sharpenStatus, ControlList &ctrls); - void applyDPC(const struct DpcStatus *dpcStatus, ControlList &ctrls); - void applyLS(const struct AlscStatus *lsStatus, ControlList &ctrls); + + size_t platformParamsBytesUsed() const override + { + return (ispParams_.has_value()) ? ispParams_->bytesused() : 0; + } + + void applyAWB(const struct AwbStatus *awbStatus, Bcm2835Params ¶ms); + void applyDG(double digitalGain, const struct AwbStatus *awbStatus, Bcm2835Params ¶ms); + void applyCCM(const struct CcmStatus *ccmStatus, Bcm2835Params ¶ms); + void applyBlackLevel(const struct BlackLevelStatus *blackLevelStatus, Bcm2835Params ¶ms); + void applyGamma(const struct ContrastStatus *contrastStatus, Bcm2835Params ¶ms); + void applyGEQ(const struct GeqStatus *geqStatus, Bcm2835Params ¶ms); + void applyDenoise(const struct DenoiseStatus *denoiseStatus, Bcm2835Params ¶ms); + void applySharpen(const struct SharpenStatus *sharpenStatus, Bcm2835Params ¶ms); + void applyDPC(const struct DpcStatus *dpcStatus, Bcm2835Params ¶ms); + void applyLS(const struct AlscStatus *lsStatus, Bcm2835Params ¶ms); void applyAF(const struct AfStatus *afStatus, ControlList &lensCtrls); void resampleTable(uint16_t dest[], const std::vector &src, int destW, int destH); - /* VC4 ISP controls. */ - ControlInfoMap ispCtrls_; - ControlList ctrls_; - /* LS table allocation passed in from the pipeline handler. */ SharedFD lsTableHandle_; void *lsTable_; + /* Params buffer for the current frame. */ + std::optional ispParams_; + /* Remember the most recent AWB values. */ AwbStatus lastAwbStatus_; }; @@ -113,13 +119,6 @@ int32_t IpaVc4::platformStart([[maybe_unused]] const ControlList &controls, int32_t IpaVc4::platformConfigure(const ConfigParams ¶ms, [[maybe_unused]] ConfigResult *result) { - ispCtrls_ = params.ispControls; - ctrls_ = ControlList(ispCtrls_); - if (!validateIspControls()) { - LOG(IPARPI, Error) << "ISP control validation failed."; - return -1; - } - /* Store the lens shading table pointer and handle if available. */ if (params.lsTableHandle.isValid()) { /* Remove any previous table, if there was one. */ @@ -144,51 +143,55 @@ int32_t IpaVc4::platformConfigure(const ConfigParams ¶ms, [[maybe_unused]] C return 0; } +void IpaVc4::platformParamsBufferInit(Span paramsBuffer) +{ + /* Initialize the extensible parameter buffer */ + ispParams_.emplace(paramsBuffer); +} + void IpaVc4::platformPrepareIsp([[maybe_unused]] const PrepareParams ¶ms, RPiController::Metadata &rpiMetadata) { - ControlList &ctrls = ctrls_; - /* Lock the metadata buffer to avoid constant locks/unlocks. */ std::unique_lock lock(rpiMetadata); AwbStatus *awbStatus = rpiMetadata.getLocked("awb.status"); if (awbStatus) { - applyAWB(awbStatus, ctrls); + applyAWB(awbStatus, *ispParams_); lastAwbStatus_ = *awbStatus; } CcmStatus *ccmStatus = rpiMetadata.getLocked("ccm.status"); if (ccmStatus) - applyCCM(ccmStatus, ctrls); + applyCCM(ccmStatus, *ispParams_); AlscStatus *lsStatus = rpiMetadata.getLocked("alsc.status"); if (lsStatus) - applyLS(lsStatus, ctrls); + applyLS(lsStatus, *ispParams_); ContrastStatus *contrastStatus = rpiMetadata.getLocked("contrast.status"); if (contrastStatus) - applyGamma(contrastStatus, ctrls); + applyGamma(contrastStatus, *ispParams_); BlackLevelStatus *blackLevelStatus = rpiMetadata.getLocked("black_level.status"); if (blackLevelStatus) - applyBlackLevel(blackLevelStatus, ctrls); + applyBlackLevel(blackLevelStatus, *ispParams_); GeqStatus *geqStatus = rpiMetadata.getLocked("geq.status"); if (geqStatus) - applyGEQ(geqStatus, ctrls); + applyGEQ(geqStatus, *ispParams_); DenoiseStatus *denoiseStatus = rpiMetadata.getLocked("denoise.status"); if (denoiseStatus) - applyDenoise(denoiseStatus, ctrls); + applyDenoise(denoiseStatus, *ispParams_); SharpenStatus *sharpenStatus = rpiMetadata.getLocked("sharpen.status"); if (sharpenStatus) - applySharpen(sharpenStatus, ctrls); + applySharpen(sharpenStatus, *ispParams_); DpcStatus *dpcStatus = rpiMetadata.getLocked("dpc.status"); if (dpcStatus) - applyDPC(dpcStatus, ctrls); + applyDPC(dpcStatus, *ispParams_); const AfStatus *afStatus = rpiMetadata.getLocked("af.status"); if (afStatus) { @@ -205,10 +208,7 @@ void IpaVc4::platformPrepareAgc(RPiController::Metadata &rpiMetadata) double digitalGain = delayedAgcStatus ? delayedAgcStatus->digitalGain : agcStatus_.digitalGain; AwbStatus *awbStatus = rpiMetadata.getLocked("awb.status"); - applyDG(digitalGain, awbStatus, ctrls_); - - setIspControls.emit(ctrls_); - ctrls_ = ControlList(ispCtrls_); + applyDG(digitalGain, awbStatus, *ispParams_); } RPiController::StatisticsPtr IpaVc4::platformProcessStats(Span mem) @@ -325,48 +325,26 @@ void IpaVc4::handleControls(const ControlList &controls) } } -bool IpaVc4::validateIspControls() +void IpaVc4::applyAWB(const struct AwbStatus *awbStatus, Bcm2835Params ¶ms) { - static const uint32_t ctrls[] = { - V4L2_CID_RED_BALANCE, - V4L2_CID_BLUE_BALANCE, - V4L2_CID_DIGITAL_GAIN, - V4L2_CID_USER_BCM2835_ISP_CC_MATRIX, - V4L2_CID_USER_BCM2835_ISP_GAMMA, - V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL, - V4L2_CID_USER_BCM2835_ISP_GEQ, - V4L2_CID_USER_BCM2835_ISP_DENOISE, - V4L2_CID_USER_BCM2835_ISP_SHARPEN, - V4L2_CID_USER_BCM2835_ISP_DPC, - V4L2_CID_USER_BCM2835_ISP_LENS_SHADING, - V4L2_CID_USER_BCM2835_ISP_CDN, - }; - - for (auto c : ctrls) { - if (ispCtrls_.find(c) == ispCtrls_.end()) { - LOG(IPARPI, Error) << "Unable to find ISP control " - << utils::hex(c); - return false; - } - } - - return true; -} + auto block = params.block(); -void IpaVc4::applyAWB(const struct AwbStatus *awbStatus, ControlList &ctrls) -{ LOG(IPARPI, Debug) << "Applying WB R: " << awbStatus->gainR << " B: " << awbStatus->gainB; - ctrls.set(V4L2_CID_RED_BALANCE, - static_cast(awbStatus->gainR * 1000)); - ctrls.set(V4L2_CID_BLUE_BALANCE, - static_cast(awbStatus->gainB * 1000)); + block->awb_gains.r_gain.num = static_cast(awbStatus->gainR * 1000); + block->awb_gains.r_gain.den = 1000; + block->awb_gains.b_gain.num = static_cast(awbStatus->gainB * 1000); + block->awb_gains.b_gain.den = 1000; + + block.setEnabled(true); } void IpaVc4::applyDG(double digitalGain, - const struct AwbStatus *awbStatus, ControlList &ctrls) + const struct AwbStatus *awbStatus, Bcm2835Params ¶ms) { + auto block = params.block(); + if (awbStatus) { /* * We must apply sufficient extra digital gain to stop any of the channel gains being @@ -380,13 +358,16 @@ void IpaVc4::applyDG(double digitalGain, digitalGain *= extraGain; } - ctrls.set(V4L2_CID_DIGITAL_GAIN, - static_cast(digitalGain * 1000)); + block->digital_gain.gain.num = static_cast(digitalGain * 1000); + block->digital_gain.gain.den = 1000; + + block.setEnabled(true); } -void IpaVc4::applyCCM(const struct CcmStatus *ccmStatus, ControlList &ctrls) +void IpaVc4::applyCCM(const struct CcmStatus *ccmStatus, Bcm2835Params ¶ms) { - bcm2835_isp_custom_ccm ccm; + auto block = params.block(); + bcm2835_isp_custom_ccm &ccm = block->ccm; for (int i = 0; i < 9; i++) { ccm.ccm.ccm[i / 3][i % 3].den = 1000; @@ -395,30 +376,28 @@ void IpaVc4::applyCCM(const struct CcmStatus *ccmStatus, ControlList &ctrls) ccm.enabled = 1; ccm.ccm.offsets[0] = ccm.ccm.offsets[1] = ccm.ccm.offsets[2] = 0; - - ControlValue c(Span{ reinterpret_cast(&ccm), - sizeof(ccm) }); - ctrls.set(V4L2_CID_USER_BCM2835_ISP_CC_MATRIX, c); + block.setEnabled(true); } -void IpaVc4::applyBlackLevel(const struct BlackLevelStatus *blackLevelStatus, ControlList &ctrls) +void IpaVc4::applyBlackLevel(const struct BlackLevelStatus *blackLevelStatus, + Bcm2835Params ¶ms) { - bcm2835_isp_black_level blackLevel; + auto block = params.block(); + bcm2835_isp_black_level &blackLevel = block->black_level; blackLevel.enabled = 1; blackLevel.black_level_r = blackLevelStatus->blackLevelR; blackLevel.black_level_g = blackLevelStatus->blackLevelG; blackLevel.black_level_b = blackLevelStatus->blackLevelB; - - ControlValue c(Span{ reinterpret_cast(&blackLevel), - sizeof(blackLevel) }); - ctrls.set(V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL, c); + block.setEnabled(true); } -void IpaVc4::applyGamma(const struct ContrastStatus *contrastStatus, ControlList &ctrls) +void IpaVc4::applyGamma(const struct ContrastStatus *contrastStatus, + Bcm2835Params ¶ms) { const unsigned int numGammaPoints = controller_.getHardwareConfig().numGammaPoints; - struct bcm2835_isp_gamma gamma; + auto block = params.block(); + struct bcm2835_isp_gamma &gamma = block->gamma; for (unsigned int i = 0; i < numGammaPoints - 1; i++) { int x = i < 16 ? i * 1024 @@ -431,31 +410,27 @@ void IpaVc4::applyGamma(const struct ContrastStatus *contrastStatus, ControlList gamma.x[numGammaPoints - 1] = 65535; gamma.y[numGammaPoints - 1] = 65535; gamma.enabled = 1; - - ControlValue c(Span{ reinterpret_cast(&gamma), - sizeof(gamma) }); - ctrls.set(V4L2_CID_USER_BCM2835_ISP_GAMMA, c); + block.setEnabled(true); } -void IpaVc4::applyGEQ(const struct GeqStatus *geqStatus, ControlList &ctrls) +void IpaVc4::applyGEQ(const struct GeqStatus *geqStatus, Bcm2835Params ¶ms) { - bcm2835_isp_geq geq; + auto block = params.block(); + bcm2835_isp_geq &geq = block->geq; geq.enabled = 1; geq.offset = geqStatus->offset; geq.slope.den = 1000; geq.slope.num = 1000 * geqStatus->slope; - - ControlValue c(Span{ reinterpret_cast(&geq), - sizeof(geq) }); - ctrls.set(V4L2_CID_USER_BCM2835_ISP_GEQ, c); + block.setEnabled(true); } -void IpaVc4::applyDenoise(const struct DenoiseStatus *denoiseStatus, ControlList &ctrls) +void IpaVc4::applyDenoise(const struct DenoiseStatus *denoiseStatus, Bcm2835Params ¶ms) { + auto blockDenoise = params.block(); using RPiController::DenoiseMode; - bcm2835_isp_denoise denoise; + bcm2835_isp_denoise &denoise = blockDenoise->denoise; DenoiseMode mode = static_cast(denoiseStatus->mode); denoise.enabled = mode != DenoiseMode::Off; @@ -464,9 +439,11 @@ void IpaVc4::applyDenoise(const struct DenoiseStatus *denoiseStatus, ControlList denoise.slope.den = 1000; denoise.strength.num = 1000 * denoiseStatus->strength; denoise.strength.den = 1000; + blockDenoise.setEnabled(denoise.enabled); /* Set the CDN mode to match the SDN operating mode. */ - bcm2835_isp_cdn cdn; + auto blockCdn = params.block(); + bcm2835_isp_cdn &cdn = blockCdn->cdn; switch (mode) { case DenoiseMode::ColourFast: cdn.enabled = 1; @@ -479,19 +456,13 @@ void IpaVc4::applyDenoise(const struct DenoiseStatus *denoiseStatus, ControlList default: cdn.enabled = 0; } - - ControlValue c(Span{ reinterpret_cast(&denoise), - sizeof(denoise) }); - ctrls.set(V4L2_CID_USER_BCM2835_ISP_DENOISE, c); - - c = ControlValue(Span{ reinterpret_cast(&cdn), - sizeof(cdn) }); - ctrls.set(V4L2_CID_USER_BCM2835_ISP_CDN, c); + blockCdn.setEnabled(cdn.enabled); } -void IpaVc4::applySharpen(const struct SharpenStatus *sharpenStatus, ControlList &ctrls) +void IpaVc4::applySharpen(const struct SharpenStatus *sharpenStatus, Bcm2835Params ¶ms) { - bcm2835_isp_sharpen sharpen; + auto block = params.block(); + bcm2835_isp_sharpen &sharpen = block->sharpen; sharpen.enabled = 1; sharpen.threshold.num = 1000 * sharpenStatus->threshold; @@ -500,25 +471,20 @@ void IpaVc4::applySharpen(const struct SharpenStatus *sharpenStatus, ControlList sharpen.strength.den = 1000; sharpen.limit.num = 1000 * sharpenStatus->limit; sharpen.limit.den = 1000; - - ControlValue c(Span{ reinterpret_cast(&sharpen), - sizeof(sharpen) }); - ctrls.set(V4L2_CID_USER_BCM2835_ISP_SHARPEN, c); + block.setEnabled(true); } -void IpaVc4::applyDPC(const struct DpcStatus *dpcStatus, ControlList &ctrls) +void IpaVc4::applyDPC(const struct DpcStatus *dpcStatus, Bcm2835Params ¶ms) { - bcm2835_isp_dpc dpc; + auto block = params.block(); + bcm2835_isp_dpc &dpc = block->dpc; dpc.enabled = 1; dpc.strength = dpcStatus->strength; - - ControlValue c(Span{ reinterpret_cast(&dpc), - sizeof(dpc) }); - ctrls.set(V4L2_CID_USER_BCM2835_ISP_DPC, c); + block.setEnabled(true); } -void IpaVc4::applyLS(const struct AlscStatus *lsStatus, ControlList &ctrls) +void IpaVc4::applyLS(const struct AlscStatus *lsStatus, Bcm2835Params ¶ms) { /* * Program lens shading tables into pipeline. @@ -542,24 +508,24 @@ void IpaVc4::applyLS(const struct AlscStatus *lsStatus, ControlList &ctrls) /* We're going to supply corner sampled tables, 16 bit samples. */ w++, h++; - bcm2835_isp_lens_shading ls = { + if (!lsTableHandle_.isValid() || !lsTable_ || w * h * 4 * sizeof(uint16_t) > MaxLsGridSize) { + LOG(IPARPI, Error) << "Do not have a correctly allocated lens shading table!"; + return; + } + + auto block = params.block(); + block->ls = { .enabled = 1, .grid_cell_size = cellSize, .grid_width = w, .grid_stride = w, .grid_height = h, - /* .dmabuf will be filled in by pipeline handler. */ - .dmabuf = 0, + .dmabuf = lsTableHandle_.get(), .ref_transform = 0, .corner_sampled = 1, .gain_format = GAIN_FORMAT_U4P10 }; - if (!lsTable_ || w * h * 4 * sizeof(uint16_t) > MaxLsGridSize) { - LOG(IPARPI, Error) << "Do not have a correctly allocate lens shading table!"; - return; - } - if (lsStatus) { /* Format will be u4.10 */ uint16_t *grid = static_cast(lsTable_); @@ -570,9 +536,7 @@ void IpaVc4::applyLS(const struct AlscStatus *lsStatus, ControlList &ctrls) resampleTable(grid + 3 * w * h, lsStatus->b, w, h); } - ControlValue c(Span{ reinterpret_cast(&ls), - sizeof(ls) }); - ctrls.set(V4L2_CID_USER_BCM2835_ISP_LENS_SHADING, c); + block.setEnabled(true); } void IpaVc4::applyAF(const struct AfStatus *afStatus, ControlList &lensCtrls) diff --git a/src/libcamera/pipeline/rpi/common/rpi_stream.h b/src/libcamera/pipeline/rpi/common/rpi_stream.h index 300a352a7d39b7ea6908777696d9a22a4520936d..cc18dcc8fa8dd70deb030027be06ea3cd6e2677a 100644 --- a/src/libcamera/pipeline/rpi/common/rpi_stream.h +++ b/src/libcamera/pipeline/rpi/common/rpi_stream.h @@ -30,6 +30,7 @@ enum BufferMask { MaskStats = 0x010000, MaskEmbeddedData = 0x020000, MaskBayerData = 0x040000, + MaskParams = 0x080000, }; struct BufferObject { diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index 7bcba32b9b58b1a8782f956d20248e4828b2ba52..4a5dcab209890313630280d2593e08ceae39ce67 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -754,7 +754,8 @@ public: void beOutputDequeue(FrameBuffer *buffer); void processStatsComplete(const ipa::RPi::BufferIds &buffers); - void prepareIspComplete(const ipa::RPi::BufferIds &buffers, bool stitchSwapBuffers); + void prepareIspComplete(const ipa::RPi::BufferIds &buffers, bool stitchSwapBuffers, + unsigned int paramsBytesUsed); void setCameraTimeout(uint32_t maxFrameLengthMs); /* Array of CFE and ISP device streams and associated buffers/streams. */ @@ -1868,7 +1869,8 @@ void PiSPCameraData::setCameraTimeout(uint32_t maxFrameLengthMs) cfe_[Cfe::Output0].dev()->setDequeueTimeout(timeout); } -void PiSPCameraData::prepareIspComplete(const ipa::RPi::BufferIds &buffers, bool stitchSwapBuffers) +void PiSPCameraData::prepareIspComplete(const ipa::RPi::BufferIds &buffers, bool stitchSwapBuffers, + [[maybe_unused]] unsigned int paramsBytesUsed) { unsigned int embeddedId = buffers.embedded & RPi::MaskID; unsigned int bayerId = buffers.bayer & RPi::MaskID; diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index cd1ec4f486990bf620f44bc6cd3ef6ae90c91327..b403992d1f5d32fd79436d7c5652afd9715ada0b 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -32,7 +32,7 @@ using StreamParams = RPi::RPiCameraConfiguration::StreamParams; namespace { enum class Unicam : unsigned int { Image, Embedded }; -enum class Isp : unsigned int { Input, Output0, Output1, Stats }; +enum class Isp : unsigned int { Input, Output0, Output1, Stats, Params }; static constexpr unsigned int kUnicamSinkPad = 0; static constexpr unsigned int kUnicamSourceImagePad = 1; @@ -84,15 +84,15 @@ public: void ispOutputDequeue(FrameBuffer *buffer); void processStatsComplete(const ipa::RPi::BufferIds &buffers); - void prepareIspComplete(const ipa::RPi::BufferIds &buffers, bool stitchSwapBuffers); - void setIspControls(const ControlList &controls); + void prepareIspComplete(const ipa::RPi::BufferIds &buffers, bool stitchSwapBuffers, + unsigned int paramsBytesUsed); void setCameraTimeout(uint32_t maxFrameLengthMs); std::unique_ptr unicamSubdev_; /* Array of Unicam and ISP device streams and associated buffers/streams. */ RPi::Device unicam_; - RPi::Device isp_; + RPi::Device isp_; /* DMAHEAP allocation helper. */ DmaBufAllocator dmaHeap_; @@ -266,6 +266,13 @@ int PipelineHandlerVc4::allocateBuffers(Camera *camera) std::max(data->config_.minUnicamBuffers, minBuffers - numRawBuffers); + } else if (stream == &data->isp_[Isp::Params]) { + /* + * Parameter buffers are dequeued asyncrhonously by the driver + * as soon as it sends each parameter to VC4. Ideally, 1 buffer + * would be sufficient, but we alot 2 to be safe. + */ + numBuffers = 2; } else if (stream == &data->unicam_[Unicam::Embedded]) { /* * Embedded data buffers are (currently) for internal use, and @@ -302,10 +309,11 @@ int PipelineHandlerVc4::allocateBuffers(Camera *camera) } /* - * Pass the stats and embedded data buffers to the IPA. No other + * Pass the stats, embedded data and params buffers to the IPA. No other * buffers need to be passed. */ mapBuffers(camera, data->isp_[Isp::Stats].getBuffers(), RPi::MaskStats); + mapBuffers(camera, data->isp_[Isp::Params].getBuffers(), RPi::MaskParams); if (data->sensorMetadata_) mapBuffers(camera, data->unicam_[Unicam::Embedded].getBuffers(), RPi::MaskEmbeddedData); @@ -324,13 +332,14 @@ int PipelineHandlerVc4::platformRegister(std::unique_ptr &camer MediaEntity *unicamSubdev = unicam->getEntityByName("unicam"); MediaEntity *unicamImage = unicam->getEntityByName("unicam-image"); - MediaEntity *ispOutput0 = isp->getEntityByName("bcm2835-isp0-output0"); - MediaEntity *ispCapture1 = isp->getEntityByName("bcm2835-isp0-capture1"); - MediaEntity *ispCapture2 = isp->getEntityByName("bcm2835-isp0-capture2"); - MediaEntity *ispCapture3 = isp->getEntityByName("bcm2835-isp0-capture3"); - - if (!unicamSubdev || !unicamImage || !ispOutput0 || !ispCapture1 || - !ispCapture2 || !ispCapture3) + MediaEntity *ispOutput0 = isp->getEntityByName("bcm2835-isp-output0"); + MediaEntity *ispCapture0 = isp->getEntityByName("bcm2835-isp-capture0"); + MediaEntity *ispCapture1 = isp->getEntityByName("bcm2835-isp-capture1"); + MediaEntity *ispCapture2 = isp->getEntityByName("bcm2835-isp-stats2"); + MediaEntity *ispParams = isp->getEntityByName("bcm2835-isp-params"); + + if (!unicamSubdev || !unicamImage || !ispOutput0 || !ispCapture0 || + !ispCapture1 || !ispCapture2 || !ispParams) return -ENOENT; /* Create the unicam subdev and video streams. */ @@ -347,9 +356,12 @@ int PipelineHandlerVc4::platformRegister(std::unique_ptr &camer /* Tag the ISP input stream as an import stream. */ data->isp_[Isp::Input] = RPi::Stream("ISP Input", ispOutput0, StreamFlag::ImportOnly); - data->isp_[Isp::Output0] = RPi::Stream("ISP Output0", ispCapture1); - data->isp_[Isp::Output1] = RPi::Stream("ISP Output1", ispCapture2); - data->isp_[Isp::Stats] = RPi::Stream("ISP Stats", ispCapture3); + data->isp_[Isp::Output0] = RPi::Stream("ISP Output0", ispCapture0); + data->isp_[Isp::Output1] = RPi::Stream("ISP Output1", ispCapture1); + data->isp_[Isp::Stats] = RPi::Stream("ISP Stats", ispCapture2); + /* Tag the ISP params stream as MMAP (for writing into it in the IPA) and recurrent. */ + data->isp_[Isp::Params] = RPi::Stream("ISP Params", ispParams, + StreamFlag::RequiresMmap | StreamFlag::Recurrent); /* Wire up all the buffer connections. */ data->unicam_[Unicam::Image].dev()->bufferReady.connect(data, &Vc4CameraData::unicamBufferDequeue); @@ -357,6 +369,7 @@ int PipelineHandlerVc4::platformRegister(std::unique_ptr &camer data->isp_[Isp::Output0].dev()->bufferReady.connect(data, &Vc4CameraData::ispOutputDequeue); data->isp_[Isp::Output1].dev()->bufferReady.connect(data, &Vc4CameraData::ispOutputDequeue); data->isp_[Isp::Stats].dev()->bufferReady.connect(data, &Vc4CameraData::ispOutputDequeue); + data->isp_[Isp::Params].dev()->bufferReady.connect(data, &Vc4CameraData::ispOutputDequeue); if (data->sensorMetadata_ ^ !!data->unicam_[Unicam::Embedded].dev()) { LOG(RPI, Warning) << "Mismatch between Unicam and CamHelper for embedded data usage!"; @@ -398,7 +411,6 @@ int PipelineHandlerVc4::platformRegister(std::unique_ptr &camer /* Write up all the IPA connections. */ data->ipa_->processStatsComplete.connect(data, &Vc4CameraData::processStatsComplete); data->ipa_->prepareIspComplete.connect(data, &Vc4CameraData::prepareIspComplete); - data->ipa_->setIspControls.connect(data, &Vc4CameraData::setIspControls); data->ipa_->setCameraTimeout.connect(data, &Vc4CameraData::setCameraTimeout); /* @@ -757,6 +769,16 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi return ret; } + /* ISP parameters input format. */ + format = {}; + format.fourcc = V4L2PixelFormat(V4L2_META_FMT_BCM2835_ISP_PARAMS); + ret = isp_[Isp::Params].dev()->setFormat(&format); + if (ret) { + LOG(RPI, Error) << "Failed to set format on ISP params stream: " + << format; + return ret; + } + /* * Configure the Unicam embedded data output format only if the sensor * supports it. @@ -800,8 +822,6 @@ int Vc4CameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfi int Vc4CameraData::platformConfigureIpa(ipa::RPi::ConfigParams ¶ms) { - params.ispControls = isp_[Isp::Input].dev()->controls(); - /* Allocate the lens shading table via dmaHeap and pass to the IPA. */ if (!lsTable_.isValid()) { lsTable_ = SharedFD(dmaHeap_.alloc("ls_grid", ipa::RPi::MaxLsGridSize)); @@ -945,46 +965,41 @@ void Vc4CameraData::processStatsComplete(const ipa::RPi::BufferIds &buffers) } void Vc4CameraData::prepareIspComplete(const ipa::RPi::BufferIds &buffers, - [[maybe_unused]] bool stitchSwapBuffers) + [[maybe_unused]] bool stitchSwapBuffers, + unsigned int paramsBytesUsed) { unsigned int embeddedId = buffers.embedded & RPi::MaskID; - unsigned int bayer = buffers.bayer & RPi::MaskID; + unsigned int bayerId = buffers.bayer & RPi::MaskID; + unsigned int paramsId = buffers.params & RPi::MaskID; FrameBuffer *buffer; if (!isRunning()) return; - buffer = unicam_[Unicam::Image].getBuffers().at(bayer & RPi::MaskID).buffer; - LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << (bayer & RPi::MaskID) + /* Queue params buffer */ + buffer = isp_[Isp::Params].getBuffers().at(paramsId).buffer; + buffer->_d()->metadata().planes()[0].bytesused = paramsBytesUsed; + LOG(RPI, Debug) << "Params re-queue to ISP, buffer id " << paramsId + << ", timestamp: " << buffer->metadata().timestamp + << ", bytes used: " << buffer->_d()->metadata().planes()[0].bytesused; + + isp_[Isp::Params].queueBuffer(buffer); + + /* Queue input buffer */ + buffer = unicam_[Unicam::Image].getBuffers().at(bayerId).buffer; + LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << bayerId << ", timestamp: " << buffer->metadata().timestamp; isp_[Isp::Input].queueBuffer(buffer); if (sensorMetadata_ && embeddedId) { - buffer = unicam_[Unicam::Embedded].getBuffers().at(embeddedId & RPi::MaskID).buffer; + buffer = unicam_[Unicam::Embedded].getBuffers().at(embeddedId).buffer; handleStreamBuffer(buffer, &unicam_[Unicam::Embedded]); } handleState(); } -void Vc4CameraData::setIspControls(const ControlList &controls) -{ - ControlList ctrls = controls; - - if (ctrls.contains(V4L2_CID_USER_BCM2835_ISP_LENS_SHADING)) { - ControlValue &value = - const_cast(ctrls.get(V4L2_CID_USER_BCM2835_ISP_LENS_SHADING)); - Span s = value.data(); - bcm2835_isp_lens_shading *ls = - reinterpret_cast(s.data()); - ls->dmabuf = lsTable_.get(); - } - - isp_[Isp::Input].dev()->setControls(&ctrls); - handleState(); -} - void Vc4CameraData::setCameraTimeout(uint32_t maxFrameLengthMs) { /* @@ -1026,9 +1041,6 @@ void Vc4CameraData::tryRunPipeline() unsigned int bayer = unicam_[Unicam::Image].getBufferId(bayerFrame.buffer); - LOG(RPI, Debug) << "Signalling prepareIsp:" - << " Bayer buffer id: " << bayer; - ipa::RPi::PrepareParams params; params.buffers.bayer = RPi::MaskBayerData | bayer; params.sensorControls = std::move(bayerFrame.controls); @@ -1037,6 +1049,16 @@ void Vc4CameraData::tryRunPipeline() params.delayContext = bayerFrame.delayContext; params.buffers.embedded = 0; + const RPi::BufferObject ¶mBufObj = isp_[Isp::Params].acquireBuffer(); + ASSERT(paramBufObj.mapped); + + unsigned int param = isp_[Isp::Params].getBufferId(paramBufObj.buffer); + params.buffers.params = RPi::MaskParams | param; + + LOG(RPI, Debug) << "Signalling prepareIsp:" + << " Bayer buffer id: " << bayer + << " Param buffer id: " << param; + if (embeddedBuffer) { unsigned int embeddedId = unicam_[Unicam::Embedded].getBufferId(embeddedBuffer);