From patchwork Fri Dec 20 16:26:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22441 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E5ED2C327D for ; Fri, 20 Dec 2024 16:27:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8E1C6684AE; Fri, 20 Dec 2024 17:27:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ENc8XRD9"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D41F3684B0 for ; Fri, 20 Dec 2024 17:27:30 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:4eeb:7fa7:1fd0:c13d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 27D35B2B; Fri, 20 Dec 2024 17:26:51 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734712011; bh=ORIfxUJ3LyKZFWZ3XrcRzMS9DCyA9wbndIgzn5uYX8E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ENc8XRD9JRpAvEKln7SdbEp8K1RemRQcFltoFunf4VStK0qw9TcabEUEzt8TVNVoZ 4VyU75K0ArWozrV2wvFHdqpfLbp5tXx5+iVNN8oxS9o5e/VJan4JazWmkFkEeDNK3v dVtmgez61Tu/dnWOAPz6vpDzOgtUJWzkMzqkINds= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [RFC PATCH 1/7] libcamera: delayed_controls: Add push() function that accepts a sequence number Date: Fri, 20 Dec 2024 17:26:47 +0100 Message-ID: <20241220162724.756494-2-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241220162724.756494-1-stefan.klug@ideasonboard.com> References: <20241220162724.756494-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The push function is assymetric to the get() and applyControls() function in that it doesn't allow to specify a frame number. This leads to the unfortunate situation that it is very difficult to detect if anything goes out of sync. Add a version of the push() function that takes a sequence paramater and warns when the sequence provided differs from the expected sequence. Don't take any further actions for now to see where issues pop up. Signed-off-by: Stefan Klug --- include/libcamera/internal/delayed_controls.h | 1 + src/libcamera/delayed_controls.cpp | 33 +++++++++++++++++-- 2 files changed, 32 insertions(+), 2 deletions(-) diff --git a/include/libcamera/internal/delayed_controls.h b/include/libcamera/internal/delayed_controls.h index e8d3014d92cb..f1467fd79814 100644 --- a/include/libcamera/internal/delayed_controls.h +++ b/include/libcamera/internal/delayed_controls.h @@ -30,6 +30,7 @@ public: void reset(); bool push(const ControlList &controls); + bool push(uint32_t sequence, const ControlList &controls); ControlList get(uint32_t sequence); void applyControls(uint32_t sequence); diff --git a/src/libcamera/delayed_controls.cpp b/src/libcamera/delayed_controls.cpp index 94d0a575b01b..6c82d5058ae8 100644 --- a/src/libcamera/delayed_controls.cpp +++ b/src/libcamera/delayed_controls.cpp @@ -144,10 +144,39 @@ void DelayedControls::reset() * Push a set of controls to the control queue. This increases the control queue * depth by one. * + * \note The usage of this function is discouraged as it does not provide a way + * to detect double pushes for the same sequence. Better + * DelayedControls::push(uint32_t sequence, const ControlList &controls) + * instead. + * * \returns true if \a controls are accepted, or false otherwise */ -bool DelayedControls::push(const ControlList &controls) +bool DelayedControls::push(const ControlList &controls) { + LOG(DelayedControls, Debug) << "Deprecated: Push without sequence number"; + return push(queueCount_, controls); +} + +/** + * \brief Push a set of controls on the queue + * \param[in] sequence The sequence number to push for + * \param[in] controls List of controls to add to the device queue + * + * Push a set of controls to the control queue. This increases the control queue + * depth by one. + * + * The \a sequence number is used to do some sanity checks to detect double + * pushes for the same sequence (either due to a bug or a request underrun). + * + * \returns true if \a controls are accepted, or false otherwise + */ +bool DelayedControls::push(uint32_t sequence, const ControlList &controls) { + if(sequence != queueCount_) { + LOG(DelayedControls, Warning) + << "Double push for sequence " << sequence + << " current queue index: " << queueCount_; + } + /* Copy state from previous frame. */ for (auto &ctrl : values_) { Info &info = ctrl.second[queueCount_]; @@ -276,7 +305,7 @@ void DelayedControls::applyControls(uint32_t sequence) while (writeCount_ > queueCount_) { LOG(DelayedControls, Debug) << "Queue is empty, auto queue no-op."; - push({}); + push(queueCount_, {}); } device_->setControls(&out); From patchwork Fri Dec 20 16:26:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22442 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CD124C327D for ; Fri, 20 Dec 2024 16:27:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7C681684BB; Fri, 20 Dec 2024 17:27:36 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="GQB2V1MQ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C96BD684AE for ; Fri, 20 Dec 2024 17:27:32 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:4eeb:7fa7:1fd0:c13d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5CF1BC67; Fri, 20 Dec 2024 17:26:53 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734712013; bh=3XmdSfN/v8OjC8PlxJ9vF+iIzbnfS+VhnzlsHUyWUA4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GQB2V1MQEam5ZHu2RQmsTkA5/za+ZUWs7VSYPLY6CVwLjjPMSg5OfCvT8R74CRabb W3gTH2xr/KsYyYE75Kchmh1Wv+TjWAcRCzogX7pWN51d7vYZwrrimJJtzq16HDlfXG kzEeAy2Rm8oSBSMyTPotfBZ317PlSrQfm28KfQyg= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [RFC PATCH 2/7] libcamera: delayed_controls: Increase log level for dummy pushes Date: Fri, 20 Dec 2024 17:26:48 +0100 Message-ID: <20241220162724.756494-3-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241220162724.756494-1-stefan.klug@ideasonboard.com> References: <20241220162724.756494-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When an automatic no-op is queued a debug message is printed. Change the debug level to warning, because that situation is actually an indication of something going seriously out of sync. Signed-off-by: Stefan Klug --- src/libcamera/delayed_controls.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/delayed_controls.cpp b/src/libcamera/delayed_controls.cpp index 6c82d5058ae8..2f134cbe7fdd 100644 --- a/src/libcamera/delayed_controls.cpp +++ b/src/libcamera/delayed_controls.cpp @@ -303,7 +303,7 @@ void DelayedControls::applyControls(uint32_t sequence) writeCount_ = sequence + 1; while (writeCount_ > queueCount_) { - LOG(DelayedControls, Debug) + LOG(DelayedControls, Warning) << "Queue is empty, auto queue no-op."; push(queueCount_, {}); } From patchwork Fri Dec 20 16:26:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22443 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 204F0C327D for ; Fri, 20 Dec 2024 16:27:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C345A684BC; Fri, 20 Dec 2024 17:27:38 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="v8jR545r"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AA717684AE for ; Fri, 20 Dec 2024 17:27:35 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:4eeb:7fa7:1fd0:c13d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0B85DC67; Fri, 20 Dec 2024 17:26:56 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734712016; bh=J1rNZ/NREyYKPTQu6aiTkxgNCZ3O1+ARk90EFyPe+DI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=v8jR545rSCzBnsAuiMlbRsOibKUfghgmswxNs8nef1AfEJH+Eo3UoBjOHl9tyVX7U yK4qJRcWIQzsDxq7iU+YwUQS9aDbayhli7puslaMsynz8wUVexQFcXK1GNfWTFOxwY bNGIpxgWFhIk7W0dRZRvM1Szl48AlyjZZkROG13Q= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [RFC PATCH 3/7] pipeline: rkisp1: Include frame number when pushing to delayed controls Date: Fri, 20 Dec 2024 17:26:49 +0100 Message-ID: <20241220162724.756494-4-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241220162724.756494-1-stefan.klug@ideasonboard.com> References: <20241220162724.756494-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 35c793da9bba..71a07c6027b9 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -428,10 +428,10 @@ void RkISP1CameraData::paramsComputed(unsigned int frame, unsigned int bytesused selfPath_->queueBuffer(info->selfPathBuffer); } -void RkISP1CameraData::setSensorControls([[maybe_unused]] unsigned int frame, +void RkISP1CameraData::setSensorControls(unsigned int frame, const ControlList &sensorControls) { - delayedCtrls_->push(sensorControls); + delayedCtrls_->push(frame, sensorControls); } void RkISP1CameraData::metadataReady(unsigned int frame, const ControlList &metadata) From patchwork Fri Dec 20 16:26:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22444 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4825CC327D for ; Fri, 20 Dec 2024 16:27:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id F0A47684BD; Fri, 20 Dec 2024 17:27:41 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="U+xsTqY2"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6AC2B684B9 for ; Fri, 20 Dec 2024 17:27:38 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:4eeb:7fa7:1fd0:c13d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id F38E6C67; Fri, 20 Dec 2024 17:26:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734712019; bh=e3ZBBzNVHwcO14bqaUorHWfBxCEyScvaJ1hk6becrN0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=U+xsTqY2B6oNsBsK5StNet3GfdbeDj8jw1feH/9QgzoRYbQvQMs3EP5JIh9I7pgzA oNwqAuc341DL7R1hulkLZM+5rT3m65MrblOEqW5fTQ35qqLp5m+E2/o8VL7llXiJfZ iXAk08ojQr8Vp22VySR5ijLVpTyUVrc7W5xcdv54= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [RFC PATCH 4/7] pipeline: rkisp1: Refactor setControls() Date: Fri, 20 Dec 2024 17:26:50 +0100 Message-ID: <20241220162724.756494-5-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241220162724.756494-1-stefan.klug@ideasonboard.com> References: <20241220162724.756494-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" IPARkISP1::setControls() is called when new sensor controls shall be queued in the pipeline handler. It constructs a list of sensor controls and then emits the setSensorControls signal. To be able to return initial sensor controls from the IPARkISP1::start() function, similar functionality will be needed. Prepare for that by changing the setControls() function to a getSensorControls() and by moving the setSensorControls.emit() out of the function. This patch contains no functional changes. Signed-off-by: Stefan Klug --- src/ipa/rkisp1/rkisp1.cpp | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 2ffdd99b158a..45a539a61fb1 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -76,7 +76,7 @@ private: void updateControls(const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls); - void setControls(unsigned int frame); + ControlList getSensorControls(unsigned int frame); std::map buffers_; std::map mappedBuffers_; @@ -211,7 +211,8 @@ int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision, int IPARkISP1::start() { - setControls(0); + ControlList ctrls = getSensorControls(0); + setSensorControls.emit(0, ctrls); return 0; } @@ -378,7 +379,12 @@ void IPARkISP1::processStats(const uint32_t frame, const uint32_t bufferId, algo->process(context_, frame, frameContext, stats, metadata); } - setControls(frame); + /* + * \todo: Here we should do a lookahead that takes the sensor delays + * into account. + */ + ControlList ctrls = getSensorControls(frame); + setSensorControls.emit(frame, ctrls); context_.debugMetadata.moveEntries(metadata); metadataReady.emit(frame, metadata); @@ -443,13 +449,8 @@ void IPARkISP1::updateControls(const IPACameraSensorInfo &sensorInfo, *ipaControls = ControlInfoMap(std::move(ctrlMap), controls::controls); } -void IPARkISP1::setControls(unsigned int frame) +ControlList IPARkISP1::getSensorControls(unsigned int frame) { - /* - * \todo The frame number is most likely wrong here, we need to take - * internal sensor delays and other timing parameters into account. - */ - IPAFrameContext &frameContext = context_.frameContexts.get(frame); uint32_t exposure = frameContext.agc.exposure; uint32_t gain = context_.camHelper->gainCode(frameContext.agc.gain); @@ -457,8 +458,7 @@ void IPARkISP1::setControls(unsigned int frame) ControlList ctrls(sensorControls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure)); ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain)); - - setSensorControls.emit(frame, ctrls); + return ctrls; } } /* namespace ipa::rkisp1 */ From patchwork Fri Dec 20 16:26:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22445 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5839EC327D for ; Fri, 20 Dec 2024 16:27:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0E708684BD; Fri, 20 Dec 2024 17:27:44 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="AyZkZAXu"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2AAAC684B0 for ; Fri, 20 Dec 2024 17:27:41 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:4eeb:7fa7:1fd0:c13d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A87541063; Fri, 20 Dec 2024 17:27:01 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734712021; bh=6sbduZVT9S/OYs2edf3mGvXi1tBwSRHSZbb0hmQS86Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AyZkZAXuazR3ihFyIQECyGO3wWJuxIk8es8Lp1fXMHjdX23DMsNYIAmFeegio0RCO JMRPK3m9Rwvf1A4+2Z1orlDZyqhJz0pc4qloA1QaK2HnUMMvTmGS5jxlRsCOjP5U9z Y/x7gMCORSJmPhBTfWSTwK9YRRJWdPkC3cds0Bu4= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [RFC PATCH 5/7] pipeline: rkisp1: Apply initial controls Date: Fri, 20 Dec 2024 17:26:51 +0100 Message-ID: <20241220162724.756494-6-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241220162724.756494-1-stefan.klug@ideasonboard.com> References: <20241220162724.756494-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Controls passed to Camera::start() are not handled at the moment. Implement that by issuing a dummy queueRequest() to initialize the frame context and then returning the controls synchronously to the caller. In the pipeline the controls get passed to the sensor before the call to stremOn() to ensure the controls are applied right away. Passing the controls to the pipeline using the setSensorControls signal is not appropriate because it is asynchronous and would reach the sensor too late (initial controls need to be applied before the sensor starts and before delayed controls initializes it's start condition.) Signed-off-by: Stefan Klug --- include/libcamera/ipa/rkisp1.mojom | 7 ++++++- src/ipa/rkisp1/rkisp1.cpp | 15 ++++++++++----- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 11 +++++++++-- 3 files changed, 25 insertions(+), 8 deletions(-) diff --git a/include/libcamera/ipa/rkisp1.mojom b/include/libcamera/ipa/rkisp1.mojom index 043ad27ea199..86eae55cd530 100644 --- a/include/libcamera/ipa/rkisp1.mojom +++ b/include/libcamera/ipa/rkisp1.mojom @@ -14,13 +14,18 @@ struct IPAConfigInfo { uint32 paramFormat; }; +struct StartResult { + libcamera.ControlList controls; + int32 code; +}; + interface IPARkISP1Interface { init(libcamera.IPASettings settings, uint32 hwRevision, libcamera.IPACameraSensorInfo sensorInfo, libcamera.ControlInfoMap sensorControls) => (int32 ret, libcamera.ControlInfoMap ipaControls); - start() => (int32 ret); + start(libcamera.ControlList controls) => (StartResult result); stop(); configure(IPAConfigInfo configInfo, diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 45a539a61fb1..799fe0846635 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -55,7 +55,7 @@ public: const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls) override; - int start() override; + void start(const ControlList &controls, StartResult *result) override; void stop() override; int configure(const IPAConfigInfo &ipaConfig, @@ -209,12 +209,17 @@ int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision, return 0; } -int IPARkISP1::start() +void IPARkISP1::start(const ControlList &controls, StartResult *result) { - ControlList ctrls = getSensorControls(0); - setSensorControls.emit(0, ctrls); + /* + * \todo: This feels a bit counter intuitive, as the queueRequest() for frame + * 0 will be called again when the actual request for frame 0 get's + * queued. + */ + queueRequest(0, controls); - return 0; + result->controls = getSensorControls(0); + result->code = 0; } void IPARkISP1::stop() diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 71a07c6027b9..ee6ed1b9c84a 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1086,13 +1086,20 @@ int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]] const ControlL return ret; actions += [&]() { freeBuffers(camera); }; - ret = data->ipa_->start(); - if (ret) { + ControlList ctrls; + if (!!controls) + ctrls = *controls; + + ipa::rkisp1::StartResult res; + data->ipa_->start(ctrls, &res); + if (res.code) { LOG(RkISP1, Error) << "Failed to start IPA " << camera->id(); return ret; } actions += [&]() { data->ipa_->stop(); }; + data->sensor_->setControls(&res.controls); + data->delayedCtrls_->reset(); data->frame_ = 0; From patchwork Fri Dec 20 16:26:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22446 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C4A02C32A3 for ; Fri, 20 Dec 2024 16:27:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 53BD1684C4; Fri, 20 Dec 2024 17:27:49 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cLGlxOan"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E878D684B0 for ; Fri, 20 Dec 2024 17:27:43 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:4eeb:7fa7:1fd0:c13d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 797F71063; Fri, 20 Dec 2024 17:27:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734712024; bh=DOyT3FbCJSGL13W/QCnUpBIAAX2xiv0cAjDkFggXV/g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cLGlxOanUQFnR2GOnMY2LpZANoDbU+SyL+VXMbeBXH1qKzmcoI8ntSUWtoxPaabIl PGIpfDkZF1xWkd3dGctGVpNQEr+22r+HIPZdcDdh6kXE68smiXAzcaIaoqTjEDKuJU 1LTBza4Lpn2RjdTXrOEOyvbC3ztu8PplKWz+Xi6k= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [RFC PATCH 6/7] ipa: rkisp1: Add a lookahead of one frame when sending sensor controls Date: Fri, 20 Dec 2024 17:26:52 +0100 Message-ID: <20241220162724.756494-7-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241220162724.756494-1-stefan.klug@ideasonboard.com> References: <20241220162724.756494-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the delayed controls instance of the pipeline is reset, it initializes the values for frame 0 as being sent out to the sensor (which is correct). The next sequence number that can be pushed to delayed controls is therefore number 1. Ensure that the IPA never tries to queue controls for frame 0. Signed-off-by: Stefan Klug --- src/ipa/rkisp1/rkisp1.cpp | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 799fe0846635..5d439e0d727b 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -386,10 +386,13 @@ void IPARkISP1::processStats(const uint32_t frame, const uint32_t bufferId, /* * \todo: Here we should do a lookahead that takes the sensor delays - * into account. + * into account. A lookahead of 1 is the smallest lookahead possible to + * ensure we don't try to send the controls for a frame that we already + * received. */ - ControlList ctrls = getSensorControls(frame); - setSensorControls.emit(frame, ctrls); + int lookahead = 1; + ControlList ctrls = getSensorControls(frame + lookahead); + setSensorControls.emit(frame + lookahead, ctrls); context_.debugMetadata.moveEntries(metadata); metadataReady.emit(frame, metadata); From patchwork Fri Dec 20 16:26:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 22447 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A5370C327D for ; Fri, 20 Dec 2024 16:27:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C581C684C6; Fri, 20 Dec 2024 17:27:50 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="E0rmwOPn"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C4573684B0 for ; Fri, 20 Dec 2024 17:27:46 +0100 (CET) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:4eeb:7fa7:1fd0:c13d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 04073C67; Fri, 20 Dec 2024 17:27:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1734712027; bh=rRS2641M0Mw6O+i8N9WGWcaIbAsjseHCybenQx4pQwc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=E0rmwOPny3Kh5NeLiBeXLSdPnqNAmF+J2hrp0Fao0yGHnJCeEyFIcBk+te/0pXa3j p90v8a7zyZSDYBmUJtGpnfpy7/YiHIswf+vmXVDvYCpXf48o6oqSk4+2f6iJXQNpta 6JIz3a5UNI6YD4TCzj06a/iTyme/PkcTACIqKR8U= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [RFC PATCH 7/7] ipa: rkisp1: Set frameContext.agc in queueRequest for auto mode also Date: Fri, 20 Dec 2024 17:26:53 +0100 Message-ID: <20241220162724.756494-8-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241220162724.756494-1-stefan.klug@ideasonboard.com> References: <20241220162724.756494-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" If the agc is in auto mode, exposure time and gain used to be set on the frame context within prepare(). As exposure time and gain are used by getSensorControls(0) from within start() that is too late (prepare() hasn't run yet). Also prepare() is documented as the place to initialize the params buffer, not the frame context. From the pipeline point of view, prepare() gets called immediately after queueRequest(). Therefore we can safely move setting the frame context into queueRequest() to fix the sensor controls for start(). Signed-off-by: Stefan Klug --- src/ipa/rkisp1/algorithms/agc.cpp | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 40e5a8f481b2..46be1413a728 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -243,7 +243,10 @@ void Agc::queueRequest(IPAContext &context, frameContext.agc.autoEnabled = agc.autoEnabled; - if (!frameContext.agc.autoEnabled) { + if (frameContext.agc.autoEnabled) { + frameContext.agc.exposure = context.activeState.agc.automatic.exposure; + frameContext.agc.gain = context.activeState.agc.automatic.gain; + } else { frameContext.agc.exposure = agc.manual.exposure; frameContext.agc.gain = agc.manual.gain; } @@ -283,11 +286,6 @@ void Agc::queueRequest(IPAContext &context, void Agc::prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, RkISP1Params *params) { - if (frameContext.agc.autoEnabled) { - frameContext.agc.exposure = context.activeState.agc.automatic.exposure; - frameContext.agc.gain = context.activeState.agc.automatic.gain; - } - if (frame > 0 && !frameContext.agc.updateMetering) return;