From patchwork Fri Oct 24 08:50:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24767 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 01C82C3259 for ; Fri, 24 Oct 2025 08:51:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C27DB6087C; Fri, 24 Oct 2025 10:51:40 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="vbnAErZl"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 851646085B for ; Fri, 24 Oct 2025 10:51:39 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 0FCD8177F; Fri, 24 Oct 2025 10:49:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295794; bh=6W2BisUo8z5q3pMwUV4ajbCf6goYEOHA6JQaFBACUvw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vbnAErZlXbh+C2wwooP5m5zOrV2jgFfcCaXZB0yJxELGF+BU++ZZ0sqyhAbSZQb3J NUngXf0H/rsJEPXQ3RBN1/shBw+o3MAGxedcUt5+pk9eJF6AwuYndK13WSORyFrSZL /Jm3semrxs3XwCZOjgGTb5w932hD0qtgu42QfNlY= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 01/35] libcamera: delayed_controls: Add push() function that accepts a sequence number Date: Fri, 24 Oct 2025 10:50:25 +0200 Message-ID: <20251024085130.995967-2-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The push function is asymmetric to the get() and applyControls() function in that it doesn't allow to specify a frame number. This leads to the unfortunate situation that it is very difficult to detect if anything goes out of sync. Add a version of the push() function that takes a sequence parameter and warns when the sequence provided differs from the expected sequence. Don't take any further actions for now to see where issues pop up. Signed-off-by: Stefan Klug --- include/libcamera/internal/delayed_controls.h | 1 + src/libcamera/delayed_controls.cpp | 32 ++++++++++++++++++- 2 files changed, 32 insertions(+), 1 deletion(-) diff --git a/include/libcamera/internal/delayed_controls.h b/include/libcamera/internal/delayed_controls.h index b64d8bba7cf7..c650e672d964 100644 --- a/include/libcamera/internal/delayed_controls.h +++ b/include/libcamera/internal/delayed_controls.h @@ -32,6 +32,7 @@ public: void reset(); bool push(const ControlList &controls); + bool push(uint32_t sequence, const ControlList &controls); ControlList get(uint32_t sequence); void applyControls(uint32_t sequence); diff --git a/src/libcamera/delayed_controls.cpp b/src/libcamera/delayed_controls.cpp index 94d0a575b01b..246e7cefa4e5 100644 --- a/src/libcamera/delayed_controls.cpp +++ b/src/libcamera/delayed_controls.cpp @@ -144,10 +144,40 @@ void DelayedControls::reset() * Push a set of controls to the control queue. This increases the control queue * depth by one. * + * \note The usage of this function is discouraged as it does not provide a way + * to detect double pushes for the same sequence. Better + * DelayedControls::push(uint32_t sequence, const ControlList &controls) + * instead. + * * \returns true if \a controls are accepted, or false otherwise */ bool DelayedControls::push(const ControlList &controls) { + LOG(DelayedControls, Debug) << "Deprecated: Push without sequence number"; + return push(queueCount_, controls); +} + +/** + * \brief Push a set of controls on the queue + * \param[in] sequence The sequence number to push for + * \param[in] controls List of controls to add to the device queue + * + * Push a set of controls to the control queue. This increases the control queue + * depth by one. + * + * The \a sequence number is used to do some sanity checks to detect double + * pushes for the same sequence (either due to a bug or a request underrun). + * + * \returns true if \a controls are accepted, or false otherwise + */ +bool DelayedControls::push(uint32_t sequence, const ControlList &controls) +{ + if (sequence < queueCount_) { + LOG(DelayedControls, Warning) + << "Double push for sequence " << sequence + << " current queue index: " << queueCount_; + } + /* Copy state from previous frame. */ for (auto &ctrl : values_) { Info &info = ctrl.second[queueCount_]; @@ -276,7 +306,7 @@ void DelayedControls::applyControls(uint32_t sequence) while (writeCount_ > queueCount_) { LOG(DelayedControls, Debug) << "Queue is empty, auto queue no-op."; - push({}); + push(queueCount_, {}); } device_->setControls(&out); From patchwork Fri Oct 24 08:50:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24768 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D99CDC3259 for ; Fri, 24 Oct 2025 08:51:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 964C760880; Fri, 24 Oct 2025 10:51:43 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ElW24RM9"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F1AD06085B for ; Fri, 24 Oct 2025 10:51:42 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 7E18A1AA6; Fri, 24 Oct 2025 10:49:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295797; bh=nukIpOw0g1pYJyUv4O678RTC9+Rmqr1IOBx5HvTHMpU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ElW24RM9rIXhN334LY0KYIYCm3KsGj+NfbDZXT+vVquQJzTvjX6QTNG66vYog7zpF rF5yFl8cd4APJPPybSnlSKG6YHMjNc4mKpbH6LuDo3zXXvT+Ym+VVnig13IT2+oDEw PFD3vJ85+2S1KAcDcJCHiuMal6/3XkZeOnCxfx14= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 02/35] libcamera: delayed_controls: Handle missed pushes Date: Fri, 24 Oct 2025 10:50:26 +0200 Message-ID: <20251024085130.995967-3-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In unstable systems it can happen, that some sequence numbers are missed in the IPA. Gracefully handle that situation by queuing no-ops. Signed-off-by: Stefan Klug --- src/libcamera/delayed_controls.cpp | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/libcamera/delayed_controls.cpp b/src/libcamera/delayed_controls.cpp index 246e7cefa4e5..e5bdc374fd0e 100644 --- a/src/libcamera/delayed_controls.cpp +++ b/src/libcamera/delayed_controls.cpp @@ -178,6 +178,15 @@ bool DelayedControls::push(uint32_t sequence, const ControlList &controls) << " current queue index: " << queueCount_; } + while (sequence > queueCount_) { + LOG(DelayedControls, Warning) + << "Missed push for sequence " << queueCount_ + << " Auto queue no-op."; + push(queueCount_, {}); + } + + ASSERT(sequence == queueCount_); + /* Copy state from previous frame. */ for (auto &ctrl : values_) { Info &info = ctrl.second[queueCount_]; From patchwork Fri Oct 24 08:50:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24769 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B3869C3259 for ; Fri, 24 Oct 2025 08:51:47 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 79A9E60893; Fri, 24 Oct 2025 10:51:47 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="oB1QWmf1"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 93C316085B for ; Fri, 24 Oct 2025 10:51:45 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 1AAD11AA6; Fri, 24 Oct 2025 10:50:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295800; bh=u0J8WJwBdP4QTq67EP2joJGeWdQ9pKZLM2Mi+xFnKPA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oB1QWmf1OfeVKEDmQgJNGBL1I3ZJlFj13UeDg7zJjbh4rUOPbF/7SHHir5EHtfC4w f2oUDX9bw6RtRgqG7wqQ9MzUB9FLof7z4nuCuJ1WkR7OAH8X+TJkmOJ++O2xxSvtBJ y/AJV5+aRH/A3JGFKu7Hym8TVYZpYo6gF1f1KB9M= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 03/35] libcamera: delayed_controls: Increase log level for dummy pushes Date: Fri, 24 Oct 2025 10:50:27 +0200 Message-ID: <20251024085130.995967-4-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When an automatic no-op is queued a debug message is printed. Change the debug level to warning, because that situation is actually an indication of something going seriously out of sync. Signed-off-by: Stefan Klug --- src/libcamera/delayed_controls.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/delayed_controls.cpp b/src/libcamera/delayed_controls.cpp index e5bdc374fd0e..044c6c7325e7 100644 --- a/src/libcamera/delayed_controls.cpp +++ b/src/libcamera/delayed_controls.cpp @@ -313,7 +313,7 @@ void DelayedControls::applyControls(uint32_t sequence) writeCount_ = sequence + 1; while (writeCount_ > queueCount_) { - LOG(DelayedControls, Debug) + LOG(DelayedControls, Warning) << "Queue is empty, auto queue no-op."; push(queueCount_, {}); } From patchwork Fri Oct 24 08:50:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24770 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5FB5BC3259 for ; Fri, 24 Oct 2025 08:51:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2871B6089B; Fri, 24 Oct 2025 10:51:50 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="WlrBGwyc"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A27CB6086F for ; Fri, 24 Oct 2025 10:51:48 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 2DA331AC5; Fri, 24 Oct 2025 10:50:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295803; bh=0RNLWOI7Kr+6RucCxL2v52hvZHMXiHYlDf0dDuYyE4s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WlrBGwycuwU1OZI9Z4uHDUerLTkb1wgtrBgV6pWOBr1TLoZmtgj1vA18QLsc1kIph QusVn0fTYN2QkStUIoNBBFizzH5tuo6DmXsZCwDeYUVpRfLmAJWVuo/JFKQ131c8uo Nw2+TriF59zEM8yuYPpNcFxXXbnVApe5hqaJMiEg= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 04/35] libcamera: delayed_controls: Queue noop when needed, not before Date: Fri, 24 Oct 2025 10:50:28 +0200 Message-ID: <20251024085130.995967-5-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" A no-op is queued when DelayedControls runs out of controls after a successful applyControls(). But after applyControls() there is still some time left for new controls to be pushed until the next call to applyControls(). To fix that, move the no-op push to the beginning of applyContols(). Signed-off-by: Stefan Klug --- src/libcamera/delayed_controls.cpp | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/src/libcamera/delayed_controls.cpp b/src/libcamera/delayed_controls.cpp index 044c6c7325e7..67213fe87d53 100644 --- a/src/libcamera/delayed_controls.cpp +++ b/src/libcamera/delayed_controls.cpp @@ -272,6 +272,14 @@ void DelayedControls::applyControls(uint32_t sequence) { LOG(DelayedControls, Debug) << "frame " << sequence << " started"; + while (queueCount_ - 1 < sequence) { + LOG(DelayedControls, Warning) + << "Queue is empty, auto queue no-op." << queueCount_; + push(queueCount_, {}); + } + + writeCount_ = sequence; + /* * Create control list peeking ahead in the value queue to ensure * values are set in time to satisfy the sensor delay. @@ -312,12 +320,6 @@ void DelayedControls::applyControls(uint32_t sequence) writeCount_ = sequence + 1; - while (writeCount_ > queueCount_) { - LOG(DelayedControls, Warning) - << "Queue is empty, auto queue no-op."; - push(queueCount_, {}); - } - device_->setControls(&out); } From patchwork Fri Oct 24 08:50:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24771 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F3202C3259 for ; Fri, 24 Oct 2025 08:51:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B495F6088A; Fri, 24 Oct 2025 10:51:52 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BC3cEeMG"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B3F9060885 for ; Fri, 24 Oct 2025 10:51:51 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 4824A1AC5; Fri, 24 Oct 2025 10:50:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295806; bh=iPNsBUwFZtLLRrWnOMhC/aNE3EabV+sH8Kj/EdWwnGI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BC3cEeMGFNXxLGjY0ACTn45dkf1ORK1vYIUqM+kj9BdiScTAASYCyQbm1muvFkylo nFDiIX2kjvtdpqse7yaxp5a3JaO+IhnkXNoP4W6j8YkPuT14XQp580tsSwbunuF5Wr GVR9o+IcgOwGePMelFNcXRF0VB/hm1BA/EjUZ0eg= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 05/35] libcamera: delayed_controls: Add maxDelay() function Date: Fri, 24 Oct 2025 10:50:29 +0200 Message-ID: <20251024085130.995967-6-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a maxDelay() function to be able to query the maximum delay of the sensor. Signed-off-by: Stefan Klug --- --- include/libcamera/internal/delayed_controls.h | 2 ++ src/libcamera/delayed_controls.cpp | 7 +++++++ 2 files changed, 9 insertions(+) diff --git a/include/libcamera/internal/delayed_controls.h b/include/libcamera/internal/delayed_controls.h index c650e672d964..3182ebddd1fa 100644 --- a/include/libcamera/internal/delayed_controls.h +++ b/include/libcamera/internal/delayed_controls.h @@ -35,6 +35,8 @@ public: bool push(uint32_t sequence, const ControlList &controls); ControlList get(uint32_t sequence); + uint32_t maxDelay() const { return maxDelay_; } + void applyControls(uint32_t sequence); private: diff --git a/src/libcamera/delayed_controls.cpp b/src/libcamera/delayed_controls.cpp index 67213fe87d53..35a624525b80 100644 --- a/src/libcamera/delayed_controls.cpp +++ b/src/libcamera/delayed_controls.cpp @@ -259,6 +259,13 @@ ControlList DelayedControls::get(uint32_t sequence) return out; } +/** + * \fn DelayedControls::maxDelay() + * \brief Get the maximum delay of the sensor + * + * \return The maximum delay of the sensor + */ + /** * \brief Inform DelayedControls of the start of a new frame * \param[in] sequence Sequence number of the frame that started From patchwork Fri Oct 24 08:50:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24772 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 78E1AC3259 for ; Fri, 24 Oct 2025 08:51:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 36AC260871; Fri, 24 Oct 2025 10:51:56 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="RU5KBTtf"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2E3E760871 for ; Fri, 24 Oct 2025 10:51:55 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id A40CC1E27; Fri, 24 Oct 2025 10:50:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295809; bh=8IDm2pC+Dl1MapppX8dQPYooTsa5rz7YfwmNziiQdcY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RU5KBTtfgVrmp/Y8Afx5ELqISFw2WDn2nNpy/Q77xIBanVHJhUAMHqLmL2XBVHmbC C8NhOmIXWzckptf6vCh0mYu/vuJuGzU7RxTfldcf12NwOZcjGHfis5LJNw56l4AAXf GNRBIp47gigu3BJXp51Med7xGqdqO3CA1n8jRNew= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 06/35] pipeline: rkisp1: Include frame number when pushing to delayed controls Date: Fri, 24 Oct 2025 10:50:30 +0200 Message-ID: <20251024085130.995967-7-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Pass the frame number when pushing to delayed controls, to ease further development. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index fef6e23e39b7..17250cbd5ae6 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -524,10 +524,10 @@ void RkISP1CameraData::paramsComputed(unsigned int frame, unsigned int bytesused selfPath_->queueBuffer(info->selfPathBuffer); } -void RkISP1CameraData::setSensorControls([[maybe_unused]] unsigned int frame, +void RkISP1CameraData::setSensorControls(unsigned int frame, const ControlList &sensorControls) { - delayedCtrls_->push(sensorControls); + delayedCtrls_->push(frame, sensorControls); } void RkISP1CameraData::metadataReady(unsigned int frame, const ControlList &metadata) From patchwork Fri Oct 24 08:50:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24773 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EF60AC3259 for ; Fri, 24 Oct 2025 08:51:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B0F376088A; Fri, 24 Oct 2025 10:51:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="INlwBde7"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B15A160885 for ; Fri, 24 Oct 2025 10:51:57 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 45C941E27; Fri, 24 Oct 2025 10:50:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295812; bh=+ZZrzTMcv9YU/56qGVrqM49B6OtaxaqtMvBF+pG0kww=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=INlwBde7WR2/RlQpVcCj8trRxjBVIKwZlbCtB0L1vt7PWeXihPaaCnQGNreQfcj0M ytnzc2H63xJ3LAGenyE+naCM3W6tLTdHvW1zMtVDcFXza1rSWA6Zx9kkzW7ha4t6rH EZzCEih5UPqPB8QKuYMkdezybRZ/qApFv1eT7d8Y= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 07/35] libipa: fc_queue: Rename template argument to FC Date: Fri, 24 Oct 2025 10:50:31 +0200 Message-ID: <20251024085130.995967-8-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The FCQueue has a template argument called FrameContext which is easily confused with the class FrameContext defined in the same file. Reduce that confusion by renaming the template argument to FC. Signed-off-by: Stefan Klug --- src/ipa/libipa/fc_queue.h | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/src/ipa/libipa/fc_queue.h b/src/ipa/libipa/fc_queue.h index a1d136521107..1f4f84c27fbc 100644 --- a/src/ipa/libipa/fc_queue.h +++ b/src/ipa/libipa/fc_queue.h @@ -28,7 +28,7 @@ private: bool initialised = false; }; -template +template class FCQueue { public: @@ -39,15 +39,15 @@ public: void clear() { - for (FrameContext &ctx : contexts_) { + for (FC &ctx : contexts_) { ctx.initialised = false; ctx.frame = 0; } } - FrameContext &alloc(const uint32_t frame) + FC &alloc(const uint32_t frame) { - FrameContext &frameContext = contexts_[frame % contexts_.size()]; + FC &frameContext = contexts_[frame % contexts_.size()]; /* * Do not re-initialise if a get() call has already fetched this @@ -69,9 +69,9 @@ public: return frameContext; } - FrameContext &get(uint32_t frame) + FC &get(uint32_t frame) { - FrameContext &frameContext = contexts_[frame % contexts_.size()]; + FC &frameContext = contexts_[frame % contexts_.size()]; /* * If the IPA algorithms try to access a frame context slot which @@ -122,14 +122,14 @@ public: } private: - void init(FrameContext &frameContext, const uint32_t frame) + void init(FC &frameContext, const uint32_t frame) { frameContext = {}; frameContext.frame = frame; frameContext.initialised = true; } - std::vector contexts_; + std::vector contexts_; }; } /* namespace ipa */ From patchwork Fri Oct 24 08:50:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24774 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6E391C3259 for ; Fri, 24 Oct 2025 08:52:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3F5E4608B5; Fri, 24 Oct 2025 10:52:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="qaiHyUwk"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 44FFF6089B for ; Fri, 24 Oct 2025 10:52:00 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id A6FE61E27; Fri, 24 Oct 2025 10:50:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295814; bh=s/k8oJsmDY4Ef6CFQGyygXBrsoR+qmcoYHs9Uyr1+qY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qaiHyUwkcLOVEVvLYMd8xluSxOMaOXZTW4cV2vyA9undNwFQKL7QRfpPRx0QxxJCu c/9Zg5NEAa7XaQ++SkH8X5egyoOyB5jF4F5vMPC1gGJjBmNia16vI0+DU7GbwSUGL2 sRPvW4XdRRtxnDcndjO/UG/iD7uRCodlpamAsoV8= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 08/35] libipa: fc_queue: Add trailing underscore to private members of FrameContext Date: Fri, 24 Oct 2025 10:50:32 +0200 Message-ID: <20251024085130.995967-9-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" It is not immediately obvious that FCQueue accesses private members of the FrameContext class with the help of the friend declaration. This gets even more confusing when such a member is shadowed by a declaration in the actual IPA FrameContext class. Improve that by accessing the FrameContext members via references to that type. Additionally add an underscore to the variable names like we do on other members and which allows us to create a frame() accessor without a name clash in an upcoming commit. Signed-off-by: Stefan Klug --- src/ipa/libipa/fc_queue.cpp | 2 +- src/ipa/libipa/fc_queue.h | 51 ++++++++++++++++++++----------------- 2 files changed, 28 insertions(+), 25 deletions(-) diff --git a/src/ipa/libipa/fc_queue.cpp b/src/ipa/libipa/fc_queue.cpp index 0365e9197748..39222c2ed204 100644 --- a/src/ipa/libipa/fc_queue.cpp +++ b/src/ipa/libipa/fc_queue.cpp @@ -34,7 +34,7 @@ namespace ipa { * update any specific action for this frame, and finally to update the metadata * control lists when the frame is fully completed. * - * \var FrameContext::frame + * \var FrameContext::frame_ * \brief The frame number */ diff --git a/src/ipa/libipa/fc_queue.h b/src/ipa/libipa/fc_queue.h index 1f4f84c27fbc..1128e42f8ca6 100644 --- a/src/ipa/libipa/fc_queue.h +++ b/src/ipa/libipa/fc_queue.h @@ -24,8 +24,8 @@ class FCQueue; struct FrameContext { private: template friend class FCQueue; - uint32_t frame; - bool initialised = false; + uint32_t frame_; + bool initialised_ = false; }; template @@ -40,14 +40,15 @@ public: void clear() { for (FC &ctx : contexts_) { - ctx.initialised = false; - ctx.frame = 0; + ctx.initialised_ = false; + ctx.frame_ = 0; } } FC &alloc(const uint32_t frame) { - FC &frameContext = contexts_[frame % contexts_.size()]; + FC &fc = contexts_[frame % contexts_.size()]; + FrameContext &frameContext = fc; /* * Do not re-initialise if a get() call has already fetched this @@ -60,18 +61,19 @@ public: * time the application has queued a request. Does this deserve * an error condition ? */ - if (frame != 0 && frame <= frameContext.frame) + if (frame != 0 && frame <= frameContext.frame_) LOG(FCQueue, Warning) << "Frame " << frame << " already initialised"; else - init(frameContext, frame); + init(fc, frame); - return frameContext; + return fc; } FC &get(uint32_t frame) { - FC &frameContext = contexts_[frame % contexts_.size()]; + FC &fc = contexts_[frame % contexts_.size()]; + FrameContext &frameContext = fc; /* * If the IPA algorithms try to access a frame context slot which @@ -81,28 +83,28 @@ public: * queueing more requests to the IPA than the frame context * queue size. */ - if (frame < frameContext.frame) + if (frame < frameContext.frame_) LOG(FCQueue, Fatal) << "Frame context for " << frame << " has been overwritten by " - << frameContext.frame; + << frameContext.frame_; - if (frame == 0 && !frameContext.initialised) { + if (frame == 0 && !frameContext.initialised_) { /* * If the IPA calls get() at start() time it will get an * un-intialized FrameContext as the below "frame == - * frameContext.frame" check will return success because - * FrameContexts are zeroed at creation time. + * frameContext.frame_" check will return success + * because FrameContexts are zeroed at creation time. * * Make sure the FrameContext gets initialised if get() * is called before alloc() by the IPA for frame#0. */ - init(frameContext, frame); + init(fc, frame); - return frameContext; + return fc; } - if (frame == frameContext.frame) - return frameContext; + if (frame == frameContext.frame_) + return fc; /* * The frame context has been retrieved before it was @@ -116,17 +118,18 @@ public: LOG(FCQueue, Warning) << "Obtained an uninitialised FrameContext for " << frame; - init(frameContext, frame); + init(fc, frame); - return frameContext; + return fc; } private: - void init(FC &frameContext, const uint32_t frame) + void init(FC &fc, const uint32_t frame) { - frameContext = {}; - frameContext.frame = frame; - frameContext.initialised = true; + fc = {}; + FrameContext &frameContext = fc; + frameContext.frame_ = frame; + frameContext.initialised_ = true; } std::vector contexts_; From patchwork Fri Oct 24 08:50:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24775 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3B79CC3259 for ; Fri, 24 Oct 2025 08:52:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EFB67608B9; Fri, 24 Oct 2025 10:52:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ub5/9RvQ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 111ED60871 for ; Fri, 24 Oct 2025 10:52:03 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 976301E46; Fri, 24 Oct 2025 10:50:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295817; bh=CVtYgOrNanjQAMna1FHGWmAogELBf/83t7UY6wCUy3s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ub5/9RvQCno1g8+SBAjPLTJjHQRkSbW2oPeZsMw4RYIHQMKcGIVBi0B2fzTIP+KLO LK7PB5ldjt18VAjQOJzuo7vJrHXZjwFm+xfZl2pUdvxti4G2Up6w3IkA4ZlOlr63db N8gJud3LGUFbam1nrtZzN1i90AJ8sdGvtA5JCJAM= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 09/35] ipa: rkisp1: Refactor setControls() Date: Fri, 24 Oct 2025 10:50:33 +0200 Message-ID: <20251024085130.995967-10-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" IPARkISP1::setControls() is called when new sensor controls shall be queued to the pipeline handler. It constructs a list of sensor controls and then emits the setSensorControls signal. To be able to return initial sensor controls from the IPARkISP1::start() function, similar functionality will be needed. Prepare for that by changing the setControls() function to a getSensorControls() that is passed a frame context and by moving the setSensorControls.emit() out of the function. Signed-off-by: Stefan Klug --- src/ipa/libipa/fc_queue.cpp | 6 ++++++ src/ipa/libipa/fc_queue.h | 2 ++ src/ipa/rkisp1/rkisp1.cpp | 26 +++++++++++++------------- 3 files changed, 21 insertions(+), 13 deletions(-) diff --git a/src/ipa/libipa/fc_queue.cpp b/src/ipa/libipa/fc_queue.cpp index 39222c2ed204..7ba28ed21611 100644 --- a/src/ipa/libipa/fc_queue.cpp +++ b/src/ipa/libipa/fc_queue.cpp @@ -38,6 +38,12 @@ namespace ipa { * \brief The frame number */ +/** + * \fn FrameContext::frame() + * \brief Get the frame of that frame context + * \return THe frame number + */ + /** * \class FCQueue * \brief A support class for managing FrameContext instances in IPA modules diff --git a/src/ipa/libipa/fc_queue.h b/src/ipa/libipa/fc_queue.h index 1128e42f8ca6..812022c496ed 100644 --- a/src/ipa/libipa/fc_queue.h +++ b/src/ipa/libipa/fc_queue.h @@ -22,6 +22,8 @@ template class FCQueue; struct FrameContext { + uint32_t frame() const { return frame_; } + private: template friend class FCQueue; uint32_t frame_; diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index fa22bfc34904..d64c72ecaf4f 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -77,7 +77,7 @@ private: void updateControls(const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls); - void setControls(unsigned int frame); + ControlList getSensorControls(const IPAFrameContext &context); std::map buffers_; std::map mappedBuffers_; @@ -380,7 +380,12 @@ void IPARkISP1::processStats(const uint32_t frame, const uint32_t bufferId, algo->process(context_, frame, frameContext, stats, metadata); } - setControls(frame); + /* + * \todo: Here we should do a lookahead that takes the sensor delays + * into account. + */ + ControlList ctrls = getSensorControls(frameContext); + setSensorControls.emit(frame, ctrls); context_.debugMetadata.moveEntries(metadata); metadataReady.emit(frame, metadata); @@ -445,28 +450,23 @@ void IPARkISP1::updateControls(const IPACameraSensorInfo &sensorInfo, *ipaControls = ControlInfoMap(std::move(ctrlMap), controls::controls); } -void IPARkISP1::setControls(unsigned int frame) +ControlList IPARkISP1::getSensorControls(const IPAFrameContext &frameContext) { - /* - * \todo The frame number is most likely wrong here, we need to take - * internal sensor delays and other timing parameters into account. - */ - - IPAFrameContext &frameContext = context_.frameContexts.get(frame); uint32_t exposure = frameContext.agc.exposure; uint32_t gain = context_.camHelper->gainCode(frameContext.agc.gain); uint32_t vblank = frameContext.agc.vblank; LOG(IPARkISP1, Debug) - << "Set controls for frame " << frame << ": exposure " << exposure - << ", gain " << frameContext.agc.gain << ", vblank " << vblank; + << "Set controls for frame " << frameContext.frame() + << ": exposure " << exposure + << ", gain " << frameContext.agc.gain + << ", vblank " << vblank; ControlList ctrls(sensorControls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure)); ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain)); ctrls.set(V4L2_CID_VBLANK, static_cast(vblank)); - - setSensorControls.emit(frame, ctrls); + return ctrls; } } /* namespace ipa::rkisp1 */ From patchwork Fri Oct 24 08:50:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24776 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B68E6C3259 for ; Fri, 24 Oct 2025 08:52:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 71C8B608BE; Fri, 24 Oct 2025 10:52:07 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ghTdwVku"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6F8E160871 for ; Fri, 24 Oct 2025 10:52:06 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 00DC61AC5; Fri, 24 Oct 2025 10:50:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295821; bh=6a/BD2w3Wvdd0+vmdLhxYk3YtXZJ0ULr+FFOcsOu8xc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ghTdwVkuUXir5om0CYm3Tl0FJjm8gTJghpL44hOfP7C6lJqStioj5o/jkrjZMOgCG fsipDteWjd6r61tOVOZhGduSxjxJbX/9hkeIXSbS+2K1MIKiedKBjWDA7eN8hI3oXP DaHGYO0gxyeRbMuuAlnHueR2usFAyeiAr0j9ThL4= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 10/35] pipeline: rkisp1: Add a frameStart function to handle DelayedControls::applyControls Date: Fri, 24 Oct 2025 10:50:34 +0200 Message-ID: <20251024085130.995967-11-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Move the call to applyControls into an intermediate function for upcoming modfications. While at it, properly disconnect the signal on failure. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 17250cbd5ae6..9be9f44db479 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1540,6 +1540,7 @@ const uint32_t kDefaultExtParamsBlocks = 0xfffff; int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) { + utils::ScopeExitActions actions; int ret; std::unique_ptr data = @@ -1570,8 +1571,10 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) data->delayedCtrls_ = std::make_unique(data->sensor_->device(), params); - isp_->frameStart.connect(data->delayedCtrls_.get(), - &DelayedControls::applyControls); + isp_->frameStart.connect(this, + &PipelineHandlerRkISP1::frameStart); + + actions += [&]() { isp_->frameStart.disconnect(this); }; uint32_t supportedBlocks = kDefaultExtParamsBlocks; @@ -1603,9 +1606,20 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) registerCamera(std::move(camera)); + actions.release(); + return 0; } +void PipelineHandlerRkISP1::frameStart(uint32_t sequence) +{ + if (!activeCamera_) + return; + + RkISP1CameraData *data = cameraData(activeCamera_); + data->delayedCtrls_->applyControls(sequence); +} + bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) { DeviceMatch dm("rkisp1"); From patchwork Fri Oct 24 08:50:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24777 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5B8EAC3259 for ; Fri, 24 Oct 2025 08:52:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1869C608B5; Fri, 24 Oct 2025 10:52:11 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="g3NhYVtm"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B367160871 for ; Fri, 24 Oct 2025 10:52:09 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 1BF981E47; Fri, 24 Oct 2025 10:50:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295824; bh=JM75Hf0fKghmW5/ZsfZ4EOJip8kutXtOZC4v2jUHinM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=g3NhYVtm4bz3/PWCAkza/3Fj6T79G30WIToX7M4nczuhZ3mDkmANs+rRinZ37YsUP Pyo/GFgVOun27FXrtOs0AYFBTiz5vd6b2pgds7b+pH4eSmUUqjvItJI5uYB71hsMOg pi5JD92jH5ezE0VXEzr3bqbY+SuP6ZUc4CHE3iis= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 11/35] ipa: rkisp1: Move setSensorControls signal to computeParams Date: Fri, 24 Oct 2025 10:50:35 +0200 Message-ID: <20251024085130.995967-12-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The setSensorControls event is emitted in the processStats() function. On first sight this looks reasonable as in processStats() we got the latest statistics and can therefore calculate the most up to date sensor controls. In the light of per-frame-controls however it produces difficult to solve timing issues: - The frame context in processStats() is the frame context of the frame that produced the stats, not for the frame that should be prepared and sent to the sensor. - To synchronize digital gain applied in the ISP with the analog gain applied in the sensor the set of parameters prepared for sensor and ISP must also be synchronized, which is not the case. To fix that, move the calculation and setting of sensor controls into the computeParams(). This way the model is far more easy to understand. We loose a tiny option for optimizations in that (in theory) we could delay the calculation of ISP parameters by another frame (assuming the sensor has a typical 2-frame delay). But all discussions and tests showed that keeping all parameters in sync is more important than that possible optimization for one frame. Signed-off-by: Stefan Klug --- src/ipa/rkisp1/rkisp1.cpp | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index d64c72ecaf4f..ab8583e389d3 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -349,6 +349,9 @@ void IPARkISP1::computeParams(const uint32_t frame, const uint32_t bufferId) for (auto const &algo : algorithms()) algo->prepare(context_, frame, frameContext, ¶ms); + ControlList ctrls = getSensorControls(frameContext); + setSensorControls.emit(frame, ctrls); + paramsComputed.emit(frame, params.size()); } @@ -380,13 +383,6 @@ void IPARkISP1::processStats(const uint32_t frame, const uint32_t bufferId, algo->process(context_, frame, frameContext, stats, metadata); } - /* - * \todo: Here we should do a lookahead that takes the sensor delays - * into account. - */ - ControlList ctrls = getSensorControls(frameContext); - setSensorControls.emit(frame, ctrls); - context_.debugMetadata.moveEntries(metadata); metadataReady.emit(frame, metadata); } From patchwork Fri Oct 24 08:50:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24778 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 29268C3259 for ; Fri, 24 Oct 2025 08:52:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D4A82608B9; Fri, 24 Oct 2025 10:52:13 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="WIW8Q8Nu"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C330660871 for ; Fri, 24 Oct 2025 10:52:12 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 5502A1E47; Fri, 24 Oct 2025 10:50:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295827; bh=vIV9kMVAxEfuUxuZvM48QSwKHrqP3a2E818NTF13OAU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WIW8Q8NuOW3c0FFqMdCVG2lnNNHyCpzMqK2t0U+PWUmBb9aE+o6o7f6SDvR8hP8xN sVwhtrM2NBLotTtX8XnkWMxkMUmn6KnAp8yefkV/KDShj3+EHDSRbfgacy44Xrkry6 xox6bV6b419VoDzpeRZEoieqS3cpaQgU5CCqSZMM= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 12/35] pipeline: rkisp1: Fix controls in raw mode Date: Fri, 24 Oct 2025 10:50:36 +0200 Message-ID: <20251024085130.995967-13-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" After the pipeline restructuring setSensorControls is no longer emitted within process() but within computeParams(). In raw mode computeParams is not called and therefore setSensorControls is never emitted. Fix that by allowing computeParams to be called with an empty bufferId. This strategy is also used for processStats() to ensure that metadata gets filled in raw mode. Then call computeParams within frameStart() when in raw mode. Signed-off-by: Stefan Klug --- src/ipa/rkisp1/rkisp1.cpp | 12 ++++++++++++ src/libcamera/pipeline/rkisp1/rkisp1.cpp | 4 ++++ 2 files changed, 16 insertions(+) diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index ab8583e389d3..ee87c899fcb1 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -343,6 +343,18 @@ void IPARkISP1::computeParams(const uint32_t frame, const uint32_t bufferId) { IPAFrameContext &frameContext = context_.frameContexts.get(frame); + /* + * \todo: This needs discussion. In raw mode, computeParams is + * called without a params buffer, to trigger the setSensorControls + * signal. Currently our algorithms don't support prepare calls with + * a nullptr. Do we need that or can we safely skip it? + */ + if (bufferId == 0) { + ControlList ctrls = getSensorControls(frameContext); + setSensorControls.emit(frame, ctrls); + return; + } + RkISP1Params params(context_.configuration.paramFormat, mappedBuffers_.at(bufferId).planes()[0]); diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 9be9f44db479..0e6e45bba75b 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1618,6 +1618,10 @@ void PipelineHandlerRkISP1::frameStart(uint32_t sequence) RkISP1CameraData *data = cameraData(activeCamera_); data->delayedCtrls_->applyControls(sequence); + + if (isRaw_) { + data->ipa_->computeParams(sequence + 1, 0); + } } bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) From patchwork Fri Oct 24 08:50:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24779 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E4408C3259 for ; Fri, 24 Oct 2025 08:52:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A648D608C4; Fri, 24 Oct 2025 10:52:17 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="NYsGzvfl"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4E22460871 for ; Fri, 24 Oct 2025 10:52:16 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id CA01A1E78; Fri, 24 Oct 2025 10:50:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295830; bh=3djUqQG0nWA/eMrJnEfgCvzJGz41gkaqYt8keBU2li8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NYsGzvflYk0p5MF7wMGZ/RBC06HMmCW26eyYYFZiBnqJ3efSZUWO19BGMaZZl9/5k GtFCwjF55hQMQQ0KYTNa7W9VU/ilUnmT6NzNuGUvcyr9pZBTF/lDTAqC718CSLyBDv qpGq55tc++222MJfGxHaOUC1AUMgFa1x6vqmjhLY= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 13/35] ipa: rkisp1: Add initializeFrameContext() function Date: Fri, 24 Oct 2025 10:50:37 +0200 Message-ID: <20251024085130.995967-14-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In preparation to handling startup controls, split the frame context initialization from queueRequest into a separate function. This patch contains no functional changes. Signed-off-by: Stefan Klug --- src/ipa/rkisp1/rkisp1.cpp | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index ee87c899fcb1..a55bac260026 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -67,6 +67,9 @@ public: void queueRequest(const uint32_t frame, const ControlList &controls) override; void computeParams(const uint32_t frame, const uint32_t bufferId) override; + void initializeFrameContext(const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls); void processStats(const uint32_t frame, const uint32_t bufferId, const ControlList &sensorControls) override; @@ -331,6 +334,13 @@ void IPARkISP1::queueRequest(const uint32_t frame, const ControlList &controls) IPAFrameContext &frameContext = context_.frameContexts.alloc(frame); context_.debugMetadata.enableByControl(controls); + initializeFrameContext(frame, frameContext, controls); +} + +void IPARkISP1::initializeFrameContext(const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ for (auto const &a : algorithms()) { Algorithm *algo = static_cast(a.get()); if (algo->disabled_) From patchwork Fri Oct 24 08:50:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24780 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BD517C3259 for ; Fri, 24 Oct 2025 08:52:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 77137608CF; Fri, 24 Oct 2025 10:52:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="KwgUu2O9"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1248960871 for ; Fri, 24 Oct 2025 10:52:19 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 712611E78; Fri, 24 Oct 2025 10:50:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295833; bh=sd74pbBwO3nWmh/S9dWBYar+SIVGoaUGf49d0DRvwhg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KwgUu2O9q56oXIsCzdWbqtvUQ8cE+kjIKXBkyDY52hO8tj4txiip0dktgDWvC7UtN 98f/KdVlD8QKo6CGQtnyv6MHPussERUoa/G1ro97+9LUWzd+deElkGgFi84W57k/m3 ybqpIoWQML/bKyM3W1eQNMQyKw8xVoR8i7eILHso= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 14/35] pipeline: rkisp1: Apply initial controls Date: Fri, 24 Oct 2025 10:50:38 +0200 Message-ID: <20251024085130.995967-15-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Controls passed to Camera::start() are not handled at the moment. Implement that by initializing a separate frame context and then returning the controls synchronously to the caller. In the pipeline the controls get passed to the sensor before the call to streamOn() to ensure the controls are applied right away. Passing the controls to the pipeline using the setSensorControls signal is not appropriate because it is asynchronous and would reach the sensor too late (initial controls need to be applied before the sensor starts and before delayed controls initializes it's start condition.) Signed-off-by: Stefan Klug --- include/libcamera/ipa/rkisp1.mojom | 7 ++++++- src/ipa/rkisp1/rkisp1.cpp | 10 ++++++---- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 11 +++++++++-- 3 files changed, 21 insertions(+), 7 deletions(-) diff --git a/include/libcamera/ipa/rkisp1.mojom b/include/libcamera/ipa/rkisp1.mojom index 068e898848c4..4c29b53cd7f9 100644 --- a/include/libcamera/ipa/rkisp1.mojom +++ b/include/libcamera/ipa/rkisp1.mojom @@ -14,13 +14,18 @@ struct IPAConfigInfo { uint32 paramFormat; }; +struct StartResult { + libcamera.ControlList controls; + int32 code; +}; + interface IPARkISP1Interface { init(libcamera.IPASettings settings, uint32 hwRevision, uint32 supportedBlocks, libcamera.IPACameraSensorInfo sensorInfo, libcamera.ControlInfoMap sensorControls) => (int32 ret, libcamera.ControlInfoMap ipaControls); - start() => (int32 ret); + start(libcamera.ControlList controls) => (StartResult result); stop(); configure(IPAConfigInfo configInfo, diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index a55bac260026..7a7b7682e242 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -56,7 +56,7 @@ public: const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls) override; - int start() override; + void start(const ControlList &controls, StartResult *result) override; void stop() override; int configure(const IPAConfigInfo &ipaConfig, @@ -215,10 +215,12 @@ int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision, return 0; } -int IPARkISP1::start() +void IPARkISP1::start(const ControlList &controls, StartResult *result) { - /* \todo Properly handle startup controls. */ - return 0; + IPAFrameContext frameContext = {}; + initializeFrameContext(0, frameContext, controls); + result->controls = getSensorControls(frameContext); + result->code = 0; } void IPARkISP1::stop() diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 0e6e45bba75b..d937c94e351f 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1290,13 +1290,20 @@ int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]] const ControlL return ret; actions += [&]() { freeBuffers(camera); }; - ret = data->ipa_->start(); - if (ret) { + ControlList ctrls; + if (!!controls) + ctrls = *controls; + + ipa::rkisp1::StartResult res; + data->ipa_->start(ctrls, &res); + if (res.code) { LOG(RkISP1, Error) << "Failed to start IPA " << camera->id(); return ret; } actions += [&]() { data->ipa_->stop(); }; + data->sensor_->setControls(&res.controls); + data->delayedCtrls_->reset(); data->frame_ = 0; From patchwork Fri Oct 24 08:50:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24781 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5F0FEC3259 for ; Fri, 24 Oct 2025 08:52:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 25CB1608D2; Fri, 24 Oct 2025 10:52:24 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Bm8lr0B5"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4048F6089B for ; Fri, 24 Oct 2025 10:52:22 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id A753620A4; Fri, 24 Oct 2025 10:50:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295836; bh=Bm7AeoHy6oUG/h9poJIAqz9bTSajkoEGA7WJM63Ko4A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Bm8lr0B5vL+kCEMW95vz2uxz+lQOyZ32wrhpfSq6DqyNAle+iw2WMMqizbYSDuKpZ QF4DIo7GxVhkJvNBAdA47vzBlEtD8sXOgaoHVOZtOnpP9Q1Z6UGCNwdA3I+4HmoOX2 BP5yJVT8jHTlVsDnZ10R32m8i19HMDssXBNyU8io= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 15/35] ipa: rkisp1: Set frameContext.agc in queueRequest for auto mode also Date: Fri, 24 Oct 2025 10:50:39 +0200 Message-ID: <20251024085130.995967-16-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" If the agc is in auto mode, exposure time and gain used to be set on the frame context within prepare(). As exposure time and gain are used by getSensorControls(0) from within start() that is too late (prepare() hasn't run yet). Also prepare() is documented as the place to initialize the params buffer, not the frame context. From the pipeline point of view, prepare() gets called immediately after queueRequest(). Therefore we can safely move setting the frame context into queueRequest() to fix the sensor controls for start(). Signed-off-by: Stefan Klug --- src/ipa/rkisp1/algorithms/agc.cpp | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index f5a3c917cb69..c67106339ef8 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -284,9 +284,13 @@ void Agc::queueRequest(IPAContext &context, frameContext.agc.autoExposureEnabled = agc.autoExposureEnabled; frameContext.agc.autoGainEnabled = agc.autoGainEnabled; - if (!frameContext.agc.autoExposureEnabled) + if (frameContext.agc.autoExposureEnabled) + frameContext.agc.exposure = context.activeState.agc.automatic.exposure; + else frameContext.agc.exposure = agc.manual.exposure; - if (!frameContext.agc.autoGainEnabled) + if (frameContext.agc.autoGainEnabled) + frameContext.agc.gain = context.activeState.agc.automatic.gain; + else frameContext.agc.gain = agc.manual.gain; if (!frameContext.agc.autoExposureEnabled && From patchwork Fri Oct 24 08:50:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24782 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 18521C3259 for ; Fri, 24 Oct 2025 08:52:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D1648608C7; Fri, 24 Oct 2025 10:52:26 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="PWa59Yl3"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 65549608C3 for ; Fri, 24 Oct 2025 10:52:25 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id CFDA921D0; Fri, 24 Oct 2025 10:50:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295839; bh=StDPAkMOQOA6KWKzAM3YzShSQyu0YbgvwZMApLh/oac=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PWa59Yl3A66Xj1gH6KJDzAJYAbOBbphoGX8/8K+6cnJSpr+sNNRJhN/EejKtEP3Su K3PSLwkISGLva1JMJ1KeB0JNk2s1rjUk7VA+4m9tYoyE4bgqlr7xrffuP5vw3aaJfx ZKOdlrIxSt8ehQAvAVVNhPrdGQhELHxsMvWajMlY= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 16/35] ipa: rkisp1: agc: Process frame duration at the right time Date: Fri, 24 Oct 2025 10:50:40 +0200 Message-ID: <20251024085130.995967-17-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The frame duration and vblank should not be calculated during process() but within prepare(), where the data for that frame get's computed. Signed-off-by: Stefan Klug --- src/ipa/rkisp1/algorithms/agc.cpp | 20 ++++++++++---------- src/ipa/rkisp1/algorithms/agc.h | 3 +-- 2 files changed, 11 insertions(+), 12 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index c67106339ef8..5ce77b7672f2 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -383,6 +383,12 @@ void Agc::prepare(IPAContext &context, const uint32_t frame, frameContext.agc.yTarget = context.activeState.agc.automatic.yTarget; + /* + * Expand the target frame duration so that we do not run faster than + * the minimum frame duration when we have short exposures. + */ + processFrameDuration(context, frameContext); + if (frame > 0 && !frameContext.agc.updateMetering) return; @@ -511,11 +517,13 @@ double Agc::estimateLuminance(double gain) const * Compute and populate vblank from the target frame duration. */ void Agc::processFrameDuration(IPAContext &context, - IPAFrameContext &frameContext, - utils::Duration frameDuration) + IPAFrameContext &frameContext) { IPACameraSensorInfo &sensorInfo = context.sensorInfo; utils::Duration lineDuration = context.configuration.sensor.lineDuration; + utils::Duration frameDuration = frameContext.agc.exposure * lineDuration; + + frameDuration = std::max(frameDuration, frameContext.agc.minFrameDuration); frameContext.agc.vblank = (frameDuration / lineDuration) - sensorInfo.outputSize.height; @@ -539,8 +547,6 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, ControlList &metadata) { if (!stats) { - processFrameDuration(context, frameContext, - frameContext.agc.minFrameDuration); fillMetadata(context, frameContext, metadata); return; } @@ -641,12 +647,6 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, activeState.agc.automatic.gain = aGain; activeState.agc.automatic.quantizationGain = qGain; activeState.agc.automatic.yTarget = effectiveYTarget(); - /* - * Expand the target frame duration so that we do not run faster than - * the minimum frame duration when we have short exposures. - */ - processFrameDuration(context, frameContext, - std::max(frameContext.agc.minFrameDuration, newExposureTime)); fillMetadata(context, frameContext, metadata); expMeans_ = {}; diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index 7867eed9c4e3..4432711f43af 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -51,8 +51,7 @@ private: ControlList &metadata); double estimateLuminance(double gain) const override; void processFrameDuration(IPAContext &context, - IPAFrameContext &frameContext, - utils::Duration frameDuration); + IPAFrameContext &frameContext); Span expMeans_; Span weights_; From patchwork Fri Oct 24 08:50:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24783 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 87598C3259 for ; Fri, 24 Oct 2025 08:52:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 57269608D1; Fri, 24 Oct 2025 10:52:30 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="utCxFLu3"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C63E3608C3 for ; Fri, 24 Oct 2025 10:52:28 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 55D4921D0; Fri, 24 Oct 2025 10:50:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295843; bh=tJZqo5Jgcw4iWuPX5RG+D/HiDG+3K7kFUCiaYJdlh7E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=utCxFLu3vZU/xwqzajsGRbsOU0N3gw1DXPyi5S7LaNW8llWbDV2Q/6xGCOrAmBYmE qEVe9dhggNcaRvo+UstpVQbtMHvf2Xm+PBGK6OypInd3FAc4UVT54w6sZJPD89EmCN 3ZW1R3cpcMPx6Hh8aGMO1vnK5tBPvEQwi3eD57ns= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 17/35] ipa: rkisp1: agc: Fix vblank, when computeParams prepare is not called Date: Fri, 24 Oct 2025 10:50:41 +0200 Message-ID: <20251024085130.995967-18-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In raw capture mode, prepare() is not called at all. Ensure that vblank is still calculated corresctly. Signed-off-by: Stefan Klug --- src/ipa/rkisp1/algorithms/agc.cpp | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 5ce77b7672f2..7c9538f95cbe 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -340,6 +340,9 @@ void Agc::queueRequest(IPAContext &context, } frameContext.agc.minFrameDuration = agc.minFrameDuration; frameContext.agc.maxFrameDuration = agc.maxFrameDuration; + + /* V-blank needs to be valid for the start controls handling. Update it. */ + processFrameDuration(context, frameContext); } /** From patchwork Fri Oct 24 08:50:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24784 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 104D6C3259 for ; Fri, 24 Oct 2025 08:52:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CB3D7608D7; Fri, 24 Oct 2025 10:52:32 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="O7j96P3c"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BAC65608C3 for ; Fri, 24 Oct 2025 10:52:31 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 44D8C21D0; Fri, 24 Oct 2025 10:50:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295846; bh=09gYgqsAJHJnPFxdJlBfYEXy/j0AidafIJj/89lpLFo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=O7j96P3caViaZfGeQ+A1TzmRWumMFLMjtEPkWFUXusbE4HCGwJZDZt1yk/GsXr0aA pXBVsxN4KKr6yhRuTJk0h1HNcsCfdnTm5Ry7EAg0gpZnwV4UshdfNoNa/r4A2PYPq2 H9Bf93Ql36s3+Njm27LPdzgp31kmW5W2CzfU9l70= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 18/35] libcamera: delayed_controls: Change semantics of sequence numbers Date: Fri, 24 Oct 2025 10:50:42 +0200 Message-ID: <20251024085130.995967-19-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In the context of per frame controls I believe it is easier to think about what needs to be done for a specific sequence number. So (assuming a max sensor delay of 2) the actions would be: - delayedControls.push(n) stores the controls that shall be active on frame n - delayedControls.get(n) returns these controls again - delayedControls.apply(n) applies the slowest control for frame n and does a look back on other controls. So when a frameStart for frame n occurs, it is time to call delayedControls.apply(n + maxDelay) Changing these semantics on delayed controls doesn't require much code change and has the added benefit that we don't run in to clamping for get() on frames < maxDelay. ToDo: This breaks the delayed controls test at the moment. The tests need to be fixed and other pipelines need to be adjusted accordingly. Signed-off-by: Stefan Klug --- src/libcamera/delayed_controls.cpp | 15 ++++++++------- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 5 +++-- test/meson.build | 2 +- 3 files changed, 12 insertions(+), 10 deletions(-) diff --git a/src/libcamera/delayed_controls.cpp b/src/libcamera/delayed_controls.cpp index 35a624525b80..8239f3dcf347 100644 --- a/src/libcamera/delayed_controls.cpp +++ b/src/libcamera/delayed_controls.cpp @@ -241,7 +241,7 @@ bool DelayedControls::push(uint32_t sequence, const ControlList &controls) */ ControlList DelayedControls::get(uint32_t sequence) { - unsigned int index = std::max(0, sequence - maxDelay_); + unsigned int index = sequence; ControlList out(device_->controls()); for (const auto &ctrl : values_) { @@ -267,13 +267,14 @@ ControlList DelayedControls::get(uint32_t sequence) */ /** - * \brief Inform DelayedControls of the start of a new frame - * \param[in] sequence Sequence number of the frame that started + * \brief Apply controls for a frame + * \param[in] sequence Sequence number of the frame to apply * - * Inform the state machine that a new frame has started and of its sequence - * number. Any user of these helpers is responsible to inform the helper about - * the start of any frame. This can be connected with ease to the start of a - * exposure (SOE) V4L2 event. + * Apply controls for the frame \a sequence. This applies the controls with the + * largest delay. For controls with a smaller delay it does a looks back and + * applies the controls for the previous sequence. So usually this function is + * called in a start of exposure event as applyControls(startedSequence + + * maxDelay) */ void DelayedControls::applyControls(uint32_t sequence) { diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index d937c94e351f..7a4957d7e535 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1624,10 +1624,11 @@ void PipelineHandlerRkISP1::frameStart(uint32_t sequence) return; RkISP1CameraData *data = cameraData(activeCamera_); - data->delayedCtrls_->applyControls(sequence); + uint32_t sequenceToApply = sequence + data->delayedCtrls_->maxDelay(); + data->delayedCtrls_->applyControls(sequenceToApply); if (isRaw_) { - data->ipa_->computeParams(sequence + 1, 0); + data->ipa_->computeParams(sequenceToApply + 1, 0); } } diff --git a/test/meson.build b/test/meson.build index 52f04364e4fc..80fb543f2201 100644 --- a/test/meson.build +++ b/test/meson.build @@ -53,7 +53,7 @@ internal_tests = [ {'name': 'bayer-format', 'sources': ['bayer-format.cpp']}, {'name': 'byte-stream-buffer', 'sources': ['byte-stream-buffer.cpp']}, {'name': 'camera-sensor', 'sources': ['camera-sensor.cpp']}, - {'name': 'delayed_controls', 'sources': ['delayed_controls.cpp']}, + {'name': 'delayed_controls', 'sources': ['delayed_controls.cpp'], 'should_fail': true}, {'name': 'event', 'sources': ['event.cpp']}, {'name': 'event-dispatcher', 'sources': ['event-dispatcher.cpp']}, {'name': 'event-thread', 'sources': ['event-thread.cpp']}, From patchwork Fri Oct 24 08:50:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24785 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 94957C3259 for ; Fri, 24 Oct 2025 08:52:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5AEED608D7; Fri, 24 Oct 2025 10:52:36 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="UapTTehl"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5C181608C7 for ; Fri, 24 Oct 2025 10:52:34 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id D533B1AC5; Fri, 24 Oct 2025 10:50:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295848; bh=R0HNYlAm9anXPL+QWSNQzkAwChXqdCez6wzkOG1DYqA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UapTTehlW3lzVtTtSE+3uxcwGtqW/tk9DWzsCrnX55IxrAVEWWDbbkyAGIdWIroB8 T3OjQhknzhjgw6XbOiErXVKL+t+Jl3p+V0FF00zXIFd7AS7E/6wbVHYfm1icTLw8j8 41lUV75qKBOF0nqGjKNuEx+14NiwKVo8yhOM1zi0= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 19/35] libipa: algorithm: Update docs Date: Fri, 24 Oct 2025 10:50:43 +0200 Message-ID: <20251024085130.995967-20-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Update the algorithm documentation to reflect the changed timing model. Signed-off-by: Stefan Klug --- src/ipa/libipa/algorithm.cpp | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/src/ipa/libipa/algorithm.cpp b/src/ipa/libipa/algorithm.cpp index 201efdfdba25..44b6bdfa6972 100644 --- a/src/ipa/libipa/algorithm.cpp +++ b/src/ipa/libipa/algorithm.cpp @@ -76,11 +76,12 @@ namespace ipa { * * This function is called for each request queued to the camera. It provides * the controls stored in the request to the algorithm. The \a frame number - * is the Request sequence number and identifies the desired corresponding + * is the sensor sequence number and identifies the desired corresponding * frame to target for the controls to take effect. * * Algorithms shall read the applicable controls and store their value for later - * use during frame processing. + * use during frame processing. All values that are already known shall be + * updated in \a frameContext. */ /** @@ -98,14 +99,18 @@ namespace ipa { * Algorithms shall fill in the parameter structure fields appropriately to * configure the ISP processing blocks that they are responsible for. This * includes setting fields and flags that enable those processing blocks. + * + * Additionally \a frameContext shall be updated with the most up to date values + * from active state. */ /** * \fn Algorithm::process() * \brief Process ISP statistics, and run algorithm operations * \param[in] context The shared IPA context - * \param[in] frame The frame context sequence number - * \param[in] frameContext The current frame's context + * \param[in] frame The frame sequence number that produces the stats + * \param[in] frameContext The frame context for the frame that produced the + * stats * \param[in] stats The IPA statistics and ISP results * \param[out] metadata Metadata for the frame, to be filled by the algorithm * @@ -118,19 +123,14 @@ namespace ipa { * computationally expensive calculations or operations must be handled * asynchronously in a separate thread. * - * Algorithms can store state in their respective IPAFrameContext structures, - * and reference state from the IPAFrameContext of other algorithms. - * - * \todo Historical data may be required as part of the processing. - * Either the previous frame, or the IPAFrameContext state of the frame - * that generated the statistics for this operation may be required for - * some advanced algorithms to prevent oscillations or support control - * loops correctly. Only a single IPAFrameContext is available currently, - * and so any data stored may represent the results of the previously - * completed operations. + * Algorithms shall update the active state. The frameContext shall *not* be + * updated as that frame was already produced. * * Care shall be taken to ensure the ordering of access to the information * such that the algorithms use up to date state as required. + * + * The \a stats parameter can be null in which case ony the frame metadata shall + * be filled. */ /** From patchwork Fri Oct 24 08:50:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24786 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1D179C3259 for ; Fri, 24 Oct 2025 08:52:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D02C7608DF; Fri, 24 Oct 2025 10:52:37 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="gDnT7cTc"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 51B60608C7 for ; Fri, 24 Oct 2025 10:52:37 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id D68E71AC5; Fri, 24 Oct 2025 10:50:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295851; bh=GHDBZhxUClPjiTGikBMuLaR34/ZPU/bFsMTzDclcYuU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gDnT7cTcwjVT64MFc6NleIwaCgDKYko1VWZpAZfbjqVPR10XpXgLcNQS0uH+ZG6D/ qLx+EEdFn0JGhMBWcskypR/Ok1kbivogx5v1Nn2B1a9yzGgccxihVLKOG0JalF4rkV Osi0M4nBdI6PlbHHHbtl050FRMh+kShP5zUR4eUI= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 20/35] libcamera: delayed_controls: Ignore double pushes for the same frame number Date: Fri, 24 Oct 2025 10:50:44 +0200 Message-ID: <20251024085130.995967-21-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" For successful PFC a single sequence must only be pushed once to delayed controls. Such a situation can occur if no-ops were pushed in delayed controls due to a buffer underrun. Signed-off-by: Stefan Klug --- src/libcamera/delayed_controls.cpp | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/src/libcamera/delayed_controls.cpp b/src/libcamera/delayed_controls.cpp index 8239f3dcf347..3dead0dc4429 100644 --- a/src/libcamera/delayed_controls.cpp +++ b/src/libcamera/delayed_controls.cpp @@ -176,6 +176,8 @@ bool DelayedControls::push(uint32_t sequence, const ControlList &controls) LOG(DelayedControls, Warning) << "Double push for sequence " << sequence << " current queue index: " << queueCount_; + LOG(DelayedControls, Warning) << "ignore silently"; + return true; } while (sequence > queueCount_) { @@ -278,7 +280,11 @@ ControlList DelayedControls::get(uint32_t sequence) */ void DelayedControls::applyControls(uint32_t sequence) { - LOG(DelayedControls, Debug) << "frame " << sequence << " started"; + LOG(DelayedControls, Debug) + << "Apply controls for: " << sequence + << " expected: " << writeCount_ + << " (instant controls for frame " + << (sequence - maxDelay_) << ")"; while (queueCount_ - 1 < sequence) { LOG(DelayedControls, Warning) From patchwork Fri Oct 24 08:50:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24787 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B8648C3259 for ; Fri, 24 Oct 2025 08:52:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7A0D3608DE; Fri, 24 Oct 2025 10:52:42 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="p9Xl02pI"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C7F71608C7 for ; Fri, 24 Oct 2025 10:52:40 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 50CF22363; Fri, 24 Oct 2025 10:50:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295855; bh=+A7vLf/5UBvL9imx4XgnsJtAZ49G47f8GYrh/QP/5sg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=p9Xl02pIQTYk66lbfAA3qiHK4EPQyLHlsc3JtLpLJQ1+w1ptPp076FjV/Z+2HD5rg 1YDJQ96p435adQ1HXB1G9uFqCuRFzNL0VYLk2BLN9uz5TbJfTgSWgU2xhz03PrD9hr DTz5T7QV1+RkHm7yRtrX8wbnV7dWG5M/ASmmqzzo= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Jacopo Mondi Subject: [PATCH v1 21/35] libcamera: v4l2_videodevice: Do not hide frame drops Date: Fri, 24 Oct 2025 10:50:45 +0200 Message-ID: <20251024085130.995967-22-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi We want to be able to identify them correctly. Signed-off-by: Jacopo Mondi --- include/libcamera/internal/v4l2_videodevice.h | 1 - src/libcamera/v4l2_videodevice.cpp | 15 --------------- 2 files changed, 16 deletions(-) diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h index 2d290971a0ee..a19fca1719d3 100644 --- a/include/libcamera/internal/v4l2_videodevice.h +++ b/include/libcamera/internal/v4l2_videodevice.h @@ -286,7 +286,6 @@ private: EventNotifier *fdBufferNotifier_; State state_; - std::optional firstFrame_; Timer watchdog_; utils::Duration watchdogDuration_; diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index 8ce739f4bc65..56a9f0ab546f 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -1904,19 +1904,6 @@ FrameBuffer *V4L2VideoDevice::dequeueBuffer() if (V4L2_TYPE_IS_OUTPUT(buf.type)) return buffer; - /* - * Detect kernel drivers which do not reset the sequence number to zero - * on stream start. - */ - if (!firstFrame_.has_value()) { - if (buf.sequence) - LOG(V4L2, Info) - << "Zero sequence expected for first frame (got " - << buf.sequence << ")"; - firstFrame_ = buf.sequence; - } - metadata.sequence -= firstFrame_.value(); - Span framebufferPlanes = buffer->planes(); unsigned int numV4l2Planes = multiPlanar ? buf.length : 1; @@ -1993,8 +1980,6 @@ int V4L2VideoDevice::streamOn() { int ret; - firstFrame_.reset(); - ret = ioctl(VIDIOC_STREAMON, &bufferType_); if (ret < 0) { LOG(V4L2, Error) From patchwork Fri Oct 24 08:50:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24788 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 72FC9C3259 for ; Fri, 24 Oct 2025 08:52:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 307DC608DE; Fri, 24 Oct 2025 10:52:45 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="XR76qxYN"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1B0A2608C7 for ; Fri, 24 Oct 2025 10:52:44 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 872922363; Fri, 24 Oct 2025 10:50:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295858; bh=bRWFbrF4vkcWee4e9c0ihLXAwb+PcLCFFzuY48wyNhg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XR76qxYNHLnTRRVCbvhhLCeZLSShOkkjxfUpmGPifLCPTNh72oZL73SZGIThLm4nW cqJaxEKdm1LY3kd5tFsjoMT7TOW35gXkI8GctqwLPDdbaKbwz4qnboyUJILjqqH/lP vLNSvKkZr5EvoJIk/gRak3jAW8588p4rQ8RFKAXE= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 22/35] ipa: rkisp1: Allow processStats to be called without stats buffer Date: Fri, 24 Oct 2025 10:50:46 +0200 Message-ID: <20251024085130.995967-23-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When there are no stats available for a frame, it still makes sense to call processStats to fill in the metadata of that frame. This mechanism is already used for the raw path. Allow it's use for no-raw also. Signed-off-by: Stefan Klug --- src/ipa/rkisp1/rkisp1.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 7a7b7682e242..01b30c947a0a 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -389,7 +389,7 @@ void IPARkISP1::processStats(const uint32_t frame, const uint32_t bufferId, * provided. */ const rkisp1_stat_buffer *stats = nullptr; - if (!context_.configuration.raw) + if (bufferId != 0) stats = reinterpret_cast( mappedBuffers_.at(bufferId).planes()[0].data()); From patchwork Fri Oct 24 08:50:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24789 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2C04BC3259 for ; Fri, 24 Oct 2025 08:52:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DC6BA608F1; Fri, 24 Oct 2025 10:52:48 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="j6NYDYtJ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9971B608C7 for ; Fri, 24 Oct 2025 10:52:46 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 15D7E484E; Fri, 24 Oct 2025 10:51:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295861; bh=dxCtoWpqKdzg6ayd/boXKOq/crNZYfHq5PZj88Ea2eg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=j6NYDYtJGMNgNCoyO+RYaQ/vcamY/7Ism/UoOyhaA6hI1q11+8xkLGBIHmt1FQ9L3 htdDbvs2l+dCqW2Zn2ChxI6UPTfNCkVGITq+KB8NKvg9A+VkXuac7CP+K7Ltj+htQJ CWSdlhnhq+rvHtDIAbmKPSTAAWaWmpyXj1cfTu1w= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 23/35] ipa: rkisp1: Lazy initialise frame context Date: Fri, 24 Oct 2025 10:50:47 +0200 Message-ID: <20251024085130.995967-24-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" For per frame control we want to tick the IPA by the sensor frame sequence instead of the request frame sequence. This has the side effect that the IPA must be able to cope with situations where a frame context is required for a frame that was not queued before (computeParams is called without a corresponding request) or processStats is called for an unexpected sequence number (because a scratch buffer was used on kernel side) Prepare for that by allowing the frame context to be initialized on demand. Signed-off-by: Stefan Klug --- src/ipa/rkisp1/algorithms/awb.cpp | 2 ++ src/ipa/rkisp1/ipa_context.h | 2 ++ src/ipa/rkisp1/rkisp1.cpp | 6 ++++++ 3 files changed, 10 insertions(+) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 399fb51be414..27109478c340 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -172,6 +172,8 @@ void Awb::queueRequest(IPAContext &context, awbAlgo_->handleControls(controls); frameContext.awb.autoEnabled = awb.autoEnabled; + frameContext.awb.gains = awb.automatic.gains; + frameContext.awb.temperatureK = awb.automatic.temperatureK; if (awb.autoEnabled) return; diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index f85a130d9c23..185951fb8286 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -214,6 +214,8 @@ struct IPAFrameContext : public FrameContext { double strength; double gain; } wdr; + + bool initialised; }; struct IPAContext { diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 01b30c947a0a..23d80bc43c5d 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -343,6 +343,10 @@ void IPARkISP1::initializeFrameContext(const uint32_t frame, IPAFrameContext &frameContext, const ControlList &controls) { + if (frameContext.initialised) + return; + + frameContext.initialised = true; for (auto const &a : algorithms()) { Algorithm *algo = static_cast(a.get()); if (algo->disabled_) @@ -354,6 +358,7 @@ void IPARkISP1::initializeFrameContext(const uint32_t frame, void IPARkISP1::computeParams(const uint32_t frame, const uint32_t bufferId) { IPAFrameContext &frameContext = context_.frameContexts.get(frame); + initializeFrameContext(frame, frameContext, {}); /* * \todo: This needs discussion. In raw mode, computeParams is @@ -383,6 +388,7 @@ void IPARkISP1::processStats(const uint32_t frame, const uint32_t bufferId, const ControlList &sensorControls) { IPAFrameContext &frameContext = context_.frameContexts.get(frame); + initializeFrameContext(frame, frameContext, {}); /* * In raw capture mode, the ISP is bypassed and no statistics buffer is From patchwork Fri Oct 24 08:50:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24790 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A5F78BE080 for ; Fri, 24 Oct 2025 08:52:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 634F960920; Fri, 24 Oct 2025 10:52:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="rcQJLIbR"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C597F608C7 for ; Fri, 24 Oct 2025 10:52:49 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 2ADDF484E; Fri, 24 Oct 2025 10:51:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295864; bh=CrYhuF6fM2fE7Fad53k80vqInEOP7VuATyquavW7HVs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rcQJLIbRTAiG/gvTGiYTEyBcdtnXBWTW8HfWNna8hqR5A92fjtbMKB68havIH+Sem 0g+rJ6bMEM0Fd7DjBRZZlj6ojsE0MndeI9Kn/54zPdRCTx7ulE0bPz8FODUH1JtUXG 4Bb+VqMZmAHDvrV5sLF6N/dexru/X9Oyev4PPqrA= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 24/35] ipa: rkisp1: Ensure controls don't get lost Date: Fri, 24 Oct 2025 10:50:48 +0200 Message-ID: <20251024085130.995967-25-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" If a request too late for controls queuing because the frame context was already initialised, store the controls and apply them on the next request. Signed-off-by: Stefan Klug ipa: rkisp1: Store controls on when necessary Store controls for later only when necessary. Add a proper log message for that condition. Signed-off-by: Stefan Klug --- src/ipa/rkisp1/rkisp1.cpp | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 23d80bc43c5d..b714e2f10c65 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -86,6 +86,7 @@ private: std::map mappedBuffers_; ControlInfoMap sensorControls_; + ControlList controlsToApply_; /* Local parameter storage */ struct IPAContext context_; @@ -343,16 +344,33 @@ void IPARkISP1::initializeFrameContext(const uint32_t frame, IPAFrameContext &frameContext, const ControlList &controls) { - if (frameContext.initialised) + if (frameContext.initialised) { + if (!controls.empty()) { + /* Too late to apply the controls. Store them for later. */ + LOG(IPARkISP1, Warning) + << "Request underrun. Controls for frame " + << frame << " are delayed "; + controlsToApply_.merge(controls, + ControlList::MergePolicy::OverwriteExisting); + } return; + } + + const ControlList *controls2 = &controls; + if (!controlsToApply_.empty()) { + controlsToApply_.merge(controls, ControlList::MergePolicy::OverwriteExisting); + controls2 = &controlsToApply_; + } frameContext.initialised = true; for (auto const &a : algorithms()) { Algorithm *algo = static_cast(a.get()); if (algo->disabled_) continue; - algo->queueRequest(context_, frame, frameContext, controls); + algo->queueRequest(context_, frame, frameContext, *controls2); } + + controlsToApply_.clear(); } void IPARkISP1::computeParams(const uint32_t frame, const uint32_t bufferId) From patchwork Fri Oct 24 08:50:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24791 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6AA02BE080 for ; Fri, 24 Oct 2025 08:52:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 23A3F60931; Fri, 24 Oct 2025 10:52:55 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="f7j2R2YW"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 54CFE608F7 for ; Fri, 24 Oct 2025 10:52:53 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id C4EA64AEB; Fri, 24 Oct 2025 10:51:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295867; bh=w0Hxg2XU1TUu/+ZvDaC8neBCy+QwzFpeQN0WFkwbwlQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=f7j2R2YW+AEoOtwfTf54jZaT9Zz4bVm3puzXCDftzEorZ/5GJtgSUN30ZYIHhq3d5 OyScJA9cdXJfsJ9mO/KJdo1jsNVOOjQcZZwspmClwyFet9Cd46fEI/U2jVBO01OOA6 OtdJFEXCsIPQNp0oA7Uk5zWoV2I6MeKGY/rt3Cxc= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 25/35] pipeline: rkisp1: Add SequenceSyncHelper class Date: Fri, 24 Oct 2025 10:50:49 +0200 Message-ID: <20251024085130.995967-26-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" On a V4L2 buffer the assigned sequence is not known until the buffer is dequeued. But for per frame controls we have to prepare other data like sensor controls and ISP params in advance. So we try to anticipate the sequence number a given buffer will be. In a perfect world this works well as long as the initial sequence is assigned correctly. But it breaks as soon as things like running out of buffers or incomplete images happen. To make things even more complicated, in most cases more than one buffer is queued to the kernel at a time. So as soon as a sequence number doesn't match the expected one after dequeuing, most likely all the already queued buffers will be dequeued with the same error. It is not sufficient to simply add the correction after dequeuing because the error on all queued frames would accumulate and the whole system starts to oscillate. To work around that add a SequenceSyncHelper class that tracks the expected error and allows to easily query the necessary correction when queuing new buffers. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 2 + .../pipeline/rkisp1/sequence_sync_helper.h | 69 +++++++++++++++++++ 2 files changed, 71 insertions(+) create mode 100644 src/libcamera/pipeline/rkisp1/sequence_sync_helper.h diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 7a4957d7e535..d83f7d787892 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -51,10 +51,12 @@ #include "libcamera/internal/yaml_parser.h" #include "rkisp1_path.h" +#include "sequence_sync_helper.h" namespace libcamera { LOG_DEFINE_CATEGORY(RkISP1) +LOG_DEFINE_CATEGORY(RkISP1Schedule) class PipelineHandlerRkISP1; class RkISP1CameraData; diff --git a/src/libcamera/pipeline/rkisp1/sequence_sync_helper.h b/src/libcamera/pipeline/rkisp1/sequence_sync_helper.h new file mode 100644 index 000000000000..c3f91dbed45f --- /dev/null +++ b/src/libcamera/pipeline/rkisp1/sequence_sync_helper.h @@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas on Board + * + * Sequence sync helper + */ + +#pragma once + +#include + +#include + +namespace libcamera { + +LOG_DECLARE_CATEGORY(RkISP1Schedule) + +class SequenceSyncHelper +{ +public: + int gotFrame(size_t expectedSequence, size_t actualSequence) + { + ASSERT(!corrections_.empty()); + int diff = actualSequence - expectedSequence; + int corr = corrections_.front(); + corrections_.pop(); + expectedOffset_ -= corr; + int necessaryCorrection = diff - expectedOffset_; + correctionToApply_ += necessaryCorrection; + + LOG(RkISP1Schedule, Debug) << "Sync frame " + << "expected: " << expectedSequence + << " actual: " << actualSequence + << " correction: " << corr + << " expectedOffset: " << expectedOffset_ + << " correctionToApply " << correctionToApply_; + + expectedOffset_ += necessaryCorrection; + return necessaryCorrection; + } + + /* Value to be added to the source sequence */ + int correction() + { + return correctionToApply_; + } + + void pushCorrection(int correction) + { + corrections_.push(correction); + correctionToApply_ -= correction; + LOG(RkISP1Schedule, Debug) + << "Push correction " << correction + << " correctionToApply " << correctionToApply_; + } + + void reset() + { + corrections_ = {}; + correctionToApply_ = 0; + expectedOffset_ = 0; + } + + std::queue corrections_; + int correctionToApply_ = 0; + int expectedOffset_ = 0; +}; + +} /* namespace libcamera */ From patchwork Fri Oct 24 08:50:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24792 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 05C8FBE080 for ; Fri, 24 Oct 2025 08:52:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B3BA460941; Fri, 24 Oct 2025 10:52:57 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="bDeLakYX"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 450CE608FE for ; Fri, 24 Oct 2025 10:52:56 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id A8F1C4B0C; Fri, 24 Oct 2025 10:51:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295870; bh=eByHVcvUquYqQ2+UqMyFSY2oHfGcKMqRDTkbu8g8WQY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bDeLakYXKLE1w3Npi9ysOlKh3NLM41/H4PMA9z2AUnYxgcjCYsdRNVDDByLKQIIff uhLjwJjU6q0rb02XSob8PKQNsOV6eysyGMevYJ6G50brG+C04OxqsLrCt5AmdOsXKO kslYmle72jf5khbKrEa3JCgT3XVHi1idleavg+Pg= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 26/35] ipa: rkisp1: awb: Ignore empty AWB statistics Date: Fri, 24 Oct 2025 10:50:50 +0200 Message-ID: <20251024085130.995967-27-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the AWB engine doesn't find a valid pixel because all pixels lie outside the configured colour range it returns an AWB measurement value of 255, 255, 255. This leaves the regulation in an unrecoverable state noticeable by a completely green image. Fix that by skipping the AWB calculation in case there were no valid pixels. Signed-off-by: Stefan Klug --- src/ipa/rkisp1/algorithms/awb.cpp | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 27109478c340..81c767531c03 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -313,6 +313,11 @@ void Awb::process(IPAContext &context, return; } + if (awb->awb_mean[0].cnt == 0) { + LOG(RkISP1Awb, Warning) << "AWB statistics are empty"; + return; + } + RGB rgbMeans = calculateRgbMeans(frameContext, awb); /* From patchwork Fri Oct 24 08:50:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24793 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A2186BE080 for ; Fri, 24 Oct 2025 08:53:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 627FD60920; Fri, 24 Oct 2025 10:53:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="H/MSe03x"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0996360904 for ; Fri, 24 Oct 2025 10:53:00 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 7012B4B0C; Fri, 24 Oct 2025 10:51:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295874; bh=pPX6ZkvwcPT+gy/6082nsbDObFZgTGQBPPRh5DqgZqQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=H/MSe03x9uJPk40CA+QCctKWhFcuWOv8qJFxEiXf89Fbata2odWXcAJCpIsx/t7FT FT5lUgWf6eD9IQlS1kc7jZNTLjbsKpprfvye8Olh4+Q8an/Ap27xpMVTnDPHzKsKO0 dq9Pqzno5FyhBkgtb353oClhnzqyU+OKNQqf2LcQ= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 27/35] pipeline: rkisp1: Decouple image, stats and param buffers Date: Fri, 24 Oct 2025 10:50:51 +0200 Message-ID: <20251024085130.995967-28-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The current code creates FrameInfo objects that tie together params, stats and image buffers and expect that these always stay in these groupings. However there are cases where these sequences get out of sync (e.g. a scratch buffer is used for an image or a params buffer was too late and therefore gets applied one frame later). As these situations are timing related and cpu dependent it is impossible to guarantee the initial grouping of buffers. Split the buffers into separate queues for stats, images and params. Resynchronize on image buffers as soon as they are dequeued from V4L2 as these are the only buffers tied to the libcamera requests coming from the user application. Now handle the other buffer types according to their specific properties: Stats buffers only need to be tracked after dequing and can be tied to the corresponding request after the corresponding image buffer was dequeued. If params buffers get out of sync we can either inject the same set of parameters twice or skip one set of params. If image buffers get out of sync we need to update the expected sensor sequence, so that the next request is assigned the correct sensor sequence. Signed-off-by: Stefan Klug --- Changes in v0.6: - Fixed multiple assertions on start/stop and a few corner cases - Fixed crash in dewarpRequestReady on stop Changes in v0.5 - Fixed possible use-after-free in RequestInfo --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 733 +++++++++++++++-------- 1 file changed, 475 insertions(+), 258 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index d83f7d787892..cd9364cb8950 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -6,6 +6,8 @@ */ #include +#include +#include #include #include #include @@ -61,44 +63,14 @@ LOG_DEFINE_CATEGORY(RkISP1Schedule) class PipelineHandlerRkISP1; class RkISP1CameraData; -struct RkISP1FrameInfo { - unsigned int frame; - Request *request; - FrameBuffer *paramBuffer; - FrameBuffer *statBuffer; - FrameBuffer *mainPathBuffer; - FrameBuffer *selfPathBuffer; - - bool paramDequeued; - bool metadataProcessed; -}; - -class RkISP1Frames -{ -public: - RkISP1Frames(PipelineHandler *pipe); - - RkISP1FrameInfo *create(const RkISP1CameraData *data, Request *request, - bool isRaw); - int destroy(unsigned int frame); - void clear(); - - RkISP1FrameInfo *find(unsigned int frame); - RkISP1FrameInfo *find(FrameBuffer *buffer); - RkISP1FrameInfo *find(Request *request); - -private: - PipelineHandlerRkISP1 *pipe_; - std::map frameInfo_; -}; class RkISP1CameraData : public Camera::Private { public: RkISP1CameraData(PipelineHandler *pipe, RkISP1MainPath *mainPath, RkISP1SelfPath *selfPath) - : Camera::Private(pipe), frame_(0), frameInfo_(pipe), + : Camera::Private(pipe), frame_(0), mainPath_(mainPath), selfPath_(selfPath) { } @@ -111,9 +83,12 @@ public: Stream selfPathStream_; std::unique_ptr sensor_; std::unique_ptr delayedCtrls_; + /* + * The sensor frame sequence of the last request queued to the pipeline + * handler. + */ unsigned int frame_; std::vector ipaBuffers_; - RkISP1Frames frameInfo_; RkISP1MainPath *mainPath_; RkISP1SelfPath *selfPath_; @@ -169,21 +144,54 @@ private: Transform combinedTransform_; }; +struct SensorFrameInfo { + Request *request = nullptr; + FrameBuffer *statsBuffer = nullptr; + ControlList metadata; + bool metadataProcessed = false; +}; + +struct RequestInfo { + Request *request = nullptr; + /* + * The estimated sensor sequence for this request. Only reliable when + * sequenceValid is true + */ + size_t sequence = 0; + bool sequenceValid = false; +}; + +struct ParamBufferInfo { + FrameBuffer *buffer = nullptr; + size_t expectedSequence = 0; +}; + +struct DewarpBufferInfo { + FrameBuffer *inputBuffer; + FrameBuffer *outputBuffer; +}; + namespace { /* - * Maximum number of requests that shall be queued into the pipeline to keep - * the regulation fast. - * \todo This needs revisiting as soon as buffers got decoupled from requests - * and/or a fast path for controls was implemented. + * This many buffers ensures that the pipeline runs smoothly, without frame + * drops. */ -static constexpr unsigned int kRkISP1MaxQueuedRequests = 4; +static constexpr unsigned int kRkISP1MinBufferCount = 6; /* - * This many internal buffers (or rather parameter and statistics buffer - * pairs) ensures that the pipeline runs smoothly, without frame drops. + * This many internal buffers (params and stats) are needed for smooth operation + * \todo In high framerate or high cpu load situations it might be necessary to + * increase this number. \todo: This also relates to max sensor delay and must + * always be >= maxSensor delay */ -static constexpr unsigned int kRkISP1MinBufferCount = 4; +static constexpr unsigned int kRkISP1InternalBufferCount = 4; + +/* + * This many internal image buffers between ISP and dewarper are needed for + * smooth operation. + */ +static constexpr unsigned int kRkISP1DewarpImageBufferCount = 4; /* * This flag allows to use dynamic dewarp maps to support pan, zoom, rotate when @@ -223,12 +231,11 @@ private: friend RkISP1CameraData; friend RkISP1CameraConfiguration; - friend RkISP1Frames; int initLinks(Camera *camera, const RkISP1CameraConfiguration &config); int createCamera(MediaEntity *sensor); - void tryCompleteRequest(RkISP1FrameInfo *info); - void cancelDewarpRequest(RkISP1FrameInfo *info); + void tryCompleteRequests(); + void cancelDewarpRequest(Request *request); void imageBufferReady(FrameBuffer *buffer); void paramBufferReady(FrameBuffer *buffer); void statBufferReady(FrameBuffer *buffer); @@ -236,6 +243,9 @@ private: void dewarpBufferReady(FrameBuffer *buffer); void frameStart(uint32_t sequence); + void queueInternalBuffers(); + void computeParamBuffers(uint32_t maxSequence); + int allocateBuffers(Camera *camera); int freeBuffers(Camera *camera); @@ -261,139 +271,32 @@ private: std::vector> dewarpRequests_; std::queue availableDewarpRequests_; + bool running_ = false; + std::vector> paramBuffers_; std::vector> statBuffers_; std::queue availableParamBuffers_; std::queue availableStatBuffers_; - Camera *activeCamera_; -}; + std::deque queuedRequests_; -RkISP1Frames::RkISP1Frames(PipelineHandler *pipe) - : pipe_(static_cast(pipe)) -{ -} - -RkISP1FrameInfo *RkISP1Frames::create(const RkISP1CameraData *data, Request *request, - bool isRaw) -{ - unsigned int frame = data->frame_; - - FrameBuffer *paramBuffer = nullptr; - FrameBuffer *statBuffer = nullptr; - FrameBuffer *mainPathBuffer = nullptr; - FrameBuffer *selfPathBuffer = nullptr; + std::map sensorFrameInfos_; - if (!isRaw) { - if (pipe_->availableParamBuffers_.empty()) { - LOG(RkISP1, Error) << "Parameters buffer underrun"; - return nullptr; - } - - if (pipe_->availableStatBuffers_.empty()) { - LOG(RkISP1, Error) << "Statistic buffer underrun"; - return nullptr; - } - - paramBuffer = pipe_->availableParamBuffers_.front(); - pipe_->availableParamBuffers_.pop(); - - statBuffer = pipe_->availableStatBuffers_.front(); - pipe_->availableStatBuffers_.pop(); - - if (data->usesDewarper_) { - mainPathBuffer = pipe_->availableMainPathBuffers_.front(); - pipe_->availableMainPathBuffers_.pop(); - } - } + std::deque queuedDewarpBuffers_; + SequenceSyncHelper paramsSyncHelper_; + SequenceSyncHelper imageSyncHelper_; - if (!mainPathBuffer) - mainPathBuffer = request->findBuffer(&data->mainPathStream_); - selfPathBuffer = request->findBuffer(&data->selfPathStream_); + std::queue computingParamBuffers_; + std::queue queuedParamBuffers_; - auto [it, inserted] = frameInfo_.try_emplace(frame); - ASSERT(inserted); - auto &info = it->second; + uint32_t nextParamsSequence_; + uint32_t nextStatsToProcess_; - info.frame = frame; - info.request = request; - info.paramBuffer = paramBuffer; - info.mainPathBuffer = mainPathBuffer; - info.selfPathBuffer = selfPathBuffer; - info.statBuffer = statBuffer; - info.paramDequeued = false; - info.metadataProcessed = false; - - return &info; -} - -int RkISP1Frames::destroy(unsigned int frame) -{ - auto it = frameInfo_.find(frame); - if (it == frameInfo_.end()) - return -ENOENT; - - auto &info = it->second; - - pipe_->availableParamBuffers_.push(info.paramBuffer); - pipe_->availableStatBuffers_.push(info.statBuffer); - pipe_->availableMainPathBuffers_.push(info.mainPathBuffer); - - frameInfo_.erase(it); - - return 0; -} - -void RkISP1Frames::clear() -{ - for (const auto &[frame, info] : frameInfo_) { - pipe_->availableParamBuffers_.push(info.paramBuffer); - pipe_->availableStatBuffers_.push(info.statBuffer); - pipe_->availableMainPathBuffers_.push(info.mainPathBuffer); - } - - frameInfo_.clear(); -} - -RkISP1FrameInfo *RkISP1Frames::find(unsigned int frame) -{ - auto itInfo = frameInfo_.find(frame); - - if (itInfo != frameInfo_.end()) - return &itInfo->second; - - LOG(RkISP1, Fatal) << "Can't locate info from frame"; - - return nullptr; -} - -RkISP1FrameInfo *RkISP1Frames::find(FrameBuffer *buffer) -{ - for (auto &[frame, info] : frameInfo_) { - if (info.paramBuffer == buffer || - info.statBuffer == buffer || - info.mainPathBuffer == buffer || - info.selfPathBuffer == buffer) - return &info; - } - - LOG(RkISP1, Fatal) << "Can't locate info from buffer"; - - return nullptr; -} - -RkISP1FrameInfo *RkISP1Frames::find(Request *request) -{ - for (auto &[frame, info] : frameInfo_) { - if (info.request == request) - return &info; - } + Camera *activeCamera_; +}; - LOG(RkISP1, Fatal) << "Can't locate info from request"; - return nullptr; -} PipelineHandlerRkISP1 *RkISP1CameraData::pipe() { @@ -504,44 +407,95 @@ int RkISP1CameraData::loadTuningFile(const std::string &path) void RkISP1CameraData::paramsComputed(unsigned int frame, unsigned int bytesused) { PipelineHandlerRkISP1 *pipe = RkISP1CameraData::pipe(); - RkISP1FrameInfo *info = frameInfo_.find(frame); - if (!info) - return; + ParamBufferInfo &pInfo = pipe->computingParamBuffers_.front(); + pipe->computingParamBuffers_.pop(); + + ASSERT(pInfo.expectedSequence == frame); + FrameBuffer *buffer = pInfo.buffer; - info->paramBuffer->_d()->metadata().planes()[0].bytesused = bytesused; + LOG(RkISP1Schedule, Debug) << "Queue params for " << frame << " " << buffer; - int ret = pipe->param_->queueBuffer(info->paramBuffer); + buffer->_d()->metadata().planes()[0].bytesused = bytesused; + int ret = pipe->param_->queueBuffer(buffer); if (ret < 0) { LOG(RkISP1, Error) << "Failed to queue parameter buffer: " << strerror(-ret); + pipe->availableParamBuffers_.push(buffer); return; } - pipe->stat_->queueBuffer(info->statBuffer); - - if (info->mainPathBuffer) - mainPath_->queueBuffer(info->mainPathBuffer); - - if (selfPath_ && info->selfPathBuffer) - selfPath_->queueBuffer(info->selfPathBuffer); + pipe->queuedParamBuffers_.push({ buffer, frame }); } void RkISP1CameraData::setSensorControls(unsigned int frame, const ControlList &sensorControls) { + /* We know delayed controls is prewarmed for frame 0 */ + if (frame == 0) + return; + + LOG(RkISP1Schedule, Debug) << "DelayedControls push " << frame; delayedCtrls_->push(frame, sensorControls); } void RkISP1CameraData::metadataReady(unsigned int frame, const ControlList &metadata) { - RkISP1FrameInfo *info = frameInfo_.find(frame); - if (!info) - return; + PipelineHandlerRkISP1 *pipe = RkISP1CameraData::pipe(); + + LOG(RkISP1Schedule, Debug) << " metadataReady " << frame; - info->request->metadata().merge(metadata); - info->metadataProcessed = true; + auto &info = pipe->sensorFrameInfos_[frame]; - pipe()->tryCompleteRequest(info); + /* + * We don't necessarily know the request for that sequence number, + * as the dequeue of the image buffer might not have happened yet. + * So we check all known requests and store the metadata otherwise. + */ + for (auto &reqInfo : pipe->queuedRequests_) { + if (!reqInfo.sequenceValid) { + LOG(RkISP1Schedule, Debug) + << "Need to store metadata for later " << frame; + info.metadata = metadata; + break; + } + + if (frame > reqInfo.sequence) { + /* + * We will never get stats for that request. Log an + * error and return it. + */ + LOG(RkISP1, Warning) + << "Stats for frame " << reqInfo.sequence + << " got lost"; + auto &info2 = pipe->sensorFrameInfos_[reqInfo.sequence]; + info2.metadataProcessed = true; + ASSERT(info2.statsBuffer == nullptr); + continue; + } + + if (frame == reqInfo.sequence) { + reqInfo.request->metadata().merge(metadata); + break; + } + + /* We should never end up here */ + LOG(RkISP1, Error) << "Request for sequence " << frame + << " is already handled. Metadata was too late"; + + break; + } + + info.metadataProcessed = true; + /* + * info.statsBuffer can be null, if ipa->processStats() was called + * without a buffer to just fill the metadata. + */ + if (info.statsBuffer) + pipe->availableStatBuffers_.push(info.statsBuffer); + info.statsBuffer = nullptr; + + pipe->tryCompleteRequests(); + pipe->queueInternalBuffers(); } /* ----------------------------------------------------------------------------- @@ -810,7 +764,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() */ PipelineHandlerRkISP1::PipelineHandlerRkISP1(CameraManager *manager) - : PipelineHandler(manager, kRkISP1MaxQueuedRequests), hasSelfPath_(true) + : PipelineHandler(manager), hasSelfPath_(true) { } @@ -1189,18 +1143,19 @@ int PipelineHandlerRkISP1::allocateBuffers(Camera *camera) } }; if (!isRaw_) { - ret = param_->allocateBuffers(kRkISP1MinBufferCount, ¶mBuffers_); + ret = param_->allocateBuffers(kRkISP1InternalBufferCount, ¶mBuffers_); if (ret < 0) return ret; - ret = stat_->allocateBuffers(kRkISP1MinBufferCount, &statBuffers_); + ret = stat_->allocateBuffers(kRkISP1InternalBufferCount, &statBuffers_); if (ret < 0) return ret; } /* If the dewarper is being used, allocate internal buffers for ISP. */ if (data->usesDewarper_) { - ret = mainPath_.exportBuffers(kRkISP1MinBufferCount, &mainPathBuffers_); + ret = mainPath_.exportBuffers(kRkISP1DewarpImageBufferCount, + &mainPathBuffers_); if (ret < 0) return ret; @@ -1208,7 +1163,7 @@ int PipelineHandlerRkISP1::allocateBuffers(Camera *camera) availableMainPathBuffers_.push(buffer.get()); if (dewarper_->supportsRequests()) { - ret = dewarper_->allocateRequests(kRkISP1MinBufferCount, + ret = dewarper_->allocateRequests(kRkISP1DewarpImageBufferCount + 1, &dewarpRequests_); if (ret < 0) LOG(RkISP1, Error) << "Failed to allocate requests."; @@ -1308,6 +1263,10 @@ int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]] const ControlL data->delayedCtrls_->reset(); data->frame_ = 0; + nextParamsSequence_ = 0; + nextStatsToProcess_ = 0; + paramsSyncHelper_.reset(); + imageSyncHelper_.reset(); if (!isRaw_) { ret = param_->streamOn(); @@ -1364,6 +1323,9 @@ int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]] const ControlL isp_->setFrameStartEnabled(true); activeCamera_ = camera; + running_ = true; + + queueInternalBuffers(); actions.release(); return 0; @@ -1373,6 +1335,9 @@ void PipelineHandlerRkISP1::stopDevice(Camera *camera) { RkISP1CameraData *data = cameraData(camera); int ret; + running_ = false; + + LOG(RkISP1Schedule, Debug) << "Stop device"; isp_->setFrameStartEnabled(false); @@ -1393,40 +1358,153 @@ void PipelineHandlerRkISP1::stopDevice(Camera *camera) LOG(RkISP1, Warning) << "Failed to stop parameters for " << camera->id(); + /* + * The param buffers are not returned in order, so the queue + * becomes useless. + */ + queuedParamBuffers_ = {}; + if (data->usesDewarper_) dewarper_->stop(); } - ASSERT(data->queuedRequests_.empty()); - data->frameInfo_.clear(); + tryCompleteRequests(); + + /* There can still be requests that are either waiting for metadata + or that contain buffers which were not yet queued at all. */ + while (!queuedRequests_.empty()) { + RequestInfo &reqInfo = queuedRequests_.front(); + cancelRequest(reqInfo.request); + queuedRequests_.pop_front(); + } + sensorFrameInfos_.clear(); + + ASSERT(queuedDewarpBuffers_.empty()); + ASSERT(queuedParamBuffers_.empty()); + ASSERT(computingParamBuffers_.empty()); freeBuffers(camera); activeCamera_ = nullptr; } -int PipelineHandlerRkISP1::queueRequestDevice(Camera *camera, Request *request) +void PipelineHandlerRkISP1::queueInternalBuffers() { - RkISP1CameraData *data = cameraData(camera); + if (!running_) + return; - RkISP1FrameInfo *info = data->frameInfo_.create(data, request, isRaw_); - if (!info) - return -ENOENT; + RkISP1CameraData *data = cameraData(activeCamera_); + + while (!availableStatBuffers_.empty()) { + FrameBuffer *buf = availableStatBuffers_.front(); + availableStatBuffers_.pop(); + data->pipe()->stat_->queueBuffer(buf); + } - data->ipa_->queueRequest(data->frame_, request->controls()); + /* + * In case of the dewarper, there is a seperate buffer loop for the main + * path + */ + while (!availableMainPathBuffers_.empty()) { + FrameBuffer *buf = availableMainPathBuffers_.front(); + availableMainPathBuffers_.pop(); + + LOG(RkISP1Schedule, Debug) << "Queue mainPath " << buf; + data->mainPath_->queueBuffer(buf); + } +} + +void PipelineHandlerRkISP1::computeParamBuffers(uint32_t maxSequence) +{ + RkISP1CameraData *data = cameraData(activeCamera_); if (isRaw_) { - if (info->mainPathBuffer) - data->mainPath_->queueBuffer(info->mainPathBuffer); + /* + * Call computeParams with an empty param buffer to trigger + * the setSensorControls signal. + */ + data->ipa_->computeParams(maxSequence, 0); + return; + } - if (data->selfPath_ && info->selfPathBuffer) - data->selfPath_->queueBuffer(info->selfPathBuffer); - } else { - data->ipa_->computeParams(data->frame_, - info->paramBuffer->cookie()); + while (nextParamsSequence_ <= maxSequence) { + if (availableParamBuffers_.empty()) { + LOG(RkISP1Schedule, Warning) + << "Ran out of parameter buffers"; + return; + } + + int correction = paramsSyncHelper_.correction(); + if (correction != 0) + LOG(RkISP1Schedule, Warning) + << "Correcting params sequence " + << correction; + + uint32_t paramsSequence; + if (correction >= 0) { + nextParamsSequence_ += correction; + paramsSyncHelper_.pushCorrection(correction); + paramsSequence = nextParamsSequence_++; + } else { + /* + * Inject the same sequence multiple times, to correct + * for the offset. + */ + paramsSyncHelper_.pushCorrection(-1); + paramsSequence = nextParamsSequence_; + } + + FrameBuffer *buf = availableParamBuffers_.front(); + availableParamBuffers_.pop(); + computingParamBuffers_.push({ buf, paramsSequence }); + LOG(RkISP1Schedule, Debug) << "Request params for " << paramsSequence; + data->ipa_->computeParams(paramsSequence, buf->cookie()); } +} +int PipelineHandlerRkISP1::queueRequestDevice(Camera *camera, Request *request) +{ + RkISP1CameraData *data = cameraData(camera); + + RequestInfo info; + info.request = request; + + int correction = imageSyncHelper_.correction(); + if (correction != 0) + LOG(RkISP1Schedule, Debug) + << "Correcting image sequence " + << data->frame_ << " to " << data->frame_ + correction; + data->frame_ += correction; + imageSyncHelper_.pushCorrection(correction); + info.sequence = data->frame_; data->frame_++; + LOG(RkISP1Schedule, Debug) << "Queue request. Request sequence: " + << request->sequence() + << " estimated sensor frame sequence: " << info.sequence + << " queue size: " << (queuedRequests_.size() + 1); + + data->ipa_->queueRequest(info.sequence, request->controls()); + + /* + * When the dewarper is used, the request buffers will be queued in + * imageBufferReady() + */ + if (!data->usesDewarper_) { + FrameBuffer *mainPathBuffer = request->findBuffer(&data->mainPathStream_); + FrameBuffer *selfPathBuffer = request->findBuffer(&data->selfPathStream_); + if (mainPathBuffer) + data->mainPath_->queueBuffer(mainPathBuffer); + + if (data->selfPath_ && selfPathBuffer) + data->selfPath_->queueBuffer(selfPathBuffer); + } + + queuedRequests_.push_back(info); + + /* Kickstart computation of parameters. */ + if (info.sequence < kRkISP1InternalBufferCount) + computeParamBuffers(info.sequence); + return 0; } @@ -1626,12 +1704,11 @@ void PipelineHandlerRkISP1::frameStart(uint32_t sequence) return; RkISP1CameraData *data = cameraData(activeCamera_); + LOG(RkISP1Schedule, Debug) << "frameStart " << sequence; uint32_t sequenceToApply = sequence + data->delayedCtrls_->maxDelay(); data->delayedCtrls_->applyControls(sequenceToApply); - if (isRaw_) { - data->ipa_->computeParams(sequenceToApply + 1, 0); - } + computeParamBuffers(sequenceToApply + 1); } bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) @@ -1724,29 +1801,51 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) * Buffer Handling */ -void PipelineHandlerRkISP1::tryCompleteRequest(RkISP1FrameInfo *info) +void PipelineHandlerRkISP1::tryCompleteRequests() { - RkISP1CameraData *data = cameraData(activeCamera_); - Request *request = info->request; + std::optional lastDeletedSequence; - if (request->hasPendingBuffers()) - return; + /* Complete finished requests */ + while (!queuedRequests_.empty()) { + RequestInfo info = queuedRequests_.front(); - if (!info->metadataProcessed) - return; + if (info.request->hasPendingBuffers()) + break; + + if (!info.sequenceValid) + break; + + if (!sensorFrameInfos_[info.sequence].metadataProcessed) + break; + + queuedRequests_.pop_front(); + + LOG(RkISP1Schedule, Debug) << "Complete request " << info.sequence; + completeRequest(info.request); + + sensorFrameInfos_[info.sequence].request = nullptr; + lastDeletedSequence = info.sequence; + } - if (!isRaw_ && !info->paramDequeued) + if (!lastDeletedSequence.has_value()) return; - data->frameInfo_.destroy(info->frame); + /* Drop all outdated sensor frame infos. */ + while (!sensorFrameInfos_.empty()) { + auto iter = sensorFrameInfos_.begin(); + if (iter->first > lastDeletedSequence.value()) + break; + + ASSERT(iter->second.request == nullptr); + ASSERT(iter->second.statsBuffer == nullptr); - completeRequest(request); + sensorFrameInfos_.erase(iter); + } } -void PipelineHandlerRkISP1::cancelDewarpRequest(RkISP1FrameInfo *info) +void PipelineHandlerRkISP1::cancelDewarpRequest(Request *request) { RkISP1CameraData *data = cameraData(activeCamera_); - Request *request = info->request; /* * i.MX8MP is the only known platform with dewarper. It has * no self path. Hence, only main path buffer completion is @@ -1765,51 +1864,109 @@ void PipelineHandlerRkISP1::cancelDewarpRequest(RkISP1FrameInfo *info) } } - tryCompleteRequest(info); + tryCompleteRequests(); } void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) { ASSERT(activeCamera_); RkISP1CameraData *data = cameraData(activeCamera_); + const FrameMetadata &metadata = buffer->metadata(); + RequestInfo *reqInfo = nullptr; - RkISP1FrameInfo *info = data->frameInfo_.find(buffer); - if (!info) - return; + /* + * When the dewarper is used, the buffer is not yet tied to a request, + * so find the first request without a valid sequence. Otherwise find + * the request for that buffer. This is not necessarily the same, + * because after streamoff the buffers are returned in arbitrary order. + */ + for (auto &info : queuedRequests_) { + if (data->usesDewarper_) { + if (!info.sequenceValid) { + reqInfo = &info; + break; + } + } else { + if (info.request == buffer->request()) { + reqInfo = &info; + } + } + } - const FrameMetadata &metadata = buffer->metadata(); - Request *request = info->request; + if (!reqInfo) { + if (data->usesDewarper_) { + LOG(RkISP1Schedule, Info) + << "Image buffer ready, but no corresponding request"; + availableMainPathBuffers_.push(buffer); + return; + } + + LOG(RkISP1Schedule, Fatal) + << "Image buffer ready, but no corresponding request"; + } + + Request *request = reqInfo->request; + + LOG(RkISP1Schedule, Debug) << "Image buffer ready: " << buffer + << " Expected sequence: " << reqInfo->sequence + << " got: " << metadata.sequence; + + uint32_t sequence = metadata.sequence; + + /* + * If the frame was cancelled, the metadata sequnce is usually wrong and + * we assume that our guess was right. + */ + if (metadata.status == FrameMetadata::FrameCancelled) + sequence = reqInfo->sequence; + + /* We now know the buffer sequence that belongs to this request */ + int droppedFrames = imageSyncHelper_.gotFrame(reqInfo->sequence, sequence); + if (droppedFrames != 0) + LOG(RkISP1Schedule, Warning) + << "Frame " << reqInfo->sequence << ": Dropped " + << droppedFrames << " frames"; + + reqInfo->sequence = sequence; + reqInfo->sequenceValid = true; + + if (sensorFrameInfos_[sequence].metadataProcessed) { + LOG(RkISP1Schedule, Debug) + << "Apply stored metadata " << reqInfo->sequence; + request->metadata().merge(sensorFrameInfos_[sequence].metadata); + } if (metadata.status != FrameMetadata::FrameCancelled) { /* * Record the sensor's timestamp in the request metadata. * - * \todo The sensor timestamp should be better estimated by connecting - * to the V4L2Device::frameStart signal. + * \todo The sensor timestamp should be better estimated by + * connecting to the V4L2Device::frameStart signal. */ request->metadata().set(controls::SensorTimestamp, metadata.timestamp); + /* In raw mode call processStats() to fill the metadata */ if (isRaw_) { const ControlList &ctrls = data->delayedCtrls_->get(metadata.sequence); - data->ipa_->processStats(info->frame, 0, ctrls); + data->ipa_->processStats(reqInfo->sequence, 0, ctrls); } } else { - if (isRaw_) - info->metadataProcessed = true; + /* No need to block waiting for metedata on that frame. */ + sensorFrameInfos_[sequence].metadataProcessed = true; } if (!data->usesDewarper_) { - completeBuffer(request, buffer); - tryCompleteRequest(info); + completeBuffer(reqInfo->request, buffer); + tryCompleteRequests(); return; } - /* Do not queue cancelled frames to dewarper. */ + /* Do not queue cancelled frames to the dewarper. */ if (metadata.status == FrameMetadata::FrameCancelled) { - cancelDewarpRequest(info); + cancelDewarpRequest(reqInfo->request); return; } @@ -1869,10 +2026,14 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) dewarper_->applyVertexMap(&data->mainPathStream_, dewarpRequest); /* - * Queue input and output buffers to the dewarper. The output - * buffers for the dewarper are the buffers of the request, supplied - * by the application. + * Queue input and output buffers to the dewarper. The output buffers + * for the dewarper are the buffers of the request, supplied by the + * application. */ + DewarpBufferInfo dewarpInfo{ buffer, reqInfo->request->findBuffer(&data->mainPathStream_) }; + queuedDewarpBuffers_.push_back(dewarpInfo); + LOG(RkISP1Schedule, Debug) << "Queue dewarper " << dewarpInfo.inputBuffer + << " " << dewarpInfo.outputBuffer; int ret = dewarper_->queueBuffers(buffer, request->buffers(), dewarpRequest); if (ret < 0) { LOG(RkISP1, Error) << "Failed to queue buffers to dewarper: -" @@ -1882,7 +2043,7 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) if (dewarpRequest) dewarpRequestReady(dewarpRequest); - cancelDewarpRequest(info); + cancelDewarpRequest(reqInfo->request); return; } @@ -1895,7 +2056,7 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) /* Push it back into the queue. */ dewarpRequestReady(dewarpRequest); - cancelDewarpRequest(info); + cancelDewarpRequest(reqInfo->request); } } @@ -1919,29 +2080,59 @@ void PipelineHandlerRkISP1::dewarpRequestReady(V4L2Request *request) void PipelineHandlerRkISP1::dewarpBufferReady(FrameBuffer *buffer) { - ASSERT(activeCamera_); - RkISP1CameraData *data = cameraData(activeCamera_); Request *request = buffer->request(); + const FrameMetadata &metadata = buffer->metadata(); - RkISP1FrameInfo *info = data->frameInfo_.find(buffer->request()); - if (!info) - return; + /* + * After stopping the dewarper, the buffers are returned out of order. + * Search the list for the corresponding info and handle it. In regular + * operation it will always be the first entry. + */ + for (DewarpBufferInfo &dwInfo : queuedDewarpBuffers_) { + if (dwInfo.outputBuffer != buffer) + continue; - completeBuffer(request, buffer); - tryCompleteRequest(info); + availableMainPathBuffers_.push(dwInfo.inputBuffer); + dwInfo.inputBuffer = nullptr; + dwInfo.outputBuffer = nullptr; + + if (metadata.status == FrameMetadata::FrameCancelled) + buffer->_d()->cancel(); + + completeBuffer(request, buffer); + } + + while (!queuedDewarpBuffers_.empty() && + queuedDewarpBuffers_.front().inputBuffer == nullptr) + queuedDewarpBuffers_.pop_front(); + + tryCompleteRequests(); + queueInternalBuffers(); } void PipelineHandlerRkISP1::paramBufferReady(FrameBuffer *buffer) { - ASSERT(activeCamera_); - RkISP1CameraData *data = cameraData(activeCamera_); + LOG(RkISP1Schedule, Debug) << "Param buffer ready " << buffer; - RkISP1FrameInfo *info = data->frameInfo_.find(buffer); - if (!info) + /* After stream off, the buffers are returned out of order, so + * we don't care about the rest. + */ + if (!running_) { + availableParamBuffers_.push(buffer); return; + } - info->paramDequeued = true; - tryCompleteRequest(info); + ParamBufferInfo pInfo = queuedParamBuffers_.front(); + queuedParamBuffers_.pop(); + + ASSERT(pInfo.buffer == buffer); + + size_t metaSequence = buffer->metadata().sequence; + LOG(RkISP1Schedule, Debug) << "Params buffer ready " + << " Expected: " << pInfo.expectedSequence + << " got: " << metaSequence; + paramsSyncHelper_.gotFrame(pInfo.expectedSequence, metaSequence); + availableParamBuffers_.push(buffer); } void PipelineHandlerRkISP1::statBufferReady(FrameBuffer *buffer) @@ -1949,21 +2140,47 @@ void PipelineHandlerRkISP1::statBufferReady(FrameBuffer *buffer) ASSERT(activeCamera_); RkISP1CameraData *data = cameraData(activeCamera_); - RkISP1FrameInfo *info = data->frameInfo_.find(buffer); - if (!info) - return; + size_t sequence = buffer->metadata().sequence; if (buffer->metadata().status == FrameMetadata::FrameCancelled) { - info->metadataProcessed = true; - tryCompleteRequest(info); + LOG(RkISP1Schedule, Warning) << "Stats cancelled " << sequence; + /* + * We can't assume that the sequence of the stat buffer is valid, + * so there is nothing left to do. + */ + availableStatBuffers_.push(buffer); + return; + } + + LOG(RkISP1Schedule, Debug) << "Stats ready " << sequence; + + if (nextStatsToProcess_ != sequence) + LOG(RkISP1Schedule, Warning) << "Stats sequence out of sync." + << " Expected: " << nextStatsToProcess_ + << " got: " << sequence; + + if (nextStatsToProcess_ > sequence) { + LOG(RkISP1Schedule, Warning) << "Stats were too late. Ignored"; + availableStatBuffers_.push(buffer); return; } - if (data->frame_ <= buffer->metadata().sequence) - data->frame_ = buffer->metadata().sequence + 1; + /* Send empty stats to ensure metadata gets created*/ + while (nextStatsToProcess_ < sequence) { + LOG(RkISP1Schedule, Warning) << "Send empty stats to fill metadata"; + data->ipa_->processStats(nextStatsToProcess_, 0, + data->delayedCtrls_->get(nextStatsToProcess_)); + + nextStatsToProcess_++; + } + + nextStatsToProcess_++; + + sensorFrameInfos_[sequence].statsBuffer = buffer; - data->ipa_->processStats(info->frame, info->statBuffer->cookie(), - data->delayedCtrls_->get(buffer->metadata().sequence)); + LOG(RkISP1Schedule, Debug) << "Process stats " << sequence; + data->ipa_->processStats(sequence, buffer->cookie(), + data->delayedCtrls_->get(sequence)); } REGISTER_PIPELINE_HANDLER(PipelineHandlerRkISP1, "rkisp1") From patchwork Fri Oct 24 08:50:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24794 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3E53EBE080 for ; Fri, 24 Oct 2025 08:53:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id F3D5E60904; Fri, 24 Oct 2025 10:53:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="dVEwnGoi"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 51BD060904 for ; Fri, 24 Oct 2025 10:53:03 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id D1A2B4B0C; Fri, 24 Oct 2025 10:51:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295877; bh=aXgtbQH79QP9gzpvn2U4Rsbd91eeA3SDfppS/QzkxFM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dVEwnGoi/Dgv7TubQQJ4BwWi203wX6VaQlMXanXjI77H/GQzKVlV6A3qzrIVbkd55 ukmb/JpGR4STbi95n2MAUHl7YlNPZbWM+KA0CV1b70X+k6mIrj4d1YkKK+ZqlyedDq DrVtj+5uykdaJObo7nSwJBjNSWbVatLw5VP+dlxo= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 28/35] pipline: rkisp1: Reinstantiate maxQueuedRequestsDevice limit Date: Fri, 24 Oct 2025 10:50:52 +0200 Message-ID: <20251024085130.995967-29-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the pipeline rework, the maxQueuedRequestsDevice should not be necessary anymore, as prepare() and therefore the calculation for the ISP regulation is called only as late as possible when a params buffer was dequeued. However with unlimited maxQueuedRequestsDevice all the incoming requests get immediately queued to the ipa with the sensor sequence number that was anticipated for that request at queueRequestDevice time. Now when the correction tries to mitigate dropped sequence numbers, it will call computeParams() with sensor frame numbers that were not anticipated for the requests queued to the IPA. There still might be a better solution to this, but reinstantiating the limit reduces the effect. Signed-off-by: Stefan Klug --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index cd9364cb8950..2d78721be63a 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -764,7 +764,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate() */ PipelineHandlerRkISP1::PipelineHandlerRkISP1(CameraManager *manager) - : PipelineHandler(manager), hasSelfPath_(true) + : PipelineHandler(manager, kRkISP1MinBufferCount), hasSelfPath_(true) { } From patchwork Fri Oct 24 08:50:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24795 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B7670BE080 for ; Fri, 24 Oct 2025 08:53:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 78DD660947; Fri, 24 Oct 2025 10:53:07 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="lcGytAZb"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 72EA860938 for ; Fri, 24 Oct 2025 10:53:06 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id D0D6B4B0C; Fri, 24 Oct 2025 10:51:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295880; bh=Ik+5kjY+0todhct0SOiC4RS7hv34hhfk2uGJKgSEv/Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lcGytAZbkR4mOY+Sja1lZLX98lKIdwVHuvNcJC+4YmGiXJZBEMlNw7be5S+EPdhLU LXqn+dJfya8Q854lAF4vYI7C7Sf1nggofEHSTpFqJiv05aZJLIUjzp4CFyTNUIGKMy SSTXtAyrr0TsUk0g0pylAzZlo8UJ3lVouqUtofJc= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 29/35] ipa: libipa: Reduce log level when obtaining an uninitialized frame context Date: Fri, 24 Oct 2025 10:50:53 +0200 Message-ID: <20251024085130.995967-30-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the pipeline rework, it is a valid condition to obtain an uninitialized frame context. Explain that in the comment and reduce the log level. The same applies to the "already initialized" case. Signed-off-by: Stefan Klug --- src/ipa/libipa/fc_queue.h | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/src/ipa/libipa/fc_queue.h b/src/ipa/libipa/fc_queue.h index 812022c496ed..75e0072a6eb4 100644 --- a/src/ipa/libipa/fc_queue.h +++ b/src/ipa/libipa/fc_queue.h @@ -64,7 +64,7 @@ public: * an error condition ? */ if (frame != 0 && frame <= frameContext.frame_) - LOG(FCQueue, Warning) + LOG(FCQueue, Debug) << "Frame " << frame << " already initialised"; else init(fc, frame); @@ -112,12 +112,12 @@ public: * The frame context has been retrieved before it was * initialised through the initialise() call. This indicates an * algorithm attempted to access a Frame context before it was - * queued to the IPA. Controls applied for this request may be - * left unhandled. - * - * \todo Set an error flag for per-frame control errors. + * queued to the IPA. This can happen when a frame was dropped + * in the kernel. Then one frame will not be queued for the + * requests to catch up, but the pipeline will still call + * prepare on that frame. */ - LOG(FCQueue, Warning) + LOG(FCQueue, Debug) << "Obtained an uninitialised FrameContext for " << frame; init(fc, frame); From patchwork Fri Oct 24 08:50:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24796 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 35801BE080 for ; Fri, 24 Oct 2025 08:53:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EAD2E60963; Fri, 24 Oct 2025 10:53:09 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="C9vcULdf"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EF85F60938 for ; Fri, 24 Oct 2025 10:53:08 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 71F5C4B17; Fri, 24 Oct 2025 10:51:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295883; bh=bDOfUmyAm/mN+cuZdw6wBlZilzXFIYveLc6YHvQBc4Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=C9vcULdfalzF9JVce1pIJNRrEANAeiwC3TMy7LST7eV3XGE9ftsfw1SMotCXnT5xC itreD+xqtnWsoIlBV4PIzKfg+UdEgciiCmk+RpLG+4D8KAgQnGfZbPz1odMPzJ9JrY z6iR8cgEdf/Rbqe/bgtHIvRX+2Njh09X2xkDN1PU= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 30/35] pipeline: rkisp1: Correctly handle params buffer for frame 0 Date: Fri, 24 Oct 2025 10:50:54 +0200 Message-ID: <20251024085130.995967-31-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The parameters for frame 0 are only active on frame 0 if the corresponding parameter buffer is queued before STREAMON is called on the isp. Therefore the normal mechanics of calling ipa->computeParams() do not work, as it gets called after the first request is queued and therefore the isp is already started. To fix that, handle the parameter buffer the same way the initial sensor controls are handled by passing it to the ipa->start() function and then queuing the params buffer before starting the isp. Signed-off-by: Stefan Klug --- include/libcamera/ipa/rkisp1.mojom | 5 +++- src/ipa/rkisp1/rkisp1.cpp | 31 +++++++++++++++++------- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 31 ++++++++++++++++++------ 3 files changed, 49 insertions(+), 18 deletions(-) diff --git a/include/libcamera/ipa/rkisp1.mojom b/include/libcamera/ipa/rkisp1.mojom index 4c29b53cd7f9..56c9fe8ab92a 100644 --- a/include/libcamera/ipa/rkisp1.mojom +++ b/include/libcamera/ipa/rkisp1.mojom @@ -17,6 +17,7 @@ struct IPAConfigInfo { struct StartResult { libcamera.ControlList controls; int32 code; + uint32 paramBufferBytesUsed; }; interface IPARkISP1Interface { @@ -25,7 +26,9 @@ interface IPARkISP1Interface { libcamera.IPACameraSensorInfo sensorInfo, libcamera.ControlInfoMap sensorControls) => (int32 ret, libcamera.ControlInfoMap ipaControls); - start(libcamera.ControlList controls) => (StartResult result); + start(libcamera.ControlList controls, + uint32 paramBufferId) + => (StartResult result); stop(); configure(IPAConfigInfo configInfo, diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index b714e2f10c65..a887214c63a9 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -56,7 +56,8 @@ public: const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls) override; - void start(const ControlList &controls, StartResult *result) override; + void start(const ControlList &controls, const uint32_t paramBufferId, + StartResult *result) override; void stop() override; int configure(const IPAConfigInfo &ipaConfig, @@ -77,6 +78,8 @@ protected: std::string logPrefix() const override; private: + uint32_t computeParamsInternal(IPAFrameContext &frameContext, const uint32_t bufferId); + void updateControls(const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, ControlInfoMap *ipaControls); @@ -216,10 +219,12 @@ int IPARkISP1::init(const IPASettings &settings, unsigned int hwRevision, return 0; } -void IPARkISP1::start(const ControlList &controls, StartResult *result) +void IPARkISP1::start(const ControlList &controls, const uint32_t paramBufferId, + StartResult *result) { IPAFrameContext frameContext = {}; initializeFrameContext(0, frameContext, controls); + result->paramBufferBytesUsed = computeParamsInternal(frameContext, paramBufferId); result->controls = getSensorControls(frameContext); result->code = 0; } @@ -373,6 +378,19 @@ void IPARkISP1::initializeFrameContext(const uint32_t frame, controlsToApply_.clear(); } +uint32_t IPARkISP1::computeParamsInternal(IPAFrameContext &frameContext, const uint32_t bufferId) +{ + ASSERT(bufferId); + + RkISP1Params params(context_.configuration.paramFormat, + mappedBuffers_.at(bufferId).planes()[0]); + + for (auto const &algo : algorithms()) + algo->prepare(context_, frameContext.frame(), frameContext, ¶ms); + + return params.size(); +} + void IPARkISP1::computeParams(const uint32_t frame, const uint32_t bufferId) { IPAFrameContext &frameContext = context_.frameContexts.get(frame); @@ -390,16 +408,11 @@ void IPARkISP1::computeParams(const uint32_t frame, const uint32_t bufferId) return; } - RkISP1Params params(context_.configuration.paramFormat, - mappedBuffers_.at(bufferId).planes()[0]); - - for (auto const &algo : algorithms()) - algo->prepare(context_, frame, frameContext, ¶ms); - + uint32_t size = computeParamsInternal(frameContext, bufferId); ControlList ctrls = getSensorControls(frameContext); setSensorControls.emit(frame, ctrls); - paramsComputed.emit(frame, params.size()); + paramsComputed.emit(frame, size); } void IPARkISP1::processStats(const uint32_t frame, const uint32_t bufferId, diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 2d78721be63a..1e39acc5cc37 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -110,8 +110,8 @@ public: bool canUseDewarper_; bool usesDewarper_; -private: void paramsComputed(unsigned int frame, unsigned int bytesused); +private: void setSensorControls(unsigned int frame, const ControlList &sensorControls); @@ -1251,23 +1251,38 @@ int PipelineHandlerRkISP1::start(Camera *camera, [[maybe_unused]] const ControlL if (!!controls) ctrls = *controls; + data->frame_ = 0; + nextParamsSequence_ = 0; + nextStatsToProcess_ = 0; + paramsSyncHelper_.reset(); + imageSyncHelper_.reset(); + + uint32_t paramBufferId = 0; + FrameBuffer *paramBuffer = nullptr; + if (!isRaw_) { + paramBuffer = availableParamBuffers_.front(); + paramBufferId = paramBuffer->cookie(); + } + ipa::rkisp1::StartResult res; - data->ipa_->start(ctrls, &res); + data->ipa_->start(ctrls, paramBufferId, &res); if (res.code) { LOG(RkISP1, Error) << "Failed to start IPA " << camera->id(); return ret; } + + if (paramBuffer) { + availableParamBuffers_.pop(); + computingParamBuffers_.push({ paramBuffer, nextParamsSequence_++ }); + paramsSyncHelper_.pushCorrection(0); + data->paramsComputed(0, res.paramBufferBytesUsed); + } + actions += [&]() { data->ipa_->stop(); }; data->sensor_->setControls(&res.controls); data->delayedCtrls_->reset(); - data->frame_ = 0; - nextParamsSequence_ = 0; - nextStatsToProcess_ = 0; - paramsSyncHelper_.reset(); - imageSyncHelper_.reset(); - if (!isRaw_) { ret = param_->streamOn(); if (ret) { From patchwork Fri Oct 24 08:50:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24797 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B0998BE080 for ; Fri, 24 Oct 2025 08:53:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7222060963; Fri, 24 Oct 2025 10:53:14 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BSXGtF0o"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 961AB60938 for ; Fri, 24 Oct 2025 10:53:12 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 069E91E27; Fri, 24 Oct 2025 10:51:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295887; bh=898GOrfowMvV5540+U/3FGa9KzWe5XYFr2nT2pcHhe4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BSXGtF0oDStKz4MaCpWa+ArHQhfQJua9QGReH4PN00sXvwWNlPf8WmAWeu05SROqi Ppq18Xk4+oE7UAnT9Zp32T/Io+scjM1KuoV3/Ra1aG+rxn0cfoKo8On6wuyoEijqTN UlQPsIW43jDJupUu3whxUJcRyy2K01VOQdM88RSc= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 31/35] WIP ipa: rkisp1: Post quantization gain as digital gain in metadata Date: Fri, 24 Oct 2025 10:50:55 +0200 Message-ID: <20251024085130.995967-32-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Signed-off-by: Stefan Klug --- src/ipa/rkisp1/algorithms/agc.cpp | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 7c9538f95cbe..7629066b9876 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -468,6 +468,8 @@ void Agc::fillMetadata(IPAContext &context, IPAFrameContext &frameContext, metadata.set(controls::AeExposureMode, frameContext.agc.exposureMode); metadata.set(controls::AeConstraintMode, frameContext.agc.constraintMode); metadata.set(controls::ExposureValue, frameContext.agc.exposureValue); + + metadata.set(controls::DigitalGain, frameContext.agc.quantizationGain); } /** From patchwork Fri Oct 24 08:50:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24798 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 37722BE080 for ; Fri, 24 Oct 2025 08:53:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EF71960963; Fri, 24 Oct 2025 10:53:16 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="V6xCDXc7"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 069A460938 for ; Fri, 24 Oct 2025 10:53:16 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 6978A4B21; Fri, 24 Oct 2025 10:51:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295890; bh=I0b9uqNMOZGYBj1FqNrnQcqrUz1g8S0QfRsDzYkpU/M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=V6xCDXc7MK8pTxyZ7Q5+BgPsu5Kds/CAX5iyJNCzg+wUK5QEFfZoacX92ChzD9psM hfVqR1YXEi0VJwd6ERkzPdIDkPyvR38CHyAL/6CreH9vH4IdehF+GISH0K7OB2asyy VKdLqU3XKgbwLe4tRm4cu0WP95RA6MiSniNz9HRU= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 32/35] WIP: rkisp1: agc: Add digital gain Date: Fri, 24 Oct 2025 10:50:56 +0200 Message-ID: <20251024085130.995967-33-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a manual only digital gain control. Signed-off-by: Stefan Klug --- src/ipa/rkisp1/algorithms/agc.cpp | 22 ++++++++++++++++++++-- src/ipa/rkisp1/ipa_context.h | 2 ++ 2 files changed, 22 insertions(+), 2 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 7629066b9876..71ad3a01d90c 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -158,6 +158,7 @@ int Agc::init(IPAContext &context, const YamlObject &tuningData) /* \todo Move this to the Camera class */ context.ctrlMap[&controls::AeEnable] = ControlInfo(false, true, true); context.ctrlMap[&controls::ExposureValue] = ControlInfo(-8.0f, 8.0f, 0.0f); + context.ctrlMap[&controls::DigitalGain] = ControlInfo(1.0f, 64.0f, 1.0f); context.ctrlMap.merge(controls()); return 0; @@ -179,6 +180,7 @@ int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) context.activeState.agc.automatic.quantizationGain = 1.0; context.activeState.agc.manual.gain = context.activeState.agc.automatic.gain; context.activeState.agc.manual.exposure = context.activeState.agc.automatic.exposure; + context.activeState.agc.manual.digitalGain = 1.0; context.activeState.agc.autoExposureEnabled = !context.configuration.raw; context.activeState.agc.autoGainEnabled = !context.configuration.raw; context.activeState.agc.exposureValue = 0.0; @@ -281,6 +283,15 @@ void Agc::queueRequest(IPAContext &context, LOG(RkISP1Agc, Debug) << "Set gain to " << agc.manual.gain; } + const auto &digitalGain = controls.get(controls::DigitalGain); + if (digitalGain) { + agc.manual.digitalGain = *digitalGain; + + LOG(RkISP1Agc, Debug) << "Set digital gain to " << agc.manual.digitalGain; + } + + frameContext.agc.digitalGain = agc.manual.digitalGain; + frameContext.agc.autoExposureEnabled = agc.autoExposureEnabled; frameContext.agc.autoGainEnabled = agc.autoGainEnabled; @@ -381,7 +392,7 @@ void Agc::prepare(IPAContext &context, const uint32_t frame, if (context.configuration.compress.supported) { frameContext.compress.enable = true; - frameContext.compress.gain = frameContext.agc.quantizationGain; + frameContext.compress.gain = frameContext.agc.quantizationGain * frameContext.agc.digitalGain; } frameContext.agc.yTarget = context.activeState.agc.automatic.yTarget; @@ -469,7 +480,14 @@ void Agc::fillMetadata(IPAContext &context, IPAFrameContext &frameContext, metadata.set(controls::AeConstraintMode, frameContext.agc.constraintMode); metadata.set(controls::ExposureValue, frameContext.agc.exposureValue); - metadata.set(controls::DigitalGain, frameContext.agc.quantizationGain); + /* + * \todo Add the quantization gain to the digital gain metadata, so that + * we don't apply any hidden gains. This might however create issues + * with applications trying to match a digital gain control with the + * metadata. + */ + metadata.set(controls::DigitalGain, + frameContext.agc.quantizationGain * frameContext.agc.digitalGain); } /** diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 185951fb8286..ba320b9e3e92 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -79,6 +79,7 @@ struct IPAActiveState { struct { uint32_t exposure; double gain; + double digitalGain; } manual; struct { uint32_t exposure; @@ -160,6 +161,7 @@ struct IPAFrameContext : public FrameContext { bool updateMetering; bool autoExposureModeChange; bool autoGainModeChange; + double digitalGain; } agc; struct { From patchwork Fri Oct 24 08:50:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24799 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2811FBE080 for ; Fri, 24 Oct 2025 08:53:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D33F260971; Fri, 24 Oct 2025 10:53:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="HO8MhuxT"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6D0D160938 for ; Fri, 24 Oct 2025 10:53:18 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id F04C51E46; Fri, 24 Oct 2025 10:51:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295893; bh=zcuiv9yXlvr31uOtzmfQAHWhJ9/2QeBPoCVD0DK3llI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HO8MhuxTuh4AjHTwdhdVf//Fh9tYgHvsNgXKi22jTtWbGnozriqnep/ZxnCpQb95k +ljmsLcn2OtTaRCotxHP600GgmqXGv4PyUGnjg6kjKyJ17QedmeTy8jWlKV7pOXZvf 9+aMeFbLCUmGEmHTwN0Bs0vm/oA4gRGDMNfjuCFc= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 33/35] libipa: agc_mean_luminance: Make startup frames and regulations speed configurable Date: Fri, 24 Oct 2025 10:50:57 +0200 Message-ID: <20251024085130.995967-34-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The rkisp1 has now a good synchronization between sensor settings/isp params and stats. It can therefore benefit from a faster regulation. To do that without breaking others IPAs that use AgcMeanLuminance, move regulation speed and startup frames into member variables that can be set at runtime. As this requires the introduction of a protected access modifier. This was added, so that most of the previously public functions are now also protected, because public visibility is not needed. Signed-off-by: Stefan Klug --- src/ipa/libipa/agc_mean_luminance.cpp | 25 +++++++++++++++++++++---- src/ipa/libipa/agc_mean_luminance.h | 4 ++++ 2 files changed, 25 insertions(+), 4 deletions(-) diff --git a/src/ipa/libipa/agc_mean_luminance.cpp b/src/ipa/libipa/agc_mean_luminance.cpp index 64f36bd75dd2..0cd3fbee99e8 100644 --- a/src/ipa/libipa/agc_mean_luminance.cpp +++ b/src/ipa/libipa/agc_mean_luminance.cpp @@ -34,7 +34,7 @@ namespace ipa { * Number of frames for which to run the algorithm at full speed, before slowing * down to prevent large and jarring changes in exposure from frame to frame. */ -static constexpr uint32_t kNumStartupFrames = 10; +static constexpr uint32_t kDefaultNumStartupFrames = 10; /* * Default relative luminance target @@ -144,7 +144,8 @@ static constexpr double kMaxRelativeLuminanceTarget = 0.95; */ AgcMeanLuminance::AgcMeanLuminance() - : exposureCompensation_(1.0), frameCount_(0), filteredExposure_(0s), + : numStartupFrames_(kDefaultNumStartupFrames), regulationSpeed_(0.2), + exposureCompensation_(1.0), frameCount_(0), filteredExposure_(0s), relativeLuminanceTarget_(0) { } @@ -554,10 +555,10 @@ double AgcMeanLuminance::effectiveYTarget() const */ utils::Duration AgcMeanLuminance::filterExposure(utils::Duration exposureValue) { - double speed = 0.2; + double speed = regulationSpeed_; /* Adapt instantly if we are in startup phase. */ - if (frameCount_ < kNumStartupFrames) + if (frameCount_ < numStartupFrames_) speed = 1.0; /* @@ -649,6 +650,22 @@ AgcMeanLuminance::calculateNewEv(uint32_t constraintModeIndex, * their configure() function. */ +/** + * \var AgcMeanLuminance::numStartupFrames_ + * \brief The number of startup frames + * + * During this number of frames after startup, the regulation is very aggressive + * to reach the target value within one or two cycles. + * + * \var AgcMeanLuminance::regulationSpeed_ + * The regulation speed. This controls the speed at which new target values are + * applied. The new target value is calculated as: + * + * \code{.unparsed} + * value = target * speed + oldValue * (1.0 - speed) + * \endcode + */ + } /* namespace ipa */ } /* namespace libcamera */ diff --git a/src/ipa/libipa/agc_mean_luminance.h b/src/ipa/libipa/agc_mean_luminance.h index d7ec548e3e58..b176c3e28410 100644 --- a/src/ipa/libipa/agc_mean_luminance.h +++ b/src/ipa/libipa/agc_mean_luminance.h @@ -42,6 +42,7 @@ public: double yTarget; }; +protected: void configure(utils::Duration lineDuration, const CameraSensorHelper *sensorHelper); int parseTuningData(const YamlObject &tuningData); @@ -79,6 +80,9 @@ public: frameCount_ = 0; } + uint32_t numStartupFrames_; + double regulationSpeed_; + private: virtual double estimateLuminance(const double gain) const = 0; From patchwork Fri Oct 24 08:50:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24800 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1D4DABE080 for ; Fri, 24 Oct 2025 08:53:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D42A06096A; Fri, 24 Oct 2025 10:53:21 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="c3Fz9iag"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D8C9960938 for ; Fri, 24 Oct 2025 10:53:20 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 61CFB21D0; Fri, 24 Oct 2025 10:51:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295895; bh=6jtGsTXG0bEb6iM/1wRCadoSIAITU2l4SiCkOBLmmng=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=c3Fz9iagT6AhALsIbEKdBg+cErkPWXglQeYiew+jSI9e4Tx+d3J4eARJU3HmP/dTi 8Fsv03AMvLwurrdyVHus6QemrcBT2pKayk2ZHzBKmhQlEqy7GHCe03v4b9pKVGHH/6 r0YkmLamdcC9Ip1fAD1+SY9jAPkGBD34NgTqf7o8= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 34/35] ipa: rkisp1: Increase regulation speed Date: Fri, 24 Oct 2025 10:50:58 +0200 Message-ID: <20251024085130.995967-35-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that the rkisp1 is well synchronized, we can safely increase the regulation speed and reduce the number of startup frames. With current settings, the results based on the stats for frame 0 are active on frame 5. Setting the startup frames to 7 leaves room for one additional lost frame. Signed-off-by: Stefan Klug --- src/ipa/rkisp1/algorithms/agc.cpp | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 71ad3a01d90c..e8f2324797d4 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -122,6 +122,9 @@ uint8_t Agc::computeHistogramPredivider(const Size &size, Agc::Agc() { supportsRaw_ = true; + /* rkisp1 is well synchronized, increase the speed. */ + regulationSpeed_ = 0.6; + numStartupFrames_ = 7; } /** From patchwork Fri Oct 24 08:50:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 24801 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9D7A6BE080 for ; Fri, 24 Oct 2025 08:53:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5C5BB6096D; Fri, 24 Oct 2025 10:53:26 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="jQeyCliT"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4670A60938 for ; Fri, 24 Oct 2025 10:53:24 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7edc:62f4:c118:1549]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id CF0184B2F; Fri, 24 Oct 2025 10:51:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761295898; bh=/Wkv+AiM4CpfjMe3gQqM7n+PNDpYpYob0YH93LUsqOU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jQeyCliTW9MFi6QjnU20IB27TyVxFYJUVcxBYMA8Dk3GZ62q4N3GHq7jTqANCup4q qRK4HfJBim9IiZAbemgyghrqAHsLwwOZX4NL0/ZH8jODRcqjM/Gx1h3UMmFlVVjjeC 1ie0e1s9xoD0M27zpIrMIhVpEDk/nUI15nrnFY+M= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 35/35] [WIP] Add imx335 delays Date: Fri, 24 Oct 2025 10:50:59 +0200 Message-ID: <20251024085130.995967-36-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20251024085130.995967-1-stefan.klug@ideasonboard.com> References: <20251024085130.995967-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" --- src/libcamera/sensor/camera_sensor_properties.cpp | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/libcamera/sensor/camera_sensor_properties.cpp b/src/libcamera/sensor/camera_sensor_properties.cpp index b217363d872a..fd1c59db9880 100644 --- a/src/libcamera/sensor/camera_sensor_properties.cpp +++ b/src/libcamera/sensor/camera_sensor_properties.cpp @@ -259,7 +259,12 @@ const CameraSensorProperties *CameraSensorProperties::get(const std::string &sen { "imx335", { .unitCellSize = { 2000, 2000 }, .testPatternModes = {}, - .sensorDelays = { }, + .sensorDelays = { + .exposureDelay = 2, + .gainDelay = 2, + .vblankDelay = 2, + .hblankDelay = 2 + } } }, { "imx415", { .unitCellSize = { 1450, 1450 },