From patchwork Thu Jul 21 12:12:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 16713 X-Patchwork-Delegate: kieran.bingham@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4F5BFBD1F1 for ; Thu, 21 Jul 2022 12:13:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2595C63326; Thu, 21 Jul 2022 14:13:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1658405598; bh=TvAEGOpGpmpFjfSWAB3ltiYQFcd0bvglEnKX0K7YKqE=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=PtIib24QC6crQ0v2D2dNzIjzkP/A9nOscASlRwLp6/809BSifhunvuo4ApzvvTMD8 1JBT6dt9617uZqSxgB3d4ay/9m5jOSb7PV0fny8yEpahQe2K5b7/WOtfjo8xox/Z1v a/iUGOgUpOOFp/yQXh3lK/+xpsryXUDMcmz3PMBNSmAQmqXFcZPSdfLkzRRnlj5M1Q A9fw9EENs5iOxXHguSSwFFy2Z9TvD9rRhEgGvAWUaoImEou64hfJNAGJcg0eouSrkZ c4OtAcwCfKs2A4atVb6hWfUWPh8Dh7KPLe978XbARO0C0LBlLsyco/YKYvURO6etcr p54yOiq86wMHA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1F266601B8 for ; Thu, 21 Jul 2022 14:13:14 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="I1jiVEjQ"; dkim-atps=neutral Received: from Monstersaurus.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 735D29B1; Thu, 21 Jul 2022 14:13:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1658405593; bh=TvAEGOpGpmpFjfSWAB3ltiYQFcd0bvglEnKX0K7YKqE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=I1jiVEjQFiNooDoHK6fzvxI0dTiyUTbTSpm9/d2Z6AYTTOl+AJDPxqCbEDxWIUgnl V/WkHZIy4aERoq1oEJCN3Ne3lguKKtaAPCE7fO4EbpjSitkrftxpbkzSvOxUqO1Gr+ QNRd/aqrZjZFGTw0tyPSB4OmLk8m19EmNfDpmAk0= To: libcamera devel Date: Thu, 21 Jul 2022 13:12:59 +0100 Message-Id: <20220721121310.1286862-2-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> References: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 01/12] libcamera: request: Add support for error flags X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Kieran Bingham via libcamera-devel From: Kieran Bingham Reply-To: Kieran Bingham Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Paul Elder Add error flags to the Request to indicate non-fatal errors. Applications should check the error() state of the Request to determine if any fault occured while processing the request. Initially, a single error flag ControlError is added to allow a pipeline handler to report a failure to set controls on a hardware device while processing the request. ControlErrors occur when a Request was asked to set a control and the pipeline handler attempted to do so, but it was rejected by the kernel. Signed-off-by: Paul Elder Signed-off-by: Kieran Bingham --- v2: - Add documentation for Request::Private::setErrorFlags --- v2 - Extend documentation Signed-off-by: Kieran Bingham --- include/libcamera/internal/request.h | 3 ++ include/libcamera/request.h | 9 ++++++ src/libcamera/request.cpp | 46 ++++++++++++++++++++++++++++ 3 files changed, 58 insertions(+) diff --git a/include/libcamera/internal/request.h b/include/libcamera/internal/request.h index 9dadd6c60361..8e592272cfae 100644 --- a/include/libcamera/internal/request.h +++ b/include/libcamera/internal/request.h @@ -38,6 +38,7 @@ public: void complete(); void cancel(); void reuse(); + void setErrorFlags(ErrorFlags flags); void prepare(std::chrono::milliseconds timeout = 0ms); Signal<> prepared; @@ -59,6 +60,8 @@ private: std::unordered_set pending_; std::map> notifiers_; std::unique_ptr timer_; + + ErrorFlags error_; }; } /* namespace libcamera */ diff --git a/include/libcamera/request.h b/include/libcamera/request.h index dffde1536cad..992629e11aa4 100644 --- a/include/libcamera/request.h +++ b/include/libcamera/request.h @@ -15,6 +15,7 @@ #include #include +#include #include #include @@ -43,6 +44,13 @@ public: ReuseBuffers = (1 << 0), }; + enum ErrorFlag { + NoError = 0, + ControlError = (1 << 0), + }; + + using ErrorFlags = Flags; + using BufferMap = std::map; Request(Camera *camera, uint64_t cookie = 0); @@ -60,6 +68,7 @@ public: uint32_t sequence() const; uint64_t cookie() const { return cookie_; } Status status() const { return status_; } + ErrorFlags error() const; bool hasPendingBuffers() const; diff --git a/src/libcamera/request.cpp b/src/libcamera/request.cpp index d2af1d2212ad..8b82757ea7e3 100644 --- a/src/libcamera/request.cpp +++ b/src/libcamera/request.cpp @@ -162,6 +162,7 @@ void Request::Private::cancel() */ void Request::Private::reuse() { + error_ = Request::NoError; sequence_ = 0; cancelled_ = false; prepared_ = false; @@ -284,6 +285,21 @@ void Request::Private::notifierActivated(FrameBuffer *buffer) emitPrepareCompleted(); } +/** + * \brief Update the error flags of the Request + * \param[in] flags Flags to apply on the Request + * + * Apply \a flags to the Request to report to the application when the Request + * completes. + * + * Setting an Error flag does not cause a Request to fail, and once set it can + * only be cleared by the application destroying the Request or calling reuse(). + */ +void Request::Private::setErrorFlags(ErrorFlags flags) +{ + error_ |= flags; +} + void Request::Private::timeout() { /* A timeout can only happen if there are fences not yet signalled. */ @@ -318,6 +334,22 @@ void Request::Private::timeout() * Reuse the buffers that were previously added by addBuffer() */ +/** + * \enum Request::ErrorFlag + * Flags to report non-fatal errors + * \var Request::NoError + * No error + * \var Request::ControlError + * Control Error. At least on control was not able to be applied to the device. + * The application should compare the metadata to the requested control values + * to check which controls weren't applied. + */ + +/** + * \typedef Request::ErrorFlags + * The error state of the request defined by \a Request::ErrorFlag + */ + /** * \typedef Request::BufferMap * \brief A map of Stream to FrameBuffer pointers @@ -560,6 +592,20 @@ uint32_t Request::sequence() const * \return The request completion status */ +/** + * \brief Retrieve the error flags + * + * The request could complete with non-fatal error. The completion status will + * indicate success. This function returns the non-fatal errors that the + * request completed with + * + * \return Flags of non-fatal errors that the request completed with + */ +Request::ErrorFlags Request::error() const +{ + return _d()->error_; +} + /** * \brief Check if a request has buffers yet to be completed * From patchwork Thu Jul 21 12:13:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 16714 X-Patchwork-Delegate: kieran.bingham@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 29ADCBD1F1 for ; Thu, 21 Jul 2022 12:13:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5D0436331F; Thu, 21 Jul 2022 14:13:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1658405599; bh=IGBbScQGZcbTQmRMFjagqowF/O41SSAC2/ooNjSvQ8s=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=ipieflcWKf22sWHughbEW0fjqI5QsUXAXUBfqn92LkrNjJvVQ+FTq9WAYQi3pDMWs cXAkcIGFduO2xluLQ61nnSdclLxQD4vFXrQzi9aN6zxUrsN2ERFI9bAxA7RXTqpnVr uVK1R8sKGqJavfs1moMoKpkGg0CWQ8XOoIu3oy1InKGpWEJTPqbFUZD6+TGIkpvhmH fb4+mkY8nzqUTqItEHcqeBSNPAsFPr4Uz/m8ABsNpM6BTBEsnqYMRJVf9w+wjJqlDa lHuRyObpzcWtXwThrCmiusGy9H1tiCFomR7v2rj85XpwgYh4VNCvPlXB20rIEf+FEP CIBqZo9lwAaqw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2DA826330E for ; Thu, 21 Jul 2022 14:13:14 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="A67ADneJ"; dkim-atps=neutral Received: from Monstersaurus.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BDB44A38; Thu, 21 Jul 2022 14:13:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1658405593; bh=IGBbScQGZcbTQmRMFjagqowF/O41SSAC2/ooNjSvQ8s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=A67ADneJ7E7I51SRRW22et8m0Kg3eptavGa1kNovPE9xrWSjEA85HdwPKcwx1D9bW oG98SsN/kiNcYCJnwiqBa4+LR/7M5ACBk5lI+478km0tET15It8nax2OeF1RNM/vw8 dkN8G6Nbximsr7OnSToUgjbzpbLwRG1ZBrSzkV4g= To: libcamera devel Date: Thu, 21 Jul 2022 13:13:00 +0100 Message-Id: <20220721121310.1286862-3-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> References: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 02/12] libcamera: pipeline: uvcvideo: Report control errors X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Kieran Bingham via libcamera-devel From: Kieran Bingham Reply-To: Kieran Bingham Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report an error when failing to process controls, but still allow the request to process and complete where possible. The Request ControlError flag is raised on the request. Bug: https://bugs.libcamera.org/show_bug.cgi?id=135 Signed-off-by: Kieran Bingham Reviewed-by: Umang Jain Reviewed-by: Jacopo Mondi --- src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp index 53b2f23ab029..1f282f26bec3 100644 --- a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp +++ b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp @@ -26,6 +26,7 @@ #include "libcamera/internal/device_enumerator.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/request.h" #include "libcamera/internal/sysfs.h" #include "libcamera/internal/v4l2_videodevice.h" @@ -373,8 +374,10 @@ int PipelineHandlerUVC::queueRequestDevice(Camera *camera, Request *request) } int ret = processControls(data, request); - if (ret < 0) - return ret; + if (ret < 0) { + LOG(UVC, Error) << "Failed to process controls"; + request->_d()->setErrorFlags(Request::ControlError); + } ret = data->video_->queueBuffer(buffer); if (ret < 0) From patchwork Thu Jul 21 12:13:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 16715 X-Patchwork-Delegate: kieran.bingham@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F3D5FBD1F1 for ; Thu, 21 Jul 2022 12:13:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 65BAB63322; Thu, 21 Jul 2022 14:13:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1658405600; bh=ixIcOvCneuW7zcPdTSGPoMyws/19Hw9cd4rAQBHTbD8=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=b9IBjbUAmRa9ZZ8VvH6MdJ4ZXtcPamD1HOkiHUPxGd+h6Gx1ihHCXsKQM2vkZOCOw 2OeBt7gnh0xjZ9rER9VTzaz7/fuyJp1YT1Wkp0YbMsJAR7BI5odTXATxwfsDzx2Qd5 WDuToG0PkHJqjDd2CVWihjVWi/dv0lNhHmwuQz+WzRCL7Y/An9BWqtX5IR1dske+MO BtKhZPH3Td0f16zUCYRX0SZhmx7xxw97fdpkDi66zNjmr3LY/PaY9vwm3AmxRGVaSW dvKXyAosIU66Qr9h7G6IrKd5/Wi+yJaHaEk1TWJRgusZRWMzT8/VaoWrgpVQ4CMfo0 lMQ5xexoUdl9g== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 56F2D6330F for ; Thu, 21 Jul 2022 14:13:14 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="WjvbgqFe"; dkim-atps=neutral Received: from Monstersaurus.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0683A825; Thu, 21 Jul 2022 14:13:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1658405594; bh=ixIcOvCneuW7zcPdTSGPoMyws/19Hw9cd4rAQBHTbD8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WjvbgqFeVLtpMlyECEZX6GgBwpnK1Ql00aQtC8mJEV9hHpwEC6QrXAtzd5lzCyGD9 tMXobYIysz8okWaQBTacyUEDNBtUrr6pLma1IgSF9b8FW9uVcbs1qvv+aNjtUcPPhv bmgCpI0IubtFRN/L9hiVo+DllxFz12WgxmkXdC34= To: libcamera devel Date: Thu, 21 Jul 2022 13:13:01 +0100 Message-Id: <20220721121310.1286862-4-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> References: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 03/12] libcamera: request: Add PFCError flag X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Kieran Bingham via libcamera-devel From: Kieran Bingham Reply-To: Kieran Bingham Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Provide a new flag for the Request error state to indicate that a per-frame control error has occurred. One or more controls set for this request could not be guaranteed on this frame. Signed-off-by: Kieran Bingham --- include/libcamera/request.h | 1 + src/libcamera/request.cpp | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/include/libcamera/request.h b/include/libcamera/request.h index 992629e11aa4..d6d9f0d214dc 100644 --- a/include/libcamera/request.h +++ b/include/libcamera/request.h @@ -47,6 +47,7 @@ public: enum ErrorFlag { NoError = 0, ControlError = (1 << 0), + PFCError = (1 << 1), }; using ErrorFlags = Flags; diff --git a/src/libcamera/request.cpp b/src/libcamera/request.cpp index 8b82757ea7e3..509ed4ca6ed5 100644 --- a/src/libcamera/request.cpp +++ b/src/libcamera/request.cpp @@ -339,10 +339,15 @@ void Request::Private::timeout() * Flags to report non-fatal errors * \var Request::NoError * No error + * * \var Request::ControlError * Control Error. At least on control was not able to be applied to the device. * The application should compare the metadata to the requested control values * to check which controls weren't applied. + * + * \var Request::PFCError + * A per-frame-control error has occured. Controls that were expected to be set + * during the processing of this request were not processed in time. */ /** From patchwork Thu Jul 21 12:13:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 16716 X-Patchwork-Delegate: kieran.bingham@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 919E7C3274 for ; Thu, 21 Jul 2022 12:13:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6A9B26330F; Thu, 21 Jul 2022 14:13:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1658405601; bh=8pIoV3OSe56tFqhGsij1IN86VZn7OmRaT4osmx2JqsE=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=Seu9IJQtFtaBNFLs3ZHM3MGHDyE69qPaSWqpkzLQUaHWQvZRpRN1GcYIWhkY/r+LK FXSI2l5jWKijcQKIIrkZc8r0Rd2NskfWiIYsAE4fR4/DiqgBrFbrVNW3s0yVMaRcDO jXEpB2AhJGRzunt/DM8e5EP+ZhuEuSikkOEUhWOSUKl4m8KzD856lvI9JRPe8ozwWq e2oSYn6yDheb1fLYPpDtDriKQrYnwMIhwcdh91SeVpbXEIiJ5S5V12liqI3adC62ZI AputkYmscsRmq3kdpbq+0pcz2mwIdSDoalZ43X1nsXX88/7OxV6qOd6sGSf3lbvMB4 5PrVBJXD66cJg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A2FF363310 for ; Thu, 21 Jul 2022 14:13:14 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BmHTd2Av"; dkim-atps=neutral Received: from Monstersaurus.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 45AE512F3; Thu, 21 Jul 2022 14:13:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1658405594; bh=8pIoV3OSe56tFqhGsij1IN86VZn7OmRaT4osmx2JqsE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BmHTd2AvCgPi1FBCQkckIKUBAaDDCwsUUj2XIivcmV8gmM6o9KeVYoiaL3sYKZoKT y5yKwDjqkaOzNBkaSfxvHkf6Mn19kAU2BXQkFM4OWdbZwgch4meF/Z2WXR7PsYQJtD DbmSlYQvHOMOGdjAdOG7ip/SCkxFTRQFuVKkX4UU= To: libcamera devel Date: Thu, 21 Jul 2022 13:13:02 +0100 Message-Id: <20220721121310.1286862-5-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> References: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 04/12] ipa: ipu3: Move the Frame Context queue to libipa X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Kieran Bingham via libcamera-devel From: Kieran Bingham Reply-To: Kieran Bingham Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Umang Jain Provide a direct FCQueue abstraction to facilitate creating a queue of Frame Context structures. The IPAFrameContext is moved back to the ipu3/ipa_context.h and a queue added to the IPU3, but the Algorithms are not yet moved from the single FrameContext exposed by the IPA context to use the correct one from the queue until the Algorithm interfaces are fully updated. Signed-off-by: Umang Jain Signed-off-by: Kieran Bingham --- src/ipa/ipu3/ipa_context.cpp | 17 ------- src/ipa/ipu3/ipa_context.h | 15 ++---- src/ipa/ipu3/ipu3.cpp | 13 +++-- src/ipa/libipa/fc_queue.cpp | 96 ++++++++++++++++++++++++++++++++++++ src/ipa/libipa/fc_queue.h | 93 ++++++++++++++++++++++++++++++++++ src/ipa/libipa/meson.build | 2 + 6 files changed, 204 insertions(+), 32 deletions(-) create mode 100644 src/ipa/libipa/fc_queue.cpp create mode 100644 src/ipa/libipa/fc_queue.h diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 13cdb835ef7f..dd139cb4c868 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -180,27 +180,10 @@ namespace libcamera::ipa::ipu3 { * struct ipu3_uapi_gamma_corr_lut for further details. */ -/** - * \brief Default constructor for IPAFrameContext - */ -IPAFrameContext::IPAFrameContext() = default; - -/** - * \brief Construct a IPAFrameContext instance - */ -IPAFrameContext::IPAFrameContext(uint32_t id, const ControlList &reqControls) - : frame(id), frameControls(reqControls) -{ - sensor = {}; -} - /** * \var IPAFrameContext::frame * \brief The frame number * - * \var IPAFrameContext::frameControls - * \brief Controls sent in by the application while queuing the request - * * \var IPAFrameContext::sensor * \brief Effective sensor values that were applied for the frame * diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 42e11141d3a1..193fc44a821a 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -8,8 +8,6 @@ #pragma once -#include - #include #include @@ -17,13 +15,12 @@ #include #include +#include + namespace libcamera { namespace ipa::ipu3 { -/* Maximum number of frame contexts to be held */ -static constexpr uint32_t kMaxFrameContexts = 16; - struct IPASessionConfiguration { struct { ipu3_uapi_grid_config bdsGrid; @@ -77,23 +74,19 @@ struct IPAActiveState { }; struct IPAFrameContext { - IPAFrameContext(); - IPAFrameContext(uint32_t id, const ControlList &reqControls); + uint32_t frame; struct { uint32_t exposure; double gain; } sensor; - - uint32_t frame; - ControlList frameControls; }; struct IPAContext { IPASessionConfiguration configuration; IPAActiveState activeState; - std::array frameContexts; + FCQueue frameContexts; }; } /* namespace ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 2f6bb672f7bb..55e82cd404f4 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -38,6 +38,8 @@ #include "algorithms/tone_mapping.h" #include "libipa/camera_sensor_helper.h" +#include "ipa_context.h" + /* Minimum grid width, expressed as a number of cells */ static constexpr uint32_t kMinGridWidth = 16; /* Maximum grid width, expressed as a number of cells */ @@ -456,8 +458,7 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo, /* Clean IPAActiveState at each reconfiguration. */ context_.activeState = {}; - IPAFrameContext initFrameContext; - context_.frameContexts.fill(initFrameContext); + context_.frameContexts.clear(); if (!validateSensorControls()) { LOG(IPAIPU3, Error) << "Sensor control validation failed."; @@ -569,7 +570,7 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, const ipu3_uapi_stats_3a *stats = reinterpret_cast(mem.data()); - IPAFrameContext &frameContext = context_.frameContexts[frame % kMaxFrameContexts]; + IPAFrameContext &frameContext = context_.frameContexts.get(frame); if (frameContext.frame != frame) LOG(IPAIPU3, Warning) << "Frame " << frame << " does not match its frame context"; @@ -618,7 +619,11 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, void IPAIPU3::queueRequest(const uint32_t frame, const ControlList &controls) { /* \todo Start processing for 'frame' based on 'controls'. */ - context_.frameContexts[frame % kMaxFrameContexts] = { frame, controls }; + IPAFrameContext &frameContext = context_.frameContexts.initialise(frame); + + /* \todo Implement queueRequest to each algorithm. */ + (void)frameContext; + (void)controls; } /** diff --git a/src/ipa/libipa/fc_queue.cpp b/src/ipa/libipa/fc_queue.cpp new file mode 100644 index 000000000000..ecb47f287350 --- /dev/null +++ b/src/ipa/libipa/fc_queue.cpp @@ -0,0 +1,96 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2022, Google Inc. + * + * fc_queue.cpp - IPA Frame context queue + */ + +#include "fc_queue.h" + +#include + +namespace libcamera { + +LOG_DEFINE_CATEGORY(FCQueue) + +namespace ipa { + +/** + * \file fc_queue.h + * \brief Queue to access per-frame Context + */ + +/** + * \class FCQueue + * \brief A support class for queueing Frame Context instances through the IPA + * \tparam FrameContext The IPA specific Frame Context type for this queue + * + * The Frame Context Queue provides a simple circular buffer implementation to + * store IPA specific context for each frame through its lifetime within the + * IPA. + * + * FrameContexts are expected to be referenced by a monotonically increasing + * sequence count referring to a Frame sequence. + * + * A FrameContext is initialised for a given frame when the corresponding + * Request is first queued into the IPA. From that point on the FrameContext can + * be obtained by the IPA and its algorithms by referencing it from the frame + * sequence number. + * + * A frame sequence number from the image source must correspond to the request + * sequence number for this implementation to be supported in an IPA. It is + * expected that the same sequence number will be used to reference the context + * of the Frame from the point of queueing the request, specifying controls for + * a given frame, and processing of any ISP related controls and statistics for + * the same corresponding image. + * + * IPA specific FrameContexts should inherit from the IPAFrameContext to support + * the minimum required features for a FrameContext, including the frame number + * and error flags that relate to the frame. + * + * FrameContexts are overwritten when they are recycled and re-initialised by + * the first access made on them by either initialise(frame) or get(frame). If a + * FrameContext is first accessed through get(frame) before calling initialise() + * a PFCError is automatically raised on the FrameContext to be transferred to + * the Request when it completes. + */ + +/** + * \fn FCQueue::clear() + * \brief Reinitialise all data on the queue + * + * Reset the queue and ensure all FrameContext slots are re-initialised. + */ + +/** + * \fn FCQueue::initialise(uint32_t frame) + * \brief Initialize and return the Frame Context at slot specified by \a frame + * \param[in] frame The frame context sequence number + * + * The first call to obtain a FrameContext from the FCQueue should be handled + * through this call. The FrameContext will be initialised for the frame and + * returned to the caller if it was not already initialised. + * + * If the FrameContext was already initialized for this sequence, a warning will + * be reported and the previously initialized FrameContext is returned. + * + * \return A reference to the FrameContext for sequence \a frame + */ + +/** + * \fn FCQueue::get() + * \brief Obtain the Frame Context at slot specified by \a frame + * \param[in] frame The frame context sequence number + * + * Obtains an existing FrameContext from the queue and returns it to the caller. + * + * If the FrameContext is not correctly intiialised for the \a frame, it will be + * initialised and a PFCError will be flagged on the context to be transferred + * to the Request when it completes. + * + * \return A reference to the FrameContext for sequence \a frame + */ + +} /* namespace ipa */ + +} /* namespace libcamera */ diff --git a/src/ipa/libipa/fc_queue.h b/src/ipa/libipa/fc_queue.h new file mode 100644 index 000000000000..82ce36811926 --- /dev/null +++ b/src/ipa/libipa/fc_queue.h @@ -0,0 +1,93 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2022, Google Inc. + * + * fc_queue.h - IPA Frame context queue + * + */ + +#pragma once + +#include + +#include + +#include + +namespace libcamera { + +LOG_DECLARE_CATEGORY(FCQueue) + +namespace ipa { + +/* + * Maximum number of frame contexts to be held onto + * + * \todo Should be larger than ISP delay + sensor delay + */ +static constexpr uint32_t kMaxFrameContexts = 16; + +template +class FCQueue : public std::array +{ +private: + void initialise(FrameContext &frameContext, const uint32_t frame) + { + frameContext = {}; + frameContext.frame = frame; + } + +public: + void clear() + { + this->fill({}); + } + + FrameContext &initialise(const uint32_t frame) + { + FrameContext &frameContext = this->at(frame & kMaxFrameContexts); + + /* + * Do not re-initialise if a get() call has already fetched this + * frame context to preseve the error flags already raised. + */ + if (frame == frameContext.frame && frame != 0) { + LOG(FCQueue, Warning) + << "Frame " << frame << " already initialised"; + } else { + initialise(frameContext, frame); + } + + return frameContext; + } + + FrameContext &get(uint32_t frame) + { + FrameContext &frameContext = this->at(frame & kMaxFrameContexts); + + if (frame != frameContext.frame) { + LOG(FCQueue, Warning) + << "Obtained an uninitialised FrameContext for " + << frame; + + initialise(frameContext, frame); + + /* + * The frame context has been retrieved before it was + * initialised through the initialise() call. This + * indicates an algorithm attempted to access a Frame + * context before it was queued to the IPA. + * + * Controls applied for this request may be left + * unhandled. + */ + frameContext.error |= Request::PFCError; + } + + return frameContext; + } +}; + +} /* namespace ipa */ + +} /* namespace libcamera */ diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build index fb894bc614af..016b8e0ec9be 100644 --- a/src/ipa/libipa/meson.build +++ b/src/ipa/libipa/meson.build @@ -3,6 +3,7 @@ libipa_headers = files([ 'algorithm.h', 'camera_sensor_helper.h', + 'fc_queue.h', 'histogram.h', 'module.h', ]) @@ -10,6 +11,7 @@ libipa_headers = files([ libipa_sources = files([ 'algorithm.cpp', 'camera_sensor_helper.cpp', + 'fc_queue.cpp', 'histogram.cpp', 'module.cpp', ]) From patchwork Thu Jul 21 12:13:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 16717 X-Patchwork-Delegate: kieran.bingham@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2C44ABD1F1 for ; Thu, 21 Jul 2022 12:13:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 415A66331B; Thu, 21 Jul 2022 14:13:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1658405602; bh=3MunA7vIsgIBxPOjXa7eh6kSvRtNGVw534+CQjCpQYA=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=mJpfuLe7a2jXkSik78/dPidznn7JerQalyherD/YL+eLVOnyJPYm6YhG5lZAqcdtS 2TDm0DgXhk6m/eJSjo/4CD02PdDiPjzuXvRqVmZGSSeErm5smlKCiYAGZ/Y4ul1Qdg Nov11PbcC5asowrJC1wb++jmtG4BijGYMWUYyOSykmw6fbe1ViIlsgdvOF1YT+4ceI +XZbHOvhZGPWlVGNH311A46hPJ1cwUBbfCYK0iUV9u7mu9GuVbbytiNL3U2OhraZYi H11CInSXz+1Nfbg64iy8Zi0fJYFgiNC0kv46pmnMxqwLIvPKS8wQf9C3bjtoDcKQ2R 3/2uYtEfsmpLQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E35EF603F1 for ; Thu, 21 Jul 2022 14:13:14 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Rn1ahXxv"; dkim-atps=neutral Received: from Monstersaurus.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 917DD13B9; Thu, 21 Jul 2022 14:13:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1658405594; bh=3MunA7vIsgIBxPOjXa7eh6kSvRtNGVw534+CQjCpQYA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Rn1ahXxvnrbqvDA2dh+ocnFLEeaKV6cGKNCpHTPoOzJMWHI1XDPqX/dUQZNwoCboV q88iN6dgkGYgFTw5PGQn5q4QIYjunz+mnnukUC4NvKq5W5TnjSNoVudiB5vSdT69UQ 0GGrJppLKXAs3ypSN+iarOIOUJXNveUoYX4WUkTs= To: libcamera devel Date: Thu, 21 Jul 2022 13:13:03 +0100 Message-Id: <20220721121310.1286862-6-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> References: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 05/12] ipa: rkisp1: Rename frameContext to activeState X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Kieran Bingham via libcamera-devel From: Kieran Bingham Reply-To: Kieran Bingham Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Move the existing frame context structure to the IPAActiveState. This structure should store the most up to date results for the IPA which may be shared to other algorithms that operate on the data. Signed-off-by: Kieran Bingham Reviewed-by: Umang Jain Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/algorithms/agc.cpp | 20 +++++++++--------- src/ipa/rkisp1/algorithms/awb.cpp | 34 +++++++++++++++---------------- src/ipa/rkisp1/algorithms/blc.cpp | 2 +- src/ipa/rkisp1/ipa_context.h | 7 +++++-- src/ipa/rkisp1/rkisp1.cpp | 12 +++++------ 5 files changed, 39 insertions(+), 36 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index a1bb7d972926..483e941fe427 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -73,8 +73,8 @@ Agc::Agc() int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) { /* Configure the default exposure and gain. */ - context.frameContext.agc.gain = std::max(context.configuration.agc.minAnalogueGain, kMinAnalogueGain); - context.frameContext.agc.exposure = 10ms / context.configuration.sensor.lineDuration; + context.activeState.agc.gain = std::max(context.configuration.agc.minAnalogueGain, kMinAnalogueGain); + context.activeState.agc.exposure = 10ms / context.configuration.sensor.lineDuration; /* * According to the RkISP1 documentation: @@ -98,7 +98,7 @@ int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) context.configuration.agc.measureWindow.h_size = 3 * configInfo.outputSize.width / 4; context.configuration.agc.measureWindow.v_size = 3 * configInfo.outputSize.height / 4; - /* \todo Use actual frame index by populating it in the frameContext. */ + /* \todo Use actual frame index by populating it in the activeState. */ frameCount_ = 0; return 0; } @@ -140,18 +140,18 @@ utils::Duration Agc::filterExposure(utils::Duration exposureValue) /** * \brief Estimate the new exposure and gain values - * \param[inout] frameContext The shared IPA frame Context + * \param[inout] context The shared IPA Context * \param[in] yGain The gain calculated on the current brightness level * \param[in] iqMeanGain The gain calculated based on the relative luminance target */ void Agc::computeExposure(IPAContext &context, double yGain, double iqMeanGain) { IPASessionConfiguration &configuration = context.configuration; - IPAFrameContext &frameContext = context.frameContext; + IPAActiveState &activeState = context.activeState; /* Get the effective exposure and gain applied on the sensor. */ - uint32_t exposure = frameContext.sensor.exposure; - double analogueGain = frameContext.sensor.gain; + uint32_t exposure = activeState.sensor.exposure; + double analogueGain = activeState.sensor.gain; /* Use the highest of the two gain estimates. */ double evGain = std::max(yGain, iqMeanGain); @@ -216,8 +216,8 @@ void Agc::computeExposure(IPAContext &context, double yGain, double iqMeanGain) << stepGain; /* Update the estimated exposure and gain. */ - frameContext.agc.exposure = shutterTime / configuration.sensor.lineDuration; - frameContext.agc.gain = stepGain; + activeState.agc.exposure = shutterTime / configuration.sensor.lineDuration; + activeState.agc.gain = stepGain; } /** @@ -324,7 +324,7 @@ void Agc::process(IPAContext &context, */ void Agc::prepare(IPAContext &context, rkisp1_params_cfg *params) { - if (context.frameContext.frameCount > 0) + if (context.activeState.frameCount > 0) return; /* Configure the measurement window. */ diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 9f00364d12b1..427aaeb1e955 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -35,9 +35,9 @@ LOG_DEFINE_CATEGORY(RkISP1Awb) int Awb::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) { - context.frameContext.awb.gains.red = 1.0; - context.frameContext.awb.gains.blue = 1.0; - context.frameContext.awb.gains.green = 1.0; + context.activeState.awb.gains.red = 1.0; + context.activeState.awb.gains.blue = 1.0; + context.activeState.awb.gains.green = 1.0; /* * Define the measurement window for AWB as a centered rectangle @@ -72,16 +72,16 @@ uint32_t Awb::estimateCCT(double red, double green, double blue) */ void Awb::prepare(IPAContext &context, rkisp1_params_cfg *params) { - params->others.awb_gain_config.gain_green_b = 256 * context.frameContext.awb.gains.green; - params->others.awb_gain_config.gain_blue = 256 * context.frameContext.awb.gains.blue; - params->others.awb_gain_config.gain_red = 256 * context.frameContext.awb.gains.red; - params->others.awb_gain_config.gain_green_r = 256 * context.frameContext.awb.gains.green; + params->others.awb_gain_config.gain_green_b = 256 * context.activeState.awb.gains.green; + params->others.awb_gain_config.gain_blue = 256 * context.activeState.awb.gains.blue; + params->others.awb_gain_config.gain_red = 256 * context.activeState.awb.gains.red; + params->others.awb_gain_config.gain_green_r = 256 * context.activeState.awb.gains.green; /* Update the gains. */ params->module_cfg_update |= RKISP1_CIF_ISP_MODULE_AWB_GAIN; /* If we already have configured the gains and window, return. */ - if (context.frameContext.frameCount > 0) + if (context.activeState.frameCount > 0) return; /* Configure the gains to apply. */ @@ -125,7 +125,7 @@ void Awb::process([[maybe_unused]] IPAContext &context, { const rkisp1_cif_isp_stat *params = &stats->params; const rkisp1_cif_isp_awb_stat *awb = ¶ms->awb; - IPAFrameContext &frameContext = context.frameContext; + IPAActiveState &activeState = context.activeState; /* Get the YCbCr mean values */ double yMean = awb->awb_mean[0].mean_y_or_g; @@ -157,22 +157,22 @@ void Awb::process([[maybe_unused]] IPAContext &context, /* Filter the values to avoid oscillations. */ double speed = 0.2; - redGain = speed * redGain + (1 - speed) * frameContext.awb.gains.red; - blueGain = speed * blueGain + (1 - speed) * frameContext.awb.gains.blue; + redGain = speed * redGain + (1 - speed) * activeState.awb.gains.red; + blueGain = speed * blueGain + (1 - speed) * activeState.awb.gains.blue; /* * Gain values are unsigned integer value, range 0 to 4 with 8 bit * fractional part. */ - frameContext.awb.gains.red = std::clamp(redGain, 0.0, 1023.0 / 256); - frameContext.awb.gains.blue = std::clamp(blueGain, 0.0, 1023.0 / 256); + activeState.awb.gains.red = std::clamp(redGain, 0.0, 1023.0 / 256); + activeState.awb.gains.blue = std::clamp(blueGain, 0.0, 1023.0 / 256); /* Hardcode the green gain to 1.0. */ - frameContext.awb.gains.green = 1.0; + activeState.awb.gains.green = 1.0; - frameContext.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean); + activeState.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean); - LOG(RkISP1Awb, Debug) << "Gain found for red: " << context.frameContext.awb.gains.red - << " and for blue: " << context.frameContext.awb.gains.blue; + LOG(RkISP1Awb, Debug) << "Gain found for red: " << context.activeState.awb.gains.red + << " and for blue: " << context.activeState.awb.gains.blue; } REGISTER_IPA_ALGORITHM(Awb, "Awb") diff --git a/src/ipa/rkisp1/algorithms/blc.cpp b/src/ipa/rkisp1/algorithms/blc.cpp index a58569fa2dc2..4d55a2d529cb 100644 --- a/src/ipa/rkisp1/algorithms/blc.cpp +++ b/src/ipa/rkisp1/algorithms/blc.cpp @@ -68,7 +68,7 @@ int BlackLevelCorrection::init([[maybe_unused]] IPAContext &context, void BlackLevelCorrection::prepare(IPAContext &context, rkisp1_params_cfg *params) { - if (context.frameContext.frameCount > 0) + if (context.activeState.frameCount > 0) return; if (!tuningParameters_) diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index f387cacea363..341fef2e68fe 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -40,7 +40,7 @@ struct IPASessionConfiguration { } hw; }; -struct IPAFrameContext { +struct IPAActiveState { struct { uint32_t exposure; double gain; @@ -64,9 +64,12 @@ struct IPAFrameContext { unsigned int frameCount; }; +struct IPAFrameContext { +}; + struct IPAContext { IPASessionConfiguration configuration; - IPAFrameContext frameContext; + IPAActiveState activeState; }; } /* namespace ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 21166b0f1ba6..9629ccbf4247 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -224,7 +224,7 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, context_.configuration.agc.minAnalogueGain = camHelper_->gain(minGain); context_.configuration.agc.maxAnalogueGain = camHelper_->gain(maxGain); - context_.frameContext.frameCount = 0; + context_.activeState.frameCount = 0; for (auto const &algo : algorithms()) { int ret = algo->configure(context_, info); @@ -284,7 +284,7 @@ void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) algo->prepare(context_, params); paramsBufferReady.emit(frame); - context_.frameContext.frameCount++; + context_.activeState.frameCount++; } void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId, @@ -294,9 +294,9 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId reinterpret_cast( mappedBuffers_.at(bufferId).planes()[0].data()); - context_.frameContext.sensor.exposure = + context_.activeState.sensor.exposure = sensorControls.get(V4L2_CID_EXPOSURE).get(); - context_.frameContext.sensor.gain = + context_.activeState.sensor.gain = camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); unsigned int aeState = 0; @@ -311,8 +311,8 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId void IPARkISP1::setControls(unsigned int frame) { - uint32_t exposure = context_.frameContext.agc.exposure; - uint32_t gain = camHelper_->gainCode(context_.frameContext.agc.gain); + uint32_t exposure = context_.activeState.agc.exposure; + uint32_t gain = camHelper_->gainCode(context_.activeState.agc.gain); ControlList ctrls(ctrls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure)); From patchwork Thu Jul 21 12:13:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 16718 X-Patchwork-Delegate: kieran.bingham@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A2EF2C3274 for ; Thu, 21 Jul 2022 12:13:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B8BDC63329; Thu, 21 Jul 2022 14:13:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1658405602; bh=s0qvwQLa2qn33+1dktqzE6GPMRMm4dpNUx6LKdiubRM=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=xU3dcWQJ9DO8ZiNmoY2o3y8sGClkvsmRvTlDrR0jRACUkn6ob3X0fTnaIausqgd4e xu0nnVzwH/4CzttG7V/YQ+u+bHOawsS5tnvvsoOaAW7kTd+Q+Nk8lUtTNyJD3eQFFU 7+PPIcruIiDrpIG3CSEoWIwKCTYRtl9Ajmv+zwUcteCDxCC1EnuwMfn/gbfgtyBgm2 MjI1QbTiK7l58s1AxZtQ806un+fBfnffB8V9zimIjWiHyTIabOewJaje9bUpcT9dhy cIKrgg1is/XyqclVnRO8yggqhyZUJZsGrNtTpz2AIrQwLkY2gGFZW4M7GCvdi+y6qW 4VnNCx74L3Dhg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 31BDE63317 for ; Thu, 21 Jul 2022 14:13:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Q158AjIw"; dkim-atps=neutral Received: from Monstersaurus.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D22A315AF; Thu, 21 Jul 2022 14:13:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1658405594; bh=s0qvwQLa2qn33+1dktqzE6GPMRMm4dpNUx6LKdiubRM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Q158AjIwvCKAK+Md6e+spk24NHJ1P1ubY7XX9XyEDPhjiXJgsvZwg2srXT6roQiWt 11P0cc9qxxcpXPVr51dlU59i4C3FUBrpnQKhFuYPUHMqo+nR9qEWfHaHU838AGTRwd Ix+cOr57ILgadNmITqrQfiQgoQ5jA7fn8YjTZVQs= To: libcamera devel Date: Thu, 21 Jul 2022 13:13:04 +0100 Message-Id: <20220721121310.1286862-7-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> References: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 06/12] ipa: libipa: Provide a common base for FrameContexts X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Kieran Bingham via libcamera-devel From: Kieran Bingham Reply-To: Kieran Bingham Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Provide a common IPAFrameContext as a base for IPA modules to inherit from. This will allow having a common set of parameters for every FrameContext. We still have to use a templated instance of the FCQueue though to make sure we allocate the correct types. Both the RKISP1 and the IPU3 create IPA specific FrameContext structures which inherit from the IPAFrameContext. While adjusting interface to Algorithm::process() the FrameContext is converted from a pointer to a reference. Signed-off-by: Kieran Bingham Reviewed-by: Umang Jain --- src/ipa/ipu3/algorithms/af.cpp | 3 ++- src/ipa/ipu3/algorithms/af.h | 2 +- src/ipa/ipu3/algorithms/agc.cpp | 9 ++++---- src/ipa/ipu3/algorithms/agc.h | 4 ++-- src/ipa/ipu3/algorithms/awb.cpp | 3 ++- src/ipa/ipu3/algorithms/awb.h | 2 +- src/ipa/ipu3/algorithms/tone_mapping.cpp | 3 ++- src/ipa/ipu3/algorithms/tone_mapping.h | 2 +- src/ipa/ipu3/ipa_context.cpp | 26 +++++------------------- src/ipa/ipu3/ipa_context.h | 5 ++--- src/ipa/ipu3/ipu3.cpp | 6 +++--- src/ipa/ipu3/module.h | 2 +- src/ipa/libipa/algorithm.h | 2 +- src/ipa/libipa/fc_queue.cpp | 21 +++++++++++++++++++ src/ipa/libipa/fc_queue.h | 6 ++++++ src/ipa/rkisp1/algorithms/agc.cpp | 2 +- src/ipa/rkisp1/algorithms/agc.h | 2 +- src/ipa/rkisp1/algorithms/awb.cpp | 2 +- src/ipa/rkisp1/algorithms/awb.h | 2 +- src/ipa/rkisp1/ipa_context.h | 4 +++- src/ipa/rkisp1/module.h | 2 +- src/ipa/rkisp1/rkisp1.cpp | 5 ++++- 22 files changed, 67 insertions(+), 48 deletions(-) diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp index d07521a090ac..fcbf8e557b10 100644 --- a/src/ipa/ipu3/algorithms/af.cpp +++ b/src/ipa/ipu3/algorithms/af.cpp @@ -420,7 +420,8 @@ bool Af::afIsOutOfFocus(IPAContext context) * * [1] Hill Climbing Algorithm, https://en.wikipedia.org/wiki/Hill_climbing */ -void Af::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext, +void Af::process(IPAContext &context, + [[maybe_unused]] IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) { /* Evaluate the AF buffer length */ diff --git a/src/ipa/ipu3/algorithms/af.h b/src/ipa/ipu3/algorithms/af.h index ccf015f3f8f5..7a5aeb1b6bf4 100644 --- a/src/ipa/ipu3/algorithms/af.h +++ b/src/ipa/ipu3/algorithms/af.h @@ -32,7 +32,7 @@ public: void prepare(IPAContext &context, ipu3_uapi_params *params) override; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; - void process(IPAContext &context, IPAFrameContext *frameContext, + void process(IPAContext &context, IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 5bc64ae52214..92ea6249798d 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -183,13 +183,13 @@ utils::Duration Agc::filterExposure(utils::Duration exposureValue) * \param[in] yGain The gain calculated based on the relative luminance target * \param[in] iqMeanGain The gain calculated based on the relative luminance target */ -void Agc::computeExposure(IPAContext &context, IPAFrameContext *frameContext, +void Agc::computeExposure(IPAContext &context, IPU3FrameContext &frameContext, double yGain, double iqMeanGain) { const IPASessionConfiguration &configuration = context.configuration; /* Get the effective exposure and gain applied on the sensor. */ - uint32_t exposure = frameContext->sensor.exposure; - double analogueGain = frameContext->sensor.gain; + uint32_t exposure = frameContext.sensor.exposure; + double analogueGain = frameContext.sensor.gain; /* Use the highest of the two gain estimates. */ double evGain = std::max(yGain, iqMeanGain); @@ -323,7 +323,8 @@ double Agc::estimateLuminance(IPAActiveState &activeState, * Identify the current image brightness, and use that to estimate the optimal * new exposure and gain for the scene. */ -void Agc::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext, +void Agc::process(IPAContext &context, + IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) { /* diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h index 105ae0f2aac6..96585f48933f 100644 --- a/src/ipa/ipu3/algorithms/agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -28,14 +28,14 @@ public: ~Agc() = default; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; - void process(IPAContext &context, IPAFrameContext *frameContext, + void process(IPAContext &context, IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: double measureBrightness(const ipu3_uapi_stats_3a *stats, const ipu3_uapi_grid_config &grid) const; utils::Duration filterExposure(utils::Duration currentExposure); - void computeExposure(IPAContext &context, IPAFrameContext *frameContext, + void computeExposure(IPAContext &context, IPU3FrameContext &frameContext, double yGain, double iqMeanGain); double estimateLuminance(IPAActiveState &activeState, const ipu3_uapi_grid_config &grid, diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index 704267222a31..45f8e5f29119 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -387,7 +387,8 @@ void Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats) /** * \copydoc libcamera::ipa::Algorithm::process */ -void Awb::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext, +void Awb::process(IPAContext &context, + [[maybe_unused]] IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) { calculateWBGains(stats); diff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h index 0acd21480845..28e39620fd59 100644 --- a/src/ipa/ipu3/algorithms/awb.h +++ b/src/ipa/ipu3/algorithms/awb.h @@ -40,7 +40,7 @@ public: int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void prepare(IPAContext &context, ipu3_uapi_params *params) override; - void process(IPAContext &context, IPAFrameContext *frameContext, + void process(IPAContext &context, IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index f86e79b24a67..977741c5a4d4 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -78,7 +78,8 @@ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, * The tone mapping look up table is generated as an inverse power curve from * our gamma setting. */ -void ToneMapping::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext, +void ToneMapping::process(IPAContext &context, + [[maybe_unused]] IPU3FrameContext &frameContext, [[maybe_unused]] const ipu3_uapi_stats_3a *stats) { /* diff --git a/src/ipa/ipu3/algorithms/tone_mapping.h b/src/ipa/ipu3/algorithms/tone_mapping.h index d7d4800628f2..8cce38c742a6 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.h +++ b/src/ipa/ipu3/algorithms/tone_mapping.h @@ -20,7 +20,7 @@ public: int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void prepare(IPAContext &context, ipu3_uapi_params *params) override; - void process(IPAContext &context, IPAFrameContext *frameContext, + void process(IPAContext &context, IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index dd139cb4c868..536d76ecd63b 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -35,22 +35,6 @@ namespace libcamera::ipa::ipu3 { * most recently computed by the IPA algorithms. */ -/** - * \struct IPAFrameContext - * \brief Context for a frame - * - * The frame context stores data specific to a single frame processed by the - * IPA. Each frame processed by the IPA has a context associated with it, - * accessible through the IPAContext structure. - * - * Fields in the frame context should reflect values and controls - * associated with the specific frame as requested by the application, and - * as configured by the hardware. Fields can be read by algorithms to - * determine if they should update any specific action for this frame, and - * finally to update the metadata control lists when the frame is fully - * completed. - */ - /** * \struct IPAContext * \brief Global IPA context data shared between all algorithms @@ -181,16 +165,16 @@ namespace libcamera::ipa::ipu3 { */ /** - * \var IPAFrameContext::frame - * \brief The frame number + * \struct IPU3FrameContext + * \copybrief libcamera::ipa::IPAFrameContext * - * \var IPAFrameContext::sensor + * \var IPU3FrameContext::sensor * \brief Effective sensor values that were applied for the frame * - * \var IPAFrameContext::sensor.exposure + * \var IPU3FrameContext::sensor.exposure * \brief Exposure time expressed as a number of lines * - * \var IPAFrameContext::sensor.gain + * \var IPU3FrameContext::sensor.gain * \brief Analogue gain multiplier */ diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 193fc44a821a..c4aa4c3f4f6a 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -73,8 +73,7 @@ struct IPAActiveState { } toneMapping; }; -struct IPAFrameContext { - uint32_t frame; +struct IPU3FrameContext : public IPAFrameContext { struct { uint32_t exposure; @@ -86,7 +85,7 @@ struct IPAContext { IPASessionConfiguration configuration; IPAActiveState activeState; - FCQueue frameContexts; + FCQueue frameContexts; }; } /* namespace ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 55e82cd404f4..489e51bc6cf7 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -570,7 +570,7 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, const ipu3_uapi_stats_3a *stats = reinterpret_cast(mem.data()); - IPAFrameContext &frameContext = context_.frameContexts.get(frame); + IPU3FrameContext &frameContext = context_.frameContexts.get(frame); if (frameContext.frame != frame) LOG(IPAIPU3, Warning) << "Frame " << frame << " does not match its frame context"; @@ -583,7 +583,7 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, ControlList ctrls(controls::controls); for (auto const &algo : algorithms_) - algo->process(context_, &frameContext, stats); + algo->process(context_, frameContext, stats); setControls(frame); @@ -619,7 +619,7 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, void IPAIPU3::queueRequest(const uint32_t frame, const ControlList &controls) { /* \todo Start processing for 'frame' based on 'controls'. */ - IPAFrameContext &frameContext = context_.frameContexts.initialise(frame); + IPU3FrameContext &frameContext = context_.frameContexts.initialise(frame); /* \todo Implement queueRequest to each algorithm. */ (void)frameContext; diff --git a/src/ipa/ipu3/module.h b/src/ipa/ipu3/module.h index d94fc4594871..6d0d50f615d8 100644 --- a/src/ipa/ipu3/module.h +++ b/src/ipa/ipu3/module.h @@ -19,7 +19,7 @@ namespace libcamera { namespace ipa::ipu3 { -using Module = ipa::Module; } /* namespace ipa::ipu3 */ diff --git a/src/ipa/libipa/algorithm.h b/src/ipa/libipa/algorithm.h index 2a8871d831a3..adbcf69adb4f 100644 --- a/src/ipa/libipa/algorithm.h +++ b/src/ipa/libipa/algorithm.h @@ -41,7 +41,7 @@ public: } virtual void process([[maybe_unused]] typename Module::Context &context, - [[maybe_unused]] typename Module::FrameContext *frameContext, + [[maybe_unused]] typename Module::FrameContext &frameContext, [[maybe_unused]] const typename Module::Stats *stats) { } diff --git a/src/ipa/libipa/fc_queue.cpp b/src/ipa/libipa/fc_queue.cpp index ecb47f287350..627c2fa3301e 100644 --- a/src/ipa/libipa/fc_queue.cpp +++ b/src/ipa/libipa/fc_queue.cpp @@ -55,6 +55,27 @@ namespace ipa { * the Request when it completes. */ +/** + * \struct IPAFrameContext + * \brief Context for a frame + * + * The frame context stores data specific to a single frame processed by the + * IPA. Each frame processed by the IPA has a context associated with it, + * accessible through the Frame Context Queue. + * + * Fields in the frame context should reflect values and controls associated + * with the specific frame as requested by the application, and as configured by + * the hardware. Fields can be read by algorithms to determine if they should + * update any specific action for this frame, and finally to update the metadata + * control lists when the frame is fully completed. + * + * \var IPAFrameContext::frame + * \brief The frame number + * + * \var IPAFrameContext::error + * \brief The error flags associated with the frame + */ + /** * \fn FCQueue::clear() * \brief Reinitialise all data on the queue diff --git a/src/ipa/libipa/fc_queue.h b/src/ipa/libipa/fc_queue.h index 82ce36811926..6c7b8bf11ab2 100644 --- a/src/ipa/libipa/fc_queue.h +++ b/src/ipa/libipa/fc_queue.h @@ -13,6 +13,7 @@ #include #include +#include namespace libcamera { @@ -27,6 +28,11 @@ namespace ipa { */ static constexpr uint32_t kMaxFrameContexts = 16; +struct IPAFrameContext { + unsigned int frame; + Request::ErrorFlags error; +}; + template class FCQueue : public std::array { diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 483e941fe427..40d27963ef68 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -281,7 +281,7 @@ double Agc::measureBrightness(const rkisp1_cif_isp_hist_stat *hist) const * new exposure and gain for the scene. */ void Agc::process(IPAContext &context, - [[maybe_unused]] IPAFrameContext *frameContext, + [[maybe_unused]] RKISP1FrameContext &frameContext, const rkisp1_stat_buffer *stats) { const rkisp1_cif_isp_stat *params = &stats->params; diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index 22c02779d311..1a291bdf3636 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -29,7 +29,7 @@ public: int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; void prepare(IPAContext &context, rkisp1_params_cfg *params) override; - void process(IPAContext &context, IPAFrameContext *frameContext, + void process(IPAContext &context, RKISP1FrameContext &frameContext, const rkisp1_stat_buffer *stats) override; private: diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 427aaeb1e955..3beafb73ca33 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -120,7 +120,7 @@ void Awb::prepare(IPAContext &context, rkisp1_params_cfg *params) * \copydoc libcamera::ipa::Algorithm::process */ void Awb::process([[maybe_unused]] IPAContext &context, - [[maybe_unused]] IPAFrameContext *frameCtx, + [[maybe_unused]] RKISP1FrameContext &frameCtx, const rkisp1_stat_buffer *stats) { const rkisp1_cif_isp_stat *params = &stats->params; diff --git a/src/ipa/rkisp1/algorithms/awb.h b/src/ipa/rkisp1/algorithms/awb.h index 7647842f6609..259b85eebdb4 100644 --- a/src/ipa/rkisp1/algorithms/awb.h +++ b/src/ipa/rkisp1/algorithms/awb.h @@ -23,7 +23,7 @@ public: int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; void prepare(IPAContext &context, rkisp1_params_cfg *params) override; - void process(IPAContext &context, IPAFrameContext *frameCtx, + void process(IPAContext &context, RKISP1FrameContext &frameCtx, const rkisp1_stat_buffer *stats) override; private: diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 341fef2e68fe..a64dbc75fdd2 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -14,6 +14,8 @@ #include +#include + namespace libcamera { namespace ipa::rkisp1 { @@ -64,7 +66,7 @@ struct IPAActiveState { unsigned int frameCount; }; -struct IPAFrameContext { +struct RKISP1FrameContext : public IPAFrameContext { }; struct IPAContext { diff --git a/src/ipa/rkisp1/module.h b/src/ipa/rkisp1/module.h index 89f83208a75c..2568c624df0f 100644 --- a/src/ipa/rkisp1/module.h +++ b/src/ipa/rkisp1/module.h @@ -19,7 +19,7 @@ namespace libcamera { namespace ipa::rkisp1 { -using Module = ipa::Module; } /* namespace ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 9629ccbf4247..7481e67e70f6 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -301,8 +301,11 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId unsigned int aeState = 0; + /* \todo Obtain the frame context to pass to process from the FCQueue */ + RKISP1FrameContext frameContext; + for (auto const &algo : algorithms()) - algo->process(context_, nullptr, stats); + algo->process(context_, frameContext, stats); setControls(frame); From patchwork Thu Jul 21 12:13:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 16719 X-Patchwork-Delegate: kieran.bingham@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0C2AABD1F1 for ; Thu, 21 Jul 2022 12:13:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 54B8C63325; Thu, 21 Jul 2022 14:13:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1658405603; bh=lr+PMgSDOkvy+qcyZCzh0vL1OSik8zfEOPD0JrMS8CE=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=NsAuP3ODMS4lCRXEszFjEfD9+22wQnKrpkgutmuvx+h5UZDemHJOoqYsupuwISPRT LEZZhy63F+G7rOkDJC4p/Om8Ao4Xyxj93JiSqs1nmhWAjOlC6ltSkNoLdkau8ZlRDI RWki075AgraGP3Vkm016t8dcmPbCWPqcX4DEXU6GDnvf4V9NMMvcSGpCNOhg576ubN Y+v4w3GeJ1qM9NLgbYUpnWSWAsZGkOrgLyIbdOFEnvEtSKfvQCAWQJxfJXUSGfMpEJ 7SmnCqgXjoo3ZtxdRYMJa8Xv/hDqKTO0B6nqxoniF9XIO6BzXrYT6tqJipJ98h171A ck7SqSeUaGWug== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7267E63318 for ; Thu, 21 Jul 2022 14:13:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BbNUUIPT"; dkim-atps=neutral Received: from Monstersaurus.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 210F3496; Thu, 21 Jul 2022 14:13:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1658405595; bh=lr+PMgSDOkvy+qcyZCzh0vL1OSik8zfEOPD0JrMS8CE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BbNUUIPTROxOy75C25IROrRQWoH1IzclzTo7QD2kc52GVdrP8Jmq2q/ywdL5X6vRV XMmpHYsQjHcav4oWLV935vMEXmfcpEle33ocqaIzHIf2a9vrH+4o9qDUe0iFSri5r5 0GQInZE/HdDX9VoTmcTRWKOP2a/18rCozuFE8758= To: libcamera devel Date: Thu, 21 Jul 2022 13:13:05 +0100 Message-Id: <20220721121310.1286862-8-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> References: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 07/12] ipa: rkisp1: Convert to use the FCQueue X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Kieran Bingham via libcamera-devel From: Kieran Bingham Reply-To: Kieran Bingham Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Establish a queue of FrameContexts using the new FCQueue and use it to supply the FrameContext to the algorithms. The algorithms on the RKISP1 do not use this yet themselves, but are able to do so after the introduction of this patch. Signed-off-by: Kieran Bingham --- src/ipa/rkisp1/ipa_context.h | 2 ++ src/ipa/rkisp1/rkisp1.cpp | 9 +++++---- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index a64dbc75fdd2..dc4fa5d1ddba 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -72,6 +72,8 @@ struct RKISP1FrameContext : public IPAFrameContext { struct IPAContext { IPASessionConfiguration configuration; IPAActiveState activeState; + + FCQueue frameContexts; }; } /* namespace ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 7481e67e70f6..a2b7b25a53e5 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -205,7 +205,9 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, << " Gain: " << minGain << "-" << maxGain; /* Clean context at configuration */ - context_ = {}; + context_.configuration = {}; + context_.activeState = {}; + context_.frameContexts.clear(); /* Set the hardware revision for the algorithms. */ context_.configuration.hw.revision = hwRevision_; @@ -290,6 +292,8 @@ void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId, const ControlList &sensorControls) { + RKISP1FrameContext &frameContext = context_.frameContexts.get(frame); + const rkisp1_stat_buffer *stats = reinterpret_cast( mappedBuffers_.at(bufferId).planes()[0].data()); @@ -301,9 +305,6 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId unsigned int aeState = 0; - /* \todo Obtain the frame context to pass to process from the FCQueue */ - RKISP1FrameContext frameContext; - for (auto const &algo : algorithms()) algo->process(context_, frameContext, stats); From patchwork Thu Jul 21 12:13:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 16720 X-Patchwork-Delegate: kieran.bingham@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 956AFC3274 for ; Thu, 21 Jul 2022 12:13:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6370E6332D; Thu, 21 Jul 2022 14:13:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1658405604; bh=TbGX0hhDtGMUTgQRABBiAeo8L5xtOC9HWEB4qh4cchE=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=XnIcTGlpEm18r1y/KH3NMH4zWkJu76VYaHFsHmLZ9iPtOy/sGeDWx8mdZTkQVyl0U fntA+3ZLedpt/+hnZoP8E1tNHe57Uto2hid/lX76GKf1t0DRMmzXEHXrF4xzaWvBBQ e012IXcbe/vHhPQ5j9qCcFe21OdOvstfowe7dFm8nC5g04ovY0xY+6bpXZhcZo396G Z0RNuqeAuCS92GinYZuOlo2diINXTv3Jpvq+KjYG/Exx8I9hq4rHeRis+z+Ez5rxwV SBYSJzObWD9wyZ3UzTLuDL6T/KJG0TZlQrTCEJtsq+HRKy5Z4ihA2e7/idms1Bi7LS ACHxNfp+vmy3w== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B2A806330F for ; Thu, 21 Jul 2022 14:13:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="N0b9wkMC"; dkim-atps=neutral Received: from Monstersaurus.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 61578825; Thu, 21 Jul 2022 14:13:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1658405595; bh=TbGX0hhDtGMUTgQRABBiAeo8L5xtOC9HWEB4qh4cchE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=N0b9wkMCUQFfT37JKWt4n0q7V7w5m6x6PIMDFgheEGEUOD4Nw5R9Stch7uIL4W+6P ooeFCLNB2VIeVUliyalSGyHdd5Jj3Y2K1P3fUNHPA4SlTw+AqJrPn3lrTthHiReZzf 2t5IEJ/qDnKG0sWIEvaSLrcEqQOoPuLcm2+yIh2Y= To: libcamera devel Date: Thu, 21 Jul 2022 13:13:06 +0100 Message-Id: <20220721121310.1286862-9-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> References: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 08/12] ipa: libipa: algorithm: prepare(): Pass frame and frame Context X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Kieran Bingham via libcamera-devel From: Kieran Bingham Reply-To: Kieran Bingham Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Pass the current frame number, and the current FrameContext for calls to prepare. Until there is a Frame Context Queue in the RKISP1 the FrameContext will be a nullptr, and must not be dereferenced until the queue is implemented. Signed-off-by: Kieran Bingham - Obtain and use the RKISP1FrameContext Queue entry for IPARkISP1::fillParamsBuffer Signed-off-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/ipu3/algorithms/af.cpp | 5 ++++- src/ipa/ipu3/algorithms/af.h | 4 +++- src/ipa/ipu3/algorithms/awb.cpp | 5 ++++- src/ipa/ipu3/algorithms/awb.h | 5 ++++- src/ipa/ipu3/algorithms/blc.cpp | 6 +++++- src/ipa/ipu3/algorithms/blc.h | 4 +++- src/ipa/ipu3/algorithms/tone_mapping.cpp | 4 ++++ src/ipa/ipu3/algorithms/tone_mapping.h | 3 ++- src/ipa/ipu3/ipu3.cpp | 4 +++- src/ipa/libipa/algorithm.cpp | 2 ++ src/ipa/libipa/algorithm.h | 2 ++ src/ipa/rkisp1/algorithms/agc.cpp | 5 ++++- src/ipa/rkisp1/algorithms/agc.h | 4 +++- src/ipa/rkisp1/algorithms/awb.cpp | 5 ++++- src/ipa/rkisp1/algorithms/awb.h | 4 +++- src/ipa/rkisp1/algorithms/blc.cpp | 2 ++ src/ipa/rkisp1/algorithms/blc.h | 4 +++- src/ipa/rkisp1/rkisp1.cpp | 4 +++- 18 files changed, 58 insertions(+), 14 deletions(-) diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp index fcbf8e557b10..1faf92969ee5 100644 --- a/src/ipa/ipu3/algorithms/af.cpp +++ b/src/ipa/ipu3/algorithms/af.cpp @@ -116,7 +116,10 @@ Af::Af() /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Af::prepare(IPAContext &context, ipu3_uapi_params *params) +void Af::prepare(IPAContext &context, + [[maybe_unused]] unsigned int frame, + [[maybe_unused]] IPU3FrameContext &frameContext, + ipu3_uapi_params *params) { const struct ipu3_uapi_grid_config &grid = context.configuration.af.afGrid; params->acc_param.af.grid_cfg = grid; diff --git a/src/ipa/ipu3/algorithms/af.h b/src/ipa/ipu3/algorithms/af.h index 7a5aeb1b6bf4..4e2bbd08c3bf 100644 --- a/src/ipa/ipu3/algorithms/af.h +++ b/src/ipa/ipu3/algorithms/af.h @@ -30,8 +30,10 @@ public: Af(); ~Af() = default; - void prepare(IPAContext &context, ipu3_uapi_params *params) override; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; + void prepare(IPAContext &context, unsigned int frame, + IPU3FrameContext &frameContext, + ipu3_uapi_params *params) override; void process(IPAContext &context, IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index 45f8e5f29119..b110ad404fd1 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -430,7 +430,10 @@ constexpr uint16_t Awb::gainValue(double gain) /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Awb::prepare(IPAContext &context, ipu3_uapi_params *params) +void Awb::prepare(IPAContext &context, + [[maybe_unused]] unsigned int frame, + [[maybe_unused]] IPU3FrameContext &frameContext, + ipu3_uapi_params *params) { /* * Green saturation thresholds are reduced because we are using the diff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h index 28e39620fd59..731e5bae17de 100644 --- a/src/ipa/ipu3/algorithms/awb.h +++ b/src/ipa/ipu3/algorithms/awb.h @@ -39,7 +39,10 @@ public: ~Awb(); int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; - void prepare(IPAContext &context, ipu3_uapi_params *params) override; + + void prepare(IPAContext &context, unsigned int frame, + IPU3FrameContext &frameContext, + ipu3_uapi_params *params) override; void process(IPAContext &context, IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; diff --git a/src/ipa/ipu3/algorithms/blc.cpp b/src/ipa/ipu3/algorithms/blc.cpp index 78ab7bff549f..6d1084066ed4 100644 --- a/src/ipa/ipu3/algorithms/blc.cpp +++ b/src/ipa/ipu3/algorithms/blc.cpp @@ -39,13 +39,17 @@ BlackLevelCorrection::BlackLevelCorrection() /** * \brief Fill in the parameter structure, and enable black level correction * \param context The shared IPA context + * \param[in] frame The frame context sequence number + * \param[in] frameContext The Frame Context for this frame * \param params The IPU3 parameters * * Populate the IPU3 parameter structure with the correction values for each * channel and enable the corresponding ImgU block processing. */ void BlackLevelCorrection::prepare([[maybe_unused]] IPAContext &context, - ipu3_uapi_params *params) + [[maybe_unused]] unsigned int frame, + [[maybe_unused]] IPU3FrameContext &frameContext, + ipu3_uapi_params *params) { /* * The Optical Black Level correction values diff --git a/src/ipa/ipu3/algorithms/blc.h b/src/ipa/ipu3/algorithms/blc.h index d8da1748baba..36388b267eea 100644 --- a/src/ipa/ipu3/algorithms/blc.h +++ b/src/ipa/ipu3/algorithms/blc.h @@ -18,7 +18,9 @@ class BlackLevelCorrection : public Algorithm public: BlackLevelCorrection(); - void prepare(IPAContext &context, ipu3_uapi_params *params) override; + void prepare(IPAContext &context, unsigned int frame, + IPU3FrameContext &frameContext, + ipu3_uapi_params *params) override; }; } /* namespace ipa::ipu3::algorithms */ diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index 977741c5a4d4..9ca3f471cbaf 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -50,12 +50,16 @@ int ToneMapping::configure(IPAContext &context, /** * \brief Fill in the parameter structure, and enable gamma control * \param context The shared IPA context + * \param[in] frame The frame context sequence number + * \param[in] frameContext The Frame Context for this frame * \param params The IPU3 parameters * * Populate the IPU3 parameter structure with our tone mapping look up table and * enable the gamma control module in the processing blocks. */ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, + [[maybe_unused]] unsigned int frame, + [[maybe_unused]] IPU3FrameContext &frameContext, ipu3_uapi_params *params) { /* Copy the calculated LUT into the parameters buffer. */ diff --git a/src/ipa/ipu3/algorithms/tone_mapping.h b/src/ipa/ipu3/algorithms/tone_mapping.h index 8cce38c742a6..cfb3de01b7f3 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.h +++ b/src/ipa/ipu3/algorithms/tone_mapping.h @@ -19,7 +19,8 @@ public: ToneMapping(); int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; - void prepare(IPAContext &context, ipu3_uapi_params *params) override; + void prepare(IPAContext &context, unsigned int frame, + IPU3FrameContext &frameContext, ipu3_uapi_params *params) override; void process(IPAContext &context, IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 489e51bc6cf7..5663b1f4bb51 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -539,8 +539,10 @@ void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) */ params->use = {}; + IPU3FrameContext frameContext = context_.frameContexts.get(frame); + for (auto const &algo : algorithms_) - algo->prepare(context_, params); + algo->prepare(context_, frame, frameContext, params); paramsBufferReady.emit(frame); } diff --git a/src/ipa/libipa/algorithm.cpp b/src/ipa/libipa/algorithm.cpp index 8549fe3f6269..bc7c130d3e09 100644 --- a/src/ipa/libipa/algorithm.cpp +++ b/src/ipa/libipa/algorithm.cpp @@ -70,6 +70,8 @@ namespace ipa { * \fn Algorithm::prepare() * \brief Fill the \a params buffer with ISP processing parameters for a frame * \param[in] context The shared IPA context + * \param[in] frame The frame context sequence number + * \param[in] frameContext The Frame Context for this frame * \param[out] params The ISP specific parameters. * * This function is called for every frame when the camera is running before it diff --git a/src/ipa/libipa/algorithm.h b/src/ipa/libipa/algorithm.h index adbcf69adb4f..ce5c7cc491d8 100644 --- a/src/ipa/libipa/algorithm.h +++ b/src/ipa/libipa/algorithm.h @@ -36,6 +36,8 @@ public: } virtual void prepare([[maybe_unused]] typename Module::Context &context, + [[maybe_unused]] unsigned int frame, + [[maybe_unused]] typename Module::FrameContext &frameContext, [[maybe_unused]] typename Module::Params *params) { } diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 40d27963ef68..2d436511caf7 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -322,7 +322,10 @@ void Agc::process(IPAContext &context, /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Agc::prepare(IPAContext &context, rkisp1_params_cfg *params) +void Agc::prepare(IPAContext &context, + [[maybe_unused]] unsigned int frame, + [[maybe_unused]] RKISP1FrameContext &frameContext, + rkisp1_params_cfg *params) { if (context.activeState.frameCount > 0) return; diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index 1a291bdf3636..7844cd2e3b47 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -28,7 +28,9 @@ public: ~Agc() = default; int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; - void prepare(IPAContext &context, rkisp1_params_cfg *params) override; + void prepare(IPAContext &context, unsigned int frame, + RKISP1FrameContext &frameContext, + rkisp1_params_cfg *params) override; void process(IPAContext &context, RKISP1FrameContext &frameContext, const rkisp1_stat_buffer *stats) override; diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 3beafb73ca33..1af6d98c5252 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -70,7 +70,10 @@ uint32_t Awb::estimateCCT(double red, double green, double blue) /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Awb::prepare(IPAContext &context, rkisp1_params_cfg *params) +void Awb::prepare(IPAContext &context, + [[maybe_unused]] unsigned int frame, + [[maybe_unused]] RKISP1FrameContext &frameContext, + rkisp1_params_cfg *params) { params->others.awb_gain_config.gain_green_b = 256 * context.activeState.awb.gains.green; params->others.awb_gain_config.gain_blue = 256 * context.activeState.awb.gains.blue; diff --git a/src/ipa/rkisp1/algorithms/awb.h b/src/ipa/rkisp1/algorithms/awb.h index 259b85eebdb4..8fd5b706861a 100644 --- a/src/ipa/rkisp1/algorithms/awb.h +++ b/src/ipa/rkisp1/algorithms/awb.h @@ -22,7 +22,9 @@ public: ~Awb() = default; int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; - void prepare(IPAContext &context, rkisp1_params_cfg *params) override; + void prepare(IPAContext &context, unsigned int frame, + RKISP1FrameContext &frameContext, + rkisp1_params_cfg *params) override; void process(IPAContext &context, RKISP1FrameContext &frameCtx, const rkisp1_stat_buffer *stats) override; diff --git a/src/ipa/rkisp1/algorithms/blc.cpp b/src/ipa/rkisp1/algorithms/blc.cpp index 4d55a2d529cb..93c7e3145fa2 100644 --- a/src/ipa/rkisp1/algorithms/blc.cpp +++ b/src/ipa/rkisp1/algorithms/blc.cpp @@ -66,6 +66,8 @@ int BlackLevelCorrection::init([[maybe_unused]] IPAContext &context, * \copydoc libcamera::ipa::Algorithm::prepare */ void BlackLevelCorrection::prepare(IPAContext &context, + [[maybe_unused]] unsigned int frame, + [[maybe_unused]] RKISP1FrameContext &frameContext, rkisp1_params_cfg *params) { if (context.activeState.frameCount > 0) diff --git a/src/ipa/rkisp1/algorithms/blc.h b/src/ipa/rkisp1/algorithms/blc.h index c2649dd7736e..1b2abfeef800 100644 --- a/src/ipa/rkisp1/algorithms/blc.h +++ b/src/ipa/rkisp1/algorithms/blc.h @@ -24,7 +24,9 @@ public: ~BlackLevelCorrection() = default; int init(IPAContext &context, const YamlObject &tuningData) override; - void prepare(IPAContext &context, rkisp1_params_cfg *params) override; + void prepare(IPAContext &context, unsigned int frame, + RKISP1FrameContext &frameContext, + rkisp1_params_cfg *params) override; private: bool tuningParameters_; diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index a2b7b25a53e5..f97484ce5030 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -275,6 +275,8 @@ void IPARkISP1::queueRequest([[maybe_unused]] const uint32_t frame, void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) { + RKISP1FrameContext &frameContext = context_.frameContexts.get(frame); + rkisp1_params_cfg *params = reinterpret_cast( mappedBuffers_.at(bufferId).planes()[0].data()); @@ -283,7 +285,7 @@ void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) memset(params, 0, sizeof(*params)); for (auto const &algo : algorithms()) - algo->prepare(context_, params); + algo->prepare(context_, frame, frameContext, params); paramsBufferReady.emit(frame); context_.activeState.frameCount++; From patchwork Thu Jul 21 12:13:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 16721 X-Patchwork-Delegate: kieran.bingham@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3B459BD1F1 for ; Thu, 21 Jul 2022 12:13:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 70A2563328; Thu, 21 Jul 2022 14:13:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1658405605; bh=K0XorCX5KY7XxeTZMPMU0sCi5tEzVXKrVmQfZ0KyRBU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=uydiM5TRcxtnT9VQaPw6a1d5DCIjvt2a0svQg7aVHpOAVgmOTzFucnr5JhGXsmrc0 BC25e9Y8xAVUq5EGyZZmnSAnwOTGHQhfr1kNgwTcCAxFD9ciLx6iPMoLXtW1hh6/vX D47u2hC7eXEsdi2yVc+QXh1kQXWlX03wnN/MDJ0b9ZtX2YLsh9JEY37reaJfWmG3Ki h1UDrATN9eL4YfnDfhMUFGQ7uTzVmKoz7FAYKqnRs3eJBF2tO0ZY9nUVjrpF5l4lTa UaxhylEuEj6KFqTdqDCdns1a2tQxIjQfJo81J09QeA0WBIjx8ySkz66mvODqoNiWwQ mfD+Xfmd0CxZg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F23B36331B for ; Thu, 21 Jul 2022 14:13:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="WkRR/Hxz"; dkim-atps=neutral Received: from Monstersaurus.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A0637A38; Thu, 21 Jul 2022 14:13:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1658405595; bh=K0XorCX5KY7XxeTZMPMU0sCi5tEzVXKrVmQfZ0KyRBU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WkRR/HxzyID9VeiSlQLOj0ZnrlkAAr0jn9BGhK/A/Znpie89glC86zpAGIT7iGw79 uCrtH9jq5YHbneTyRJKMktB/IFliQXf88TGzOlOG3XcNC6ob5kne+9lTWB4PvipL9Z uWhOuXb9S6Up+co70cMj3UvXsWyS6AbxiZFQgAbc= To: libcamera devel Date: Thu, 21 Jul 2022 13:13:07 +0100 Message-Id: <20220721121310.1286862-10-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> References: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 09/12] ipa: libipa: algorithm: process(): Pass frame number X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Kieran Bingham via libcamera-devel From: Kieran Bingham Reply-To: Kieran Bingham Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Pass the frame number of the current frame being processed. Signed-off-by: Kieran Bingham - Use RKISP1FrameContext from the queue during IPARkISP1::processStatsBuffer Signed-off-by: Kieran Bingham --- src/ipa/ipu3/algorithms/af.cpp | 2 ++ src/ipa/ipu3/algorithms/af.h | 4 +++- src/ipa/ipu3/algorithms/agc.cpp | 2 ++ src/ipa/ipu3/algorithms/agc.h | 3 ++- src/ipa/ipu3/algorithms/awb.cpp | 1 + src/ipa/ipu3/algorithms/awb.h | 3 ++- src/ipa/ipu3/algorithms/tone_mapping.cpp | 2 ++ src/ipa/ipu3/algorithms/tone_mapping.h | 3 ++- src/ipa/ipu3/ipu3.cpp | 2 +- src/ipa/libipa/algorithm.cpp | 1 + src/ipa/libipa/algorithm.h | 1 + src/ipa/rkisp1/algorithms/agc.cpp | 1 + src/ipa/rkisp1/algorithms/agc.h | 3 ++- src/ipa/rkisp1/algorithms/awb.cpp | 1 + src/ipa/rkisp1/algorithms/awb.h | 3 ++- src/ipa/rkisp1/rkisp1.cpp | 2 +- 16 files changed, 26 insertions(+), 8 deletions(-) diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp index 1faf92969ee5..29a9f520d93c 100644 --- a/src/ipa/ipu3/algorithms/af.cpp +++ b/src/ipa/ipu3/algorithms/af.cpp @@ -409,6 +409,7 @@ bool Af::afIsOutOfFocus(IPAContext context) /** * \brief Determine the max contrast image and lens position. * \param[in] context The IPA context. + * \param[in] frame The frame context sequence number * \param[in] frameContext The current frame context * \param[in] stats The statistics buffer of IPU3. * @@ -424,6 +425,7 @@ bool Af::afIsOutOfFocus(IPAContext context) * [1] Hill Climbing Algorithm, https://en.wikipedia.org/wiki/Hill_climbing */ void Af::process(IPAContext &context, + [[maybe_unused]] unsigned int frame, [[maybe_unused]] IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) { diff --git a/src/ipa/ipu3/algorithms/af.h b/src/ipa/ipu3/algorithms/af.h index 4e2bbd08c3bf..977112506c98 100644 --- a/src/ipa/ipu3/algorithms/af.h +++ b/src/ipa/ipu3/algorithms/af.h @@ -31,10 +31,12 @@ public: ~Af() = default; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; + void prepare(IPAContext &context, unsigned int frame, IPU3FrameContext &frameContext, ipu3_uapi_params *params) override; - void process(IPAContext &context, IPU3FrameContext &frameContext, + void process(IPAContext &context, unsigned int frame, + IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 92ea6249798d..00ef54d45574 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -317,6 +317,7 @@ double Agc::estimateLuminance(IPAActiveState &activeState, /** * \brief Process IPU3 statistics, and run AGC operations * \param[in] context The shared IPA context + * \param[in] frame The current frame sequence number * \param[in] frameContext The current frame context * \param[in] stats The IPU3 statistics and ISP results * @@ -324,6 +325,7 @@ double Agc::estimateLuminance(IPAActiveState &activeState, * new exposure and gain for the scene. */ void Agc::process(IPAContext &context, + [[maybe_unused]] unsigned int frame, IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) { diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h index 96585f48933f..5bb9dc8f610e 100644 --- a/src/ipa/ipu3/algorithms/agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -28,7 +28,8 @@ public: ~Agc() = default; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; - void process(IPAContext &context, IPU3FrameContext &frameContext, + void process(IPAContext &context, unsigned int frame, + IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index b110ad404fd1..8943ead6fc4c 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -388,6 +388,7 @@ void Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats) * \copydoc libcamera::ipa::Algorithm::process */ void Awb::process(IPAContext &context, + [[maybe_unused]] unsigned int frame, [[maybe_unused]] IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) { diff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h index 731e5bae17de..094b177672ca 100644 --- a/src/ipa/ipu3/algorithms/awb.h +++ b/src/ipa/ipu3/algorithms/awb.h @@ -43,7 +43,8 @@ public: void prepare(IPAContext &context, unsigned int frame, IPU3FrameContext &frameContext, ipu3_uapi_params *params) override; - void process(IPAContext &context, IPU3FrameContext &frameContext, + void process(IPAContext &context, unsigned int frame, + IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index 9ca3f471cbaf..e545011ce203 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -76,6 +76,7 @@ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, /** * \brief Calculate the tone mapping look up table * \param context The shared IPA context + * \param frame The current frame sequence number * \param frameContext The current frame context * \param stats The IPU3 statistics and ISP results * @@ -83,6 +84,7 @@ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, * our gamma setting. */ void ToneMapping::process(IPAContext &context, + [[maybe_unused]] unsigned int frame, [[maybe_unused]] IPU3FrameContext &frameContext, [[maybe_unused]] const ipu3_uapi_stats_3a *stats) { diff --git a/src/ipa/ipu3/algorithms/tone_mapping.h b/src/ipa/ipu3/algorithms/tone_mapping.h index cfb3de01b7f3..a3ab285c927f 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.h +++ b/src/ipa/ipu3/algorithms/tone_mapping.h @@ -21,7 +21,8 @@ public: int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void prepare(IPAContext &context, unsigned int frame, IPU3FrameContext &frameContext, ipu3_uapi_params *params) override; - void process(IPAContext &context, IPU3FrameContext &frameContext, + void process(IPAContext &context, unsigned int frame, + IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 5663b1f4bb51..4d7a54f8d1a5 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -585,7 +585,7 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, ControlList ctrls(controls::controls); for (auto const &algo : algorithms_) - algo->process(context_, frameContext, stats); + algo->process(context_, frame, frameContext, stats); setControls(frame); diff --git a/src/ipa/libipa/algorithm.cpp b/src/ipa/libipa/algorithm.cpp index bc7c130d3e09..65e9e8a56fc4 100644 --- a/src/ipa/libipa/algorithm.cpp +++ b/src/ipa/libipa/algorithm.cpp @@ -87,6 +87,7 @@ namespace ipa { * \fn Algorithm::process() * \brief Process ISP statistics, and run algorithm operations * \param[in] context The shared IPA context + * \param[in] frame The frame context sequence number * \param[in] frameContext The current frame's context * \param[in] stats The IPA statistics and ISP results * diff --git a/src/ipa/libipa/algorithm.h b/src/ipa/libipa/algorithm.h index ce5c7cc491d8..1d2544a767eb 100644 --- a/src/ipa/libipa/algorithm.h +++ b/src/ipa/libipa/algorithm.h @@ -43,6 +43,7 @@ public: } virtual void process([[maybe_unused]] typename Module::Context &context, + [[maybe_unused]] unsigned int frame, [[maybe_unused]] typename Module::FrameContext &frameContext, [[maybe_unused]] const typename Module::Stats *stats) { diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 2d436511caf7..60018db73260 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -281,6 +281,7 @@ double Agc::measureBrightness(const rkisp1_cif_isp_hist_stat *hist) const * new exposure and gain for the scene. */ void Agc::process(IPAContext &context, + [[maybe_unused]] unsigned int frame, [[maybe_unused]] RKISP1FrameContext &frameContext, const rkisp1_stat_buffer *stats) { diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index 7844cd2e3b47..05bda50c5962 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -31,7 +31,8 @@ public: void prepare(IPAContext &context, unsigned int frame, RKISP1FrameContext &frameContext, rkisp1_params_cfg *params) override; - void process(IPAContext &context, RKISP1FrameContext &frameContext, + void process(IPAContext &context, unsigned int frame, + RKISP1FrameContext &frameContext, const rkisp1_stat_buffer *stats) override; private: diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 1af6d98c5252..b3ffa6cda4ea 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -123,6 +123,7 @@ void Awb::prepare(IPAContext &context, * \copydoc libcamera::ipa::Algorithm::process */ void Awb::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] unsigned int frame, [[maybe_unused]] RKISP1FrameContext &frameCtx, const rkisp1_stat_buffer *stats) { diff --git a/src/ipa/rkisp1/algorithms/awb.h b/src/ipa/rkisp1/algorithms/awb.h index 8fd5b706861a..510fbad9958e 100644 --- a/src/ipa/rkisp1/algorithms/awb.h +++ b/src/ipa/rkisp1/algorithms/awb.h @@ -25,7 +25,8 @@ public: void prepare(IPAContext &context, unsigned int frame, RKISP1FrameContext &frameContext, rkisp1_params_cfg *params) override; - void process(IPAContext &context, RKISP1FrameContext &frameCtx, + void process(IPAContext &context, unsigned int frame, + RKISP1FrameContext &frameCtx, const rkisp1_stat_buffer *stats) override; private: diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index f97484ce5030..b468e08a67fa 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -308,7 +308,7 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId unsigned int aeState = 0; for (auto const &algo : algorithms()) - algo->process(context_, frameContext, stats); + algo->process(context_, frame, frameContext, stats); setControls(frame); From patchwork Thu Jul 21 12:13:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 16722 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0A7EDC3274 for ; Thu, 21 Jul 2022 12:13:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B234863317; Thu, 21 Jul 2022 14:13:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1658405606; bh=Qyt/OUdZg+2y+xNvn48MTOqODy1zqpfhIIuhwHcTC60=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=th3Unazmo73kOsp6oyehcPti4RL4PLzuZtjS170F8Sp7upBLGJzsZYahwmcYOsDRY T7kxCncdOND4TV09wE+YidLiz0wvpYHRBfL+I31UmkkwBpnW+q5nnO9HN/vdyWx4fZ qbbsckcJwNkejetHrwIAHr+JozPg8rznP4BSBGwJXvFUIEneSbrqB0FIh5eVoERPZN G1/Zl+u7bwXvVQTYeC+9sjGH93EhV9AJNgIanEmO5zfOdlhqWIrS8xx2rYf3Q3O1st +1QLKgeZVxrY+q9E2cbYl1t3YyQZS832Ss8WRp+Ib4Qx1dxjNNttqIWWVfVH1jHomG +nWcYXaLFqwdg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 413376331D for ; Thu, 21 Jul 2022 14:13:16 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="khQQI69S"; dkim-atps=neutral Received: from Monstersaurus.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DC1AD9B1; Thu, 21 Jul 2022 14:13:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1658405596; bh=Qyt/OUdZg+2y+xNvn48MTOqODy1zqpfhIIuhwHcTC60=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=khQQI69S10Qirrtntq0n81HM2LSw2yiL9ERvzNu0nzQ7zDcaCMijBc0RisRuUukZH 2TLEyK7mKjGXcce7nbUCPqT6rz65R7UzRpwd56hmd5WwRpiBTfFCcQWUvDxIX3ZKH+ qhfg006bl5vBmO6BRdysb/1P1jXplPL9gOUkEXyY= To: libcamera devel Date: Thu, 21 Jul 2022 13:13:08 +0100 Message-Id: <20220721121310.1286862-11-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> References: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 10/12] ipa: libipa: algorithm: Add queueRequest() to the Algorithm class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Kieran Bingham via libcamera-devel From: Kieran Bingham Reply-To: Kieran Bingham Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Florian Sylvestre Add queueRequest() function to the Algorithm class. The queueRequest() function provides controls values coming from the application to each algorithm. Each algorithm is responsible for retrieving the controls associated to them. Signed-off-by: Florian Sylvestre Signed-off-by: Kieran Bingham --- v2: (Kieran) - Adjust function ordering - Add FrameContext to queueRequest - Removed Module::ControlList and using libcamera::ControlList directly Signed-off-by: Kieran Bingham --- src/ipa/libipa/algorithm.cpp | 17 +++++++++++++++++ src/ipa/libipa/algorithm.h | 9 +++++++++ 2 files changed, 26 insertions(+) diff --git a/src/ipa/libipa/algorithm.cpp b/src/ipa/libipa/algorithm.cpp index 65e9e8a56fc4..020b0b57b632 100644 --- a/src/ipa/libipa/algorithm.cpp +++ b/src/ipa/libipa/algorithm.cpp @@ -83,6 +83,23 @@ namespace ipa { * includes setting fields and flags that enable those processing blocks. */ +/** + * \fn Algorithm::queueRequest() + * \brief Provide control values to the algorithm + * \param[in] context The shared IPA context + * \param[in] frame The frame context sequence number + * \param[in] frameContext The current frame's context + * \param[in] controls The list of user controls + * + * This function is called for each request queued to the camera. It provides + * the controls stored in the request to the algorithm. The \a frame number + * is the Request sequence number and identifies the desired corresponding + * frame to target for the controls to take effect. + * + * Algorithms shall read the applicable controls and store their value for later + * use during frame processing. + */ + /** * \fn Algorithm::process() * \brief Process ISP statistics, and run algorithm operations diff --git a/src/ipa/libipa/algorithm.h b/src/ipa/libipa/algorithm.h index 1d2544a767eb..5ccc640955c4 100644 --- a/src/ipa/libipa/algorithm.h +++ b/src/ipa/libipa/algorithm.h @@ -11,6 +11,7 @@ namespace libcamera { +class ControlList; class YamlObject; namespace ipa { @@ -35,6 +36,14 @@ public: return 0; } + + virtual void queueRequest([[maybe_unused]] typename Module::Context &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] typename Module::FrameContext &frameContext, + [[maybe_unused]] const ControlList &controls) + { + } + virtual void prepare([[maybe_unused]] typename Module::Context &context, [[maybe_unused]] unsigned int frame, [[maybe_unused]] typename Module::FrameContext &frameContext, From patchwork Thu Jul 21 12:13:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 16723 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 88C8ABD1F1 for ; Thu, 21 Jul 2022 12:13:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4528063330; Thu, 21 Jul 2022 14:13:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1658405607; bh=Lu3nDaJ8+ZeFIP/DzBydOoTiShcPM5Lm4EsmHyyJxY8=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=SmeC9Pbthqvo5WzgmVkBHHbbCkrUJ9npFdA0DGybPdPPRR/h4PzDjTvcWNOOsxnpi QFCIeSBf4nJZfUbCeSeQlrjLYZ8muOIFbpRic+hRdz0CwOOskv1gz8VdX4QXNbslfK vjt6zUamrcaGc1qXGBiMLNptnnTF/OdU/FoMu+YUTQGBQr/s7W9dWFg+yp9NaYb9JV 4bWfD3QXxKpGgRkhFPV/dJGLmVAEpXlAMMX7nG4VtuPvVRR3omJMfcnv9LilktLrFq tcVm4rIhghlvTyPQ2rE/5cFOrJujv14bGNNZQ4ewcLrnvrxR1o4KNP6Woa51b1/gds 8pmjatAZlcJ7Q== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A350663310 for ; Thu, 21 Jul 2022 14:13:16 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="HWGDYRND"; dkim-atps=neutral Received: from Monstersaurus.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2D00212F3; Thu, 21 Jul 2022 14:13:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1658405596; bh=Lu3nDaJ8+ZeFIP/DzBydOoTiShcPM5Lm4EsmHyyJxY8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HWGDYRNDVcTTlC9gbKujSlhsMT3GWA2BixNA6ZvGg5BTVrJfsLLYPuMPNRKcNQ24Y 5HCxJKkSJRCpy8m0yurW0D6feHWQf5K7KIqODMY4BiIugvwVsPHrocTHu6mc+kKfun 2UFmO2Sgm4XViqKpli1Ox6sVmF7lSLodipMiB8vw= To: libcamera devel Date: Thu, 21 Jul 2022 13:13:09 +0100 Message-Id: <20220721121310.1286862-12-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> References: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 11/12] ipa: rkisp1: Transfer queueRequest() call to each algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Kieran Bingham via libcamera-devel From: Kieran Bingham Reply-To: Kieran Bingham Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Florian Sylvestre Implement rkisp1 queueRequest() function to update each algorithm with user controls. Signed-off-by: Florian Sylvestre Signed-off-by: Kieran Bingham --- v2: (Kieran) - Provide frame, and frame context to the algorithms. - Use RKISP1FrameContext type now Signed-off-by: Kieran Bingham --- src/ipa/rkisp1/rkisp1.cpp | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index b468e08a67fa..c2bd5ef36c1b 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -267,10 +267,13 @@ void IPARkISP1::unmapBuffers(const std::vector &ids) } } -void IPARkISP1::queueRequest([[maybe_unused]] const uint32_t frame, - [[maybe_unused]] const ControlList &controls) +void IPARkISP1::queueRequest(const uint32_t frame, + const ControlList &controls) { - /* \todo Start processing for 'frame' based on 'controls'. */ + RKISP1FrameContext &frameContext = context_.frameContexts.initialise(frame); + + for (auto const &algo : algorithms()) + algo->queueRequest(context_, frame, frameContext, controls); } void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) From patchwork Thu Jul 21 12:13:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 16724 X-Patchwork-Delegate: kieran.bingham@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 52A5EC3274 for ; Thu, 21 Jul 2022 12:13:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EBBBB63331; Thu, 21 Jul 2022 14:13:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1658405608; bh=IiWy8sLuefx8n77tSE4lJXPzHFE5gucSi08EqBnmJFM=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=GN6wZIB2/s7AjWoHaDJZ84qul8VPP/Ql7DUVYwcX2XUTZl1cYHgIwGTQC7PEdHKnl 9Z8BbdyeBQpCi8+iJF3GYnLWGAsTZyrDc/J/ob9TxA4gRgkaF9BB4QAboAWUM0snp6 B+CMoK7P1BdaMnvAXM+QUXdbJHFkhDD2dp0vN7Fd2P8sy8ht+TjOhiceMY4XbB9n9L 8NYKQYp/I1DM86/kMUpVKwetb54Lspkateb2rZ1itqboDI8aHwfAOwkaUIanp0YbvY tacePHUpzQFg3kMHRXUPZF3oWc1A8wf2vPbHd1pyQMS4DOUoofcH1JM6Cp/Uc4WEeP C/ycm7eAUmgvA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DF86D63311 for ; Thu, 21 Jul 2022 14:13:16 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="e+hVOipS"; dkim-atps=neutral Received: from Monstersaurus.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 74A49496; Thu, 21 Jul 2022 14:13:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1658405596; bh=IiWy8sLuefx8n77tSE4lJXPzHFE5gucSi08EqBnmJFM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=e+hVOipSQmnZfLe/ohLz/acV7iO6CJEPoIXjuvIilIYty+TrO+T7Ka1B89IZge+Ph MLZ/jwSca9wnYy/1LusHDZCfrCnijHCiccZl8MPt+4o+PWABB+BRVYXI/iK6Q6rQuF BsV/rbcH8WR5U2/WHpIR1RLNE3BuvbdLmmDQyXoM= To: libcamera devel Date: Thu, 21 Jul 2022 13:13:10 +0100 Message-Id: <20220721121310.1286862-13-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> References: <20220721121310.1286862-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 12/12] ipa: ipu3: Transfer queueRequest() call to each algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Kieran Bingham via libcamera-devel From: Kieran Bingham Reply-To: Kieran Bingham Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement the IPU3 queueRequest() function to allow each algorithm to process the user controls that it supports. Signed-off-by: Kieran Bingham --- src/ipa/ipu3/ipu3.cpp | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 4d7a54f8d1a5..ee28044fc40e 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -620,12 +620,10 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, */ void IPAIPU3::queueRequest(const uint32_t frame, const ControlList &controls) { - /* \todo Start processing for 'frame' based on 'controls'. */ IPU3FrameContext &frameContext = context_.frameContexts.initialise(frame); - /* \todo Implement queueRequest to each algorithm. */ - (void)frameContext; - (void)controls; + for (auto const &algo : algorithms_) + algo->queueRequest(context_, frame, frameContext, controls); } /**