From patchwork Mon Oct 31 11:45:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17736 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D8AE0C3285 for ; Mon, 31 Oct 2022 11:45:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 00D2163035; Mon, 31 Oct 2022 12:45:30 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1667216730; bh=94lFACRMLlJSvs2JAH2MVfht2+K/TNuOHg6pDaIUxFM=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=3Sd0O5Jhu82eCrluf2Qp53x7zIZ4KSol9fIfq4TOiPxL+25YCjWiFoTeKLKt2jUkc 5YGgOueyL+FjIAJet9N4rVpURksu1LPCt5O3CUwSDspBVwk14NI/YuBaM8p+Amwimd LIf/K0i581UPHzHrNbf4szpgZuKke2PsWWjlk+nYXw0YvFT+nKzgdaLzF2doG4XB5h EYXnCp9ipWOcr+tTAxzqtp86BzDK/YoishQ4czlsJsP89CTRcRGq6gFXf5PhlWeUl/ 3KM8xofM69OglsBorndqDnjINyG9SRubWf6yE3H7OvRJ2NybtFONwPqq6LJueaPIjn UqUoMDIwP5S+A== Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D7DC961F46 for ; Mon, 31 Oct 2022 12:45:28 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Tw0mQnaz"; dkim-atps=neutral Received: by mail-wr1-x42d.google.com with SMTP id bs21so15587813wrb.4 for ; Mon, 31 Oct 2022 04:45:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JPw0yACzhH/JPVM8B5gaYJbcuHwNFRb/r1SBK2+U6XM=; b=Tw0mQnazQMBMMBOaBFgYinlSjDkdVcga9+oy1+cN+GttecdOztgomH9cFn4T2dXOY4 s/3ETiFaJGrbMML+bcl5nvj+y7n+Oa6LsIo/HONfif9pozvIyLjiXlydrGnjyYHuF0RL 14UYTqNpeUR0YQbmG3veSe4pIQLQxhZjXl8sjAAmlXQ2w9J3kr1MTCjm9z2cD6ZiwE9U 7WgWoMOAdmjuFPZEVqtj3ESWErrJ6BwiSlHOCiTPqEvyzGs2QccrWyFI1cR1IRX78DVs dy7bDO6e2CUKRrmD/vM86xs7zchtTKYLDdqM8h1N1ZP8vpSqDeJxA0aUgDmFh+diAizA H03g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JPw0yACzhH/JPVM8B5gaYJbcuHwNFRb/r1SBK2+U6XM=; b=TzxFnHZDFSNQZyhiqmypDkpUv245zQe+nSIc6Z2cJ/y3GF/8Toxc0A4hg+EleQwuMQ 6ubaS5LzBGjlZ1cY1TSXPZX0xXWkGQy7SVvqyYwrO0WlJ5GKwycVJgr1GCYFUcUzvcR1 X79xpkzgagxWZX2DLPoviAqByMwumndMyZ434iTfjcQ52mqLkuR/530HbEM5C4CNMlAM 5KVqCBc14Taqom3u+SK8OsR0LTwmszvXfXeJP3bRTRDLJDmKiJPNN4GYsh+vVjqGfpI+ 9U49qUCI5mOZfVc/1kh0gonllyhhSfZ+/S4OL9UC6YIihzeNUi7pFGJn0GMVsaPwu4Ok 6JnQ== X-Gm-Message-State: ACrzQf16cRdqZDZAuhpROBExTcPTAurqFRdXV2uVQt0P9na8u/VpwSJN 4WxBD5tLYZVlS/nzJd3iSe/9MzHoF0aLsA== X-Google-Smtp-Source: AMsMyM7as4+D+pU6RuqYK9Zx67wS/TgsVR6r1pi8n4c6timO2gzK6BDWiyiwB4OqF/kaFQz5gkafkA== X-Received: by 2002:a5d:4410:0:b0:236:a8f9:268f with SMTP id z16-20020a5d4410000000b00236a8f9268fmr7816879wrq.405.1667216728178; Mon, 31 Oct 2022 04:45:28 -0700 (PDT) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id h11-20020adff18b000000b0023677e1157fsm6936354wro.56.2022.10.31.04.45.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Oct 2022 04:45:27 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Mon, 31 Oct 2022 11:45:16 +0000 Message-Id: <20221031114522.14215-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221031114522.14215-1-naush@raspberrypi.com> References: <20221031114522.14215-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 1/7] delayed_controls: Template the ControlRingBuffer class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Convert ControlRingBuffer to a templated class to allow arbitrary ring buffer array types to be defined. Rename ControlRingBuffer to RingBuffer to indicate this. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: David Plowman Reviewed-by: Kieran Bingham --- include/libcamera/internal/delayed_controls.h | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/include/libcamera/internal/delayed_controls.h b/include/libcamera/internal/delayed_controls.h index aef37077103c..de8026e3e4f0 100644 --- a/include/libcamera/internal/delayed_controls.h +++ b/include/libcamera/internal/delayed_controls.h @@ -53,17 +53,18 @@ private: /* \todo Make the listSize configurable at instance creation time. */ static constexpr int listSize = 16; - class ControlRingBuffer : public std::array + template + class RingBuffer : public std::array { public: - Info &operator[](unsigned int index) + T &operator[](unsigned int index) { - return std::array::operator[](index % listSize); + return std::array::operator[](index % listSize); } - const Info &operator[](unsigned int index) const + const T &operator[](unsigned int index) const { - return std::array::operator[](index % listSize); + return std::array::operator[](index % listSize); } }; @@ -75,7 +76,7 @@ private: uint32_t queueCount_; uint32_t writeCount_; /* \todo Evaluate if we should index on ControlId * or unsigned int */ - std::unordered_map values_; + std::unordered_map> values_; }; } /* namespace libcamera */ From patchwork Mon Oct 31 11:45:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17737 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 348B2BDB16 for ; Mon, 31 Oct 2022 11:45:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C5B9763047; Mon, 31 Oct 2022 12:45:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1667216733; bh=3OkQGGuHDaXKnQLqJ7ahIMxqVij/7kQdZAl0mQRualY=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=G2/4IS9n3ZQyQEEiak/qTJr/xKAOXIfPH4ZAbSGF4leSj2RWVc3sDVTHaHn3Q49PE zwppSnAOmbt2cI+3J6oktFGl5Ha/0lZe5hWXAs2Llp7fjsVRlzbydVAVgtYPRuy/kY VX07hDWbhDb4iBs7fink7KXTa+fKw/yFyfB9gPmw25vpKtkxQV0in28BFStV6NIbaG 40eNJ+3TZtJQKf6busZnGrh29ocY88TueCzvFT8xqok7Uc028Mkt6dsOvKLKFzi+IY c0Rar9cPcWCmlzt01ghY/S07OWMG82dHQicAOut5r6BSOk0cUw5TPUWWnbVfgeHvwW aafeiFI3vkV1A== Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 13EEE6303C for ; Mon, 31 Oct 2022 12:45:30 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="D6pFidVE"; dkim-atps=neutral Received: by mail-wm1-x333.google.com with SMTP id l16-20020a05600c4f1000b003c6c0d2a445so7829417wmq.4 for ; Mon, 31 Oct 2022 04:45:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=19O3ykJgy/boyGdyqfUfUKxSJwPQG6EIUI1ltOgJCc4=; b=D6pFidVEA+b3+qjLrOSnPP/jTE5yVYVqPtKfx3WVmL5dZ6NBKFxkKh4Kh8YFJteVCs RVCCdvOq4vGBiX21SzA8FMd8BjMlnhqaY/KcNCyasswgVgZzX+KKtHcOXAFXsshZ06wN P6fW7j3HedxmjbbVhzqHT7Fl0YiKovPiQ1wC9+ZLRwOv61uxooOxfy//AJiul6EEL8tE 6cPTv7e0JiEOoC1oEW3X/NOtAq+k6cJMzHo0g3PRDHXI3fdpw6a4hYqinHSV6VihN+0S qeudl3FpQbPZTkLUUApVJTkJIncq2X7DwViFrIrlxby6QNG2lcuz1JfcJKnlbtmdKAlM m3kA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=19O3ykJgy/boyGdyqfUfUKxSJwPQG6EIUI1ltOgJCc4=; b=Mth0AMVS+n8Htet6SSlYNcAiYIPeOAR+xNsI/4rrAe2le6Ltk0leM9REZV4pUN1IhF iJjrpy0vTCBeFLNHk54Mmy+hFeKCYf4eSRHICONz3JYFk+IxWKw4bdchtwwmmz5MKhg2 jbh6Rke+ctA7XtDLhCbHolvdDjxgaduhkN24wrDIdfEicHu4YcOB4qpRrKdw27+UFhu4 SvOhn9xuBn62g0aYCJSZSEZlspCIo83l7i6BTkOyTomaQY0EdPoQ37c26LKBknPnyKUy DCH+WaHfGrCV8huzp3r9dfThlEZbL00Kk11+YQB1TwHXjElwcsxATUuRnSOLU8lU73fd w8xw== X-Gm-Message-State: ACrzQf3n6VovA1dgTVDQBfNke0X0gKUjgwyzp3x7QZ22i9SROMQnCKGv U1Rk/sTS9jBxSWj1vCEDr+cJqER5PK7q6Q== X-Google-Smtp-Source: AMsMyM4Go6ZeYV2BLR3MGXOEMynTKcC7jiI3J+h4V+1BJOLyQysHmy6G4yXGF+8DS2Vy7dgzn0+SWQ== X-Received: by 2002:a05:600c:a4b:b0:3b9:859d:7ed6 with SMTP id c11-20020a05600c0a4b00b003b9859d7ed6mr17964247wmq.169.1667216729061; Mon, 31 Oct 2022 04:45:29 -0700 (PDT) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id h11-20020adff18b000000b0023677e1157fsm6936354wro.56.2022.10.31.04.45.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Oct 2022 04:45:28 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Mon, 31 Oct 2022 11:45:17 +0000 Message-Id: <20221031114522.14215-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221031114522.14215-1-naush@raspberrypi.com> References: <20221031114522.14215-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 2/7] delayed_controls: Add user cookie to DelayedControls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Allow the caller to provide a cookie value to DelayedControls::reset and DelayedControls::push. This cookie value is returned from DelayedControls::get for the frame that has the control values applied to it. The cookie value is useful in tracking when a set of controls has been applied to a frame. In a subsequent commit, it will be used by the Raspberry Pi IPA to track the IPA context used when setting the control values. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: David Plowman Reviewed-by: Kieran Bingham --- include/libcamera/internal/delayed_controls.h | 8 ++++--- src/libcamera/delayed_controls.cpp | 22 ++++++++++------- src/libcamera/pipeline/ipu3/ipu3.cpp | 9 +++---- .../pipeline/raspberrypi/raspberrypi.cpp | 6 ++--- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 7 +++--- test/delayed_controls.cpp | 24 +++++++++---------- 6 files changed, 43 insertions(+), 33 deletions(-) diff --git a/include/libcamera/internal/delayed_controls.h b/include/libcamera/internal/delayed_controls.h index de8026e3e4f0..14470f3966ac 100644 --- a/include/libcamera/internal/delayed_controls.h +++ b/include/libcamera/internal/delayed_controls.h @@ -9,6 +9,7 @@ #include #include +#include #include @@ -27,10 +28,10 @@ public: DelayedControls(V4L2Device *device, const std::unordered_map &controlParams); - void reset(); + void reset(unsigned int cookie); - bool push(const ControlList &controls); - ControlList get(uint32_t sequence); + bool push(const ControlList &controls, unsigned int cookie); + std::pair get(uint32_t sequence); void applyControls(uint32_t sequence); @@ -77,6 +78,7 @@ private: uint32_t writeCount_; /* \todo Evaluate if we should index on ControlId * or unsigned int */ std::unordered_map> values_; + RingBuffer cookies_; }; } /* namespace libcamera */ diff --git a/src/libcamera/delayed_controls.cpp b/src/libcamera/delayed_controls.cpp index 777441e8222a..26243dc9e939 100644 --- a/src/libcamera/delayed_controls.cpp +++ b/src/libcamera/delayed_controls.cpp @@ -104,19 +104,21 @@ DelayedControls::DelayedControls(V4L2Device *device, maxDelay_ = std::max(maxDelay_, controlParams_[id].delay); } - reset(); + reset(0); } /** * \brief Reset state machine + * \param[in] cookie User supplied reset cookie value * * Resets the state machine to a starting position based on control values * retrieved from the device. */ -void DelayedControls::reset() +void DelayedControls::reset(unsigned int cookie) { queueCount_ = 1; writeCount_ = 0; + cookies_[0] = cookie; /* Retrieve control as reported by the device. */ std::vector ids; @@ -140,13 +142,15 @@ void DelayedControls::reset() /** * \brief Push a set of controls on the queue * \param[in] controls List of controls to add to the device queue + * \param[in] cookie User supplied cookie value for \a controls * * Push a set of controls to the control queue. This increases the control queue - * depth by one. + * depth by one. The \a cookie value will be subsequently returned from + * \a get() for the frame with all controls applied. * * \returns true if \a controls are accepted, or false otherwise */ -bool DelayedControls::push(const ControlList &controls) +bool DelayedControls::push(const ControlList &controls, const unsigned int cookie) { /* Copy state from previous frame. */ for (auto &ctrl : values_) { @@ -180,6 +184,7 @@ bool DelayedControls::push(const ControlList &controls) << " at index " << queueCount_; } + cookies_[queueCount_] = cookie; queueCount_++; return true; @@ -198,9 +203,10 @@ bool DelayedControls::push(const ControlList &controls) * push(). The max history from the current sequence number that yields valid * values are thus 16 minus number of controls pushed. * - * \return The controls at \a sequence number + * \return The controls at \a sequence number and associated user supplied + * cookie value. */ -ControlList DelayedControls::get(uint32_t sequence) +std::pair DelayedControls::get(uint32_t sequence) { unsigned int index = std::max(0, sequence - maxDelay_); @@ -217,7 +223,7 @@ ControlList DelayedControls::get(uint32_t sequence) << " at index " << index; } - return out; + return { out, cookies_[index] }; } /** @@ -276,7 +282,7 @@ void DelayedControls::applyControls(uint32_t sequence) while (writeCount_ > queueCount_) { LOG(DelayedControls, Debug) << "Queue is empty, auto queue no-op."; - push({}); + push({}, cookies_[queueCount_ - 1]); } device_->setControls(&out); diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 3b892d9671c5..8a5de76aa1b8 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -789,7 +789,7 @@ int PipelineHandlerIPU3::start(Camera *camera, [[maybe_unused]] const ControlLis if (ret) goto error; - data->delayedCtrls_->reset(); + data->delayedCtrls_->reset(0); /* * Start the ImgU video devices, buffers will be queued to the @@ -1265,11 +1265,11 @@ int IPU3CameraData::loadIPA() return 0; } -void IPU3CameraData::setSensorControls([[maybe_unused]] unsigned int id, +void IPU3CameraData::setSensorControls(unsigned int id, const ControlList &sensorControls, const ControlList &lensControls) { - delayedCtrls_->push(sensorControls); + delayedCtrls_->push(sensorControls, id); CameraLens *focusLens = cio2_.sensor()->focusLens(); if (!focusLens) @@ -1389,7 +1389,8 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) request->metadata().set(controls::SensorTimestamp, buffer->metadata().timestamp); - info->effectiveSensorControls = delayedCtrls_->get(buffer->metadata().sequence); + auto [controls, cookie] = delayedCtrls_->get(buffer->metadata().sequence); + info->effectiveSensorControls = std::move(controls); if (request->findBuffer(&rawStream_)) pipe()->completeBuffer(request, buffer); diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 343f8cb2c7ed..29626c0ef14e 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -1064,7 +1064,7 @@ int PipelineHandlerRPi::start(Camera *camera, const ControlList *controls) * Reset the delayed controls with the gain and exposure values set by * the IPA. */ - data->delayedCtrls_->reset(); + data->delayedCtrls_->reset(0); data->state_ = RPiCameraData::State::Idle; @@ -1794,7 +1794,7 @@ void RPiCameraData::setIspControls(const ControlList &controls) void RPiCameraData::setDelayedControls(const ControlList &controls) { - if (!delayedCtrls_->push(controls)) + if (!delayedCtrls_->push(controls, 0)) LOG(RPI, Error) << "V4L2 DelayedControl set failed"; handleState(); } @@ -1867,7 +1867,7 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) * Lookup the sensor controls used for this frame sequence from * DelayedControl and queue them along with the frame buffer. */ - ControlList ctrl = delayedCtrls_->get(buffer->metadata().sequence); + auto [ctrl, cookie] = delayedCtrls_->get(buffer->metadata().sequence); /* * Add the frame timestamp to the ControlList for the IPA to use * as it does not receive the FrameBuffer object. diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 455ee2a0a711..6f3c34e764fd 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -376,10 +376,10 @@ void RkISP1CameraData::paramFilled(unsigned int frame) selfPath_->queueBuffer(info->selfPathBuffer); } -void RkISP1CameraData::setSensorControls([[maybe_unused]] unsigned int frame, +void RkISP1CameraData::setSensorControls(unsigned int frame, const ControlList &sensorControls) { - delayedCtrls_->push(sensorControls); + delayedCtrls_->push(sensorControls, frame); } void RkISP1CameraData::metadataReady(unsigned int frame, const ControlList &metadata) @@ -1191,8 +1191,9 @@ void PipelineHandlerRkISP1::statReady(FrameBuffer *buffer) if (data->frame_ <= buffer->metadata().sequence) data->frame_ = buffer->metadata().sequence + 1; + auto [controls, cookie] = data->delayedCtrls_->get(buffer->metadata().sequence); data->ipa_->processStatsBuffer(info->frame, info->statBuffer->cookie(), - data->delayedCtrls_->get(buffer->metadata().sequence)); + controls); } REGISTER_PIPELINE_HANDLER(PipelineHandlerRkISP1) diff --git a/test/delayed_controls.cpp b/test/delayed_controls.cpp index a8ce9828d73d..26037268f245 100644 --- a/test/delayed_controls.cpp +++ b/test/delayed_controls.cpp @@ -82,7 +82,7 @@ protected: /* Reset control to value not used in test. */ ctrls.set(V4L2_CID_BRIGHTNESS, 1); dev_->setControls(&ctrls); - delayed->reset(); + delayed->reset(0); /* Trigger the first frame start event */ delayed->applyControls(0); @@ -92,11 +92,11 @@ protected: int32_t value = 100 + i; ctrls.set(V4L2_CID_BRIGHTNESS, value); - delayed->push(ctrls); + delayed->push(ctrls, i); delayed->applyControls(i); - ControlList result = delayed->get(i); + auto [result, cookie] = delayed->get(i); int32_t brightness = result.get(V4L2_CID_BRIGHTNESS).get(); if (brightness != value) { cerr << "Failed single control without delay" @@ -124,7 +124,7 @@ protected: int32_t expected = 4; ctrls.set(V4L2_CID_BRIGHTNESS, expected); dev_->setControls(&ctrls); - delayed->reset(); + delayed->reset(0); /* Trigger the first frame start event */ delayed->applyControls(0); @@ -134,11 +134,11 @@ protected: int32_t value = 10 + i; ctrls.set(V4L2_CID_BRIGHTNESS, value); - delayed->push(ctrls); + delayed->push(ctrls, i); delayed->applyControls(i); - ControlList result = delayed->get(i); + auto [result, cookie] = delayed->get(i); int32_t brightness = result.get(V4L2_CID_BRIGHTNESS).get(); if (brightness != expected) { cerr << "Failed single control with delay" @@ -172,7 +172,7 @@ protected: ctrls.set(V4L2_CID_BRIGHTNESS, expected); ctrls.set(V4L2_CID_CONTRAST, expected + 1); dev_->setControls(&ctrls); - delayed->reset(); + delayed->reset(0); /* Trigger the first frame start event */ delayed->applyControls(0); @@ -183,11 +183,11 @@ protected: ctrls.set(V4L2_CID_BRIGHTNESS, value); ctrls.set(V4L2_CID_CONTRAST, value + 1); - delayed->push(ctrls); + delayed->push(ctrls, i); delayed->applyControls(i); - ControlList result = delayed->get(i); + auto [result, cookie] = delayed->get(i); int32_t brightness = result.get(V4L2_CID_BRIGHTNESS).get(); int32_t contrast = result.get(V4L2_CID_CONTRAST).get(); if (brightness != expected || contrast != expected + 1) { @@ -223,7 +223,7 @@ protected: ctrls.set(V4L2_CID_BRIGHTNESS, expected); ctrls.set(V4L2_CID_CONTRAST, expected); dev_->setControls(&ctrls); - delayed->reset(); + delayed->reset(0); /* Trigger the first frame start event */ delayed->applyControls(0); @@ -238,7 +238,7 @@ protected: ctrls.set(V4L2_CID_BRIGHTNESS, value); ctrls.set(V4L2_CID_CONTRAST, value); - delayed->push(ctrls); + delayed->push(ctrls, i); } /* Process all queued controls. */ @@ -247,7 +247,7 @@ protected: delayed->applyControls(i); - ControlList result = delayed->get(i); + auto [result, cookie] = delayed->get(i); int32_t brightness = result.get(V4L2_CID_BRIGHTNESS).get(); int32_t contrast = result.get(V4L2_CID_CONTRAST).get(); From patchwork Mon Oct 31 11:45:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17738 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 24559C3286 for ; Mon, 31 Oct 2022 11:45:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AF07A6304B; Mon, 31 Oct 2022 12:45:34 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1667216734; bh=YgGiKgMXIahXa21Lh6ZR4WbW075cAGL5YbEd7/YWnH0=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=XBVUGKoqSBpcp2Dq/xQErvEtFyax82Vc1g/w7jvdALMLdNwmliAl5EWsgLlzdkTy2 39aEM4oibjqTiZfSpRAnRurCGLlk1wx9uDAMGHdEvFtGzleqVO3yokYT+mkc00Zx9R knJor4xFoCzCWCZEEX5/CqlLWy348WIZAPsCYW9HYPEkViALfjWSLfwAXOG7CA/7xi 8gPH64ZRH72zQSFBBQPcRSGRn+jTDEHIsaWyyXlRQVpCR0oiEB8d9pgoWGyLm0JQRZ eHZV0ZnktMHw6fT5ot0Y5bQORmVACpjHDzYoFa6W0pbvzD800kFTPKNFDSVhIDEF40 SmdsqCnrAl8oQ== Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 892E163041 for ; Mon, 31 Oct 2022 12:45:30 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="FelnHXry"; dkim-atps=neutral Received: by mail-wr1-x42b.google.com with SMTP id g12so15565478wrs.10 for ; Mon, 31 Oct 2022 04:45:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3Hub9huiA2Fp5q/ldBR20j+lQfHfA0OwAhO+e2na52o=; b=FelnHXryaept3bVwmE6Ay19SG9LY3VOtLbNxW3F0notR/kOU0o/V4pL51BrYmFgn93 ukZC5cVHIddtTM/4Lf/rgiKT4kz+IACMtGOdlJHkyXeECGgv3nxSp9s9RILdY/Z45mdl ZRd4NovKzMwDlWxpSvbyoyOW6jpIMHjjAVcoj3m0hEhzKE2+Pv8yoqjKBAGg5TPtzf8k APrrPudEEObINzjqVedB2A7fgltLtPynjfdExRJoHbaLcdwx87pTaVn+dYrXFwlzrsrl ipfvxHZjDri+y2IjqWIY+42DUrHxv7bNZkuj4rpFF3Yy95Hkd/XqYhk9fosLp8UAf7fW uLMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3Hub9huiA2Fp5q/ldBR20j+lQfHfA0OwAhO+e2na52o=; b=J80LSBZq+sVAnI9ZpBU0HPn8+u6kgWjssPgOAMV1GDAUMOmZhsonDyv9fRby+65Ivt N6tKkVz8McP85uYU7l9je3mNeRTBLCabVlQ1BeRspj9BihIUsTcgau1+fvG2vq3+khm7 Srl5XdW21LAauWe27tB38hHq2I7ElvJLnkcW9LYK2P7jU25idBpK56fEnTieAl7T1rhH 09HfvglJrSjKB0DfEJp/bUwtM/NFJyplSJwWRMVkzkCocf1PP0hSPD2sc9CcF3Vn3Bqs Vn0rIJIWsphgaJDOlD2oe8cl/J2x/xpAk6xAfz1PUlPyhUEiaMdy3LvnuE2BaIY4Z/ax roYQ== X-Gm-Message-State: ACrzQf2u2G3RnitGEQIksTaxx7n7/bDdTmf+PDc5zNcQkh+cmSNlc1rb WVeqWN7N5HJHdLUFSTtilyyr9NApFF1Y7g== X-Google-Smtp-Source: AMsMyM4KE1gzZGVWuM/7OpDp9vukE1ea+fqXws1GhH59PAtRkdhkYzeCsvJT5DTtoqgxOg7PitxVDA== X-Received: by 2002:a5d:6743:0:b0:236:6301:918 with SMTP id l3-20020a5d6743000000b0023663010918mr7830577wrw.247.1667216729648; Mon, 31 Oct 2022 04:45:29 -0700 (PDT) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id h11-20020adff18b000000b0023677e1157fsm6936354wro.56.2022.10.31.04.45.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Oct 2022 04:45:29 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Mon, 31 Oct 2022 11:45:18 +0000 Message-Id: <20221031114522.14215-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221031114522.14215-1-naush@raspberrypi.com> References: <20221031114522.14215-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 3/7] tests: delayed_controls: Add cookie tests X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a test for passing and returning cookie values in DelayedControls. Add a test to simulate frame skips ensuring cookie values are returned correctly. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- test/delayed_controls.cpp | 97 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 97 insertions(+) diff --git a/test/delayed_controls.cpp b/test/delayed_controls.cpp index 26037268f245..387958a339a2 100644 --- a/test/delayed_controls.cpp +++ b/test/delayed_controls.cpp @@ -267,6 +267,93 @@ protected: return TestPass; } + int cookieValue() + { + std::unordered_map delays = { + { V4L2_CID_BRIGHTNESS, { 1, false } }, + }; + std::unique_ptr delayed = + std::make_unique(dev_.get(), delays); + ControlList ctrls; + + /* Set a cookie to the reset value. */ + const unsigned int startCookie = 0x1234; + ctrls.set(V4L2_CID_BRIGHTNESS, 1); + dev_->setControls(&ctrls); + delayed->reset(startCookie); + + /* Trigger the first frame start event */ + delayed->applyControls(0); + + for (unsigned int i = 1; i < 100; i++) { + ctrls.set(V4L2_CID_BRIGHTNESS, 1); + delayed->push(ctrls, startCookie + i); + + delayed->applyControls(i); + + auto [result, cookie] = delayed->get(i); + unsigned int expected = startCookie + i - 1; + if (cookie != expected) { + cerr << "Failed cookie value" + << " frame " << i + << " expected cookie " << expected + << " got cookie " << cookie + << endl; + return TestFail; + } + } + + return TestPass; + } + + int cookieSkip() + { + std::unordered_map delays = { + { V4L2_CID_BRIGHTNESS, { 1, false } }, + }; + std::unique_ptr delayed = + std::make_unique(dev_.get(), delays); + ControlList ctrls; + + /* Set a cookie to the reset value. */ + unsigned int cookie = 0x1234; + ctrls.set(V4L2_CID_BRIGHTNESS, 1); + dev_->setControls(&ctrls); + delayed->reset(cookie); + + /* Trigger the first frame start event */ + delayed->applyControls(0); + + unsigned int expected = cookie++; + for (unsigned int i = 1; i < 100; i++) { + /* + * Send a control for 2 out of every 3 frames, simulating + * some frame skips. + */ + if (i % 3) { + ctrls.set(V4L2_CID_BRIGHTNESS, 1); + delayed->push(ctrls, cookie); + } + + delayed->applyControls(i); + + auto [result, getCookie] = delayed->get(i); + if (expected != getCookie) { + cerr << "Failed cookie value" + << " frame " << i + << " expected cookie " << expected + << " got cookie " << getCookie + << endl; + return TestFail; + } + + if (i % 3) + expected = cookie++; + } + + return TestPass; + } + int run() override { int ret; @@ -291,6 +378,16 @@ protected: if (ret) return ret; + /* Test cookie values. */ + ret = cookieValue(); + if (ret) + return ret; + + /* Test cookie values with frame skips. */ + ret = cookieSkip(); + if (ret) + return ret; + return TestPass; } From patchwork Mon Oct 31 11:45:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17739 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6082BC3287 for ; Mon, 31 Oct 2022 11:45:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A73306304D; Mon, 31 Oct 2022 12:45:35 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1667216735; bh=QlbSyZb55JOooihaGg2jancKeP/f5baLUDuOMGUyyuI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=m5F8uDtV1ENXY7zVhypBBV08948D/jtt/ABFcSJYyv5UK/PRDQpB+eKl2fO1JOkSb GGeBdQE2Z/Sb092iXK/fVRcpBUO9RzWLsa7X+hYK0x+eMINc8elyik6A7qBgIeXohK A1tYOuXcZPjmLOen9qHSyrA/0WzOLdp2vZ34xEpAb80guygLquBGG023uUdEYSGA2j oySlM9pGnsY1PmvbavwlE9+Nw+D0TbbZKVXvbmd19Lwe7wNCyRzdkv5H8SkcAcSThb hiFv4jLNRvdPkgyXurlZo45rWc6a1fM9j6nnBAM1XJbqnO+Dinl9prbKI8qGUdHfu4 qCRaGoo8nPh+A== Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 26D1963032 for ; Mon, 31 Oct 2022 12:45:31 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="UGZIgmqC"; dkim-atps=neutral Received: by mail-wr1-x432.google.com with SMTP id a14so15581626wru.5 for ; Mon, 31 Oct 2022 04:45:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yn36HTpXKWZYQWKnBoWmhX3QM0Hd57lWMTJ6qGaKsMY=; b=UGZIgmqCHxJEqx10cMo4+Qs2nLCn2CKBdn2jTMlIZmaU4stPAbosAiCQx0WqKYjRDd w8SseGLHbliVoOr/4p+5gzdn9+hW9q/9531B3q35VrRn/78mxOamt34ryIGZfaE11g8k JkgDgXOTqDczAUMmKlj0O7MSxNmtSvbzrVW9CGuUCpk9IgKSofVfO42k+2/J9AE5wQYn /owKN0EWxTPhEC2GPF5SycHcJ+oH9TgRuQsgpvL7WxKprt1Eq/PWOknn+y/eZPuamC9R c7O/EzBPJbG9D29ruReXlD2XHdeJl4smjVzP6QZoFThM3pbs16ErHliEI29fRc0ev5Oh lI4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yn36HTpXKWZYQWKnBoWmhX3QM0Hd57lWMTJ6qGaKsMY=; b=msBLdZEcG7L5nn+0XsoYdrZp5lA0PlIyzxbBbwAbVD3wI9Pwag9I6WtY31sTnjTQDc Ga8TSGP6yebdjIts4dFqhgN8pxsBB+xKAG/fDym0gBeGxhGc+vDmE8brOZqUyGU8OzDO ng986pkBtse1YIu+nkfm9U+gmOmU0pcgEiohcBHADRzKQu/MZGkoO/AJWq/QgqdkWaK1 mKIkwXu702K6tQv/H2YeWjwGljWConn4cFQOJsd5CNGO7mOvDtgCNQpHzUtAP15/zljC Whk4pf6lPNSp/72K8vkhyvPCKSo4+15l3m6uDyPjomCCqm0d82WHbOh11MZzhwPZS/pm viOA== X-Gm-Message-State: ACrzQf1eagUzG5w+T1LWBLh6PmS+2wxeNZOXCVp/uqJp4gKfwgY/3uVf +GC0qGsmSynH8RDgYrihiZUNC4Zew8tJdQ== X-Google-Smtp-Source: AMsMyM6ALfszR/ykSZ0uZxX1zrp0URaFPSBXtAACh1LujF/0XWzLPn99EdF9r3jvgdZfD0TBApgPCw== X-Received: by 2002:a05:6000:4086:b0:236:b659:471d with SMTP id da6-20020a056000408600b00236b659471dmr6336829wrb.563.1667216730394; Mon, 31 Oct 2022 04:45:30 -0700 (PDT) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id h11-20020adff18b000000b0023677e1157fsm6936354wro.56.2022.10.31.04.45.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Oct 2022 04:45:29 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Mon, 31 Oct 2022 11:45:19 +0000 Message-Id: <20221031114522.14215-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221031114522.14215-1-naush@raspberrypi.com> References: <20221031114522.14215-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 4/7] ipa: raspberrypi: Add RPiController::Metadata::mergeCopy X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a new member function to RPiController::Metadata that copies unique key/value pairs from one object to the other. This is different from std::map::merge that would remove the key/value pairs from the source object. Signed-off-by: Naushir Patuck Reviewed-by: Kieran Bingham --- src/ipa/raspberrypi/controller/metadata.h | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/ipa/raspberrypi/controller/metadata.h b/src/ipa/raspberrypi/controller/metadata.h index 0f7ebfaf4c25..ddce491784e9 100644 --- a/src/ipa/raspberrypi/controller/metadata.h +++ b/src/ipa/raspberrypi/controller/metadata.h @@ -78,6 +78,16 @@ public: data_.merge(other.data_); } + void mergeCopy(const Metadata &other) + { + std::scoped_lock lock(mutex_, other.mutex_); + /* + * If the metadata key exists, ignore this item and copy only + * unique key/value pairs. + */ + data_.insert(other.data_.begin(), other.data_.end()); + } + template T *getLocked(std::string const &tag) { From patchwork Mon Oct 31 11:45:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17740 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2F6F5C3285 for ; Mon, 31 Oct 2022 11:45:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AF44C6304E; Mon, 31 Oct 2022 12:45:36 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1667216736; bh=jlRfxeMNJmHpGoiVrybdPgGhig7tY5AWa9GoNAXL7UI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=s4OyrKp9dGja1M5UpbchwNY8/i/Pmn1WPXAN7m+6Ag4XDDFEQJZGAM3xc6fziqIOf +aEV3pMw0Q22dJ5RNT4rPgGE6tSo6dN3uRmh9DsL/Wqfw3HAZ+ZiTxiiLjZxeV0dzZ RXGt/RkzhgdiYp6yU92tWoIRp/XxOrf1wRpMLVFcw6yTs6hs8Dbyn/cie2n01iTsJN l6ncXw1ULlsI/WDo9WgTbdDikYAySLwGAlb8vgUTI1B4jF3Xx9afjeDwbjLxaPqqY+ sVID/T5+JE1Fq2qDsELO9+YfTlR4SjNAIkP4HBMRiZCi8P13Fj8hDvekMphfmR6DAt JOEK52vGaU2qA== Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7A00B6303C for ; Mon, 31 Oct 2022 12:45:32 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="REFz0rPK"; dkim-atps=neutral Received: by mail-wm1-x330.google.com with SMTP id p16so1402787wmc.3 for ; Mon, 31 Oct 2022 04:45:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7pCZVHEGkH4rkfudxa5QoNd2EeOZUCc279G5kmfq7Bk=; b=REFz0rPK51+ZG3NnStbLtSHWUbFCCvOmh0FGGt3X1wiFwmp6sbndUA9ME7z3RCEPZ8 byiFnxymPiFD6JwFfKyGY3IEBCz9d+yy8t0qYx+2Ipq/yqlK0hcAsoQNvZyueq1AOvBT quTbzqRklaDVd3XkGwvWywcfMNl6uraHEcrWifebsRj64bejaSIsZk4m1AEI9iJNBiIe w/oPCQh3g5QKsZ3bgUJFRfDHvgBwRvmBOvyvvrXmxwvtZpSmpefEjuH2ff9jxL657TCe 7WmVvGaGtZgMckQHRip5QaRBYHAPQxWY5M1mjtanLWhsERP/wpIJBEz0KQqMpIoUBpPP +BPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7pCZVHEGkH4rkfudxa5QoNd2EeOZUCc279G5kmfq7Bk=; b=gdXcVF1KFWBRLtYe1jebJ8o0JbsVPVbn9kXdTHS/25f7Cb5mon7OyrxqxYr/rJRUb5 9qSb/ojWi3khu+7FskkZFyLFfsryLFlcoqG/br95JbOtMQz6FLKWh2Rvy9z4ARdXmx/R o/Z2lW7XU6+wWKcWBDEa2dy/jq8hlURcuJQlp4ygBPnhnYoRNXQYMPx536jvmGH+GFgq pO8ANdbTZB6ij8ExYrBeC69BGL80DFdV5AtojjI1w81Px2+ehRsMpEErVR4oKMnlMVmh IQKyENY5sjTXS3ik8VNhO7mKSvsy6oJowfhvdhdyF9d0EUMkRnk0Bl8tFdtOA+krxmRY R07w== X-Gm-Message-State: ACrzQf1Xy9zenmWKhBed2ID5zWExF5lh7Y6fEDuM1w4i8wubY/MYM8BE 736/pNM1Mz7q6kKrKqsZVOBS9yRAkaVnOg== X-Google-Smtp-Source: AMsMyM5G9mH4irr4K6PaMKGJonz7+jzqqc41u1A2/1p+5avfDtyQe09nkVFtk+QKxSXOnYq2ERN0iw== X-Received: by 2002:a05:600c:28ce:b0:3cf:678c:885f with SMTP id h14-20020a05600c28ce00b003cf678c885fmr6768540wmd.154.1667216731584; Mon, 31 Oct 2022 04:45:31 -0700 (PDT) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id h11-20020adff18b000000b0023677e1157fsm6936354wro.56.2022.10.31.04.45.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Oct 2022 04:45:30 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Mon, 31 Oct 2022 11:45:20 +0000 Message-Id: <20221031114522.14215-6-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221031114522.14215-1-naush@raspberrypi.com> References: <20221031114522.14215-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 5/7] ipa: raspberrypi: Use an array of RPiController::Metadata objects X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Allow the IPA to cycle through an array of RPiController::Metadata objects, one per prepare()/process() invocation, when running the controller algorithms. This allows historical metadata objects to be retained, and subsequently passed into the controller algorithms on future frames. At present, only a single index into this array is used. This change provides a route to fixing a problem with the AGC algorithm, where if a manual shutter/gain is requested, the algorithm does not currently retain any context of the total exposure that it has calculated. As a result, the wrong digital gain would be applied when the frame with the manual shutter/gain is processed by the ISP. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: David Plowman Reviewed-by: Kieran Bingham --- src/ipa/raspberrypi/raspberrypi.cpp | 63 ++++++++++++++++------------- 1 file changed, 35 insertions(+), 28 deletions(-) diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index b74f1ecf738f..799a4fe70000 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -57,6 +57,9 @@ namespace libcamera { using namespace std::literals::chrono_literals; using utils::Duration; +/* Number of metadata objects available in the context list. */ +constexpr unsigned int numMetadataContexts = 16; + /* Configure the sensor with these values initially. */ constexpr double defaultAnalogueGain = 1.0; constexpr Duration defaultExposureTime = 20.0ms; @@ -163,7 +166,7 @@ private: /* Raspberry Pi controller specific defines. */ std::unique_ptr helper_; RPiController::Controller controller_; - RPiController::Metadata rpiMetadata_; + std::array rpiMetadata_; /* * We count frames to decide if the frame must be hidden (e.g. from @@ -539,14 +542,15 @@ void IPARPi::signalIspPrepare(const ISPConfig &data) void IPARPi::reportMetadata() { - std::unique_lock lock(rpiMetadata_); + RPiController::Metadata &rpiMetadata = rpiMetadata_[0]; + std::unique_lock lock(rpiMetadata); /* * Certain information about the current frame and how it will be * processed can be extracted and placed into the libcamera metadata * buffer, where an application could query it. */ - DeviceStatus *deviceStatus = rpiMetadata_.getLocked("device.status"); + DeviceStatus *deviceStatus = rpiMetadata.getLocked("device.status"); if (deviceStatus) { libcameraMetadata_.set(controls::ExposureTime, deviceStatus->shutterSpeed.get()); @@ -557,24 +561,24 @@ void IPARPi::reportMetadata() libcameraMetadata_.set(controls::SensorTemperature, *deviceStatus->sensorTemperature); } - AgcStatus *agcStatus = rpiMetadata_.getLocked("agc.status"); + AgcStatus *agcStatus = rpiMetadata.getLocked("agc.status"); if (agcStatus) { libcameraMetadata_.set(controls::AeLocked, agcStatus->locked); libcameraMetadata_.set(controls::DigitalGain, agcStatus->digitalGain); } - LuxStatus *luxStatus = rpiMetadata_.getLocked("lux.status"); + LuxStatus *luxStatus = rpiMetadata.getLocked("lux.status"); if (luxStatus) libcameraMetadata_.set(controls::Lux, luxStatus->lux); - AwbStatus *awbStatus = rpiMetadata_.getLocked("awb.status"); + AwbStatus *awbStatus = rpiMetadata.getLocked("awb.status"); if (awbStatus) { libcameraMetadata_.set(controls::ColourGains, { static_cast(awbStatus->gainR), static_cast(awbStatus->gainB) }); libcameraMetadata_.set(controls::ColourTemperature, awbStatus->temperatureK); } - BlackLevelStatus *blackLevelStatus = rpiMetadata_.getLocked("black_level.status"); + BlackLevelStatus *blackLevelStatus = rpiMetadata.getLocked("black_level.status"); if (blackLevelStatus) libcameraMetadata_.set(controls::SensorBlackLevels, { static_cast(blackLevelStatus->blackLevelR), @@ -582,7 +586,7 @@ void IPARPi::reportMetadata() static_cast(blackLevelStatus->blackLevelG), static_cast(blackLevelStatus->blackLevelB) }); - FocusStatus *focusStatus = rpiMetadata_.getLocked("focus.status"); + FocusStatus *focusStatus = rpiMetadata.getLocked("focus.status"); if (focusStatus && focusStatus->num == 12) { /* * We get a 4x3 grid of regions by default. Calculate the average @@ -593,7 +597,7 @@ void IPARPi::reportMetadata() libcameraMetadata_.set(controls::FocusFoM, focusFoM); } - CcmStatus *ccmStatus = rpiMetadata_.getLocked("ccm.status"); + CcmStatus *ccmStatus = rpiMetadata.getLocked("ccm.status"); if (ccmStatus) { float m[9]; for (unsigned int i = 0; i < 9; i++) @@ -1006,9 +1010,10 @@ void IPARPi::prepareISP(const ISPConfig &data) { int64_t frameTimestamp = data.controls.get(controls::SensorTimestamp).value_or(0); RPiController::Metadata lastMetadata; + RPiController::Metadata &rpiMetadata = rpiMetadata_[0]; Span embeddedBuffer; - lastMetadata = std::move(rpiMetadata_); + lastMetadata = std::move(rpiMetadata); fillDeviceStatus(data.controls); if (data.embeddedBufferPresent) { @@ -1025,7 +1030,7 @@ void IPARPi::prepareISP(const ISPConfig &data) * This may overwrite the DeviceStatus using values from the sensor * metadata, and may also do additional custom processing. */ - helper_->prepare(embeddedBuffer, rpiMetadata_); + helper_->prepare(embeddedBuffer, rpiMetadata); /* Done with embedded data now, return to pipeline handler asap. */ if (data.embeddedBufferPresent) @@ -1041,7 +1046,7 @@ void IPARPi::prepareISP(const ISPConfig &data) * current frame, or any other bits of metadata that were added * in helper_->Prepare(). */ - rpiMetadata_.merge(lastMetadata); + rpiMetadata.merge(lastMetadata); processPending_ = false; return; } @@ -1051,48 +1056,48 @@ void IPARPi::prepareISP(const ISPConfig &data) ControlList ctrls(ispCtrls_); - controller_.prepare(&rpiMetadata_); + controller_.prepare(&rpiMetadata); /* Lock the metadata buffer to avoid constant locks/unlocks. */ - std::unique_lock lock(rpiMetadata_); + std::unique_lock lock(rpiMetadata); - AwbStatus *awbStatus = rpiMetadata_.getLocked("awb.status"); + AwbStatus *awbStatus = rpiMetadata.getLocked("awb.status"); if (awbStatus) applyAWB(awbStatus, ctrls); - CcmStatus *ccmStatus = rpiMetadata_.getLocked("ccm.status"); + CcmStatus *ccmStatus = rpiMetadata.getLocked("ccm.status"); if (ccmStatus) applyCCM(ccmStatus, ctrls); - AgcStatus *dgStatus = rpiMetadata_.getLocked("agc.status"); + AgcStatus *dgStatus = rpiMetadata.getLocked("agc.status"); if (dgStatus) applyDG(dgStatus, ctrls); - AlscStatus *lsStatus = rpiMetadata_.getLocked("alsc.status"); + AlscStatus *lsStatus = rpiMetadata.getLocked("alsc.status"); if (lsStatus) applyLS(lsStatus, ctrls); - ContrastStatus *contrastStatus = rpiMetadata_.getLocked("contrast.status"); + ContrastStatus *contrastStatus = rpiMetadata.getLocked("contrast.status"); if (contrastStatus) applyGamma(contrastStatus, ctrls); - BlackLevelStatus *blackLevelStatus = rpiMetadata_.getLocked("black_level.status"); + BlackLevelStatus *blackLevelStatus = rpiMetadata.getLocked("black_level.status"); if (blackLevelStatus) applyBlackLevel(blackLevelStatus, ctrls); - GeqStatus *geqStatus = rpiMetadata_.getLocked("geq.status"); + GeqStatus *geqStatus = rpiMetadata.getLocked("geq.status"); if (geqStatus) applyGEQ(geqStatus, ctrls); - DenoiseStatus *denoiseStatus = rpiMetadata_.getLocked("denoise.status"); + DenoiseStatus *denoiseStatus = rpiMetadata.getLocked("denoise.status"); if (denoiseStatus) applyDenoise(denoiseStatus, ctrls); - SharpenStatus *sharpenStatus = rpiMetadata_.getLocked("sharpen.status"); + SharpenStatus *sharpenStatus = rpiMetadata.getLocked("sharpen.status"); if (sharpenStatus) applySharpen(sharpenStatus, ctrls); - DpcStatus *dpcStatus = rpiMetadata_.getLocked("dpc.status"); + DpcStatus *dpcStatus = rpiMetadata.getLocked("dpc.status"); if (dpcStatus) applyDPC(dpcStatus, ctrls); @@ -1116,11 +1121,13 @@ void IPARPi::fillDeviceStatus(const ControlList &sensorControls) LOG(IPARPI, Debug) << "Metadata - " << deviceStatus; - rpiMetadata_.set("device.status", deviceStatus); + rpiMetadata_[0].set("device.status", deviceStatus); } void IPARPi::processStats(unsigned int bufferId) { + RPiController::Metadata &rpiMetadata = rpiMetadata_[0]; + auto it = buffers_.find(bufferId); if (it == buffers_.end()) { LOG(IPARPI, Error) << "Could not find stats buffer!"; @@ -1130,11 +1137,11 @@ void IPARPi::processStats(unsigned int bufferId) Span mem = it->second.planes()[0]; bcm2835_isp_stats *stats = reinterpret_cast(mem.data()); RPiController::StatisticsPtr statistics = std::make_shared(*stats); - helper_->process(statistics, rpiMetadata_); - controller_.process(statistics, &rpiMetadata_); + helper_->process(statistics, rpiMetadata); + controller_.process(statistics, &rpiMetadata); struct AgcStatus agcStatus; - if (rpiMetadata_.get("agc.status", agcStatus) == 0) { + if (rpiMetadata.get("agc.status", agcStatus) == 0) { ControlList ctrls(sensorCtrls_); applyAGC(&agcStatus, ctrls); From patchwork Mon Oct 31 11:45:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17741 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 41694C3288 for ; Mon, 31 Oct 2022 11:45:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9C8A56304C; Mon, 31 Oct 2022 12:45:40 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1667216740; bh=3Jo1qVg3Sfuw1vzQ2V5aXJtvy5yWBNhAi4JDuOuniPU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=is+Eb1tkuOVY6GNp5NJbmsBCGPiEYlQphMJv2jv+cDQhMCNsIdIka5+rQOSjtKO/x mut+xA+g9K3E4QCW7Xlirp/oqU+cKZKw4/KDoOWEuqvY44fxu8DeEt9Kd+EJcCZW8b PgdpXwk99jXVITUXLZwGMpipw0zFa1hUt0tW6RtSQnu5Q5T7kFfeXPOzkLniNoJMSw ZZUNNxj2gZraNLbUP0Sq6Cg/eUkz2FRDUhUcr0TpaQQe51eWkWafluDccAEgw8PlMp HZyZSQRNQED42tMYX7ZLSvzmXIuCNw2x3aHCVoi1AcFrPSEbz0BdHVqfJvd4+FjhbK BKaKIbMLEtH3w== Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7A85863044 for ; Mon, 31 Oct 2022 12:45:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="AhDeLbUU"; dkim-atps=neutral Received: by mail-wr1-x432.google.com with SMTP id bk15so15536335wrb.13 for ; Mon, 31 Oct 2022 04:45:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5snii4aeFSeyDQEkTWsmNtv8fYZz92tBHLMau6EEI/s=; b=AhDeLbUUBigs5jAZeLKHfapt/5wozo1N4I5Ct2dW5+ieDW+vCORkC3f0pgbKX8Fskn mV9PPDuGPy1PuO3bDGzlhEiIOjjkzQv3BrXfG7SZcMaEtHevtm8807b/3yMeHeWEONKy d7eXgJZNqZyxQMkcRC4oOjExt1RGz8QsI5yHprF8Klaasdr//TPoD3BR5RLaVnm2wnSk YuMe5r4KsvSC2qjYDF9wx16piBtHTBwOmUDyvg1mmJTjSU0AaPsBgm/n0WCSTlNYAuO1 +DASMyNWOZ11LhAWIfF9tVyLLBHF2gDiPEpYWTBY/Lu1QTutoPlTM3b+SoAwXaA1sZcC UTOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5snii4aeFSeyDQEkTWsmNtv8fYZz92tBHLMau6EEI/s=; b=MKKbbFLJrMINDIv6x+OD6k08s0xpoyt5Knf3AfT/vVqY2TB+cGFo3qv7t4RH9i4zI7 +lgNx9TBaRWVtaP6d/vkXOk4565qln/vZ2yQeAksbx1o8ZRiHnXgXsxMMXuNOdljnQ3y A79VG4nY1ND5CQfvvYnnnrMntEVGYSEQMcfS/3DxGeQ+NX57KgohOJ437+eU9/kouOOZ DjOouiCIz1ECBbkg/r1nLoryyzTHAJmehwrlgNsi3KDCy4qsPP6yo40UQthrojEVsyRf ZhJTlqbAEjrPLdoUqDIpy7hsHM/x23JI+fmRrmo/MtDc53gImiug2ZaC3ipT+q5iqLV3 tdUw== X-Gm-Message-State: ACrzQf3QHEeh5UP3NTBTp4WxjJwCwiw/Vxa6kMF+owKgVOAE7tONUEIi KV8Ci+FrUgB6ybEfGftko1z8NYv4e+mGOg== X-Google-Smtp-Source: AMsMyM4oPcnPpkg7PGNAWFCC6EONDNp95A37W9JDinmGEwdNQDS2yXsN2hMOLRJlWV80kFAknjxhQQ== X-Received: by 2002:adf:c082:0:b0:236:d057:3fff with SMTP id d2-20020adfc082000000b00236d0573fffmr1979121wrf.206.1667216732784; Mon, 31 Oct 2022 04:45:32 -0700 (PDT) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id h11-20020adff18b000000b0023677e1157fsm6936354wro.56.2022.10.31.04.45.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Oct 2022 04:45:31 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Mon, 31 Oct 2022 11:45:21 +0000 Message-Id: <20221031114522.14215-7-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221031114522.14215-1-naush@raspberrypi.com> References: <20221031114522.14215-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 6/7] pipeline: ipa: raspberrypi: Use IPA cookies X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Pass an IPA cookie from the pipeline handler to the IPA and eventually back to the pipeline handler through the setDelayedControls signal. This cookie is used to index the RPiController::Metadata object to be used for the frame. The IPA cookie is then returned from DelayedControls when the frame with the applied controls has been returned from the sensor, and eventually passed back to the IPA from the signalIspPrepare signal. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: David Plowman Reviewed-by: Kieran Bingham --- include/libcamera/ipa/raspberrypi.mojom | 6 ++- src/ipa/raspberrypi/raspberrypi.cpp | 42 ++++++++++--------- .../pipeline/raspberrypi/raspberrypi.cpp | 16 ++++--- 3 files changed, 37 insertions(+), 27 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index 40f78d9e3b3f..325d2d855bc0 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -37,6 +37,8 @@ struct ISPConfig { uint32 bayerBufferId; bool embeddedBufferPresent; libcamera.ControlList controls; + uint32 ipaContext; + uint32 delayContext; }; struct IPAConfig { @@ -127,7 +129,7 @@ interface IPARPiInterface { */ unmapBuffers(array ids); - [async] signalStatReady(uint32 bufferId); + [async] signalStatReady(uint32 bufferId, uint32 ipaContext); [async] signalQueueRequest(libcamera.ControlList controls); [async] signalIspPrepare(ISPConfig data); }; @@ -137,5 +139,5 @@ interface IPARPiEventInterface { runIsp(uint32 bufferId); embeddedComplete(uint32 bufferId); setIspControls(libcamera.ControlList controls); - setDelayedControls(libcamera.ControlList controls); + setDelayedControls(libcamera.ControlList controls, uint32 delayContext); }; diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 799a4fe70000..194171a8bc96 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -126,7 +126,7 @@ public: ControlList *controls, IPAConfigResult *result) override; void mapBuffers(const std::vector &buffers) override; void unmapBuffers(const std::vector &ids) override; - void signalStatReady(const uint32_t bufferId) override; + void signalStatReady(const uint32_t bufferId, uint32_t ipaContext) override; void signalQueueRequest(const ControlList &controls) override; void signalIspPrepare(const ISPConfig &data) override; @@ -137,9 +137,9 @@ private: void queueRequest(const ControlList &controls); void returnEmbeddedBuffer(unsigned int bufferId); void prepareISP(const ISPConfig &data); - void reportMetadata(); - void fillDeviceStatus(const ControlList &sensorControls); - void processStats(unsigned int bufferId); + void reportMetadata(unsigned int ipaContext); + void fillDeviceStatus(const ControlList &sensorControls, unsigned int ipaContext); + void processStats(unsigned int bufferId, unsigned int ipaContext); void applyFrameDurations(Duration minFrameDuration, Duration maxFrameDuration); void applyAGC(const struct AgcStatus *agcStatus, ControlList &ctrls); void applyAWB(const struct AwbStatus *awbStatus, ControlList &ctrls); @@ -509,14 +509,16 @@ void IPARPi::unmapBuffers(const std::vector &ids) } } -void IPARPi::signalStatReady(uint32_t bufferId) +void IPARPi::signalStatReady(uint32_t bufferId, uint32_t ipaContext) { + unsigned int context = ipaContext % rpiMetadata_.size(); + if (++checkCount_ != frameCount_) /* assert here? */ LOG(IPARPI, Error) << "WARNING: Prepare/Process mismatch!!!"; if (processPending_ && frameCount_ > mistrustCount_) - processStats(bufferId); + processStats(bufferId, context); - reportMetadata(); + reportMetadata(context); statsMetadataComplete.emit(bufferId & MaskID, libcameraMetadata_); } @@ -540,9 +542,9 @@ void IPARPi::signalIspPrepare(const ISPConfig &data) runIsp.emit(data.bayerBufferId & MaskID); } -void IPARPi::reportMetadata() +void IPARPi::reportMetadata(unsigned int ipaContext) { - RPiController::Metadata &rpiMetadata = rpiMetadata_[0]; + RPiController::Metadata &rpiMetadata = rpiMetadata_[ipaContext]; std::unique_lock lock(rpiMetadata); /* @@ -1009,12 +1011,12 @@ void IPARPi::returnEmbeddedBuffer(unsigned int bufferId) void IPARPi::prepareISP(const ISPConfig &data) { int64_t frameTimestamp = data.controls.get(controls::SensorTimestamp).value_or(0); - RPiController::Metadata lastMetadata; - RPiController::Metadata &rpiMetadata = rpiMetadata_[0]; + unsigned int ipaContext = data.ipaContext % rpiMetadata_.size(); + RPiController::Metadata &rpiMetadata = rpiMetadata_[ipaContext]; Span embeddedBuffer; - lastMetadata = std::move(rpiMetadata); - fillDeviceStatus(data.controls); + rpiMetadata.clear(); + fillDeviceStatus(data.controls, ipaContext); if (data.embeddedBufferPresent) { /* @@ -1046,7 +1048,9 @@ void IPARPi::prepareISP(const ISPConfig &data) * current frame, or any other bits of metadata that were added * in helper_->Prepare(). */ - rpiMetadata.merge(lastMetadata); + RPiController::Metadata &lastMetadata = + rpiMetadata_[ipaContext ? ipaContext : rpiMetadata_.size()]; + rpiMetadata.mergeCopy(lastMetadata); processPending_ = false; return; } @@ -1105,7 +1109,7 @@ void IPARPi::prepareISP(const ISPConfig &data) setIspControls.emit(ctrls); } -void IPARPi::fillDeviceStatus(const ControlList &sensorControls) +void IPARPi::fillDeviceStatus(const ControlList &sensorControls, unsigned int ipaContext) { DeviceStatus deviceStatus = {}; @@ -1121,12 +1125,12 @@ void IPARPi::fillDeviceStatus(const ControlList &sensorControls) LOG(IPARPI, Debug) << "Metadata - " << deviceStatus; - rpiMetadata_[0].set("device.status", deviceStatus); + rpiMetadata_[ipaContext].set("device.status", deviceStatus); } -void IPARPi::processStats(unsigned int bufferId) +void IPARPi::processStats(unsigned int bufferId, unsigned int ipaContext) { - RPiController::Metadata &rpiMetadata = rpiMetadata_[0]; + RPiController::Metadata &rpiMetadata = rpiMetadata_[ipaContext]; auto it = buffers_.find(bufferId); if (it == buffers_.end()) { @@ -1145,7 +1149,7 @@ void IPARPi::processStats(unsigned int bufferId) ControlList ctrls(sensorCtrls_); applyAGC(&agcStatus, ctrls); - setDelayedControls.emit(ctrls); + setDelayedControls.emit(ctrls, ipaContext); } } diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 29626c0ef14e..cae1f7b9aea3 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -206,7 +206,7 @@ public: void runIsp(uint32_t bufferId); void embeddedComplete(uint32_t bufferId); void setIspControls(const ControlList &controls); - void setDelayedControls(const ControlList &controls); + void setDelayedControls(const ControlList &controls, uint32_t delayContext); void setSensorControls(ControlList &controls); void unicamTimeout(); @@ -262,6 +262,7 @@ public: struct BayerFrame { FrameBuffer *buffer; ControlList controls; + unsigned int delayContext; }; std::queue bayerQueue_; @@ -1792,9 +1793,9 @@ void RPiCameraData::setIspControls(const ControlList &controls) handleState(); } -void RPiCameraData::setDelayedControls(const ControlList &controls) +void RPiCameraData::setDelayedControls(const ControlList &controls, uint32_t delayContext) { - if (!delayedCtrls_->push(controls, 0)) + if (!delayedCtrls_->push(controls, delayContext)) LOG(RPI, Error) << "V4L2 DelayedControl set failed"; handleState(); } @@ -1867,13 +1868,13 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) * Lookup the sensor controls used for this frame sequence from * DelayedControl and queue them along with the frame buffer. */ - auto [ctrl, cookie] = delayedCtrls_->get(buffer->metadata().sequence); + auto [ctrl, delayContext] = delayedCtrls_->get(buffer->metadata().sequence); /* * Add the frame timestamp to the ControlList for the IPA to use * as it does not receive the FrameBuffer object. */ ctrl.set(controls::SensorTimestamp, buffer->metadata().timestamp); - bayerQueue_.push({ buffer, std::move(ctrl) }); + bayerQueue_.push({ buffer, std::move(ctrl), delayContext }); } else { embeddedQueue_.push(buffer); } @@ -1923,7 +1924,8 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) * application until after the IPA signals so. */ if (stream == &isp_[Isp::Stats]) { - ipa_->signalStatReady(ipa::RPi::MaskStats | static_cast(index)); + ipa_->signalStatReady(ipa::RPi::MaskStats | static_cast(index), + requestQueue_.front()->sequence()); } else { /* Any other ISP output can be handed back to the application now. */ handleStreamBuffer(buffer, stream); @@ -2168,6 +2170,8 @@ void RPiCameraData::tryRunPipeline() ipa::RPi::ISPConfig ispPrepare; ispPrepare.bayerBufferId = ipa::RPi::MaskBayerData | bayerId; ispPrepare.controls = std::move(bayerFrame.controls); + ispPrepare.ipaContext = request->sequence(); + ispPrepare.delayContext = bayerFrame.delayContext; if (embeddedBuffer) { unsigned int embeddedId = unicam_[Unicam::Embedded].getBufferId(embeddedBuffer); From patchwork Mon Oct 31 11:45:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17742 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CEB71C3289 for ; Mon, 31 Oct 2022 11:45:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3600763044; Mon, 31 Oct 2022 12:45:41 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1667216741; bh=YhIxN2di9TyNXMbx5fMifQcQb3B04ncA8Y/T2+CW3v4=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=PlAf2+JMx1uQhSR6UDYm1U5uEas/v+hxpKKhh0DrYzGavFT5REsyVNBtoz7qaRxgc LzotytDHpMDFFcpT2QF6mi+7pxk6XPrTAHnqGHqudW8eF+gAvf+LvJswitkT1eNsjm Y+nO+4r5DJE4i0bViAPc8b1B4QOTZlEp269wDdxkx3ZUfWrw8r8UYNjp/JumR6gmJY ELjLy17oD6ua1eQnbFgv+C8u51bVgSiYL94gipg68usgLjngkFpnNrCiE//+Pggb1D j3mLVJpSRg6+lO5ZOjD0B0Bdr1PgjQ+rI5cu4kz5XUwGBFfKWL8dcbLzrArTNK6618 nKDwIiQ0lnBBg== Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9BE466304A for ; Mon, 31 Oct 2022 12:45:34 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="SzHvhgnr"; dkim-atps=neutral Received: by mail-wr1-x42d.google.com with SMTP id y16so15541192wrt.12 for ; Mon, 31 Oct 2022 04:45:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zdhz+1j/NkLDJJXH7usJMOs+Zyvf3jcELZrgcTTFIAE=; b=SzHvhgnrmwkVeMJAnSqa49KEkX+Tgdo+9L821PlYWqGqNGzVimg5RNXlnR74OFc6Mz gqIALBJw29PnR3+3AMG2WIfnbUEHMAh8IO8awejn5Oy+giNdp4Pv7y7Y1p0NGZiwY5xG 1+Xm/P0BBgwn9Zet4cNRUK1LUxxwouH409qwUR+ura16ZRc/hJ+kXqhz/hZx1d0zAKGm SbMeuYXo/0MLE3voYXOsnE40vEDQKiocc7XuUi6YLHUmx0qa8x/LEdMuqXhZ/tmVdFaG 1wYvPkrVdWl+4Mi3akbe3BMJ1ayB8+pnz4rXKp5YpsQBwcpTmQOo8Qe9rgDWjh74Mvhi mwVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zdhz+1j/NkLDJJXH7usJMOs+Zyvf3jcELZrgcTTFIAE=; b=BMYYjlkYWWkmC8HSUOkHVdSzSgyEi78CpqC5OmZnI/QUcY9OPX+A5tQggg15M0+BbC dQbFby5B7+AWzgVIInyF4vrUsPwEqcViZtwXXwSqCz9z/2Uuv1morZ+cv4eU40dqSmZm isDJwXMZYqfDAPKyXahGsMRxmY9p1vW8RSFWahjgp4G9u7ORoGFUMuV8DD/RlgA1rJ6x X/D/mGWnvIYkhdhoumzhU6/VE04R/wX+uv+zVoJt20dFReTgKNcMW/sGSnb6DW+/+ND7 ZQyWlujzsi4PZs1g+2sb/IyDyElNvGtVPxeoUl/1nKIoZulO2yPTUY+4odqPxQMsNc4B NddA== X-Gm-Message-State: ACrzQf1/YMOtLRgLguRwnnTZPJFaVhlMSeachVVZ5FayghByKGrkMglJ hCUEEei5OE5pj9WxTwu1E0u0PQCpw8auRw== X-Google-Smtp-Source: AMsMyM5scar6dX258fobLOYECkBrPtOgSE8GPOmWrlEZr9x5PYXumD2thclZp+Ow+sIpbEFEYxShDw== X-Received: by 2002:a5d:698e:0:b0:236:4b95:17ff with SMTP id g14-20020a5d698e000000b002364b9517ffmr8087204wru.196.1667216733681; Mon, 31 Oct 2022 04:45:33 -0700 (PDT) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id h11-20020adff18b000000b0023677e1157fsm6936354wro.56.2022.10.31.04.45.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Oct 2022 04:45:33 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Mon, 31 Oct 2022 11:45:22 +0000 Message-Id: <20221031114522.14215-8-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221031114522.14215-1-naush@raspberrypi.com> References: <20221031114522.14215-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 7/7] ipa: raspberrypi: agc: Fix digital gain calculation for manual mode X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The digital gain calculation uses a total exposure value computed with the current AGC state. However, this is wrong in the case of manual shutter/gain controls, as the total exposure value used must be the value computed when the AGC sent the manual shutter/gain controls to the pipeline handler to action. To fix this, the IPA now adds the historical AgcStatus structure to the metadata (tagged with "agc.delayed_status"). This historical AgcStatus structure contains the total exposure value calculated when the AGC sent the manual shutter/gain controls to the pipeline handler. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: David Plowman Reviewed-by: Kieran Bingham --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 10 ++++++++-- src/ipa/raspberrypi/raspberrypi.cpp | 11 +++++++++++ 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index bd54a639d637..b18bd7b5b19e 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -408,6 +408,12 @@ void Agc::switchMode(CameraMode const &cameraMode, void Agc::prepare(Metadata *imageMetadata) { + Duration totalExposureValue = status_.totalExposureValue; + AgcStatus delayedStatus; + + if (!imageMetadata->get("agc.delayed_status", delayedStatus)) + totalExposureValue = delayedStatus.totalExposureValue; + status_.digitalGain = 1.0; fetchAwbStatus(imageMetadata); /* always fetch it so that Process knows it's been done */ @@ -418,8 +424,8 @@ void Agc::prepare(Metadata *imageMetadata) Duration actualExposure = deviceStatus.shutterSpeed * deviceStatus.analogueGain; if (actualExposure) { - status_.digitalGain = status_.totalExposureValue / actualExposure; - LOG(RPiAgc, Debug) << "Want total exposure " << status_.totalExposureValue; + status_.digitalGain = totalExposureValue / actualExposure; + LOG(RPiAgc, Debug) << "Want total exposure " << totalExposureValue; /* * Never ask for a gain < 1.0, and also impose * some upper limit. Make it customisable? diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 194171a8bc96..7904f2b696dd 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -1028,6 +1028,17 @@ void IPARPi::prepareISP(const ISPConfig &data) embeddedBuffer = it->second.planes()[0]; } + /* + * AGC wants to know the algorithm status from the time it actioned the + * sensor exposure/gain changes. So fetch it from the metadata list + * indexed by the IPA cookie returned, and put it in the current frame + * metadata. + */ + AgcStatus agcStatus; + RPiController::Metadata &delayedMetadata = rpiMetadata_[data.delayContext]; + if (!delayedMetadata.get("agc.status", agcStatus)) + rpiMetadata.set("agc.delayed_status", agcStatus); + /* * This may overwrite the DeviceStatus using values from the sensor * metadata, and may also do additional custom processing.