From patchwork Fri Sep 2 12:08:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17274 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A467CC327D for ; Fri, 2 Sep 2022 12:08:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6826C61FDF; Fri, 2 Sep 2022 14:08:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662120498; bh=gBEVAsghRyLUxKvoBP8RRSW7iyAeDStcB1KGpH1JaZU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=fWCCFafZMDV1UWnq0PTXMMEdCpKUz6HIuRFOsNAR8ls9Gz45b0HHGbIkayRxfXj13 rULVhpp93EjRn0ZQmPUaQorVseQ2mRUSyv2/YQg8kkzFaCgWaNJorcVhQC5og/1+6n 2N7V9cyRcb9Pjjw7/Ad4CzJWTJjSGBtirdJ0XHsamo6ShpzYm1K6do+eXuzoK0gDty 5s0KqCVskrHd3yksck+alb4avHFu9HtLX/JlT/vuRUZ1jwLtWjZW40AdeIkFH4Hhxe tbs8YTenjZvvSWuEXh5ggiXDYfN3b+mxXgVI3FpE8+cD0XqG3z9EcRHxhyt4ooA8vK 40TiPkmj1sN4w== Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D62FF603E1 for ; Fri, 2 Sep 2022 14:08:16 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="NILdb5al"; dkim-atps=neutral Received: by mail-wm1-x32a.google.com with SMTP id k17so1132858wmr.2 for ; Fri, 02 Sep 2022 05:08:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=YJjap5Vlhvd2qi+OflmsVt8uhRG4R/2mis0V9sewWMk=; b=NILdb5aloWYnTaJCZrGezyD2nxg5OB5WjEW5Rbqrh1qpeHIhHtKQdIO2aXjd1Mp6hx OZUbp8BIUBZW9+cA7Y/Tu9tJXZlcXCJooJNczXbTjoNC5TtCZVPJ7J6gm3hgZlLDYOos loNJMAtbll91VV0JZLtZ7/Zn7kvTHwIEBDejSOaE/AbwCuOeAWesZlKgtFmYEiWAy49J uIf3cC9riGDGBvV+nFMN6//w4MQ+NBKu8p8dbICR/xyEFM8ipWy8CUxUIr4y+jDYNhPY pykPquEaQh1wQkmGWVW6rpdhPmEkqWpY1WG2YO6pCMsj/K93j5j7SAuyjwRoUpcaWknr tAzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=YJjap5Vlhvd2qi+OflmsVt8uhRG4R/2mis0V9sewWMk=; b=X7gRv8LqzHBmJNqNYjFjno091BiJOEsUCat2GeUL4XOPNNcCcoocb9weltzTj2o7J0 kTAjFW1TxVQiZw8khaDQ51d5VucSfVUCpr+j391bm8k9mnWko4Y93/ARR0SC6Yyl/Z5C xiO254m8IO1yCWDiyy11Gjl+RPyFOYg4/jSK3SgUAHdRbcV6xsij9JQfw76wTpxl5ykH kphz6M2c4Q3rXNV8l7+bmTpJyfcG+hukU2C7BXsPnu3w49NdJ6ljNoUWssUuEs6v3KLj hZ5pWRoWLM5XfGVGCa5arNVviR2JcPdmsRn7esXTXFuLlSBHylqTy5FH3wo1wC7fLEez sP4Q== X-Gm-Message-State: ACgBeo06cojEOskGw4MyP9888X8ebYpWR5l3N21B7oi64Q4mLY3CY9FT D0ECQTkaMbEFPLXuWaMD3onTMNLD5f2oFqWU X-Google-Smtp-Source: AA6agR7syQlFPTT+++I5hyugMczTA0UncrZSmQ5a5wQu9QL++DgSflU7WqrfX1YWSlo2UFsJHnhCxg== X-Received: by 2002:a05:600c:4e8d:b0:3a5:f7ed:873 with SMTP id f13-20020a05600c4e8d00b003a5f7ed0873mr2703331wmq.170.1662120496217; Fri, 02 Sep 2022 05:08:16 -0700 (PDT) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id h2-20020a5d4302000000b0021e51c039c5sm1397809wrq.80.2022.09.02.05.08.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Sep 2022 05:08:15 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 2 Sep 2022 13:08:03 +0100 Message-Id: <20220902120808.842-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220902120808.842-1-naush@raspberrypi.com> References: <20220902120808.842-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 1/6] delayed_controls: Template the ControlRingBuffer class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Convert ControlRingBuffer to a templated class to allow arbitrary ring buffer array types to be defined. Signed-off-by: Naushir Patuck --- include/libcamera/internal/delayed_controls.h | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/include/libcamera/internal/delayed_controls.h b/include/libcamera/internal/delayed_controls.h index aef37077103c..f6e622f1dfba 100644 --- a/include/libcamera/internal/delayed_controls.h +++ b/include/libcamera/internal/delayed_controls.h @@ -53,17 +53,18 @@ private: /* \todo Make the listSize configurable at instance creation time. */ static constexpr int listSize = 16; - class ControlRingBuffer : public std::array + template + class ControlRingBuffer : public std::array { public: - Info &operator[](unsigned int index) + T &operator[](unsigned int index) { - return std::array::operator[](index % listSize); + return std::array::operator[](index % listSize); } - const Info &operator[](unsigned int index) const + const T &operator[](unsigned int index) const { - return std::array::operator[](index % listSize); + return std::array::operator[](index % listSize); } }; @@ -75,7 +76,7 @@ private: uint32_t queueCount_; uint32_t writeCount_; /* \todo Evaluate if we should index on ControlId * or unsigned int */ - std::unordered_map values_; + std::unordered_map> values_; }; } /* namespace libcamera */ From patchwork Fri Sep 2 12:08:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17275 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C22E1C0DA4 for ; Fri, 2 Sep 2022 12:08:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E7DDD61FE3; Fri, 2 Sep 2022 14:08:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662120500; bh=g+yojow+QSITK4eB7dyNWvdns/J7/9ZmNwJ1ghjHEIY=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=pnKmR74Yk5lNZrXafAVXsIPJ+fh49TdOz5bBwXEzRgb9FaHgVFKafAQ0HjIg/FXwv OUPDK8k5YFTodMgdZiWEduCO/CT2IuE0sf3yF+QFhLF4TAEGfkrYApIXIcCcAhL53H Jg0X3GtFJRtPx/Xh45TzRWXHNb6WkBMpb11aANrDKXYvo3Se3EsgyXHtjENXwmfq+f OckPGpPCIF9DgOQO1CueFG4V8M7uVpveybVeB8mumGmHArwgwkraOXAoIYCMuu5oIu Cs2RjlVSmLsxL4PEk2l0IUZdQT+yUpiVqyl9SIQYMhxrfs57e9DYxRDMUtgCu/aWLV zQx1Jejrp2DPw== Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BCCD461FDE for ; Fri, 2 Sep 2022 14:08:17 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="h13YdhOh"; dkim-atps=neutral Received: by mail-wm1-x334.google.com with SMTP id i188-20020a1c3bc5000000b003a7b6ae4eb2so3207218wma.4 for ; Fri, 02 Sep 2022 05:08:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=TLd3IVP2iO3I4bZNxOYJauQGUqiKFviOy3ElyLrY/78=; b=h13YdhOhNKnO856Arbhty28zkJO5AUbbf53EukhhS57hpxaXE16yWnIKiF9enTuWbf S5PaGP5iJui7iKsa2cHymW9dA0RgcS/t8mQvOm3Ys5nipE3ZIr6Pu2hUs3WunXl/c/8u ug8a66gO2A5sgWm1SdxlqbqPCAeplySoplXuA2LYPjebEFTqLm9z1WBBCZonAvqL2peV sq9d8qrNymO44vjD9LTQsW+WTat0/N62qB+h+4yrbBdBu1oBhU5exWnJRRQsZM8g+BOq OlqfAQgzjJGqchhyaKxioI7WJ7hU6Y565bJga2ufCkI/wLdo7XPloueLafSFVNJ/sd9L JF2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=TLd3IVP2iO3I4bZNxOYJauQGUqiKFviOy3ElyLrY/78=; b=pgYFBzwwrFK81zWdOZn3esoi/K4FUehoOVLxvqL9JnQS0C5zPjLiJAELztxp2KU+J7 HigcGj9s73nTMTQRChycMx3X24Ik3Sq2W8MUUPRTUJWmztDcvzz8ao0A7a+TvbhfNQWf 0Kziv5YGHs30Pcp68HaccZcl1r2UNXckuxTjUglsaFwPLWiOLo9z7ud17YhLyL4ExTLL G7Qxu5gYBr294r++0NPBzsHPck30SUvraPevyXNty3/ex990Q0mO3fNnmjuiLDNQnfdA uhSAQVEyz8ATCDKDhg/hbkZdyuCCJyg23x+9pzb991w4DVtD+7O9TejDlkyl2BEPcF/e PFBA== X-Gm-Message-State: ACgBeo3g8z56gdbf7HwezcwkNxf0ZOTRjA4tTYqoXvkcsIsgKPHgeoDY GTg8RE0lMc7y6leoMLRmuVgcyUnWnKGL1AZG X-Google-Smtp-Source: AA6agR7wxSdKAqwMPJGx1UqSLHCZs7BuRGGZ3AWdcvbFIRox8BGIUltDh5CIXT6C3x5CQAcxafK4oA== X-Received: by 2002:a05:600c:3ac9:b0:3a5:f114:1f8 with SMTP id d9-20020a05600c3ac900b003a5f11401f8mr2557143wms.204.1662120497174; Fri, 02 Sep 2022 05:08:17 -0700 (PDT) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id h2-20020a5d4302000000b0021e51c039c5sm1397809wrq.80.2022.09.02.05.08.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Sep 2022 05:08:16 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 2 Sep 2022 13:08:04 +0100 Message-Id: <20220902120808.842-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220902120808.842-1-naush@raspberrypi.com> References: <20220902120808.842-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 2/6] delayed_controls: Add user cookie to DelayedControls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Allow the caller to provide a cookie value to DelayedControls::reset and DelayedControls::push. This cookie value is returned from DelayedControls::get for the frame that has the control values applied to it. The cookie value is useful in tracking when a set of controls has been applied to a frame. In a subsequent commit, it will be used by the Raspberry Pi IPA to track the IPA context used when setting the control values. Signed-off-by: Naushir Patuck --- include/libcamera/internal/delayed_controls.h | 8 +++++--- src/libcamera/delayed_controls.cpp | 17 +++++++++++------ src/libcamera/pipeline/ipu3/ipu3.cpp | 3 ++- .../pipeline/raspberrypi/raspberrypi.cpp | 2 +- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 3 ++- test/delayed_controls.cpp | 8 ++++---- 6 files changed, 25 insertions(+), 16 deletions(-) diff --git a/include/libcamera/internal/delayed_controls.h b/include/libcamera/internal/delayed_controls.h index f6e622f1dfba..fa222bb17d23 100644 --- a/include/libcamera/internal/delayed_controls.h +++ b/include/libcamera/internal/delayed_controls.h @@ -9,6 +9,7 @@ #include #include +#include #include @@ -27,10 +28,10 @@ public: DelayedControls(V4L2Device *device, const std::unordered_map &controlParams); - void reset(); + void reset(unsigned int cookie = 0); - bool push(const ControlList &controls); - ControlList get(uint32_t sequence); + bool push(const ControlList &controls, unsigned int cookie = 0); + std::pair get(uint32_t sequence); void applyControls(uint32_t sequence); @@ -77,6 +78,7 @@ private: uint32_t writeCount_; /* \todo Evaluate if we should index on ControlId * or unsigned int */ std::unordered_map> values_; + ControlRingBuffer cookies_; }; } /* namespace libcamera */ diff --git a/src/libcamera/delayed_controls.cpp b/src/libcamera/delayed_controls.cpp index 777441e8222a..c2198d0e8d77 100644 --- a/src/libcamera/delayed_controls.cpp +++ b/src/libcamera/delayed_controls.cpp @@ -109,14 +109,16 @@ DelayedControls::DelayedControls(V4L2Device *device, /** * \brief Reset state machine + * \param[in] cookie User supplied reset cookie value * * Resets the state machine to a starting position based on control values * retrieved from the device. */ -void DelayedControls::reset() +void DelayedControls::reset(unsigned int cookie) { queueCount_ = 1; writeCount_ = 0; + cookies_[0] = cookie; /* Retrieve control as reported by the device. */ std::vector ids; @@ -140,13 +142,14 @@ void DelayedControls::reset() /** * \brief Push a set of controls on the queue * \param[in] controls List of controls to add to the device queue + * \param[in] cookie User supplied cookie value for \a controls * * Push a set of controls to the control queue. This increases the control queue * depth by one. * * \returns true if \a controls are accepted, or false otherwise */ -bool DelayedControls::push(const ControlList &controls) +bool DelayedControls::push(const ControlList &controls, const unsigned int cookie) { /* Copy state from previous frame. */ for (auto &ctrl : values_) { @@ -180,6 +183,7 @@ bool DelayedControls::push(const ControlList &controls) << " at index " << queueCount_; } + cookies_[queueCount_] = cookie; queueCount_++; return true; @@ -198,9 +202,10 @@ bool DelayedControls::push(const ControlList &controls) * push(). The max history from the current sequence number that yields valid * values are thus 16 minus number of controls pushed. * - * \return The controls at \a sequence number + * \return The controls at \a sequence number and associated user supplied + * cookie value. */ -ControlList DelayedControls::get(uint32_t sequence) +std::pair DelayedControls::get(uint32_t sequence) { unsigned int index = std::max(0, sequence - maxDelay_); @@ -217,7 +222,7 @@ ControlList DelayedControls::get(uint32_t sequence) << " at index " << index; } - return out; + return { out, cookies_[index] }; } /** @@ -276,7 +281,7 @@ void DelayedControls::applyControls(uint32_t sequence) while (writeCount_ > queueCount_) { LOG(DelayedControls, Debug) << "Queue is empty, auto queue no-op."; - push({}); + push({}, cookies_[queueCount_ - 1]); } device_->setControls(&out); diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 93219a6c1134..b0f0b22c5298 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -1385,7 +1385,8 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) request->metadata().set(controls::SensorTimestamp, buffer->metadata().timestamp); - info->effectiveSensorControls = delayedCtrls_->get(buffer->metadata().sequence); + auto [controls, cookie] = delayedCtrls_->get(buffer->metadata().sequence); + info->effectiveSensorControls = std::move(controls); if (request->findBuffer(&rawStream_)) pipe()->completeBuffer(request, buffer); diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index b4094898ca6c..d34a906c3cea 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -1848,7 +1848,7 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) * Lookup the sensor controls used for this frame sequence from * DelayedControl and queue them along with the frame buffer. */ - ControlList ctrl = delayedCtrls_->get(buffer->metadata().sequence); + auto [ctrl, cookie] = delayedCtrls_->get(buffer->metadata().sequence); /* * Add the frame timestamp to the ControlList for the IPA to use * as it does not receive the FrameBuffer object. diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 03bbe6b467ae..cec90810ff95 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1159,8 +1159,9 @@ void PipelineHandlerRkISP1::statReady(FrameBuffer *buffer) if (data->frame_ <= buffer->metadata().sequence) data->frame_ = buffer->metadata().sequence + 1; + auto [controls, cooke] = data->delayedCtrls_->get(buffer->metadata().sequence); data->ipa_->processStatsBuffer(info->frame, info->statBuffer->cookie(), - data->delayedCtrls_->get(buffer->metadata().sequence)); + controls); } REGISTER_PIPELINE_HANDLER(PipelineHandlerRkISP1) diff --git a/test/delayed_controls.cpp b/test/delayed_controls.cpp index a8ce9828d73d..322c545998b2 100644 --- a/test/delayed_controls.cpp +++ b/test/delayed_controls.cpp @@ -96,7 +96,7 @@ protected: delayed->applyControls(i); - ControlList result = delayed->get(i); + auto [result, cookie] = delayed->get(i); int32_t brightness = result.get(V4L2_CID_BRIGHTNESS).get(); if (brightness != value) { cerr << "Failed single control without delay" @@ -138,7 +138,7 @@ protected: delayed->applyControls(i); - ControlList result = delayed->get(i); + auto [result, cookie] = delayed->get(i); int32_t brightness = result.get(V4L2_CID_BRIGHTNESS).get(); if (brightness != expected) { cerr << "Failed single control with delay" @@ -187,7 +187,7 @@ protected: delayed->applyControls(i); - ControlList result = delayed->get(i); + auto [result, cookie] = delayed->get(i); int32_t brightness = result.get(V4L2_CID_BRIGHTNESS).get(); int32_t contrast = result.get(V4L2_CID_CONTRAST).get(); if (brightness != expected || contrast != expected + 1) { @@ -247,7 +247,7 @@ protected: delayed->applyControls(i); - ControlList result = delayed->get(i); + auto [result, cookie] = delayed->get(i); int32_t brightness = result.get(V4L2_CID_BRIGHTNESS).get(); int32_t contrast = result.get(V4L2_CID_CONTRAST).get(); From patchwork Fri Sep 2 12:08:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17276 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 625CDC327D for ; Fri, 2 Sep 2022 12:08:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 728E361FE9; Fri, 2 Sep 2022 14:08:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662120500; bh=YG8zKuN4cx17jT26zExVUEUn1t8fvoswfulGuF4ti20=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=ONDYJhGWfspdS61ki9ykU/bF7LNFXokF7bWrHSoL14a8IBBDZJ77sHOxMCknseqKc 6g9ET/ckSa2W1bDMz6C5pPGDXM68bUeeNPlUrFJA1K4R28yVCp/1OOKgL7EE6g7ZK2 JySDXeWX5O+qs/Re5JdeuoKF8BrUbXOdlMTEChF9Fjp4uJG2rH2VmWD62U8jQzAp/a vIdn11Ivmr5K9+bycAOXfhkwHsTeaTaDH1VO4oZLMVrHzW4aOONMmp7wTs9Op8sKsN Hpn0V1hiHvrkYti1BP51FL5DWSmD0XbeJrFOKYddzgZsBsiLkcYPykPUNGahoeZIpb xGYTe4PLIQfbw== Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D53C761FE3 for ; Fri, 2 Sep 2022 14:08:18 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="kmyfU4tl"; dkim-atps=neutral Received: by mail-wm1-x333.google.com with SMTP id c131-20020a1c3589000000b003a84b160addso2517911wma.2 for ; Fri, 02 Sep 2022 05:08:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=CiHEOsMF91m92inxh4BZs7UxuXbwFOzk4SqKgH3uS1Q=; b=kmyfU4tl7LjBkEWM06Khb3KtGaZYzmfL1xsmuMikFpkffKdbPH1LA9LNiS0Ypaabri sYsIWfMZ2kDKoQPZA1on0txgyHXeLmg6KpePkUtptmOI+C0hrL3uJhspcYEMBqLTuswh Zhlc6dV1rx4wV8TV6gHOzMRneMUzX/3z209Uozuy2cdFT/tDHRFfwG2hdX59kDWTZJNN iX+f89QZ4qpxcnr+fERtgddA5iE40MGkiJzLRBziNywcPXxFkxxABEjX3q4prZxgEBVs 7td3OR5stvkYrFeYlyh5j9DfDv/ujiAQd46iYjXjTAMBU/rL1CGnZyBgCwiot99U3Orm oonQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=CiHEOsMF91m92inxh4BZs7UxuXbwFOzk4SqKgH3uS1Q=; b=2gNB1GmABskDMMf/XKwbWoJDvqdLkl3vthwfz2cOAG2Jue/v1JKiOpW/3KUgn/hpjX wPAGMSO6PHU9wBJbqJEiUlKILKTPX9h9J4S18qCF9lnpDHrl9WpQVXpPin/Tayzvd5Dm aEaulDqRiIOoUaPa8d50xMVIo7+iajGRtowPpSHzCcUUptr/9HkS3fTGkP94vIj36Teq SEYcyMo3zbhohdXFN6nPCrL6g85uZp6/cVL+F56K1F9H31VVy5He0xGOkEMXUCNshgj9 0VQY3k4wwx3s7gpOsW7wy6CmvxNJveaqG7h+mHgGgpOxaybmnFgoQYRGWbmS7FILV/c+ /cNA== X-Gm-Message-State: ACgBeo0PnYuMvuc3gvLkliWkf/gI8eeAfddwTs057a1GUekFMKs0zeyq eDWOKyfuuwfAL40L2z4DRwVr82VlBpALf5Ex X-Google-Smtp-Source: AA6agR7QvNgUJEqlZNjoocw1nXVcHBeYu75N0XE8tcf/wpDpQcX0/WTuNj/bzpesuMYi9a9Xo8IYgQ== X-Received: by 2002:a05:600c:22c8:b0:3a5:c134:1f50 with SMTP id 8-20020a05600c22c800b003a5c1341f50mr2693161wmg.55.1662120498006; Fri, 02 Sep 2022 05:08:18 -0700 (PDT) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id h2-20020a5d4302000000b0021e51c039c5sm1397809wrq.80.2022.09.02.05.08.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Sep 2022 05:08:17 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 2 Sep 2022 13:08:05 +0100 Message-Id: <20220902120808.842-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220902120808.842-1-naush@raspberrypi.com> References: <20220902120808.842-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 3/6] tests: delayed_controls: Add cookie value test X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a test for passing and returning cookie values in DelayedControls. Signed-off-by: Naushir Patuck --- test/delayed_controls.cpp | 44 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/test/delayed_controls.cpp b/test/delayed_controls.cpp index 322c545998b2..5cc7d3aed4fd 100644 --- a/test/delayed_controls.cpp +++ b/test/delayed_controls.cpp @@ -267,6 +267,45 @@ protected: return TestPass; } + int cookieValue() + { + std::unordered_map delays = { + { V4L2_CID_BRIGHTNESS, { 1, false } }, + }; + std::unique_ptr delayed = + std::make_unique(dev_.get(), delays); + ControlList ctrls; + + /* Set a cookie to the reset value. */ + const unsigned int startCookie = 0x1234; + ctrls.set(V4L2_CID_BRIGHTNESS, 1); + dev_->setControls(&ctrls); + delayed->reset(startCookie); + + /* Trigger the first frame start event */ + delayed->applyControls(0); + + for (unsigned int i = 1; i < 100; i++) { + ctrls.set(V4L2_CID_BRIGHTNESS, 1); + delayed->push(ctrls, startCookie + i); + + delayed->applyControls(i); + + auto [result, cookie] = delayed->get(i); + unsigned int expected = startCookie + i - 1; + if (cookie != expected) { + cerr << "Failed cookie value" + << " frame " << i + << " expected cookie " << expected + << " got cookie " << cookie + << endl; + return TestFail; + } + } + + return TestPass; + } + int run() override { int ret; @@ -291,6 +330,11 @@ protected: if (ret) return ret; + /* Test cookie values. */ + ret = cookieValue(); + if (ret) + return ret; + return TestPass; } From patchwork Fri Sep 2 12:08:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17277 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 31A7EC0DA4 for ; Fri, 2 Sep 2022 12:08:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E52E661FF4; Fri, 2 Sep 2022 14:08:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662120502; bh=+KFgwAseA9vrO4LqNwq2KCX8NbA7gihlGm+CKv6ZPPU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=eLQKwOHPehPsJkRok1tvbaSFNwPrGd/Mf6WxvdTQkXxh8vb8hwrXnp2KNq+LBJQg5 X/+Xje1gZ+9bockeRl0gWEFN4xVCfxFn/fcQpNnImYH7LCPo6oTG4jahVOsOVFGytK GFrm9x57EVnUBPFNhkDqr/t2s2tzMlmTTmNMr5wlSp3fqZVl3H/yM5BOfnSix8sAjg 8tdOy/JZgU7/8w1n2433JSKQwIsjH8NDQ0XyDCwOB8tBZCMyDZBU4HM7QwS8xJWB8r 3VEEWlzGcWFTGCYDFbQSjYPeyHnkLeaStcRXmdDFJIsJHD+xLEJfvu9VFRDWQE42jY cxKXb3wbHauCQ== Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6064761FDD for ; Fri, 2 Sep 2022 14:08:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="oxMqo9N8"; dkim-atps=neutral Received: by mail-wm1-x32a.google.com with SMTP id d12-20020a05600c34cc00b003a83d20812fso1298734wmq.1 for ; Fri, 02 Sep 2022 05:08:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=Is7m4sXIxNED6YCMQjr83qnL3GhcGkJ4WiwA5GgyiYE=; b=oxMqo9N8Db0/oa+TY47zcuq6olOxMfoUtAt8+QexslfKUYGqnQwjZLQP3RS9yp7o0n +d+Pz9vrB6bzchIa+7mJjQ24ZFwpzEv3mZfoQ5XX6uAPlPeFSaV66WsNZdcEiWId5Vqa mazEGUbtrYPjTFSOf9p2rgL5kPoEdOny74GxfZXM+EOlg/OmMhHwV4w0wxJ7f9neTO6L c95dw3+kIFEoIQfVaVz6f3ai1Azz/rXuaEYz0W1iI5ZpmtHZSeK/GrfqiHF+TQzM+1yN TKhjMfb1++l+7UxosrCMVfr61Uwz2PKlzPSlmRMG/ImVU90+Uz8YR48/bxDFRYjsx/fq 1onQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=Is7m4sXIxNED6YCMQjr83qnL3GhcGkJ4WiwA5GgyiYE=; b=uCYcFhnB7xiff481jS5mzuJkUsINtDSKmxqn0wtYKpYxuJv7uUeQGWFbGzkEPwszjL YSWw9N3emuLR1toDY0N4PHn4EoA25XhdlyTRgUCoeRC5x107BQEEFwpoPKbS865x1w37 wjzRujvMw9U+dz1ECLtpcJLIk0byoSt31INLUiUb3D5ytshCGaYHF408AmwXngUTzRm7 ma0+t0ryUuCeFtaRzNSDOkdkV37LNkAh0nRiAU1gn3P1iuEzWghVyajA1tkUH9WNdGII QWJ4qrvOUD9iVxlUjbFBr2RLwuhOLkmc9gMJRBJ3Z/EaY7ml9j4+OikYp01rf+AnLBlI mXaw== X-Gm-Message-State: ACgBeo0G3AhR9LhdD9GsF9DuV2hK70WWqkM2BdE1UmXR8HsDIBUKgkjJ qHqVbWw7hRGYfKGAHtRgUhdbt65oW7+K/2Vk X-Google-Smtp-Source: AA6agR6Cb40cbvBkIpqDw+zTsrfI0ieeNRZFFiot91scPxhrzRO1qoFl58Z7nx4XbVig/BOj6wGF3Q== X-Received: by 2002:a1c:a187:0:b0:3a5:e055:715b with SMTP id k129-20020a1ca187000000b003a5e055715bmr2666455wme.171.1662120498630; Fri, 02 Sep 2022 05:08:18 -0700 (PDT) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id h2-20020a5d4302000000b0021e51c039c5sm1397809wrq.80.2022.09.02.05.08.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Sep 2022 05:08:18 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 2 Sep 2022 13:08:06 +0100 Message-Id: <20220902120808.842-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220902120808.842-1-naush@raspberrypi.com> References: <20220902120808.842-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 4/6] ipa: raspberrypi: Use an array of RPiController::Metadata objects X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Cycle through an array of RPiController::Metadata objects, one per prepare()/process() invocation, when running the controller algorithms. This allows historical metadata objects to be retained, and subsequently passed into the controller algorithms on future frames. This change provides a route to fixing a problem with the AGC algorithm, where if a manual shutter/gain is requested, the algorithm does not currently retain any context of the total exposure that it has calculated. As a result, the wrong digital gain would be applied when the frame with the manual shutter/gain is processed by the ISP. Signed-off-by: Naushir Patuck --- src/ipa/raspberrypi/raspberrypi.cpp | 69 +++++++++++++++++------------ 1 file changed, 40 insertions(+), 29 deletions(-) diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 8d731435764e..424ca6c1c596 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -57,6 +57,9 @@ namespace libcamera { using namespace std::literals::chrono_literals; using utils::Duration; +/* Number of metadata objects available in the context list. */ +constexpr unsigned int numMetadataContexts = 16; + /* Configure the sensor with these values initially. */ constexpr double defaultAnalogueGain = 1.0; constexpr Duration defaultExposureTime = 20.0ms; @@ -163,7 +166,8 @@ private: /* Raspberry Pi controller specific defines. */ std::unique_ptr helper_; RPiController::Controller controller_; - RPiController::Metadata rpiMetadata_; + std::array rpiMetadata_; + unsigned int metadataIdx_; /* * We count frames to decide if the frame must be hidden (e.g. from @@ -319,6 +323,7 @@ void IPARPi::start(const ControlList &controls, StartConfig *startConfig) firstStart_ = false; lastRunTimestamp_ = 0; + metadataIdx_ = 0; } void IPARPi::setMode(const IPACameraSensorInfo &sensorInfo) @@ -513,6 +518,7 @@ void IPARPi::signalStatReady(uint32_t bufferId) reportMetadata(); statsMetadataComplete.emit(bufferId & MaskID, libcameraMetadata_); + metadataIdx_ = (metadataIdx_ + 1) % rpiMetadata_.size(); } void IPARPi::signalQueueRequest(const ControlList &controls) @@ -536,14 +542,15 @@ void IPARPi::signalIspPrepare(const ISPConfig &data) void IPARPi::reportMetadata() { - std::unique_lock lock(rpiMetadata_); + RPiController::Metadata &rpiMetadata = rpiMetadata_[metadataIdx_]; + std::unique_lock lock(rpiMetadata); /* * Certain information about the current frame and how it will be * processed can be extracted and placed into the libcamera metadata * buffer, where an application could query it. */ - DeviceStatus *deviceStatus = rpiMetadata_.getLocked("device.status"); + DeviceStatus *deviceStatus = rpiMetadata.getLocked("device.status"); if (deviceStatus) { libcameraMetadata_.set(controls::ExposureTime, deviceStatus->shutterSpeed.get()); @@ -554,17 +561,17 @@ void IPARPi::reportMetadata() libcameraMetadata_.set(controls::SensorTemperature, *deviceStatus->sensorTemperature); } - AgcStatus *agcStatus = rpiMetadata_.getLocked("agc.status"); + AgcStatus *agcStatus = rpiMetadata.getLocked("agc.status"); if (agcStatus) { libcameraMetadata_.set(controls::AeLocked, agcStatus->locked); libcameraMetadata_.set(controls::DigitalGain, agcStatus->digitalGain); } - LuxStatus *luxStatus = rpiMetadata_.getLocked("lux.status"); + LuxStatus *luxStatus = rpiMetadata.getLocked("lux.status"); if (luxStatus) libcameraMetadata_.set(controls::Lux, luxStatus->lux); - AwbStatus *awbStatus = rpiMetadata_.getLocked("awb.status"); + AwbStatus *awbStatus = rpiMetadata.getLocked("awb.status"); if (awbStatus) { libcameraMetadata_.set(controls::ColourGains, Span({ static_cast(awbStatus->gainR), @@ -572,7 +579,7 @@ void IPARPi::reportMetadata() libcameraMetadata_.set(controls::ColourTemperature, awbStatus->temperatureK); } - BlackLevelStatus *blackLevelStatus = rpiMetadata_.getLocked("black_level.status"); + BlackLevelStatus *blackLevelStatus = rpiMetadata.getLocked("black_level.status"); if (blackLevelStatus) libcameraMetadata_.set(controls::SensorBlackLevels, Span({ static_cast(blackLevelStatus->blackLevelR), @@ -580,7 +587,7 @@ void IPARPi::reportMetadata() static_cast(blackLevelStatus->blackLevelG), static_cast(blackLevelStatus->blackLevelB) })); - FocusStatus *focusStatus = rpiMetadata_.getLocked("focus.status"); + FocusStatus *focusStatus = rpiMetadata.getLocked("focus.status"); if (focusStatus && focusStatus->num == 12) { /* * We get a 4x3 grid of regions by default. Calculate the average @@ -591,7 +598,7 @@ void IPARPi::reportMetadata() libcameraMetadata_.set(controls::FocusFoM, focusFoM); } - CcmStatus *ccmStatus = rpiMetadata_.getLocked("ccm.status"); + CcmStatus *ccmStatus = rpiMetadata.getLocked("ccm.status"); if (ccmStatus) { float m[9]; for (unsigned int i = 0; i < 9; i++) @@ -1002,10 +1009,10 @@ void IPARPi::returnEmbeddedBuffer(unsigned int bufferId) void IPARPi::prepareISP(const ISPConfig &data) { int64_t frameTimestamp = data.controls.get(controls::SensorTimestamp).value_or(0); - RPiController::Metadata lastMetadata; + RPiController::Metadata &rpiMetadata = rpiMetadata_[metadataIdx_]; Span embeddedBuffer; - lastMetadata = std::move(rpiMetadata_); + rpiMetadata.clear(); fillDeviceStatus(data.controls); if (data.embeddedBufferPresent) { @@ -1022,7 +1029,7 @@ void IPARPi::prepareISP(const ISPConfig &data) * This may overwrite the DeviceStatus using values from the sensor * metadata, and may also do additional custom processing. */ - helper_->prepare(embeddedBuffer, rpiMetadata_); + helper_->prepare(embeddedBuffer, rpiMetadata); /* Done with embedded data now, return to pipeline handler asap. */ if (data.embeddedBufferPresent) @@ -1038,7 +1045,9 @@ void IPARPi::prepareISP(const ISPConfig &data) * current frame, or any other bits of metadata that were added * in helper_->Prepare(). */ - rpiMetadata_.merge(lastMetadata); + RPiController::Metadata &lastMetadata = + rpiMetadata_[metadataIdx_ == 0 ? rpiMetadata_.size() - 1 : metadataIdx_ - 1]; + rpiMetadata.merge(lastMetadata); processPending_ = false; return; } @@ -1048,48 +1057,48 @@ void IPARPi::prepareISP(const ISPConfig &data) ControlList ctrls(ispCtrls_); - controller_.prepare(&rpiMetadata_); + controller_.prepare(&rpiMetadata); /* Lock the metadata buffer to avoid constant locks/unlocks. */ - std::unique_lock lock(rpiMetadata_); + std::unique_lock lock(rpiMetadata); - AwbStatus *awbStatus = rpiMetadata_.getLocked("awb.status"); + AwbStatus *awbStatus = rpiMetadata.getLocked("awb.status"); if (awbStatus) applyAWB(awbStatus, ctrls); - CcmStatus *ccmStatus = rpiMetadata_.getLocked("ccm.status"); + CcmStatus *ccmStatus = rpiMetadata.getLocked("ccm.status"); if (ccmStatus) applyCCM(ccmStatus, ctrls); - AgcStatus *dgStatus = rpiMetadata_.getLocked("agc.status"); + AgcStatus *dgStatus = rpiMetadata.getLocked("agc.status"); if (dgStatus) applyDG(dgStatus, ctrls); - AlscStatus *lsStatus = rpiMetadata_.getLocked("alsc.status"); + AlscStatus *lsStatus = rpiMetadata.getLocked("alsc.status"); if (lsStatus) applyLS(lsStatus, ctrls); - ContrastStatus *contrastStatus = rpiMetadata_.getLocked("contrast.status"); + ContrastStatus *contrastStatus = rpiMetadata.getLocked("contrast.status"); if (contrastStatus) applyGamma(contrastStatus, ctrls); - BlackLevelStatus *blackLevelStatus = rpiMetadata_.getLocked("black_level.status"); + BlackLevelStatus *blackLevelStatus = rpiMetadata.getLocked("black_level.status"); if (blackLevelStatus) applyBlackLevel(blackLevelStatus, ctrls); - GeqStatus *geqStatus = rpiMetadata_.getLocked("geq.status"); + GeqStatus *geqStatus = rpiMetadata.getLocked("geq.status"); if (geqStatus) applyGEQ(geqStatus, ctrls); - DenoiseStatus *denoiseStatus = rpiMetadata_.getLocked("denoise.status"); + DenoiseStatus *denoiseStatus = rpiMetadata.getLocked("denoise.status"); if (denoiseStatus) applyDenoise(denoiseStatus, ctrls); - SharpenStatus *sharpenStatus = rpiMetadata_.getLocked("sharpen.status"); + SharpenStatus *sharpenStatus = rpiMetadata.getLocked("sharpen.status"); if (sharpenStatus) applySharpen(sharpenStatus, ctrls); - DpcStatus *dpcStatus = rpiMetadata_.getLocked("dpc.status"); + DpcStatus *dpcStatus = rpiMetadata.getLocked("dpc.status"); if (dpcStatus) applyDPC(dpcStatus, ctrls); @@ -1111,11 +1120,13 @@ void IPARPi::fillDeviceStatus(const ControlList &sensorControls) LOG(IPARPI, Debug) << "Metadata - " << deviceStatus; - rpiMetadata_.set("device.status", deviceStatus); + rpiMetadata_[metadataIdx_].set("device.status", deviceStatus); } void IPARPi::processStats(unsigned int bufferId) { + RPiController::Metadata &rpiMetadata = rpiMetadata_[metadataIdx_]; + auto it = buffers_.find(bufferId); if (it == buffers_.end()) { LOG(IPARPI, Error) << "Could not find stats buffer!"; @@ -1125,11 +1136,11 @@ void IPARPi::processStats(unsigned int bufferId) Span mem = it->second.planes()[0]; bcm2835_isp_stats *stats = reinterpret_cast(mem.data()); RPiController::StatisticsPtr statistics = std::make_shared(*stats); - helper_->process(statistics, rpiMetadata_); - controller_.process(statistics, &rpiMetadata_); + helper_->process(statistics, rpiMetadata); + controller_.process(statistics, &rpiMetadata); struct AgcStatus agcStatus; - if (rpiMetadata_.get("agc.status", agcStatus) == 0) { + if (rpiMetadata.get("agc.status", agcStatus) == 0) { ControlList ctrls(sensorCtrls_); applyAGC(&agcStatus, ctrls); From patchwork Fri Sep 2 12:08:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17278 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D0BE0C327D for ; Fri, 2 Sep 2022 12:08:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9CCA361FF0; Fri, 2 Sep 2022 14:08:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662120503; bh=BGta78LDjF7JV8+i1H+doqCf/m6mwDDIL7HTKbqQKc8=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=YYE1LfobIO6K4goW5nNAb/RdZR92hE9pid9aQuxrpju6mhG57xWPsNp2RQSnF+Bxz xoJag172oYU+WzDQDuolYx55jwVdlxS7zugjcYzXiYip15z5xnJxCOHwiiIL4dG41A GfSzbajRbKAHT/3NO3ijxzOhTAoIfiCem6oiNaAy/he13+MXY0WeWCjQfs0TAJlA+w g9AXWGiayo8z9eyiiJ1+Z0CTgnh632dvf/bbGUyvm37MLdETJz5jGo4O0qzT/j8PJd rhaXHY2RYjcNEr5Y3ptjZ5yyMMBpvmn5raplHqysOJJ7rJBSC1zsbwTZvTYNI4d78Z 5vQLjC4ESAmZQ== Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 59F2961FE4 for ; Fri, 2 Sep 2022 14:08:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="BswpqBIK"; dkim-atps=neutral Received: by mail-wm1-x333.google.com with SMTP id c131-20020a1c3589000000b003a84b160addso2517947wma.2 for ; Fri, 02 Sep 2022 05:08:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=i4Y4yviSj+cU6g+Xf/dahLktTgYTzN/ra6Hm4N5ijGQ=; b=BswpqBIKUxlQC5/10y9VRzPTNmZPzJLmutsOK1iRmWzvPM42fFW+mo7xwKs43Bvul7 9PwXEhyRT+spaHOexOmHPYmbxT2O4LpvfkQOsMlfgu4yb8yf2Roe55ITtHwL/XDBvb4l z74eG01wdLgEtOJh2lk6K1CCkW5SyozC4oA/0XNh/TdDgGKGebXHMkPk3hP0qDMqPSnR K3x1vH9XTovXhNL2ttB8ZnJKtOjQAZMpoIcma7dGwVm2TSzN/7864cUimltXwSS0JFmk PdueuxfilnE5tFyTdujUM5vrVkO+K/3VtUQXvtSgWRWNS0xBBB3uRaVBbepaEvuN1f7x lJ3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=i4Y4yviSj+cU6g+Xf/dahLktTgYTzN/ra6Hm4N5ijGQ=; b=V48G+eQiC5dGG7Yiy+Y91RfC+305PEp+/DFgH7k21/+e5tF1NN5Rvvhn9EWt0GvqJL tWkblydCAS7eMM8su9teGcrzIr/czmqM/HybqV6mbIdYiu7SKHqz5eJ1yiza62YO8tcP FjXIOsNiCLzAXOwA2g/P2+HSmDxdZEoREBiCDS+x122koSPuvxQa8pp/M0dm8P7mRq9R kLbZaPUGidcIvhrnX38k/W8zWNGcURTEYyjg8Z84YMpL3r0Zi2h3QZcwTYx6lw0WGSmB vG1WwnDbNdeP4Owl1nNZuvTAUwlg4PyUiAwKFM9xcrgtrY/kR93TCSDDeVGzDMZNpR/C 8QLQ== X-Gm-Message-State: ACgBeo0CmkSpaStzhYFxAW5DXErJrEMjLI8iEggdUaVylP4M9ngIGRN+ +GU8ANb3EGTQMF/ZROKF70nz/hPBd6gQtgiP X-Google-Smtp-Source: AA6agR4qCGlBwdtYM2znoHprbOCZPPQbDRdfdSMC54fQTbsqsbNmfGfhG/BFDz1dY3gCxYoSHUJdcw== X-Received: by 2002:a05:600c:582:b0:3a5:4f7b:3146 with SMTP id o2-20020a05600c058200b003a54f7b3146mr2644936wmd.152.1662120499715; Fri, 02 Sep 2022 05:08:19 -0700 (PDT) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id h2-20020a5d4302000000b0021e51c039c5sm1397809wrq.80.2022.09.02.05.08.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Sep 2022 05:08:18 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 2 Sep 2022 13:08:07 +0100 Message-Id: <20220902120808.842-6-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220902120808.842-1-naush@raspberrypi.com> References: <20220902120808.842-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 5/6] pipeline: ipa: raspberrypi: Use IPA cookies X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Pass an IPA cookie from the IPA to the pipeline handler through the setDelayedControls signal. This cookie is the index of the current RPiController::Metadata context used for the frame. The cookie is subsequently passed into DelayedControls together with the controls to action. The IPA cookie is then returned from DelayedControls when the frame with the applied controls has been returned from the sensor, and eventually passed back to the IPA from the signalIspPrepare signal. Signed-off-by: Naushir Patuck --- include/libcamera/ipa/raspberrypi.mojom | 3 ++- src/ipa/raspberrypi/raspberrypi.cpp | 2 +- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 12 +++++++----- 3 files changed, 10 insertions(+), 7 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index c0de435b7b33..fb806a1518d5 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -36,6 +36,7 @@ struct ISPConfig { uint32 bayerBufferId; bool embeddedBufferPresent; libcamera.ControlList controls; + uint32 ipaCookie; }; struct IPAConfig { @@ -136,5 +137,5 @@ interface IPARPiEventInterface { runIsp(uint32 bufferId); embeddedComplete(uint32 bufferId); setIspControls(libcamera.ControlList controls); - setDelayedControls(libcamera.ControlList controls); + setDelayedControls(libcamera.ControlList controls, uint32 ipaCookie); }; diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 424ca6c1c596..3faa3d490606 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -1144,7 +1144,7 @@ void IPARPi::processStats(unsigned int bufferId) ControlList ctrls(sensorCtrls_); applyAGC(&agcStatus, ctrls); - setDelayedControls.emit(ctrls); + setDelayedControls.emit(ctrls, metadataIdx_); } } diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index d34a906c3cea..4d4c9e1fb9c4 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -208,7 +208,7 @@ public: void runIsp(uint32_t bufferId); void embeddedComplete(uint32_t bufferId); void setIspControls(const ControlList &controls); - void setDelayedControls(const ControlList &controls); + void setDelayedControls(const ControlList &controls, uint32_t ipaCookie); void setSensorControls(ControlList &controls); void unicamTimeout(); @@ -259,6 +259,7 @@ public: struct BayerFrame { FrameBuffer *buffer; ControlList controls; + unsigned int ipaCookie; }; std::queue bayerQueue_; @@ -1784,9 +1785,9 @@ void RPiCameraData::setIspControls(const ControlList &controls) handleState(); } -void RPiCameraData::setDelayedControls(const ControlList &controls) +void RPiCameraData::setDelayedControls(const ControlList &controls, uint32_t ipaCookie) { - if (!delayedCtrls_->push(controls)) + if (!delayedCtrls_->push(controls, ipaCookie)) LOG(RPI, Error) << "V4L2 DelayedControl set failed"; handleState(); } @@ -1848,13 +1849,13 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) * Lookup the sensor controls used for this frame sequence from * DelayedControl and queue them along with the frame buffer. */ - auto [ctrl, cookie] = delayedCtrls_->get(buffer->metadata().sequence); + auto [ctrl, ipaCookie] = delayedCtrls_->get(buffer->metadata().sequence); /* * Add the frame timestamp to the ControlList for the IPA to use * as it does not receive the FrameBuffer object. */ ctrl.set(controls::SensorTimestamp, buffer->metadata().timestamp); - bayerQueue_.push({ buffer, std::move(ctrl) }); + bayerQueue_.push({ buffer, std::move(ctrl), ipaCookie }); } else { embeddedQueue_.push(buffer); } @@ -2148,6 +2149,7 @@ void RPiCameraData::tryRunPipeline() ipa::RPi::ISPConfig ispPrepare; ispPrepare.bayerBufferId = ipa::RPi::MaskBayerData | bayerId; ispPrepare.controls = std::move(bayerFrame.controls); + ispPrepare.ipaCookie = bayerFrame.ipaCookie; if (embeddedBuffer) { unsigned int embeddedId = unicam_[Unicam::Embedded].getBufferId(embeddedBuffer); From patchwork Fri Sep 2 12:08:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17279 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5CB89C327E for ; Fri, 2 Sep 2022 12:08:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 07F4E61FDD; Fri, 2 Sep 2022 14:08:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662120504; bh=zws5OoSrDMBlk0wAhq+tw+ujsBf5kF6gB6btg7hIiA8=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=EuBwoxlPMSOu0aQnopvn9YcmTkA14LC1IF7nwTko+pRVYownaHWT54cAw0IlrVvu1 VhCW+5OkTSGq53C3Z+wGTtLYSJerbGz/EIlJrplpURxAgGqRnBpTxaSSJq/wx7jFN7 OMfi3s5izfVynUQE5+MB957BPMFEKKEZybmEWhyjFs0cs4pprgeq7X7HHnEI0jS2Aj iwl+TfcJ1DOSzJYFVqt3GKz4GzgDBW/UP7gHClJyX1RGaDBUy5Sp3pvlJm3SheET4K AYTuw0/Zf+f15L6eo45izEBkrn3kserutsToSQqE2krMrdSUi9txhpSeUawWK/7AUM qRlvNqH3WN/Bg== Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 31A7C61FDE for ; Fri, 2 Sep 2022 14:08:21 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="cZwpeiDD"; dkim-atps=neutral Received: by mail-wr1-x42c.google.com with SMTP id az27so2047914wrb.6 for ; Fri, 02 Sep 2022 05:08:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=PWHtvpbu5t7AV24xAifkrTb/5HFHtuUs1P0hdp1Tv9c=; b=cZwpeiDDZrVE6wC24h8oYZd4GJmJiTm1/YcfCX9PeY7ZsI2e+qUYkOMrWLfoJIjNog qOg4186+xjqnhiQbXR43FW6vGByB7Kd1vPw8K+frtPoKn5s5YwQfNHoAn2rv1PwG7GO6 laZwXjPxKLaaDPD5R96c/46zzlB07owk3LBPQd6UwT8f+AR4pXCiXmU2nlaH+Gpow5hS TbpBb0KzRIVoyX7G+s+3J+xHM6+JpGC8KYtgiM1dD1pO9xMPC4XsRzMlWXQfOUCDTLgf G7QlRbBnrW+XgCXRZvtq7AKR7PByMatjNstC2zUGiHW0bXi2D3m1xIadD4ss4hLej9lH vahw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=PWHtvpbu5t7AV24xAifkrTb/5HFHtuUs1P0hdp1Tv9c=; b=C1cYmxllhroYwCH64MjlH85W1xJct5aa+5KKonQk0wzcaFPI5eOyJdargs1LC9ZVNG dN4XssDmPdGHKaqgUCojmjbdfjQuf6WzTjQfj2fV5vkDvBms+QL0fqKItv+gwOUmv2sF qC+zeL8mRj5DpPAbkRT/9p5NlpfFhQ5OfkaGPETf+9v9SLy2GNvuSzRwqNwKwI565j3a zJbi+beMsGOf8PNjXSXWfGAIE1+nCeuHLBcTUtplvu2qagNLt8Nx3a0XalzxNHjDWv90 jKgcHALKQ4xQL9JbqcEc0k9xjQgXatH06/WGUBRcBkCLqSlijyD9GyoV5a8RI74Ivk6E eqQQ== X-Gm-Message-State: ACgBeo2cvVNX7T6oCLg9k5NX/ovsFJVnVQOxgYn8kZ0o7EDLFqmpPVpP m/Hww3gZSWA75Xx4809HDl/VDKZnwfbmHmXE X-Google-Smtp-Source: AA6agR6eaCW65rP7JkR5+Fo6Zm4qDHZaIoP1eLZ18mVPG/i5gjDtnUGz1JOvRRpobCq70TKfD9KciQ== X-Received: by 2002:a05:6000:2a8:b0:226:e711:67f5 with SMTP id l8-20020a05600002a800b00226e71167f5mr9711480wry.359.1662120500614; Fri, 02 Sep 2022 05:08:20 -0700 (PDT) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id h2-20020a5d4302000000b0021e51c039c5sm1397809wrq.80.2022.09.02.05.08.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Sep 2022 05:08:19 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 2 Sep 2022 13:08:08 +0100 Message-Id: <20220902120808.842-7-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220902120808.842-1-naush@raspberrypi.com> References: <20220902120808.842-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 6/6] ipa: raspberrypi: agc: Fix digital gain calculation for manual mode X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The digital gain calculation uses a total exposure value computed with the current AGC state. However, this is wrong in the case of manual shutter/gain controls, as the total exposure value used must be the value computed when the AGC sent the manual shutter/gain controls to the pipeline handler to action. To fix this, the IPA now adds the historical AgcStatus structure to the metadata (tagged with "agc.delayed_status"). This historical AgcStatus structure contains the total exposure value calculated when the AGC sent the manual shutter/gain controls to the pipeline handler. Signed-off-by: Naushir Patuck --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 10 ++++++++-- src/ipa/raspberrypi/raspberrypi.cpp | 11 +++++++++++ 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index bd54a639d637..b18bd7b5b19e 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -408,6 +408,12 @@ void Agc::switchMode(CameraMode const &cameraMode, void Agc::prepare(Metadata *imageMetadata) { + Duration totalExposureValue = status_.totalExposureValue; + AgcStatus delayedStatus; + + if (!imageMetadata->get("agc.delayed_status", delayedStatus)) + totalExposureValue = delayedStatus.totalExposureValue; + status_.digitalGain = 1.0; fetchAwbStatus(imageMetadata); /* always fetch it so that Process knows it's been done */ @@ -418,8 +424,8 @@ void Agc::prepare(Metadata *imageMetadata) Duration actualExposure = deviceStatus.shutterSpeed * deviceStatus.analogueGain; if (actualExposure) { - status_.digitalGain = status_.totalExposureValue / actualExposure; - LOG(RPiAgc, Debug) << "Want total exposure " << status_.totalExposureValue; + status_.digitalGain = totalExposureValue / actualExposure; + LOG(RPiAgc, Debug) << "Want total exposure " << totalExposureValue; /* * Never ask for a gain < 1.0, and also impose * some upper limit. Make it customisable? diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 3faa3d490606..0c057f2689cc 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -1025,6 +1025,17 @@ void IPARPi::prepareISP(const ISPConfig &data) embeddedBuffer = it->second.planes()[0]; } + /* + * AGC wants to know the algorithm status from the time it actioned the + * sensor exposure/gain changes. So fetch it from the metadata list + * indexed by the IPA cookie returned, and put it in the current frame + * metadata. + */ + AgcStatus agcStatus; + RPiController::Metadata &delayedMetadata = rpiMetadata_[data.ipaCookie]; + if (!delayedMetadata.get("agc.status", agcStatus)) + rpiMetadata.set("agc.delayed_status", agcStatus); + /* * This may overwrite the DeviceStatus using values from the sensor * metadata, and may also do additional custom processing.