From patchwork Mon May 3 10:41:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12170 X-Patchwork-Delegate: laurent.pinchart@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3DD57BDE78 for ; Mon, 3 May 2021 10:41:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8C7606891E; Mon, 3 May 2021 12:41:13 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5B3EF60511 for ; Mon, 3 May 2021 12:41:12 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 27F86FF807 for ; Mon, 3 May 2021 10:41:11 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Mon, 3 May 2021 12:41:36 +0200 Message-Id: <20210503104152.34048-2-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210503104152.34048-1-jacopo@jmondi.org> References: <20210503104152.34048-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 01/17] libcamera: controls: Add a function to merge two control lists X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Laurent Pinchart Add a new ControlList::merge() function to merge two control lists by copying in the values in the list passed as parameters. This can be used by pipeline handlers to merge metadata they populate with metadata received from an IPA. Signed-off-by: Laurent Pinchart [reimplement the function by not using std::unordered_map::merge()] Signed-off-by: Jacopo Mondi Reviewed-by: Kieran Bingham Reviewed-by: Hirokazu Honda --- include/libcamera/controls.h | 2 ++ src/libcamera/controls.cpp | 30 ++++++++++++++++++++++++++++++ 2 files changed, 32 insertions(+) diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h index 1a5690a5ccbe..1c9b37e617bc 100644 --- a/include/libcamera/controls.h +++ b/include/libcamera/controls.h @@ -363,7 +363,9 @@ public: bool empty() const { return controls_.empty(); } std::size_t size() const { return controls_.size(); } + void clear() { controls_.clear(); } + void merge(const ControlList &source); bool contains(const ControlId &id) const; bool contains(unsigned int id) const; diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp index c58ed3946f3b..d8f104e3734b 100644 --- a/src/libcamera/controls.cpp +++ b/src/libcamera/controls.cpp @@ -874,6 +874,36 @@ ControlList::ControlList(const ControlInfoMap &infoMap, ControlValidator *valida * \brief Removes all controls from the list */ +/** + * \brief Merge the \a source into the ControlList + * \param[in] source The ControlList to merge into this object + * + * Merging two control lists copies elements from the \a source and inserts + * them in *this. If the \a source contains elements whose key is already + * present in *this, then those elements are not overwritten. + * + * Only control lists created from the same ControlIdMap or ControlInfoMap may + * be merged. Attempting to do otherwise results in undefined behaviour. + * + * \todo Reimplement or implement an overloaded version which internally uses + * std::unordered_map::merge() and accepts a non-cost argument. + */ +void ControlList::merge(const ControlList &source) +{ + ASSERT(idmap_ == source.idmap_); + + for (const auto &ctrl : source) { + if (contains(ctrl.first)) { + const ControlId *id = idmap_->at(ctrl.first); + LOG(Controls, Warning) + << "Control " << id->name() << " not overwritten"; + continue; + } + + set(ctrl.first, ctrl.second); + } +} + /** * \brief Check if the list contains a control with the specified \a id * \param[in] id The control ID From patchwork Mon May 3 10:41:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12171 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2366CBDE78 for ; Mon, 3 May 2021 10:41:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B701E68922; Mon, 3 May 2021 12:41:13 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AD2DA60511 for ; Mon, 3 May 2021 12:41:12 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 78547FF805 for ; Mon, 3 May 2021 10:41:12 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Mon, 3 May 2021 12:41:37 +0200 Message-Id: <20210503104152.34048-3-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210503104152.34048-1-jacopo@jmondi.org> References: <20210503104152.34048-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 02/17] test: control_list: Check for Brightness presence X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The current test verifies that after adding a new control to a list already populated with one control the new one is present. However the test wrongly tests for its presence twice instead of making sure the existing control is still there. Fix this by checking for the presence of both controls after the update, and fix the error message accordingly. Fixes: a8c40942b99e ("libcamera: controls: Improve the API towards applications") Reviewed-by: Niklas Söderlund Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi --- test/controls/control_list.cpp | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/test/controls/control_list.cpp b/test/controls/control_list.cpp index b5a49dc17000..2b321ddd6fa4 100644 --- a/test/controls/control_list.cpp +++ b/test/controls/control_list.cpp @@ -108,9 +108,10 @@ protected: list.set(controls::Brightness, 0.0f); list.set(controls::Contrast, 1.5f); - if (!list.contains(controls::Contrast) || + if (!list.contains(controls::Brightness) || !list.contains(controls::Contrast)) { - cout << "List should contain Contrast control" << endl; + cout << "List should contain Brightness and Contrast controls" + << endl; return TestFail; } From patchwork Mon May 3 10:41:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12172 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E9429BDE78 for ; Mon, 3 May 2021 10:41:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3E5E2688E4; Mon, 3 May 2021 12:41:14 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 12B7260511 for ; Mon, 3 May 2021 12:41:13 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id D4E83FF805 for ; Mon, 3 May 2021 10:41:12 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Mon, 3 May 2021 12:41:38 +0200 Message-Id: <20210503104152.34048-4-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210503104152.34048-1-jacopo@jmondi.org> References: <20210503104152.34048-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 03/17] test: control_list: Test ControlList::merge() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Test the ControlList::merge() method by creating a new list and merging it with the existing one. Test that the merged list contains all the controls, the existing values do not get overwritten and the ones copied are not changed. Reviewed-by: Niklas Söderlund Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi --- test/controls/control_list.cpp | 46 ++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/test/controls/control_list.cpp b/test/controls/control_list.cpp index 2b321ddd6fa4..70cf61b85cbc 100644 --- a/test/controls/control_list.cpp +++ b/test/controls/control_list.cpp @@ -150,6 +150,52 @@ protected: return TestFail; } + /* + * Create a new list with a new control and merge it with the + * existing one, verifying that the existing controls + * values don't get overwritten. + */ + ControlList mergeList(controls::controls, &validator); + mergeList.set(controls::Brightness, 0.7f); + mergeList.set(controls::Saturation, 0.4f); + + mergeList.merge(list); + if (mergeList.size() != 3) { + cout << "Merged list should contain three elements" << endl; + return TestFail; + } + + if (list.size() != 2) { + cout << "The list to merge should contain two elements" + << endl; + return TestFail; + } + + if (!mergeList.contains(controls::Brightness) || + !mergeList.contains(controls::Contrast) || + !mergeList.contains(controls::Saturation)) { + cout << "Merged list does not contain all controls" << endl; + return TestFail; + } + + if (mergeList.get(controls::Brightness) != 0.7f) { + cout << "Brightness control value changed after merging lists" + << endl; + return TestFail; + } + + if (mergeList.get(controls::Contrast) != 1.1f) { + cout << "Contrast control value changed after merging lists" + << endl; + return TestFail; + } + + if (mergeList.get(controls::Saturation) != 0.4f) { + cout << "Saturation control value changed after merging lists" + << endl; + return TestFail; + } + return TestPass; } }; From patchwork Mon May 3 10:41:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12173 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 96347BDE78 for ; Mon, 3 May 2021 10:41:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 692646892F; Mon, 3 May 2021 12:41:17 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 940BB68920 for ; Mon, 3 May 2021 12:41:13 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 367A1FF805 for ; Mon, 3 May 2021 10:41:13 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Mon, 3 May 2021 12:41:39 +0200 Message-Id: <20210503104152.34048-5-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210503104152.34048-1-jacopo@jmondi.org> References: <20210503104152.34048-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 04/17] libcamera: controls: Destage 'SensorTimestamp' X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Destage the 'SensorTimestamp' control, which is used by pipeline handlers to report the time when the first active line of the sensor's pixel array is exposed. Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Hirokazu Honda Signed-off-by: Jacopo Mondi --- src/libcamera/control_ids.yaml | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml index b4771f9def89..88d81ac4cccc 100644 --- a/src/libcamera/control_ids.yaml +++ b/src/libcamera/control_ids.yaml @@ -360,6 +360,20 @@ controls: size: [2] + - SensorTimestamp: + type: int64_t + description: | + The time when the first row of the image sensor active array is exposed. + + The timestamp, expressed in nanoseconds, represents a monotonically + increasing counter since the system boot time, as defined by the + Linux-specific CLOCK_BOOTTIME clock id. + + The SensorTimestamp control can only be returned in metadata. + + \todo Define how the sensor timestamp has to be used in the reprocessing + use case. + # ---------------------------------------------------------------------------- # Draft controls section @@ -547,13 +561,6 @@ controls: value: 3 description: The AWB algorithm is locked. - - SensorTimestamp: - type: int64_t - draft: true - description: | - Control to report the start of exposure of the first row of the captured - image. Currently identical to ANDROID_SENSOR_TIMESTAMP. - - SensorRollingShutterSkew: type: int64_t draft: true From patchwork Mon May 3 10:41:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12174 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 405D3BDE78 for ; Mon, 3 May 2021 10:41:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9987168931; Mon, 3 May 2021 12:41:17 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CA87968923 for ; Mon, 3 May 2021 12:41:13 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 932EBFF802 for ; Mon, 3 May 2021 10:41:13 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Mon, 3 May 2021 12:41:40 +0200 Message-Id: <20210503104152.34048-6-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210503104152.34048-1-jacopo@jmondi.org> References: <20210503104152.34048-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 05/17] libcamera: ipu3: Report sensor timestamp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report the sensor's timestamp in the Request metadata by using the CIO2 buffer timestamp as an initial approximation. The buffer's timestamp is recorded at DMA-transfer time, and it does not theoretically matches the 'start of exposure' definition, but when used to compare two consecutive frames it gives an acceptable estimation of the sensor frame period duration. Reviewed-by: Kieran Bingham Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/ipu3/ipu3.cpp | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 73306cea6b37..88b7bd1e52c3 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -1255,6 +1255,15 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) Request *request = info->request; + /* + * Record the sensor's timestamp in the request metadata. + * + * \todo The sensor timestamp should be better estimated by connecting + * to the V4L2Device::frameStart signal. + */ + request->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + /* If the buffer is cancelled force a complete of the whole request. */ if (buffer->metadata().status == FrameMetadata::FrameCancelled) { for (auto it : request->buffers()) { From patchwork Mon May 3 10:41:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12175 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A9C4ABDE78 for ; Mon, 3 May 2021 10:41:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 365506892A; Mon, 3 May 2021 12:41:18 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2DC486891C for ; Mon, 3 May 2021 12:41:14 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id EC114FF802 for ; Mon, 3 May 2021 10:41:13 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Mon, 3 May 2021 12:41:41 +0200 Message-Id: <20210503104152.34048-7-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210503104152.34048-1-jacopo@jmondi.org> References: <20210503104152.34048-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 06/17] libcamera: ipu3: Merge IPA metadata controls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a Request is completed upon receiving the IPA produced metadata, they should be merged with the metadata set by the pipeline handler before reporting them to applications. Use ControlList::merge() to collect the IPA produced controls to the request's metadata. Signed-off-by: Jacopo Mondi Reviewed-by: Kieran Bingham Reviewed-by: Hirokazu Honda Reviewed-by: Niklas Söderlund --- src/libcamera/pipeline/ipu3/ipu3.cpp | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 88b7bd1e52c3..ade8ffbddb0e 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -1190,11 +1190,8 @@ void IPU3CameraData::queueFrameAction(unsigned int id, if (!info) break; - /* - * \todo Parse the value of the controls returned by the IPA - * in action.controls to register additional metadata. - */ Request *request = info->request; + request->metadata().merge(action.controls); info->metadataProcessed = true; if (frameInfos_.tryComplete(info)) From patchwork Mon May 3 10:41:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12176 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 00DE1BDE79 for ; Mon, 3 May 2021 10:41:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7A7B368936; Mon, 3 May 2021 12:41:18 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7D3196891D for ; Mon, 3 May 2021 12:41:14 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 49A1BFF802 for ; Mon, 3 May 2021 10:41:14 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Mon, 3 May 2021 12:41:42 +0200 Message-Id: <20210503104152.34048-8-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210503104152.34048-1-jacopo@jmondi.org> References: <20210503104152.34048-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 07/17] libcamera: uvc: Report sensor timestamp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report the sensor's timestamp in the Request metadata using the completed buffer timestamp. The UVC driver reports timestamps of SOE event through metadata, for which there is no support in the current pipeline implementation. Reviewed-by: Hirokazu Honda Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp index b6c6ade5ebaf..faa8d6b05f46 100644 --- a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp +++ b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp @@ -662,6 +662,10 @@ void UVCCameraData::bufferReady(FrameBuffer *buffer) { Request *request = buffer->request(); + /* \todo Use the UVC metadata to calculate a more precise timestamp */ + request->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + pipe_->completeBuffer(request, buffer); pipe_->completeRequest(request); } From patchwork Mon May 3 10:41:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12177 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 50A89BDE78 for ; Mon, 3 May 2021 10:41:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B2FCD68937; Mon, 3 May 2021 12:41:18 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CC82760511 for ; Mon, 3 May 2021 12:41:14 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 9AACFFF802 for ; Mon, 3 May 2021 10:41:14 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Mon, 3 May 2021 12:41:43 +0200 Message-Id: <20210503104152.34048-9-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210503104152.34048-1-jacopo@jmondi.org> References: <20210503104152.34048-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 08/17] libcamera: rkisp1: Do not over-write metadata X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a Request is completed upon receiving the IPA produced metadata, the metadata associated with the Request are over-written, deleting the information set at output buffer completion, such as the SensorTimestamp. This commit applies to the RkISP1 pipeline handler the same change applied to IPU3 in commit 2ee3dd57e2fc ("libcamera: ipu3: Merge IPA metadata controls"). Signed-off-by: Jacopo Mondi Reviewed-by: Kieran Bingham Reviewed-by: Hirokazu Honda --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index c75666391222..26f708242523 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -365,7 +365,7 @@ void RkISP1CameraData::metadataReady(unsigned int frame, const ControlList &meta if (!info) return; - info->request->metadata() = metadata; + info->request->metadata().merge(metadata); info->metadataProcessed = true; pipe->tryCompleteRequest(info->request); From patchwork Mon May 3 10:41:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12178 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3515ABDE78 for ; Mon, 3 May 2021 10:41:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E4C0368923; Mon, 3 May 2021 12:41:20 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 33E6C68927 for ; Mon, 3 May 2021 12:41:15 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id ED06CFF805 for ; Mon, 3 May 2021 10:41:14 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Mon, 3 May 2021 12:41:44 +0200 Message-Id: <20210503104152.34048-10-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210503104152.34048-1-jacopo@jmondi.org> References: <20210503104152.34048-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 09/17] libcamera: rkisp1: Report sensor timestamp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report the sensor's timestamp in the Request metadata using the completed buffer timestamp. The buffer's timestamp is recorded at DMA-transfer time, and it does not theoretically matches the 'start of exposure' definition. Record this with a \todo entry. Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 26f708242523..cee6d85fac47 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1070,6 +1070,15 @@ void PipelineHandlerRkISP1::bufferReady(FrameBuffer *buffer) { Request *request = buffer->request(); + /* + * Record the sensor's timestamp in the request metadata. + * + * \todo The sensor timestamp should be better estimated by connecting + * to the V4L2Device::frameStart signal. + */ + request->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + completeBuffer(request, buffer); tryCompleteRequest(request); } From patchwork Mon May 3 10:41:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12180 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D30D9BDE78 for ; Mon, 3 May 2021 10:41:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 77E2768927; Mon, 3 May 2021 12:41:21 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 89E4F6890C for ; Mon, 3 May 2021 12:41:15 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 57270FF805 for ; Mon, 3 May 2021 10:41:15 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Mon, 3 May 2021 12:41:45 +0200 Message-Id: <20210503104152.34048-11-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210503104152.34048-1-jacopo@jmondi.org> References: <20210503104152.34048-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 10/17] libcamera: simple: Report sensor timestamp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report the sensor's timestamp in the Request metadata using the completed buffer timestamp. The buffer's timestamp is recorded at DMA-transfer time, and it does not theoretically matches the 'start of exposure' definition. Record this with a \todo entry. Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/simple/simple.cpp | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index f6095d38e97a..e1ee640b0a92 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -22,6 +22,7 @@ #include #include +#include #include #include @@ -1116,6 +1117,16 @@ void SimplePipelineHandler::bufferReady(FrameBuffer *buffer) return; } + /* + * Record the sensor's timestamp in the request metadata. + * + * \todo The sensor timestamp should be better estimated by connecting + * to the V4L2Device::frameStart signal if the platform provides it. + */ + Request *request = buffer->request(); + request->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + /* * Queue the captured and the request buffer to the converter if format * conversion is needed. If there's no queued request, just requeue the @@ -1133,7 +1144,6 @@ void SimplePipelineHandler::bufferReady(FrameBuffer *buffer) } /* Otherwise simply complete the request. */ - Request *request = buffer->request(); completeBuffer(request, buffer); completeRequest(request); } From patchwork Mon May 3 10:41:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12179 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 69E71BDE79 for ; Mon, 3 May 2021 10:41:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 285C26893B; Mon, 3 May 2021 12:41:21 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DEB8368920 for ; Mon, 3 May 2021 12:41:15 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id AC03EFF805 for ; Mon, 3 May 2021 10:41:15 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Mon, 3 May 2021 12:41:46 +0200 Message-Id: <20210503104152.34048-12-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210503104152.34048-1-jacopo@jmondi.org> References: <20210503104152.34048-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 11/17] libcamera: vimc: Report sensor timestamp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report the sensor's timestamp in the Request metadata using the completed buffer timestamp. The buffer's timestamp reports the video capture buffer processing time, and it does not theoretically matches the 'start of exposure' definition. VIMC being a testing platform and the test driver completes the buffers for each media entity connected in the pipeline one after the other, the current solution is acceptable for the pipeline. Reviewed-by: Hirokazu Honda Reviewed-by: Niklas Söderlund Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/vimc/vimc.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/libcamera/pipeline/vimc/vimc.cpp b/src/libcamera/pipeline/vimc/vimc.cpp index 8f5f4ba30953..ce83dcaab8ea 100644 --- a/src/libcamera/pipeline/vimc/vimc.cpp +++ b/src/libcamera/pipeline/vimc/vimc.cpp @@ -523,6 +523,10 @@ void VimcCameraData::bufferReady(FrameBuffer *buffer) { Request *request = buffer->request(); + /* Record the sensor's timestamp in the request metadata. */ + request->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + pipe_->completeBuffer(request, buffer); pipe_->completeRequest(request); } From patchwork Mon May 3 10:41:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12181 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3DC12BDE79 for ; Mon, 3 May 2021 10:41:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0443368921; Mon, 3 May 2021 12:41:22 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4738668923 for ; Mon, 3 May 2021 12:41:16 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 15727FF805 for ; Mon, 3 May 2021 10:41:15 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Mon, 3 May 2021 12:41:47 +0200 Message-Id: <20210503104152.34048-13-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210503104152.34048-1-jacopo@jmondi.org> References: <20210503104152.34048-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 12/17] libcamera: buffer: Re-work setRequest() documentation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" I got fooled by the documentation of setRequest() implying that the function is meant to be called by pipeline handlers only, which it is used in the Request class at Request::addBuffer() and Request::reuse() time. Rework the documentation to report that. Reviewed-by: Niklas Söderlund Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi --- src/libcamera/buffer.cpp | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/libcamera/buffer.cpp b/src/libcamera/buffer.cpp index 7635226b9045..ca999429dce5 100644 --- a/src/libcamera/buffer.cpp +++ b/src/libcamera/buffer.cpp @@ -191,8 +191,9 @@ FrameBuffer::FrameBuffer(const std::vector &planes, unsigned int cookie) * \brief Set the request this buffer belongs to * \param[in] request Request to set * - * The intended callers of this method are pipeline handlers and only for - * buffers that are internal to the pipeline. + * For buffers added to requests by applications, this method is called by + * Request::addBuffer() or Request::reuse(). For buffers internal to pipeline + * handlers, it is called by the pipeline handlers themselves. * * \todo Shall be hidden from applications with a d-pointer design. */ From patchwork Mon May 3 10:41:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12182 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7E35DBDE78 for ; Mon, 3 May 2021 10:41:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 411C86893E; Mon, 3 May 2021 12:41:22 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A27DB6891C for ; Mon, 3 May 2021 12:41:16 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 6C399FF807 for ; Mon, 3 May 2021 10:41:16 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Mon, 3 May 2021 12:41:48 +0200 Message-Id: <20210503104152.34048-14-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210503104152.34048-1-jacopo@jmondi.org> References: <20210503104152.34048-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 13/17] libcamera: raspberrypi: Do not over-write metadata X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a Request is completed upon receiving the IPA produced metadata, the metadata associated with the Request are over-written, deleting the information set at output buffer completion, such as the SensorTimestamp. This commit applies to the RaspberryPi pipeline handler the same change applied to IPU3 in commit 2ee3dd57e2fc ("libcamera: ipu3: Merge IPA metadata controls"). Reviewed-by: Naushir Patuck Signed-off-by: Jacopo Mondi Reviewed-by: Hirokazu Honda --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 2a917455500f..91f44f6844b3 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -1312,9 +1312,9 @@ void RPiCameraData::statsMetadataComplete(uint32_t bufferId, const ControlList & handleStreamBuffer(buffer, &isp_[Isp::Stats]); - /* Fill the Request metadata buffer with what the IPA has provided */ + /* Add to the Request metadata buffer what the IPA has provided. */ Request *request = requestQueue_.front(); - request->metadata() = controls; + request->metadata().merge(controls); /* * Also update the ScalerCrop in the metadata with what we actually From patchwork Mon May 3 10:41:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12183 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CE01ABDE79 for ; Mon, 3 May 2021 10:41:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 74BCA68940; Mon, 3 May 2021 12:41:22 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 001FE60511 for ; Mon, 3 May 2021 12:41:16 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id C1DD5FF802 for ; Mon, 3 May 2021 10:41:16 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Mon, 3 May 2021 12:41:49 +0200 Message-Id: <20210503104152.34048-15-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210503104152.34048-1-jacopo@jmondi.org> References: <20210503104152.34048-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 14/17] libcamera: raspberry: Report sensor timestamp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report the sensor's timestamp in the Request metadata by using the Unicam::Image buffer timestamp as an initial approximation. The buffer's timestamp is recorded at DMA-transfer time, and it does not theoretically matches the 'start of exposure' definition, but when used to compare two consecutive frames it gives an acceptable estimation of the sensor frame period duration. Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart Reviewed-by: Naushir Patuck Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 91f44f6844b3..b22564938cdc 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -1409,6 +1409,18 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) << ", timestamp: " << buffer->metadata().timestamp; if (stream == &unicam_[Unicam::Image]) { + /* + * Record the sensor timestamp in the Request. + * + * \todo Do not assume the request in the front of the queue + * is the correct one + */ + Request *request = requestQueue_.front(); + ASSERT(request); + + request->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + /* * Lookup the sensor controls used for this frame sequence from * DelayedControl and queue them along with the frame buffer. From patchwork Mon May 3 10:41:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12184 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 18516BDE7A for ; Mon, 3 May 2021 10:41:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A192268943; Mon, 3 May 2021 12:41:22 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5A30C68911 for ; Mon, 3 May 2021 12:41:17 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 25F4CFF802 for ; Mon, 3 May 2021 10:41:16 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Mon, 3 May 2021 12:41:50 +0200 Message-Id: <20210503104152.34048-16-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210503104152.34048-1-jacopo@jmondi.org> References: <20210503104152.34048-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 15/17] cam: Add option to print the Request metadata X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add the "--metadata" option to the cam tool, which will be used to print the metadata associated with a completed capture request. Reviewed-by: Niklas Söderlund Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi --- src/cam/main.cpp | 3 +++ src/cam/main.h | 1 + 2 files changed, 4 insertions(+) diff --git a/src/cam/main.cpp b/src/cam/main.cpp index 994fbb343029..70e9f62c7cb0 100644 --- a/src/cam/main.cpp +++ b/src/cam/main.cpp @@ -197,6 +197,9 @@ int CamApp::parseOptions(int argc, char *argv[]) parser.addOption(OptStrictFormats, OptionNone, "Do not allow requested stream format(s) to be adjusted", "strict-formats"); + parser.addOption(OptMetadata, OptionNone, + "Print the metadata for completed requests", + "metadata"); options_ = parser.parse(argc, argv); if (!options_.valid()) diff --git a/src/cam/main.h b/src/cam/main.h index ea8dfd330830..d22451f59817 100644 --- a/src/cam/main.h +++ b/src/cam/main.h @@ -19,6 +19,7 @@ enum { OptStream = 's', OptListControls = 256, OptStrictFormats = 257, + OptMetadata = 258, }; #endif /* __CAM_MAIN_H__ */ From patchwork Mon May 3 10:41:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12186 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 34F40BDE78 for ; Mon, 3 May 2021 10:41:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 04C2F68911; Mon, 3 May 2021 12:41:25 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AE8D26891D for ; Mon, 3 May 2021 12:41:17 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 79AD1FF802 for ; Mon, 3 May 2021 10:41:17 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Mon, 3 May 2021 12:41:51 +0200 Message-Id: <20210503104152.34048-17-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210503104152.34048-1-jacopo@jmondi.org> References: <20210503104152.34048-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 16/17] cam: Implement OptMetadata X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement support for the new '--metadata' option by printing the value of each metadata entry associated with a completed Request. As sample of the output, running on raspberry pi, looks like the following: 3050.205672 (30.01 fps) stream0 seq: 000033 bytesused: 720000 ScalerCrop = (0x2)/3280x2460 ExposureTime = 13969 AeLocked = true DigitalGain = 1.000721 Lux = 771.204224 ColourGains = [ 1.561101, 1.629698 ] ColourTemperature = 4289 SensorBlackLevels = [ 4096, 4096, 4096, 4096 ] ColourCorrectionMatrix = [ 1.691066, -0.599756, -0.091317, -0.437452, 1.983766, -0.546314, -0.083429, -0.722407, 1.805836 ] AnalogueGain = 2.000000 SensorTimestamp = 3050205672000 3050.238999 (30.01 fps) stream0 seq: 000034 bytesused: 720000 ScalerCrop = (0x2)/3280x2460 ExposureTime = 13969 AeLocked = true DigitalGain = 1.000709 Lux = 771.232422 ColourGains = [ 1.560868, 1.630029 ] ColourTemperature = 4289 SensorBlackLevels = [ 4096, 4096, 4096, 4096 ] ColourCorrectionMatrix = [ 1.691081, -0.599726, -0.091362, -0.437497, 1.983627, -0.546130, -0.083420, -0.722523, 1.805943 ] AnalogueGain = 2.000000 SensorTimestamp = 3050238999000 Reviewed-by: Niklas Söderlund Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi --- src/cam/capture.cpp | 15 ++++++++++++++- src/cam/capture.h | 1 + 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/src/cam/capture.cpp b/src/cam/capture.cpp index 7b55fc677022..3c3e3a53adf7 100644 --- a/src/cam/capture.cpp +++ b/src/cam/capture.cpp @@ -10,6 +10,8 @@ #include #include +#include + #include "capture.h" #include "main.h" @@ -18,7 +20,8 @@ using namespace libcamera; Capture::Capture(std::shared_ptr camera, CameraConfiguration *config, EventLoop *loop) : camera_(camera), config_(config), writer_(nullptr), last_(0), loop_(loop), - queueCount_(0), captureCount_(0), captureLimit_(0) + queueCount_(0), captureCount_(0), captureLimit_(0), + printMetadata_(false) { } @@ -29,6 +32,7 @@ int Capture::run(const OptionsParser::Options &options) queueCount_ = 0; captureCount_ = 0; captureLimit_ = options[OptCapture].toInteger(); + printMetadata_ = options.isSet(OptMetadata); if (!camera_) { std::cout << "Can't capture without a camera" << std::endl; @@ -217,6 +221,15 @@ void Capture::processRequest(Request *request) std::cout << info.str() << std::endl; + if (printMetadata_) { + const ControlList &requestMetadata = request->metadata(); + for (const auto &ctrl : requestMetadata) { + const ControlId *id = controls::controls.at(ctrl.first); + std::cout << "\t" << id->name() << " = " + << ctrl.second.toString() << std::endl; + } + } + captureCount_++; if (captureLimit_ && captureCount_ >= captureLimit_) { loop_->exit(0); diff --git a/src/cam/capture.h b/src/cam/capture.h index c7c9dc00d30f..59d138766b1e 100644 --- a/src/cam/capture.h +++ b/src/cam/capture.h @@ -47,6 +47,7 @@ private: unsigned int queueCount_; unsigned int captureCount_; unsigned int captureLimit_; + bool printMetadata_; std::vector> requests_; }; From patchwork Mon May 3 10:41:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12185 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6757BBDE78 for ; Mon, 3 May 2021 10:41:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C86C468945; Mon, 3 May 2021 12:41:22 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2170368932 for ; Mon, 3 May 2021 12:41:18 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id CF494FF802 for ; Mon, 3 May 2021 10:41:17 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Mon, 3 May 2021 12:41:52 +0200 Message-Id: <20210503104152.34048-18-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210503104152.34048-1-jacopo@jmondi.org> References: <20210503104152.34048-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 17/17] android: camera_device: Use controls::SensorTimestamp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use the controls::SensorTimestamp value to populate ANDROID_SENSOR_TIMESTAMP result metadata. The Camera is assumed to provide the control in the Request metadata. Reviewed-by: Hirokazu Honda Reviewed-by: Niklas Söderlund Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi --- src/android/camera_device.cpp | 21 ++++++++------------- src/android/camera_device.h | 3 +-- 2 files changed, 9 insertions(+), 15 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index a71aee2fc734..f1f26c775611 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -2025,7 +2025,6 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques void CameraDevice::requestComplete(Request *request) { - const Request::BufferMap &buffers = request->buffers(); camera3_buffer_status status = CAMERA3_BUFFER_STATUS_OK; std::unique_ptr resultMetadata; @@ -2053,14 +2052,7 @@ void CameraDevice::requestComplete(Request *request) LOG(HAL, Debug) << "Request " << request->cookie() << " completed with " << descriptor.buffers_.size() << " streams"; - /* - * \todo The timestamp used for the metadata is currently always taken - * from the first buffer (which may be the first stream) in the Request. - * It might be appropriate to return a 'correct' (as determined by - * pipeline handlers) timestamp in the Request itself. - */ - uint64_t timestamp = buffers.begin()->second->metadata().timestamp; - resultMetadata = getResultMetadata(descriptor, timestamp); + resultMetadata = getResultMetadata(descriptor); /* Handle any JPEG compression. */ for (camera3_stream_buffer_t &buffer : descriptor.buffers_) { @@ -2105,6 +2097,9 @@ void CameraDevice::requestComplete(Request *request) captureResult.output_buffers = descriptor.buffers_.data(); if (status == CAMERA3_BUFFER_STATUS_OK) { + uint64_t timestamp = + static_cast(request->metadata() + .get(controls::SensorTimestamp)); notifyShutter(descriptor.frameNumber_, timestamp); captureResult.partial_result = 1; @@ -2164,8 +2159,7 @@ void CameraDevice::notifyError(uint32_t frameNumber, camera3_stream_t *stream) * Produce a set of fixed result metadata. */ std::unique_ptr -CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor, - int64_t timestamp) const +CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) const { const ControlList &metadata = descriptor.request_->metadata(); const CameraMetadata &settings = descriptor.settings_; @@ -2291,8 +2285,6 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor, resultMetadata->addEntry(ANDROID_SENSOR_TEST_PATTERN_MODE, &value32, 1); - resultMetadata->addEntry(ANDROID_SENSOR_TIMESTAMP, ×tamp, 1); - value = ANDROID_STATISTICS_FACE_DETECT_MODE_OFF; resultMetadata->addEntry(ANDROID_STATISTICS_FACE_DETECT_MODE, &value, 1); @@ -2318,6 +2310,9 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor, &rolling_shutter_skew, 1); /* Add metadata tags reported by libcamera. */ + const int64_t timestamp = metadata.get(controls::SensorTimestamp); + resultMetadata->addEntry(ANDROID_SENSOR_TIMESTAMP, ×tamp, 1); + if (metadata.contains(controls::draft::PipelineDepth)) { uint8_t pipeline_depth = metadata.get(controls::draft::PipelineDepth); diff --git a/src/android/camera_device.h b/src/android/camera_device.h index c63e8e21726e..23457e47767a 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -107,8 +107,7 @@ private: libcamera::PixelFormat toPixelFormat(int format) const; int processControls(Camera3RequestDescriptor *descriptor); std::unique_ptr getResultMetadata( - const Camera3RequestDescriptor &descriptor, - int64_t timestamp) const; + const Camera3RequestDescriptor &descriptor) const; unsigned int id_; camera3_device_t camera3Device_;