From patchwork Fri Apr 30 16:00:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12140 X-Patchwork-Delegate: laurent.pinchart@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2A4C2BDE4C for ; Fri, 30 Apr 2021 15:59:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E4BE368915; Fri, 30 Apr 2021 17:59:50 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 51664688AC for ; Fri, 30 Apr 2021 17:59:48 +0200 (CEST) X-Originating-IP: 93.61.96.190 Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 9ADF5FF808; Fri, 30 Apr 2021 15:59:47 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Fri, 30 Apr 2021 18:00:11 +0200 Message-Id: <20210430160026.190724-2-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210430160026.190724-1-jacopo@jmondi.org> References: <20210430160026.190724-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 01/16] libcamera: controls: Add a function to merge two control lists X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Laurent Pinchart Add a new ControlList::merge() function to merge two control lists by copying in the values in the list passed as parameters. This can be used by pipeline handlers to merge metadata they populate with metadata received from an IPA. Signed-off-by: Laurent Pinchart [reimplement the function by not using std::unordered_map::merge()] Signed-off-by: Jacopo Mondi --- include/libcamera/controls.h | 2 ++ src/libcamera/controls.cpp | 28 ++++++++++++++++++++++++++++ 2 files changed, 30 insertions(+) diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h index 1a5690a5ccbe..1c9b37e617bc 100644 --- a/include/libcamera/controls.h +++ b/include/libcamera/controls.h @@ -363,7 +363,9 @@ public: bool empty() const { return controls_.empty(); } std::size_t size() const { return controls_.size(); } + void clear() { controls_.clear(); } + void merge(const ControlList &source); bool contains(const ControlId &id) const; bool contains(unsigned int id) const; diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp index c58ed3946f3b..154ca7bfe7c1 100644 --- a/src/libcamera/controls.cpp +++ b/src/libcamera/controls.cpp @@ -874,6 +874,34 @@ ControlList::ControlList(const ControlInfoMap &infoMap, ControlValidator *valida * \brief Removes all controls from the list */ +/** + * \brief Merge the \a source into the ControlList + * \param[in] source The ControlList to merge into this object + * + * Merging two control lists copies elements from the \a source and insert + * them in *this. If the \a source contains elements whose key is already + * present in *this, then those elements are not overwritten. + * identical to std::unordered_map::merge() in that only internal pointers to + * nodes are updated, without copying or moving the elements. + * + * Only control lists created from the same ControlIdMap or ControlInfoMap may + * be merged. Attempting to do otherwise results in undefined behaviour. + * + * \todo Reimplement or implement an overloaded version which internally uses + * std::unordered_map::merge() and accepts a non-cost argument. + */ +void ControlList::merge(const ControlList &source) +{ + ASSERT(idmap_ == source.idmap_); + + for (const auto &ctrl : source) { + if (contains(ctrl.first)) + continue; + + set(ctrl.first, ctrl.second); + } +} + /** * \brief Check if the list contains a control with the specified \a id * \param[in] id The control ID From patchwork Fri Apr 30 16:00:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12141 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D53D1BDE4C for ; Fri, 30 Apr 2021 15:59:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 38D6C6891A; Fri, 30 Apr 2021 17:59:52 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C4546688E4 for ; Fri, 30 Apr 2021 17:59:49 +0200 (CEST) X-Originating-IP: 93.61.96.190 Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 83653FF80B; Fri, 30 Apr 2021 15:59:48 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Fri, 30 Apr 2021 18:00:12 +0200 Message-Id: <20210430160026.190724-3-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210430160026.190724-1-jacopo@jmondi.org> References: <20210430160026.190724-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 02/16] test: control_list: Check for Brightness presence X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The current test verifies that after adding a new control to a list already populated with one control the new one is present. However the test wrongly tests for its presence twice instead of making sure the existing control is still there. Fix this by checking for the presence of both controls after the update, and fix the error message accordingly. Fixes: a8c40942b99e ("libcamera: controls: Improve the API towards applications") Reviewed-by: Niklas Söderlund Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi --- test/controls/control_list.cpp | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/test/controls/control_list.cpp b/test/controls/control_list.cpp index b5a49dc17000..2b321ddd6fa4 100644 --- a/test/controls/control_list.cpp +++ b/test/controls/control_list.cpp @@ -108,9 +108,10 @@ protected: list.set(controls::Brightness, 0.0f); list.set(controls::Contrast, 1.5f); - if (!list.contains(controls::Contrast) || + if (!list.contains(controls::Brightness) || !list.contains(controls::Contrast)) { - cout << "List should contain Contrast control" << endl; + cout << "List should contain Brightness and Contrast controls" + << endl; return TestFail; } From patchwork Fri Apr 30 16:00:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12142 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8DA79BDE4C for ; Fri, 30 Apr 2021 15:59:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5F3136891F; Fri, 30 Apr 2021 17:59:55 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E24F068914 for ; Fri, 30 Apr 2021 17:59:50 +0200 (CEST) X-Originating-IP: 93.61.96.190 Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id D9114FF804; Fri, 30 Apr 2021 15:59:49 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Fri, 30 Apr 2021 18:00:13 +0200 Message-Id: <20210430160026.190724-4-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210430160026.190724-1-jacopo@jmondi.org> References: <20210430160026.190724-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 03/16] test: control_list: Test ControlList::merge() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Test the ControlList::merge() method by creating a new list and merging it with the existing one. Test that the merged list contains all the controls, the existing values do not get overwritten and the ones copied are not changed. Reviewed-by: Niklas Söderlund Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi --- test/controls/control_list.cpp | 46 ++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/test/controls/control_list.cpp b/test/controls/control_list.cpp index 2b321ddd6fa4..70cf61b85cbc 100644 --- a/test/controls/control_list.cpp +++ b/test/controls/control_list.cpp @@ -150,6 +150,52 @@ protected: return TestFail; } + /* + * Create a new list with a new control and merge it with the + * existing one, verifying that the existing controls + * values don't get overwritten. + */ + ControlList mergeList(controls::controls, &validator); + mergeList.set(controls::Brightness, 0.7f); + mergeList.set(controls::Saturation, 0.4f); + + mergeList.merge(list); + if (mergeList.size() != 3) { + cout << "Merged list should contain three elements" << endl; + return TestFail; + } + + if (list.size() != 2) { + cout << "The list to merge should contain two elements" + << endl; + return TestFail; + } + + if (!mergeList.contains(controls::Brightness) || + !mergeList.contains(controls::Contrast) || + !mergeList.contains(controls::Saturation)) { + cout << "Merged list does not contain all controls" << endl; + return TestFail; + } + + if (mergeList.get(controls::Brightness) != 0.7f) { + cout << "Brightness control value changed after merging lists" + << endl; + return TestFail; + } + + if (mergeList.get(controls::Contrast) != 1.1f) { + cout << "Contrast control value changed after merging lists" + << endl; + return TestFail; + } + + if (mergeList.get(controls::Saturation) != 0.4f) { + cout << "Saturation control value changed after merging lists" + << endl; + return TestFail; + } + return TestPass; } }; From patchwork Fri Apr 30 16:00:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12143 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D08F6BDE4D for ; Fri, 30 Apr 2021 15:59:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9A0B168917; Fri, 30 Apr 2021 17:59:55 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 15F2468918 for ; Fri, 30 Apr 2021 17:59:52 +0200 (CEST) X-Originating-IP: 93.61.96.190 Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 175D0FF80B; Fri, 30 Apr 2021 15:59:50 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Fri, 30 Apr 2021 18:00:14 +0200 Message-Id: <20210430160026.190724-5-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210430160026.190724-1-jacopo@jmondi.org> References: <20210430160026.190724-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 04/16] libcamera: controls: Destage 'SensorTimestamp' X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Destage the 'SensorTimestamp' control, which is used by pipeline handlers to report the time when the first active line of the sensor's pixel array is exposed. Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Hirokazu Honda Signed-off-by: Jacopo Mondi --- src/libcamera/control_ids.yaml | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml index b4771f9def89..88d81ac4cccc 100644 --- a/src/libcamera/control_ids.yaml +++ b/src/libcamera/control_ids.yaml @@ -360,6 +360,20 @@ controls: size: [2] + - SensorTimestamp: + type: int64_t + description: | + The time when the first row of the image sensor active array is exposed. + + The timestamp, expressed in nanoseconds, represents a monotonically + increasing counter since the system boot time, as defined by the + Linux-specific CLOCK_BOOTTIME clock id. + + The SensorTimestamp control can only be returned in metadata. + + \todo Define how the sensor timestamp has to be used in the reprocessing + use case. + # ---------------------------------------------------------------------------- # Draft controls section @@ -547,13 +561,6 @@ controls: value: 3 description: The AWB algorithm is locked. - - SensorTimestamp: - type: int64_t - draft: true - description: | - Control to report the start of exposure of the first row of the captured - image. Currently identical to ANDROID_SENSOR_TIMESTAMP. - - SensorRollingShutterSkew: type: int64_t draft: true From patchwork Fri Apr 30 16:00:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12144 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B3040BDE4C for ; Fri, 30 Apr 2021 15:59:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2FF5268909; Fri, 30 Apr 2021 17:59:56 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 57AAC68909 for ; Fri, 30 Apr 2021 17:59:53 +0200 (CEST) X-Originating-IP: 93.61.96.190 Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 3E1EAFF808; Fri, 30 Apr 2021 15:59:52 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Fri, 30 Apr 2021 18:00:15 +0200 Message-Id: <20210430160026.190724-6-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210430160026.190724-1-jacopo@jmondi.org> References: <20210430160026.190724-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 05/16] libcamera: ipu3: Report sensor timestamp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report the sensor's timestamp in the Request metadata by using the CIO2 buffer timestamp as an initial approximation. The buffer's timestamp is recorded at DMA-transfer time, and it does not theoretically matches the 'start of exposure' definition, but when used to compare two consecutive frames it gives an acceptable estimation of the sensor frame period duration. Reviewed-by: Kieran Bingham Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi Reviewed-by: Niklas Söderlund --- src/libcamera/pipeline/ipu3/ipu3.cpp | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 73306cea6b37..88b7bd1e52c3 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -1255,6 +1255,15 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) Request *request = info->request; + /* + * Record the sensor's timestamp in the request metadata. + * + * \todo The sensor timestamp should be better estimated by connecting + * to the V4L2Device::frameStart signal. + */ + request->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + /* If the buffer is cancelled force a complete of the whole request. */ if (buffer->metadata().status == FrameMetadata::FrameCancelled) { for (auto it : request->buffers()) { From patchwork Fri Apr 30 16:00:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12145 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6C596BDE4C for ; Fri, 30 Apr 2021 15:59:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 16A456891E; Fri, 30 Apr 2021 17:59:59 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5359968909 for ; Fri, 30 Apr 2021 17:59:54 +0200 (CEST) X-Originating-IP: 93.61.96.190 Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 5CDE0FF803; Fri, 30 Apr 2021 15:59:53 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Fri, 30 Apr 2021 18:00:16 +0200 Message-Id: <20210430160026.190724-7-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210430160026.190724-1-jacopo@jmondi.org> References: <20210430160026.190724-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 06/16] libcamera: uvc: Report sensor timestamp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report the sensor's timestamp in the Request metadata using the completed buffer timestamp. The UVC driver reports timestamps of SOE event through metadata, for which there is no support in the current pipeline implementation. Reviewed-by: Hirokazu Honda Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi Reviewed-by: Niklas Söderlund --- src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp index b6c6ade5ebaf..faa8d6b05f46 100644 --- a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp +++ b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp @@ -662,6 +662,10 @@ void UVCCameraData::bufferReady(FrameBuffer *buffer) { Request *request = buffer->request(); + /* \todo Use the UVC metadata to calculate a more precise timestamp */ + request->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + pipe_->completeBuffer(request, buffer); pipe_->completeRequest(request); } From patchwork Fri Apr 30 16:00:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12146 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6BC1EBDE4C for ; Fri, 30 Apr 2021 16:00:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D504068925; Fri, 30 Apr 2021 17:59:59 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DBA3768909 for ; Fri, 30 Apr 2021 17:59:54 +0200 (CEST) X-Originating-IP: 93.61.96.190 Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 763E0FF803; Fri, 30 Apr 2021 15:59:54 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Fri, 30 Apr 2021 18:00:17 +0200 Message-Id: <20210430160026.190724-8-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210430160026.190724-1-jacopo@jmondi.org> References: <20210430160026.190724-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 07/16] libcamera: rkisp1: Do not over-write metadata X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a Request is completed upon receiving the IPA produced metadata, the metadata associated with the Request are over-written, deleting the information set at output buffer completion, such as the SensorTimestamp. This commit applies to the RkISP1 pipeline handler the same change applied to IPU3 in commit 13a7ed7b1f1f ("libcamera: ipu3: Do not over-write metadata") but compared to that commit it uses the newly introduced ControlList::merge() function. Signed-off-by: Jacopo Mondi Reviewed-by: Kieran Bingham Reviewed-by: Niklas Söderlund --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index c75666391222..26f708242523 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -365,7 +365,7 @@ void RkISP1CameraData::metadataReady(unsigned int frame, const ControlList &meta if (!info) return; - info->request->metadata() = metadata; + info->request->metadata().merge(metadata); info->metadataProcessed = true; pipe->tryCompleteRequest(info->request); From patchwork Fri Apr 30 16:00:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12147 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CDA59BDE4C for ; Fri, 30 Apr 2021 16:00:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 26E1E6890E; Fri, 30 Apr 2021 18:00:01 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DAC0768923 for ; Fri, 30 Apr 2021 17:59:55 +0200 (CEST) X-Originating-IP: 93.61.96.190 Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 0C9C9FF803; Fri, 30 Apr 2021 15:59:54 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Fri, 30 Apr 2021 18:00:18 +0200 Message-Id: <20210430160026.190724-9-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210430160026.190724-1-jacopo@jmondi.org> References: <20210430160026.190724-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 08/16] libcamera: rkisp1: Report sensor timestamp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report the sensor's timestamp in the Request metadata using the completed buffer timestamp. The buffer's timestamp is recorded at DMA-transfer time, and it does not theoretically matches the 'start of exposure' definition. Record this with a \todo entry. Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi Reviewed-by: Niklas Söderlund --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 26f708242523..cee6d85fac47 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1070,6 +1070,15 @@ void PipelineHandlerRkISP1::bufferReady(FrameBuffer *buffer) { Request *request = buffer->request(); + /* + * Record the sensor's timestamp in the request metadata. + * + * \todo The sensor timestamp should be better estimated by connecting + * to the V4L2Device::frameStart signal. + */ + request->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + completeBuffer(request, buffer); tryCompleteRequest(request); } From patchwork Fri Apr 30 16:00:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12148 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A73A4BDE4C for ; Fri, 30 Apr 2021 16:00:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5CF76688A5; Fri, 30 Apr 2021 18:00:04 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CA6AB68919 for ; Fri, 30 Apr 2021 17:59:56 +0200 (CEST) X-Originating-IP: 93.61.96.190 Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 05DCBFF803; Fri, 30 Apr 2021 15:59:55 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Fri, 30 Apr 2021 18:00:19 +0200 Message-Id: <20210430160026.190724-10-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210430160026.190724-1-jacopo@jmondi.org> References: <20210430160026.190724-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 09/16] libcamera: simple: Report sensor timestamp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report the sensor's timestamp in the Request metadata using the completed buffer timestamp. The buffer's timestamp is recorded at DMA-transfer time, and it does not theoretically matches the 'start of exposure' definition. Record this with a \todo entry. Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi Reviewed-by: Niklas Söderlund --- src/libcamera/pipeline/simple/simple.cpp | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index f6095d38e97a..e1ee640b0a92 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -22,6 +22,7 @@ #include #include +#include #include #include @@ -1116,6 +1117,16 @@ void SimplePipelineHandler::bufferReady(FrameBuffer *buffer) return; } + /* + * Record the sensor's timestamp in the request metadata. + * + * \todo The sensor timestamp should be better estimated by connecting + * to the V4L2Device::frameStart signal if the platform provides it. + */ + Request *request = buffer->request(); + request->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + /* * Queue the captured and the request buffer to the converter if format * conversion is needed. If there's no queued request, just requeue the @@ -1133,7 +1144,6 @@ void SimplePipelineHandler::bufferReady(FrameBuffer *buffer) } /* Otherwise simply complete the request. */ - Request *request = buffer->request(); completeBuffer(request, buffer); completeRequest(request); } From patchwork Fri Apr 30 16:00:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12149 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BCF63BDE4C for ; Fri, 30 Apr 2021 16:00:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 56A8E68913; Fri, 30 Apr 2021 18:00:06 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 03493688A5 for ; Fri, 30 Apr 2021 17:59:58 +0200 (CEST) X-Originating-IP: 93.61.96.190 Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 008F3FF804; Fri, 30 Apr 2021 15:59:56 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Fri, 30 Apr 2021 18:00:20 +0200 Message-Id: <20210430160026.190724-11-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210430160026.190724-1-jacopo@jmondi.org> References: <20210430160026.190724-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 10/16] libcamera: vimc: Report sensor timestamp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report the sensor's timestamp in the Request metadata using the completed buffer timestamp. The buffer's timestamp reports the video capture buffer processing time, and it does not theoretically matches the 'start of exposure' definition. VIMC being a testing platform and the test driver completes the buffers for each media entity connected in the pipeline one after the other, the current solution is acceptable for the pipeline. Reviewed-by: Hirokazu Honda Reviewed-by: Niklas Söderlund Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/vimc/vimc.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/libcamera/pipeline/vimc/vimc.cpp b/src/libcamera/pipeline/vimc/vimc.cpp index 8f5f4ba30953..ce83dcaab8ea 100644 --- a/src/libcamera/pipeline/vimc/vimc.cpp +++ b/src/libcamera/pipeline/vimc/vimc.cpp @@ -523,6 +523,10 @@ void VimcCameraData::bufferReady(FrameBuffer *buffer) { Request *request = buffer->request(); + /* Record the sensor's timestamp in the request metadata. */ + request->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + pipe_->completeBuffer(request, buffer); pipe_->completeRequest(request); } From patchwork Fri Apr 30 16:00:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12150 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 71D4CBDE4C for ; Fri, 30 Apr 2021 16:00:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 287A468913; Fri, 30 Apr 2021 18:00:09 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0B21768915 for ; Fri, 30 Apr 2021 17:59:59 +0200 (CEST) X-Originating-IP: 93.61.96.190 Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 33048FF80B; Fri, 30 Apr 2021 15:59:57 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Fri, 30 Apr 2021 18:00:21 +0200 Message-Id: <20210430160026.190724-12-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210430160026.190724-1-jacopo@jmondi.org> References: <20210430160026.190724-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 11/16] libcamera: buffer: Re-work setRequest() documentation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" I got fooled by the documentation of setRequest() implying that the function is meant to be called by pipeline handlers only, which it is used in the Request class at Request::addBuffer() and Request::reuse() time. Rework the documentation to report that. Reviewed-by: Niklas Söderlund Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi --- src/libcamera/buffer.cpp | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/libcamera/buffer.cpp b/src/libcamera/buffer.cpp index 7635226b9045..ca999429dce5 100644 --- a/src/libcamera/buffer.cpp +++ b/src/libcamera/buffer.cpp @@ -191,8 +191,9 @@ FrameBuffer::FrameBuffer(const std::vector &planes, unsigned int cookie) * \brief Set the request this buffer belongs to * \param[in] request Request to set * - * The intended callers of this method are pipeline handlers and only for - * buffers that are internal to the pipeline. + * For buffers added to requests by applications, this method is called by + * Request::addBuffer() or Request::reuse(). For buffers internal to pipeline + * handlers, it is called by the pipeline handlers themselves. * * \todo Shall be hidden from applications with a d-pointer design. */ From patchwork Fri Apr 30 16:00:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12151 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3DEB6BDE4C for ; Fri, 30 Apr 2021 16:00:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CC34168924; Fri, 30 Apr 2021 18:00:09 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D18C068924 for ; Fri, 30 Apr 2021 17:59:59 +0200 (CEST) X-Originating-IP: 93.61.96.190 Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 33A7EFF804; Fri, 30 Apr 2021 15:59:59 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Fri, 30 Apr 2021 18:00:22 +0200 Message-Id: <20210430160026.190724-13-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210430160026.190724-1-jacopo@jmondi.org> References: <20210430160026.190724-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 12/16] libcamera: raspberrypi: Do not over-write metadata X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a Request is completed upon receiving the IPA produced metadata, the metadata associated with the Request are over-written, deleting the information set at output buffer completion, such as the SensorTimestamp. This commit applies to the RaspberryPi pipeline handler the same change applied to IPU3 in commit 13a7ed7b1f1f ("libcamera: ipu3: Do not over-write metadata") but compared to that commit it uses the newly introduced ControlList::merge() function. Reviewed-by: Naushir Patuck Signed-off-by: Jacopo Mondi Reviewed-by: Niklas Söderlund --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 2a917455500f..46f418aa8d73 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -1312,9 +1312,14 @@ void RPiCameraData::statsMetadataComplete(uint32_t bufferId, const ControlList & handleStreamBuffer(buffer, &isp_[Isp::Stats]); - /* Fill the Request metadata buffer with what the IPA has provided */ + /* + * Add to the Request metadata buffer what the IPA has provided. + * + * Do not overwrite controls set by the pipeline handler, for example + * SensorTimestamp. + */ Request *request = requestQueue_.front(); - request->metadata() = controls; + request->metadata().merge(controls); /* * Also update the ScalerCrop in the metadata with what we actually From patchwork Fri Apr 30 16:00:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12152 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 378A2BDE4C for ; Fri, 30 Apr 2021 16:00:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C554768919; Fri, 30 Apr 2021 18:00:10 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2289968909 for ; Fri, 30 Apr 2021 18:00:01 +0200 (CEST) X-Originating-IP: 93.61.96.190 Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 08F45FF804; Fri, 30 Apr 2021 15:59:59 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Fri, 30 Apr 2021 18:00:23 +0200 Message-Id: <20210430160026.190724-14-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210430160026.190724-1-jacopo@jmondi.org> References: <20210430160026.190724-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 13/16] libcamera: raspberry: Report sensor timestamp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report the sensor's timestamp in the Request metadata by using the Unicam::Image buffer timestamp as an initial approximation. The buffer's timestamp is recorded at DMA-transfer time, and it does not theoretically matches the 'start of exposure' definition, but when used to compare two consecutive frames it gives an acceptable estimation of the sensor frame period duration. Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart Reviewed-by: Naushir Patuck Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 46f418aa8d73..0a8bc2f3c249 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -1414,6 +1414,18 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) << ", timestamp: " << buffer->metadata().timestamp; if (stream == &unicam_[Unicam::Image]) { + /* + * Record the sensor timestamp in the Request. + * + * \todo Do not assume the request in the front of the queue + * is the correct one + */ + Request *request = requestQueue_.front(); + ASSERT(request); + + request->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + /* * Lookup the sensor controls used for this frame sequence from * DelayedControl and queue them along with the frame buffer. From patchwork Fri Apr 30 16:00:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12153 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 37B6ABDE4C for ; Fri, 30 Apr 2021 16:00:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D26C66891A; Fri, 30 Apr 2021 18:00:11 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F2A3968909 for ; Fri, 30 Apr 2021 18:00:02 +0200 (CEST) X-Originating-IP: 93.61.96.190 Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 6E3C2FF814; Fri, 30 Apr 2021 16:00:01 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Fri, 30 Apr 2021 18:00:24 +0200 Message-Id: <20210430160026.190724-15-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210430160026.190724-1-jacopo@jmondi.org> References: <20210430160026.190724-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 14/16] cam: Add option to print the Request metadata X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add the "--metadata" option to the cam tool, which will be used to print the metadata associated with a completed capture request. Reviewed-by: Niklas Söderlund Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi --- src/cam/main.cpp | 3 +++ src/cam/main.h | 1 + 2 files changed, 4 insertions(+) diff --git a/src/cam/main.cpp b/src/cam/main.cpp index 994fbb343029..70e9f62c7cb0 100644 --- a/src/cam/main.cpp +++ b/src/cam/main.cpp @@ -197,6 +197,9 @@ int CamApp::parseOptions(int argc, char *argv[]) parser.addOption(OptStrictFormats, OptionNone, "Do not allow requested stream format(s) to be adjusted", "strict-formats"); + parser.addOption(OptMetadata, OptionNone, + "Print the metadata for completed requests", + "metadata"); options_ = parser.parse(argc, argv); if (!options_.valid()) diff --git a/src/cam/main.h b/src/cam/main.h index ea8dfd330830..d22451f59817 100644 --- a/src/cam/main.h +++ b/src/cam/main.h @@ -19,6 +19,7 @@ enum { OptStream = 's', OptListControls = 256, OptStrictFormats = 257, + OptMetadata = 258, }; #endif /* __CAM_MAIN_H__ */ From patchwork Fri Apr 30 16:00:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12154 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 10690BDE4C for ; Fri, 30 Apr 2021 16:00:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5845F68917; Fri, 30 Apr 2021 18:00:12 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A3FB96890E for ; Fri, 30 Apr 2021 18:00:04 +0200 (CEST) X-Originating-IP: 93.61.96.190 Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id CCD91FF804; Fri, 30 Apr 2021 16:00:02 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Fri, 30 Apr 2021 18:00:25 +0200 Message-Id: <20210430160026.190724-16-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210430160026.190724-1-jacopo@jmondi.org> References: <20210430160026.190724-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 15/16] cam: Implement OptMetadata X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement support for the new '--metadata' option by printing the value of each metadata entry associated with a completed Request. As sample of the output, running on raspberry pi, looks like the following: 3050.205672 (30.01 fps) stream0 seq: 000033 bytesused: 720000 ScalerCrop = (0x2)/3280x2460 ExposureTime = 13969 AeLocked = true DigitalGain = 1.000721 Lux = 771.204224 ColourGains = [ 1.561101, 1.629698 ] ColourTemperature = 4289 SensorBlackLevels = [ 4096, 4096, 4096, 4096 ] ColourCorrectionMatrix = [ 1.691066, -0.599756, -0.091317, -0.437452, 1.983766, -0.546314, -0.083429, -0.722407, 1.805836 ] AnalogueGain = 2.000000 SensorTimestamp = 3050205672000 3050.238999 (30.01 fps) stream0 seq: 000034 bytesused: 720000 ScalerCrop = (0x2)/3280x2460 ExposureTime = 13969 AeLocked = true DigitalGain = 1.000709 Lux = 771.232422 ColourGains = [ 1.560868, 1.630029 ] ColourTemperature = 4289 SensorBlackLevels = [ 4096, 4096, 4096, 4096 ] ColourCorrectionMatrix = [ 1.691081, -0.599726, -0.091362, -0.437497, 1.983627, -0.546130, -0.083420, -0.722523, 1.805943 ] AnalogueGain = 2.000000 SensorTimestamp = 3050238999000 Reviewed-by: Niklas Söderlund Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi --- src/cam/capture.cpp | 15 ++++++++++++++- src/cam/capture.h | 1 + 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/src/cam/capture.cpp b/src/cam/capture.cpp index 7b55fc677022..3c3e3a53adf7 100644 --- a/src/cam/capture.cpp +++ b/src/cam/capture.cpp @@ -10,6 +10,8 @@ #include #include +#include + #include "capture.h" #include "main.h" @@ -18,7 +20,8 @@ using namespace libcamera; Capture::Capture(std::shared_ptr camera, CameraConfiguration *config, EventLoop *loop) : camera_(camera), config_(config), writer_(nullptr), last_(0), loop_(loop), - queueCount_(0), captureCount_(0), captureLimit_(0) + queueCount_(0), captureCount_(0), captureLimit_(0), + printMetadata_(false) { } @@ -29,6 +32,7 @@ int Capture::run(const OptionsParser::Options &options) queueCount_ = 0; captureCount_ = 0; captureLimit_ = options[OptCapture].toInteger(); + printMetadata_ = options.isSet(OptMetadata); if (!camera_) { std::cout << "Can't capture without a camera" << std::endl; @@ -217,6 +221,15 @@ void Capture::processRequest(Request *request) std::cout << info.str() << std::endl; + if (printMetadata_) { + const ControlList &requestMetadata = request->metadata(); + for (const auto &ctrl : requestMetadata) { + const ControlId *id = controls::controls.at(ctrl.first); + std::cout << "\t" << id->name() << " = " + << ctrl.second.toString() << std::endl; + } + } + captureCount_++; if (captureLimit_ && captureCount_ >= captureLimit_) { loop_->exit(0); diff --git a/src/cam/capture.h b/src/cam/capture.h index c7c9dc00d30f..59d138766b1e 100644 --- a/src/cam/capture.h +++ b/src/cam/capture.h @@ -47,6 +47,7 @@ private: unsigned int queueCount_; unsigned int captureCount_; unsigned int captureLimit_; + bool printMetadata_; std::vector> requests_; }; From patchwork Fri Apr 30 16:00:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12155 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D0617BDE4C for ; Fri, 30 Apr 2021 16:00:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7043A68913; Fri, 30 Apr 2021 18:00:13 +0200 (CEST) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 43B5C688E4 for ; Fri, 30 Apr 2021 18:00:06 +0200 (CEST) X-Originating-IP: 93.61.96.190 Received: from uno.LocalDomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) (Authenticated sender: jacopo@jmondi.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id ABE8EFF816; Fri, 30 Apr 2021 16:00:04 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Fri, 30 Apr 2021 18:00:26 +0200 Message-Id: <20210430160026.190724-17-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210430160026.190724-1-jacopo@jmondi.org> References: <20210430160026.190724-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 16/16] android: camera_device: Use controls::SensorTimestamp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use the controls::SensorTimestamp value to populate ANDROID_SENSOR_TIMESTAMP result metadata. The Camera is assumed to provide the control in the Request metadata. Reviewed-by: Hirokazu Honda Reviewed-by: Niklas Söderlund Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi --- src/android/camera_device.cpp | 21 ++++++++------------- src/android/camera_device.h | 3 +-- 2 files changed, 9 insertions(+), 15 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index a71aee2fc734..f1f26c775611 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -2025,7 +2025,6 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques void CameraDevice::requestComplete(Request *request) { - const Request::BufferMap &buffers = request->buffers(); camera3_buffer_status status = CAMERA3_BUFFER_STATUS_OK; std::unique_ptr resultMetadata; @@ -2053,14 +2052,7 @@ void CameraDevice::requestComplete(Request *request) LOG(HAL, Debug) << "Request " << request->cookie() << " completed with " << descriptor.buffers_.size() << " streams"; - /* - * \todo The timestamp used for the metadata is currently always taken - * from the first buffer (which may be the first stream) in the Request. - * It might be appropriate to return a 'correct' (as determined by - * pipeline handlers) timestamp in the Request itself. - */ - uint64_t timestamp = buffers.begin()->second->metadata().timestamp; - resultMetadata = getResultMetadata(descriptor, timestamp); + resultMetadata = getResultMetadata(descriptor); /* Handle any JPEG compression. */ for (camera3_stream_buffer_t &buffer : descriptor.buffers_) { @@ -2105,6 +2097,9 @@ void CameraDevice::requestComplete(Request *request) captureResult.output_buffers = descriptor.buffers_.data(); if (status == CAMERA3_BUFFER_STATUS_OK) { + uint64_t timestamp = + static_cast(request->metadata() + .get(controls::SensorTimestamp)); notifyShutter(descriptor.frameNumber_, timestamp); captureResult.partial_result = 1; @@ -2164,8 +2159,7 @@ void CameraDevice::notifyError(uint32_t frameNumber, camera3_stream_t *stream) * Produce a set of fixed result metadata. */ std::unique_ptr -CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor, - int64_t timestamp) const +CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) const { const ControlList &metadata = descriptor.request_->metadata(); const CameraMetadata &settings = descriptor.settings_; @@ -2291,8 +2285,6 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor, resultMetadata->addEntry(ANDROID_SENSOR_TEST_PATTERN_MODE, &value32, 1); - resultMetadata->addEntry(ANDROID_SENSOR_TIMESTAMP, ×tamp, 1); - value = ANDROID_STATISTICS_FACE_DETECT_MODE_OFF; resultMetadata->addEntry(ANDROID_STATISTICS_FACE_DETECT_MODE, &value, 1); @@ -2318,6 +2310,9 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor, &rolling_shutter_skew, 1); /* Add metadata tags reported by libcamera. */ + const int64_t timestamp = metadata.get(controls::SensorTimestamp); + resultMetadata->addEntry(ANDROID_SENSOR_TIMESTAMP, ×tamp, 1); + if (metadata.contains(controls::draft::PipelineDepth)) { uint8_t pipeline_depth = metadata.get(controls::draft::PipelineDepth); diff --git a/src/android/camera_device.h b/src/android/camera_device.h index c63e8e21726e..23457e47767a 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -107,8 +107,7 @@ private: libcamera::PixelFormat toPixelFormat(int format) const; int processControls(Camera3RequestDescriptor *descriptor); std::unique_ptr getResultMetadata( - const Camera3RequestDescriptor &descriptor, - int64_t timestamp) const; + const Camera3RequestDescriptor &descriptor) const; unsigned int id_; camera3_device_t camera3Device_;