From patchwork Wed Apr 21 16:03:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12033 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EBA93BDB17 for ; Wed, 21 Apr 2021 16:02:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B876868854; Wed, 21 Apr 2021 18:02:45 +0200 (CEST) Received: from relay10.mail.gandi.net (relay10.mail.gandi.net [217.70.178.230]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 62B626884C for ; Wed, 21 Apr 2021 18:02:43 +0200 (CEST) Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay10.mail.gandi.net (Postfix) with ESMTPSA id 3C86124000D; Wed, 21 Apr 2021 16:02:42 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 21 Apr 2021 18:03:04 +0200 Message-Id: <20210421160319.42251-2-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210421160319.42251-1-jacopo@jmondi.org> References: <20210421160319.42251-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 01/16] libcamera: controls: Add a function to merge two control lists X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Laurent Pinchart Add a new ControlList::merge() function to merge two control lists. The semantics is identical to std::unordered_map::merge(). This can be used by pipeline handlers to merge metadata they populate with metadata received from an IPA. Signed-off-by: Laurent Pinchart Reviewed-by: Niklas Söderlund Signed-off-by: Jacopo Mondi Reviewed-by: Hirokazu Honda --- include/libcamera/controls.h | 2 ++ src/libcamera/controls.cpp | 20 ++++++++++++++++++++ 2 files changed, 22 insertions(+) diff --git a/include/libcamera/controls.h b/include/libcamera/controls.h index 1a5690a5ccbe..16726ce9b1ed 100644 --- a/include/libcamera/controls.h +++ b/include/libcamera/controls.h @@ -363,7 +363,9 @@ public: bool empty() const { return controls_.empty(); } std::size_t size() const { return controls_.size(); } + void clear() { controls_.clear(); } + void merge(ControlList &source); bool contains(const ControlId &id) const; bool contains(unsigned int id) const; diff --git a/src/libcamera/controls.cpp b/src/libcamera/controls.cpp index c58ed3946f3b..58555ea64c95 100644 --- a/src/libcamera/controls.cpp +++ b/src/libcamera/controls.cpp @@ -874,6 +874,26 @@ ControlList::ControlList(const ControlInfoMap &infoMap, ControlValidator *valida * \brief Removes all controls from the list */ +/** + * \brief Merge the \a source into the ControlList + * \param[in] source The ControlList to merge into this object + * + * Merging two control lists extracts elements from the \a source and insert + * them in *this. If the \a source contains elements whose key is already + * present in *this, then those elements are not extracted. The semantics is + * identical to std::unordered_map::merge() in that only internal pointers to + * nodes are updated, without copying or moving the elements. + * + * Only control lists created from the same ControlIdMap or ControlInfoMap may + * be merged. Attempting to do otherwise results in undefined behaviour. + */ +void ControlList::merge(ControlList &source) +{ + ASSERT(idmap_ == source.idmap_); + + controls_.merge(source.controls_); +} + /** * \brief Check if the list contains a control with the specified \a id * \param[in] id The control ID From patchwork Wed Apr 21 16:03:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12034 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 98A02BDB17 for ; Wed, 21 Apr 2021 16:02:48 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 45BD16885E; Wed, 21 Apr 2021 18:02:48 +0200 (CEST) Received: from relay10.mail.gandi.net (relay10.mail.gandi.net [217.70.178.230]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BC85C68835 for ; Wed, 21 Apr 2021 18:02:43 +0200 (CEST) Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay10.mail.gandi.net (Postfix) with ESMTPSA id 5F852240002; Wed, 21 Apr 2021 16:02:43 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 21 Apr 2021 18:03:05 +0200 Message-Id: <20210421160319.42251-3-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210421160319.42251-1-jacopo@jmondi.org> References: <20210421160319.42251-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 02/16] test: control_list: Check for Brightness presence X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The current test verifies that after adding a new control to a list already populated with one control the new one is present. However the test wrongly tests for its presence twice instead of making sure the existing control is still there. Fix this by checking for the presence of both controls after the update, and fix the error message accordingly. Signed-off-by: Jacopo Mondi Reviewed-by: Niklas Söderlund Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart --- test/controls/control_list.cpp | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/test/controls/control_list.cpp b/test/controls/control_list.cpp index b5a49dc17000..2b321ddd6fa4 100644 --- a/test/controls/control_list.cpp +++ b/test/controls/control_list.cpp @@ -108,9 +108,10 @@ protected: list.set(controls::Brightness, 0.0f); list.set(controls::Contrast, 1.5f); - if (!list.contains(controls::Contrast) || + if (!list.contains(controls::Brightness) || !list.contains(controls::Contrast)) { - cout << "List should contain Contrast control" << endl; + cout << "List should contain Brightness and Contrast controls" + << endl; return TestFail; } From patchwork Wed Apr 21 16:03:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12035 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 77564BDB17 for ; Wed, 21 Apr 2021 16:02:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EFF2068860; Wed, 21 Apr 2021 18:02:48 +0200 (CEST) Received: from relay10.mail.gandi.net (relay10.mail.gandi.net [217.70.178.230]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5550A68846 for ; Wed, 21 Apr 2021 18:02:44 +0200 (CEST) Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay10.mail.gandi.net (Postfix) with ESMTPSA id E1FE924000D; Wed, 21 Apr 2021 16:02:43 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 21 Apr 2021 18:03:06 +0200 Message-Id: <20210421160319.42251-4-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210421160319.42251-1-jacopo@jmondi.org> References: <20210421160319.42251-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 03/16] test: control_list: Test ControlList::merge() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Test the newly introduce ControlList::merge() method by creating a new list with a single control and merging it with the existing one. Test that the merged list contains all the controls and their values are not changed. Signed-off-by: Jacopo Mondi Reviewed-by: Niklas Söderlund Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart --- test/controls/control_list.cpp | 40 ++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/test/controls/control_list.cpp b/test/controls/control_list.cpp index 2b321ddd6fa4..91b38993abf9 100644 --- a/test/controls/control_list.cpp +++ b/test/controls/control_list.cpp @@ -150,6 +150,46 @@ protected: return TestFail; } + /* + * Create a new list with a new control and merge it with the + * existing one, verifying that the existing controls + * values does not get changed. + */ + ControlList mergeList(controls::controls, &validator); + mergeList.set(controls::Saturation, 0.4f); + + mergeList.merge(list); + if (mergeList.size() != 3) { + cout << "Merged list should contain three elements" << endl; + return TestFail; + } + + if (!mergeList.contains(controls::Brightness) || + !mergeList.contains(controls::Contrast) || + !mergeList.contains(controls::Saturation)) { + cout << "Merged list does not contain all controls it should" + << endl; + return TestFail; + } + + if (mergeList.get(controls::Brightness) != 0.5f) { + cout << "Brightness control value changed after merging lists" + << endl; + return TestFail; + } + + if (mergeList.get(controls::Contrast) != 1.1f) { + cout << "Contrast control value changed after merging lists" + << endl; + return TestFail; + } + + if (mergeList.get(controls::Saturation) != 0.4f) { + cout << "Saturation control value changed after merging lists" + << endl; + return TestFail; + } + return TestPass; } }; From patchwork Wed Apr 21 16:03:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12036 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 76189BDB17 for ; Wed, 21 Apr 2021 16:02:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DB14968851; Wed, 21 Apr 2021 18:02:49 +0200 (CEST) Received: from relay10.mail.gandi.net (relay10.mail.gandi.net [217.70.178.230]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 694AA68852 for ; Wed, 21 Apr 2021 18:02:45 +0200 (CEST) Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay10.mail.gandi.net (Postfix) with ESMTPSA id 7AC8524000D; Wed, 21 Apr 2021 16:02:44 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 21 Apr 2021 18:03:07 +0200 Message-Id: <20210421160319.42251-5-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210421160319.42251-1-jacopo@jmondi.org> References: <20210421160319.42251-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 04/16] libcamera: controls: Destage 'SensorTimestamp' X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Destage the 'SensorTimestamp' control, which is used by pipeline handlers to report the time when the first active line of the sensor's pixel array is exposed. Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Hirokazu Honda Signed-off-by: Jacopo Mondi --- src/libcamera/control_ids.yaml | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml index b4771f9def89..88d81ac4cccc 100644 --- a/src/libcamera/control_ids.yaml +++ b/src/libcamera/control_ids.yaml @@ -360,6 +360,20 @@ controls: size: [2] + - SensorTimestamp: + type: int64_t + description: | + The time when the first row of the image sensor active array is exposed. + + The timestamp, expressed in nanoseconds, represents a monotonically + increasing counter since the system boot time, as defined by the + Linux-specific CLOCK_BOOTTIME clock id. + + The SensorTimestamp control can only be returned in metadata. + + \todo Define how the sensor timestamp has to be used in the reprocessing + use case. + # ---------------------------------------------------------------------------- # Draft controls section @@ -547,13 +561,6 @@ controls: value: 3 description: The AWB algorithm is locked. - - SensorTimestamp: - type: int64_t - draft: true - description: | - Control to report the start of exposure of the first row of the captured - image. Currently identical to ANDROID_SENSOR_TIMESTAMP. - - SensorRollingShutterSkew: type: int64_t draft: true From patchwork Wed Apr 21 16:03:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12037 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3CE91BDB17 for ; Wed, 21 Apr 2021 16:02:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 43B9568864; Wed, 21 Apr 2021 18:02:50 +0200 (CEST) Received: from relay10.mail.gandi.net (relay10.mail.gandi.net [217.70.178.230]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8094368851 for ; Wed, 21 Apr 2021 18:02:46 +0200 (CEST) Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay10.mail.gandi.net (Postfix) with ESMTPSA id 92388240008; Wed, 21 Apr 2021 16:02:45 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 21 Apr 2021 18:03:08 +0200 Message-Id: <20210421160319.42251-6-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210421160319.42251-1-jacopo@jmondi.org> References: <20210421160319.42251-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 05/16] libcamera: ipu3: Report sensor timestamp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report the sensor's timestamp in the Request metadata by using the CIO2 buffer timestamp as an initial approximation. The buffer's timestamp is recorded at DMA-transfer time, and it does not theoretically matches the 'start of exposure' definition, but when used to compare two consecutive frames it gives an acceptable estimation of the sensor frame period duration. Reviewed-by: Kieran Bingham Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/ipu3/ipu3.cpp | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 51446fcf5bc1..28e849a43a3e 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -1255,6 +1255,15 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) Request *request = info->request; + /* + * Record the sensor's timestamp in the request metadata. + * + * \todo The sensor timestamp should be better estimated by connecting + * to the V4L2Device::frameStart signal. + */ + request->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + /* If the buffer is cancelled force a complete of the whole request. */ if (buffer->metadata().status == FrameMetadata::FrameCancelled) { for (auto it : request->buffers()) From patchwork Wed Apr 21 16:03:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12038 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3DEAFBDB17 for ; Wed, 21 Apr 2021 16:02:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DF4DC68858; Wed, 21 Apr 2021 18:02:50 +0200 (CEST) Received: from relay10.mail.gandi.net (relay10.mail.gandi.net [217.70.178.230]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6E9B66884A for ; Wed, 21 Apr 2021 18:02:47 +0200 (CEST) Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay10.mail.gandi.net (Postfix) with ESMTPSA id A519824000D; Wed, 21 Apr 2021 16:02:46 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 21 Apr 2021 18:03:09 +0200 Message-Id: <20210421160319.42251-7-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210421160319.42251-1-jacopo@jmondi.org> References: <20210421160319.42251-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 06/16] libcamera: uvc: Report sensor timestamp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report the sensor's timestamp in the Request metadata using the completed buffer timestamp. The UVC driver reports timestamps of SOE event through metadata, for which there is no support in the current pipeline implementation. Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi Reviewed-by: Hirokazu Honda --- src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp index b6c6ade5ebaf..faa8d6b05f46 100644 --- a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp +++ b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp @@ -662,6 +662,10 @@ void UVCCameraData::bufferReady(FrameBuffer *buffer) { Request *request = buffer->request(); + /* \todo Use the UVC metadata to calculate a more precise timestamp */ + request->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + pipe_->completeBuffer(request, buffer); pipe_->completeRequest(request); } From patchwork Wed Apr 21 16:03:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12039 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 03D54BDB17 for ; Wed, 21 Apr 2021 16:02:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 900386885C; Wed, 21 Apr 2021 18:02:52 +0200 (CEST) Received: from relay10.mail.gandi.net (relay10.mail.gandi.net [217.70.178.230]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F33B46884C for ; Wed, 21 Apr 2021 18:02:47 +0200 (CEST) Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay10.mail.gandi.net (Postfix) with ESMTPSA id 9934424000C; Wed, 21 Apr 2021 16:02:47 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 21 Apr 2021 18:03:10 +0200 Message-Id: <20210421160319.42251-8-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210421160319.42251-1-jacopo@jmondi.org> References: <20210421160319.42251-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 07/16] libcamera: rkisp1: Do not over-write metadata X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a Request is completed upon receiving the IPA produced metadata, the metadata associated with the Request are over-written, deleting the information set at output buffer completion, such as the SensorTimestamp. This commit applies to the RkISP1 pipeline handler the same change applied to IPU3 in commit 13a7ed7b1f1f ("libcamera: ipu3: Do not over-write metadata") but compared to that commit it uses the newly introduced ControlList::merge() function. Fixes: 0eb65e14e18d ("libcamera: pipeline: rkisp1: Attach to an IPA") Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 549f4a4e61a8..c3d390f775f2 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -362,7 +362,7 @@ void RkISP1CameraData::metadataReady(unsigned int frame, const ControlList &meta if (!info) return; - info->request->metadata() = metadata; + info->request->metadata().merge(const_cast(metadata)); info->metadataProcessed = true; pipe->tryCompleteRequest(info->request); From patchwork Wed Apr 21 16:03:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12040 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EF309BDB17 for ; Wed, 21 Apr 2021 16:02:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C0B556885B; Wed, 21 Apr 2021 18:02:55 +0200 (CEST) Received: from relay10.mail.gandi.net (relay10.mail.gandi.net [217.70.178.230]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AED6068854 for ; Wed, 21 Apr 2021 18:02:48 +0200 (CEST) Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay10.mail.gandi.net (Postfix) with ESMTPSA id 240E3240003; Wed, 21 Apr 2021 16:02:47 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 21 Apr 2021 18:03:11 +0200 Message-Id: <20210421160319.42251-9-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210421160319.42251-1-jacopo@jmondi.org> References: <20210421160319.42251-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 08/16] libcamera: rkisp1: Report sensor timestamp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report the sensor's timestamp in the Request metadata using the completed buffer timestamp. The buffer's timestamp is recorded at DMA-transfer time, and it does not theoretically matches the 'start of exposure' definition. Record this with a \todo entry. Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi Reviewed-by: Hirokazu Honda --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index c3d390f775f2..da57624a2817 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1067,6 +1067,15 @@ void PipelineHandlerRkISP1::bufferReady(FrameBuffer *buffer) { Request *request = buffer->request(); + /* + * Record the sensor's timestamp in the request metadata. + * + * \todo The sensor timestamp should be better estimated by connecting + * to the V4L2Device::frameStart signal. + */ + request->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + completeBuffer(request, buffer); tryCompleteRequest(request); } From patchwork Wed Apr 21 16:03:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12041 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 71625BDB17 for ; Wed, 21 Apr 2021 16:02:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0549868850; Wed, 21 Apr 2021 18:02:57 +0200 (CEST) Received: from relay10.mail.gandi.net (relay10.mail.gandi.net [217.70.178.230]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4482568850 for ; Wed, 21 Apr 2021 18:02:49 +0200 (CEST) Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay10.mail.gandi.net (Postfix) with ESMTPSA id DD61F240012; Wed, 21 Apr 2021 16:02:48 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 21 Apr 2021 18:03:12 +0200 Message-Id: <20210421160319.42251-10-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210421160319.42251-1-jacopo@jmondi.org> References: <20210421160319.42251-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 09/16] libcamera: simple: Report sensor timestamp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report the sensor's timestamp in the Request metadata using the completed buffer timestamp. The buffer's timestamp is recorded at DMA-transfer time, and it does not theoretically matches the 'start of exposure' definition. Record this with a \todo entry. Signed-off-by: Jacopo Mondi Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/simple/simple.cpp | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index f6095d38e97a..e1ee640b0a92 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -22,6 +22,7 @@ #include #include +#include #include #include @@ -1116,6 +1117,16 @@ void SimplePipelineHandler::bufferReady(FrameBuffer *buffer) return; } + /* + * Record the sensor's timestamp in the request metadata. + * + * \todo The sensor timestamp should be better estimated by connecting + * to the V4L2Device::frameStart signal if the platform provides it. + */ + Request *request = buffer->request(); + request->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + /* * Queue the captured and the request buffer to the converter if format * conversion is needed. If there's no queued request, just requeue the @@ -1133,7 +1144,6 @@ void SimplePipelineHandler::bufferReady(FrameBuffer *buffer) } /* Otherwise simply complete the request. */ - Request *request = buffer->request(); completeBuffer(request, buffer); completeRequest(request); } From patchwork Wed Apr 21 16:03:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12042 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6A7E5BDB17 for ; Wed, 21 Apr 2021 16:02:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E9E6068858; Wed, 21 Apr 2021 18:02:57 +0200 (CEST) Received: from relay10.mail.gandi.net (relay10.mail.gandi.net [217.70.178.230]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 054E768862 for ; Wed, 21 Apr 2021 18:02:50 +0200 (CEST) Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay10.mail.gandi.net (Postfix) with ESMTPSA id 6B2B3240003; Wed, 21 Apr 2021 16:02:49 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 21 Apr 2021 18:03:13 +0200 Message-Id: <20210421160319.42251-11-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210421160319.42251-1-jacopo@jmondi.org> References: <20210421160319.42251-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 10/16] libcamera: vimc: Report sensor timestamp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report the sensor's timestamp in the Request metadata using the completed buffer timestamp. The buffer's timestamp reports the video capture buffer processing time, and it does not theoretically matches the 'start of exposure' definition. VIMC being a testing platform and the test driver completes the buffers for each media entity connected in the pipeline one after the other, the current solution is acceptable for the pipeline. Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi Reviewed-by: Niklas Söderlund Reviewed-by: Hirokazu Honda --- src/libcamera/pipeline/vimc/vimc.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/libcamera/pipeline/vimc/vimc.cpp b/src/libcamera/pipeline/vimc/vimc.cpp index 8f5f4ba30953..ce83dcaab8ea 100644 --- a/src/libcamera/pipeline/vimc/vimc.cpp +++ b/src/libcamera/pipeline/vimc/vimc.cpp @@ -523,6 +523,10 @@ void VimcCameraData::bufferReady(FrameBuffer *buffer) { Request *request = buffer->request(); + /* Record the sensor's timestamp in the request metadata. */ + request->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + pipe_->completeBuffer(request, buffer); pipe_->completeRequest(request); } From patchwork Wed Apr 21 16:03:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12044 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 79C42BDB17 for ; Wed, 21 Apr 2021 16:03:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 44B4A6886B; Wed, 21 Apr 2021 18:03:00 +0200 (CEST) Received: from relay10.mail.gandi.net (relay10.mail.gandi.net [217.70.178.230]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B321668840 for ; Wed, 21 Apr 2021 18:02:50 +0200 (CEST) Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay10.mail.gandi.net (Postfix) with ESMTPSA id 2A76B240003; Wed, 21 Apr 2021 16:02:49 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 21 Apr 2021 18:03:14 +0200 Message-Id: <20210421160319.42251-12-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210421160319.42251-1-jacopo@jmondi.org> References: <20210421160319.42251-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 11/16] libcamera: buffer: Re-work setRequest() documentation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" I got fooled by the documentation of setRequest() implying that the function is meant to be called by pipeline handlers only, which it is used in the Request class at Request::addBuffer() and Request::reuse() time. Rework the documentation to report that. Fixes: 125be3436ae0 ("libcamera: FrameBuffer: Add a setRequest() interface") Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi Reviewed-by: Niklas Söderlund Reviewed-by: Hirokazu Honda --- src/libcamera/buffer.cpp | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/libcamera/buffer.cpp b/src/libcamera/buffer.cpp index 75b2693281a7..5baccfa99f10 100644 --- a/src/libcamera/buffer.cpp +++ b/src/libcamera/buffer.cpp @@ -191,8 +191,9 @@ FrameBuffer::FrameBuffer(const std::vector &planes, unsigned int cookie) * \brief Set the request this buffer belongs to * \param[in] request Request to set * - * The intended callers of this method are pipeline handlers and only for - * buffers that are internal to the pipeline. + * For buffers added to requests by applications, this method is called by + * Request::addBuffer() or Request::reuse(). For buffers internal to pipeline + * handlers, it is called by the pipeline handlers themselves. * * \todo Shall be hidden from applications with a d-pointer design. */ From patchwork Wed Apr 21 16:03:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12043 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6C762BDB17 for ; Wed, 21 Apr 2021 16:02:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 21C7568851; Wed, 21 Apr 2021 18:02:59 +0200 (CEST) Received: from relay10.mail.gandi.net (relay10.mail.gandi.net [217.70.178.230]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 991726884A for ; Wed, 21 Apr 2021 18:02:51 +0200 (CEST) Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay10.mail.gandi.net (Postfix) with ESMTPSA id DE16D240003; Wed, 21 Apr 2021 16:02:50 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 21 Apr 2021 18:03:15 +0200 Message-Id: <20210421160319.42251-13-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210421160319.42251-1-jacopo@jmondi.org> References: <20210421160319.42251-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 12/16] libcamera: raspberrypi: Do not over-write metadata X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a Request is completed upon receiving the IPA produced metadata, the metadata associated with the Request are over-written, deleting the information set at output buffer completion, such as the SensorTimestamp. This commit applies to the RaspberryPi pipeline handler the same change applied to IPU3 in commit 13a7ed7b1f1f ("libcamera: ipu3: Do not over-write metadata") but compared to that commit it uses the newly introduced ControlList::merge() function. Reviewed-by: Naushir Patuck Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 2a917455500f..6c6d31f78c88 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -1312,9 +1312,14 @@ void RPiCameraData::statsMetadataComplete(uint32_t bufferId, const ControlList & handleStreamBuffer(buffer, &isp_[Isp::Stats]); - /* Fill the Request metadata buffer with what the IPA has provided */ + /* + * Add to the Request metadata buffer what the IPA has provided. + * + * Do not overwrite controls set by the pipeline handler, for example + * SensorTimestamp. + */ Request *request = requestQueue_.front(); - request->metadata() = controls; + request->metadata().merge(const_cast(controls)); /* * Also update the ScalerCrop in the metadata with what we actually From patchwork Wed Apr 21 16:03:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12045 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D4791BDB18 for ; Wed, 21 Apr 2021 16:03:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 994C268870; Wed, 21 Apr 2021 18:03:00 +0200 (CEST) Received: from relay10.mail.gandi.net (relay10.mail.gandi.net [217.70.178.230]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5327668853 for ; Wed, 21 Apr 2021 18:02:52 +0200 (CEST) Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay10.mail.gandi.net (Postfix) with ESMTPSA id B1843240008; Wed, 21 Apr 2021 16:02:51 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 21 Apr 2021 18:03:16 +0200 Message-Id: <20210421160319.42251-14-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210421160319.42251-1-jacopo@jmondi.org> References: <20210421160319.42251-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 13/16] libcamera: raspberry: Report sensor timestamp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Report the sensor's timestamp in the Request metadata by using the Unicam::Image buffer timestamp as an initial approximation. The buffer's timestamp is recorded at DMA-transfer time, and it does not theoretically matches the 'start of exposure' definition, but when used to compare two consecutive frames it gives an acceptable estimation of the sensor frame period duration. Reviewed-by: Naushir Patuck Signed-off-by: Jacopo Mondi Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 6c6d31f78c88..ac135f95de12 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -1414,6 +1414,18 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) << ", timestamp: " << buffer->metadata().timestamp; if (stream == &unicam_[Unicam::Image]) { + /* + * Record the sensor timestamp in the Request. + * + * \todo Do not assume the request in the front of the queue + * is the correct one + */ + Request *request = requestQueue_.front(); + ASSERT(request); + + request->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + /* * Lookup the sensor controls used for this frame sequence from * DelayedControl and queue them along with the frame buffer. From patchwork Wed Apr 21 16:03:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12048 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 22E11BDB17 for ; Wed, 21 Apr 2021 16:03:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AB6CD6885C; Wed, 21 Apr 2021 18:03:02 +0200 (CEST) Received: from relay10.mail.gandi.net (relay10.mail.gandi.net [217.70.178.230]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C928C68852 for ; Wed, 21 Apr 2021 18:02:52 +0200 (CEST) Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay10.mail.gandi.net (Postfix) with ESMTPSA id 6B855240003; Wed, 21 Apr 2021 16:02:52 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 21 Apr 2021 18:03:17 +0200 Message-Id: <20210421160319.42251-15-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210421160319.42251-1-jacopo@jmondi.org> References: <20210421160319.42251-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 14/16] cam: Add option to print the Request metadata X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add the "--metadata" option to the cam tool, which will be used to print the metadata associated with a completed capture request. Signed-off-by: Jacopo Mondi Reviewed-by: Niklas Söderlund Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart --- src/cam/main.cpp | 3 +++ src/cam/main.h | 1 + 2 files changed, 4 insertions(+) diff --git a/src/cam/main.cpp b/src/cam/main.cpp index 994fbb343029..687142d8b1a8 100644 --- a/src/cam/main.cpp +++ b/src/cam/main.cpp @@ -197,6 +197,9 @@ int CamApp::parseOptions(int argc, char *argv[]) parser.addOption(OptStrictFormats, OptionNone, "Do not allow requested stream format(s) to be adjusted", "strict-formats"); + parser.addOption(OptMetadata, OptionNone, + "Print the list of metadata associated with each completed capture request", + "metadata"); options_ = parser.parse(argc, argv); if (!options_.valid()) diff --git a/src/cam/main.h b/src/cam/main.h index ea8dfd330830..d22451f59817 100644 --- a/src/cam/main.h +++ b/src/cam/main.h @@ -19,6 +19,7 @@ enum { OptStream = 's', OptListControls = 256, OptStrictFormats = 257, + OptMetadata = 258, }; #endif /* __CAM_MAIN_H__ */ From patchwork Wed Apr 21 16:03:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12046 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 876A9BDB17 for ; Wed, 21 Apr 2021 16:03:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 328FE68854; Wed, 21 Apr 2021 18:03:01 +0200 (CEST) Received: from relay10.mail.gandi.net (relay10.mail.gandi.net [217.70.178.230]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 79EDA68859 for ; Wed, 21 Apr 2021 18:02:53 +0200 (CEST) Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay10.mail.gandi.net (Postfix) with ESMTPSA id 13E9F240003; Wed, 21 Apr 2021 16:02:52 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 21 Apr 2021 18:03:18 +0200 Message-Id: <20210421160319.42251-16-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210421160319.42251-1-jacopo@jmondi.org> References: <20210421160319.42251-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 15/16] cam: Implement OptMetadata X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement support for the new '--metadata' option by printing the value of each metadata entry associated with a completed Request. As sample of the output, running on raspberry pi, looks like the following: 3050.205672 (30.01 fps) stream0 seq: 000033 bytesused: 720000 ScalerCrop = (0x2)/3280x2460 ExposureTime = 13969 AeLocked = true DigitalGain = 1.000721 Lux = 771.204224 ColourGains = [ 1.561101, 1.629698 ] ColourTemperature = 4289 SensorBlackLevels = [ 4096, 4096, 4096, 4096 ] ColourCorrectionMatrix = [ 1.691066, -0.599756, -0.091317, -0.437452, 1.983766, -0.546314, -0.083429, -0.722407, 1.805836 ] AnalogueGain = 2.000000 SensorTimestamp = 3050205672000 3050.238999 (30.01 fps) stream0 seq: 000034 bytesused: 720000 ScalerCrop = (0x2)/3280x2460 ExposureTime = 13969 AeLocked = true DigitalGain = 1.000709 Lux = 771.232422 ColourGains = [ 1.560868, 1.630029 ] ColourTemperature = 4289 SensorBlackLevels = [ 4096, 4096, 4096, 4096 ] ColourCorrectionMatrix = [ 1.691081, -0.599726, -0.091362, -0.437497, 1.983627, -0.546130, -0.083420, -0.722523, 1.805943 ] AnalogueGain = 2.000000 SensorTimestamp = 3050238999000 Signed-off-by: Jacopo Mondi Reviewed-by: Niklas Söderlund Reviewed-by: Hirokazu Honda Reviewed-by: Laurent Pinchart --- src/cam/capture.cpp | 15 ++++++++++++++- src/cam/capture.h | 1 + 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/src/cam/capture.cpp b/src/cam/capture.cpp index 7b55fc677022..7213abd934ec 100644 --- a/src/cam/capture.cpp +++ b/src/cam/capture.cpp @@ -10,6 +10,8 @@ #include #include +#include + #include "capture.h" #include "main.h" @@ -18,7 +20,8 @@ using namespace libcamera; Capture::Capture(std::shared_ptr camera, CameraConfiguration *config, EventLoop *loop) : camera_(camera), config_(config), writer_(nullptr), last_(0), loop_(loop), - queueCount_(0), captureCount_(0), captureLimit_(0) + queueCount_(0), captureCount_(0), captureLimit_(0), + printMetadata_(false) { } @@ -29,6 +32,7 @@ int Capture::run(const OptionsParser::Options &options) queueCount_ = 0; captureCount_ = 0; captureLimit_ = options[OptCapture].toInteger(); + printMetadata_ = options.isSet(OptMetadata); if (!camera_) { std::cout << "Can't capture without a camera" << std::endl; @@ -217,6 +221,15 @@ void Capture::processRequest(Request *request) std::cout << info.str() << std::endl; + if (printMetadata_) { + const ControlList &requestMetadata = request->metadata(); + for (const auto ctrl : requestMetadata) { + const ControlId *id = controls::controls.at(ctrl.first); + std::cout << "\t" << id->name() << " = " + << ctrl.second.toString() << std::endl; + } + } + captureCount_++; if (captureLimit_ && captureCount_ >= captureLimit_) { loop_->exit(0); diff --git a/src/cam/capture.h b/src/cam/capture.h index c7c9dc00d30f..59d138766b1e 100644 --- a/src/cam/capture.h +++ b/src/cam/capture.h @@ -47,6 +47,7 @@ private: unsigned int queueCount_; unsigned int captureCount_; unsigned int captureLimit_; + bool printMetadata_; std::vector> requests_; }; From patchwork Wed Apr 21 16:03:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 12047 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 24A69BDB17 for ; Wed, 21 Apr 2021 16:03:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C32A068872; Wed, 21 Apr 2021 18:03:01 +0200 (CEST) Received: from relay10.mail.gandi.net (relay10.mail.gandi.net [217.70.178.230]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 39B0368864 for ; Wed, 21 Apr 2021 18:02:54 +0200 (CEST) Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay10.mail.gandi.net (Postfix) with ESMTPSA id 9ECA1240003; Wed, 21 Apr 2021 16:02:53 +0000 (UTC) From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Date: Wed, 21 Apr 2021 18:03:19 +0200 Message-Id: <20210421160319.42251-17-jacopo@jmondi.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210421160319.42251-1-jacopo@jmondi.org> References: <20210421160319.42251-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 16/16] android: camera_device: Use controls::SensorTimestamp X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use the controls::SensorTimestamp value to populate ANDROID_SENSOR_TIMESTAMP result metadata. The Camera is assumed to provide the control in the Request metadata. Reviewed-by: Laurent Pinchart Signed-off-by: Jacopo Mondi Reviewed-by: Niklas Söderlund Reviewed-by: Hirokazu Honda --- src/android/camera_device.cpp | 19 ++++++------------- src/android/camera_device.h | 3 +-- 2 files changed, 7 insertions(+), 15 deletions(-) diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp index a71aee2fc734..97ce34fd65a0 100644 --- a/src/android/camera_device.cpp +++ b/src/android/camera_device.cpp @@ -2025,7 +2025,6 @@ int CameraDevice::processCaptureRequest(camera3_capture_request_t *camera3Reques void CameraDevice::requestComplete(Request *request) { - const Request::BufferMap &buffers = request->buffers(); camera3_buffer_status status = CAMERA3_BUFFER_STATUS_OK; std::unique_ptr resultMetadata; @@ -2053,14 +2052,7 @@ void CameraDevice::requestComplete(Request *request) LOG(HAL, Debug) << "Request " << request->cookie() << " completed with " << descriptor.buffers_.size() << " streams"; - /* - * \todo The timestamp used for the metadata is currently always taken - * from the first buffer (which may be the first stream) in the Request. - * It might be appropriate to return a 'correct' (as determined by - * pipeline handlers) timestamp in the Request itself. - */ - uint64_t timestamp = buffers.begin()->second->metadata().timestamp; - resultMetadata = getResultMetadata(descriptor, timestamp); + resultMetadata = getResultMetadata(descriptor); /* Handle any JPEG compression. */ for (camera3_stream_buffer_t &buffer : descriptor.buffers_) { @@ -2105,6 +2097,7 @@ void CameraDevice::requestComplete(Request *request) captureResult.output_buffers = descriptor.buffers_.data(); if (status == CAMERA3_BUFFER_STATUS_OK) { + int64_t timestamp = request->metadata().get(controls::SensorTimestamp); notifyShutter(descriptor.frameNumber_, timestamp); captureResult.partial_result = 1; @@ -2164,8 +2157,7 @@ void CameraDevice::notifyError(uint32_t frameNumber, camera3_stream_t *stream) * Produce a set of fixed result metadata. */ std::unique_ptr -CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor, - int64_t timestamp) const +CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor) const { const ControlList &metadata = descriptor.request_->metadata(); const CameraMetadata &settings = descriptor.settings_; @@ -2291,8 +2283,6 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor, resultMetadata->addEntry(ANDROID_SENSOR_TEST_PATTERN_MODE, &value32, 1); - resultMetadata->addEntry(ANDROID_SENSOR_TIMESTAMP, ×tamp, 1); - value = ANDROID_STATISTICS_FACE_DETECT_MODE_OFF; resultMetadata->addEntry(ANDROID_STATISTICS_FACE_DETECT_MODE, &value, 1); @@ -2318,6 +2308,9 @@ CameraDevice::getResultMetadata(const Camera3RequestDescriptor &descriptor, &rolling_shutter_skew, 1); /* Add metadata tags reported by libcamera. */ + int64_t timestamp = metadata.get(controls::SensorTimestamp); + resultMetadata->addEntry(ANDROID_SENSOR_TIMESTAMP, ×tamp, 1); + if (metadata.contains(controls::draft::PipelineDepth)) { uint8_t pipeline_depth = metadata.get(controls::draft::PipelineDepth); diff --git a/src/android/camera_device.h b/src/android/camera_device.h index c63e8e21726e..23457e47767a 100644 --- a/src/android/camera_device.h +++ b/src/android/camera_device.h @@ -107,8 +107,7 @@ private: libcamera::PixelFormat toPixelFormat(int format) const; int processControls(Camera3RequestDescriptor *descriptor); std::unique_ptr getResultMetadata( - const Camera3RequestDescriptor &descriptor, - int64_t timestamp) const; + const Camera3RequestDescriptor &descriptor) const; unsigned int id_; camera3_device_t camera3Device_;