From patchwork Wed May 12 08:47:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 12251 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AA50AC31E3 for ; Wed, 12 May 2021 08:47:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0A51668920; Wed, 12 May 2021 10:47:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="qDa0Fasy"; dkim-atps=neutral Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7D0CA688E4 for ; Wed, 12 May 2021 10:47:49 +0200 (CEST) Received: by mail-wr1-x42c.google.com with SMTP id q5so3298588wrs.4 for ; Wed, 12 May 2021 01:47:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=AJ6XpHi6pLrZNinv13+3uvM+Ynhndyf1PXAUaNkGF1c=; b=qDa0FasyUNKRJ36FpC3Q7Mz52otpUBD1vW6z3qrk6chBU2eAU+pKbfNNDJ+OF/bJNv b1T6Yq+5/c5Ql2PUjAKd4tUT+P7Mac33zuMHZdxlWh3s20+NLfhuWOpUihv5FdeJ1Dev bLxPohDF8cCwPPGlZHWO6yBlyIXf5lOY/n6l9K/IvKzlvQ70dm/6AYyb6WUuGoz+XsFl Xe0t0XM/FHsht52Teo5BGnXi8lF4AEOtdxA8215JRMdmIT9VNDjvohTpGbk3Dp1sgE7k 9uV57sgIJLiPLgZqFSpjA11L5tP3FBqAncbQJC20YLs+j+weAH1e/FaAHzSyuL5Fg1zx uJFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=AJ6XpHi6pLrZNinv13+3uvM+Ynhndyf1PXAUaNkGF1c=; b=Z8biXyxC2jf99svDlvmWORQlzAprLxP5xavOyLwTccFTxjGaX6kRTD47+fd6bLsmbA L3GGYPyNe4GGZ0QpPO+ey6vhOdXzhPQhkVAO/wpAYI7/iDLhOLbx304+6kRX7VBvfe7G uOT1RJCG7xXa6F/inBsp3s+AudsRmxn55184JUOAbkI+NxkM44Z+aCrHAkMGZrxpGPuE NvRnIaFb+xSLuo7YW6Ddn6pcceRVHIHA3K6xmkZmGmbWzR3DqcPDAT/IlrHPQmkIKY2T 42spfTJFGiFFCVojOUV1w0Xu0XQLvf+edfDe195Y3NptcJIvwd6eTwHv/mN3KtoYeiMS 2J6A== X-Gm-Message-State: AOAM532FgiOiCj1j2x4Z8K/raeX5hTLfiqO/v+7wPAecGBOztZBkjWRu O3RhGTskT16zjg8I5BdbA87xTKVj2aQUXw== X-Google-Smtp-Source: ABdhPJx5rwiuWmvZVr0ky4BYxjfGEdrFzbGrA1GXffJccbU7NVHM6GHMCiUBcs/aKmrWK6qAM89yMA== X-Received: by 2002:a5d:524f:: with SMTP id k15mr43414469wrc.412.1620809268941; Wed, 12 May 2021 01:47:48 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:d29a:cbe2:1c54:7a47]) by smtp.gmail.com with ESMTPSA id d3sm16370759wrs.41.2021.05.12.01.47.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 May 2021 01:47:48 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Wed, 12 May 2021 09:47:43 +0100 Message-Id: <20210512084744.1499469-1-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/2] pipeline: raspberrypi: Store timestamp in the correct Request metadata X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Write the controls::SensorTimestamp value in the Request metadata when the request is popped from the queue ready to run the pipeline. This ensures that the timestamp is written to the correct Request item, which may not be at the top of the queue when the Unicam buffer dequeue occurs. Fixes: fcfb1dc02a6b ("libcamera: raspberry: Report sensor timestamp") Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: David Plowman Reviewed-by: Kieran Bingham --- .../pipeline/raspberrypi/raspberrypi.cpp | 29 +++++++++++-------- 1 file changed, 17 insertions(+), 12 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 6fbdba0487bf..eb6d31670567 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -221,6 +221,8 @@ public: private: void checkRequestCompleted(); + void fillRequestMetadata(const ControlList &bufferControls, + Request *request); void tryRunPipeline(); bool findMatchingBuffers(BayerFrame &bayerFrame, FrameBuffer *&embeddedBuffer); @@ -1416,18 +1418,6 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) << ", timestamp: " << buffer->metadata().timestamp; if (stream == &unicam_[Unicam::Image]) { - /* - * Record the sensor timestamp in the Request. - * - * \todo Do not assume the request in the front of the queue - * is the correct one - */ - Request *request = requestQueue_.front(); - ASSERT(request); - - request->metadata().set(controls::SensorTimestamp, - buffer->metadata().timestamp); - /* * Lookup the sensor controls used for this frame sequence from * DelayedControl and queue them along with the frame buffer. @@ -1689,6 +1679,13 @@ void RPiCameraData::applyScalerCrop(const ControlList &controls) } } +void RPiCameraData::fillRequestMetadata(const ControlList &bufferControls, + Request *request) +{ + request->metadata().set(controls::SensorTimestamp, + bufferControls.get(controls::SensorTimestamp)); +} + void RPiCameraData::tryRunPipeline() { FrameBuffer *embeddedBuffer; @@ -1708,6 +1705,14 @@ void RPiCameraData::tryRunPipeline() /* See if a new ScalerCrop value needs to be applied. */ applyScalerCrop(request->controls()); + /* + * Clear the request metadata and fill it with some initial non-IPA + * related controls. We clear it first because the request metadata + * may have been populated if we have dropped the previous frame. + */ + request->metadata().clear(); + fillRequestMetadata(bayerFrame.controls, request); + /* * Process all the user controls by the IPA. Once this is complete, we * queue the ISP output buffer listed in the request to start the HW From patchwork Wed May 12 08:47:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 12252 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5425AC31E5 for ; Wed, 12 May 2021 08:47:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7C9786891A; Wed, 12 May 2021 10:47:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Ew0OSv96"; dkim-atps=neutral Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 39A7F688E4 for ; Wed, 12 May 2021 10:47:50 +0200 (CEST) Received: by mail-wm1-x32c.google.com with SMTP id n17-20020a7bc5d10000b0290169edfadac9so2071988wmk.1 for ; Wed, 12 May 2021 01:47:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5g00rQh09hp0CWGm/UKrsjSeTYX4ad+gKpUJXc2JwTg=; b=Ew0OSv96vkD49OuMk7NUuzeMMlQVIUy+GhLiLZKwI0ygWoa0NcdkBbMpT+GrON7JaX XA90hOwX4OqT+8Tv+5/v9Jt7ItNgk4zpcmoW1tpjeNmXB5Uacey5ZYziLSz3aQumHgYs A7MShhGhztpsIOMg5sbqSBosokoG2mi3upv4fEhbUJ/zkYVs5OEpiAZ2zUEVeuaGYT9P +GtNBw5vh4jDIdGEdPswVY4YgrZxdN1/xuo99kQdyxlWnbuqraTk1eYZi3iNcgyapO2f XglpJcqOgM4nNjsS3u14qljpowsh201EazrwC5Vu2WfS9/X5zXMjWtUWY0FG40bO8/JB 5oPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5g00rQh09hp0CWGm/UKrsjSeTYX4ad+gKpUJXc2JwTg=; b=eAh/dPra+nZMOeEP7+Ktc57Lcf8amNSWB8AuL6DeElqgJiIbnpJOefnf0o3lulvoGs 2PtrXPc2qkn+v+Um37Jbimh1UcOsF3mAAfuhlllL3XCsGwPYP2+q8EPQ7VgKj844pxC0 FYQClBsuKh1CP9Jy39XwaD+8vX5tWRbCz9GQNATbmrQGr93eV7nOa8MTpwaKEwh1myGM Pzhp5Fk1A2aVV4KqdYJCp6JD03kVZ014N7x4Y0a7WyxYolt3o+o1LogsgX/XCOjULU0j NFv+knJKO2rqj1FU62KyCG7EELhcVZkFwc+HWgjk6WtFL4pxbpA8btsMHfFf1Ps5ALsH zZQg== X-Gm-Message-State: AOAM530E7SjEBagXKsxsmqi48Kr83ySC5MsHLy0JKVBSkIlh/Q9/9Y// u0p7FRrE7z4Wf6r6KZs0MEh1kvjn0/JszA== X-Google-Smtp-Source: ABdhPJwCFVEZD9K/rmL8YV7O+ENd1EzlGU6NCl97ArUM80FzW8HMSjPzIALXlF8xVuPJLJsCVpnBTQ== X-Received: by 2002:a1c:750b:: with SMTP id o11mr10357862wmc.188.1620809269709; Wed, 12 May 2021 01:47:49 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:d29a:cbe2:1c54:7a47]) by smtp.gmail.com with ESMTPSA id d3sm16370759wrs.41.2021.05.12.01.47.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 May 2021 01:47:49 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Wed, 12 May 2021 09:47:44 +0100 Message-Id: <20210512084744.1499469-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210512084744.1499469-1-naush@raspberrypi.com> References: <20210512084744.1499469-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/2] pipeline raspberrypi: Move adding of ScalerCrop to the Request metadata X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the recent change to merge existing Request metadata with the ControlList provided by the IPA in commit fcfb1dc02a6b ("libcamera: raspberry: Report sensor timestamp"), we can now write the controls::ScalerCrop value at the start of the pipeline instead of at the end. This change simplifies the logic slightly, and allows us to write all metadata items to the Request in one place. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: David Plowman Reviewed-by: Kieran Bingham --- .../pipeline/raspberrypi/raspberrypi.cpp | 32 +++++++------------ 1 file changed, 12 insertions(+), 20 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index eb6d31670567..0a71325ad7c0 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -140,7 +140,7 @@ public: RPiCameraData(PipelineHandler *pipe) : CameraData(pipe), state_(State::Stopped), supportsFlips_(false), flipsAlterBayerOrder_(false), - updateScalerCrop_(true), dropFrameCount_(0), ispOutputCount_(0) + dropFrameCount_(0), ispOutputCount_(0) { } @@ -214,7 +214,6 @@ public: CameraSensorInfo sensorInfo_; Rectangle ispCrop_; /* crop in ISP (camera mode) pixels */ Rectangle scalerCrop_; /* crop in sensor native pixels */ - bool updateScalerCrop_; Size ispMinCropSize_; unsigned int dropFrameCount_; @@ -1325,23 +1324,7 @@ void RPiCameraData::statsMetadataComplete(uint32_t bufferId, const ControlList & Request *request = requestQueue_.front(); request->metadata().merge(controls); - /* - * Also update the ScalerCrop in the metadata with what we actually - * used. But we must first rescale that from ISP (camera mode) pixels - * back into sensor native pixels. - * - * Sending this information on every frame may be helpful. - */ - if (updateScalerCrop_) { - updateScalerCrop_ = false; - scalerCrop_ = ispCrop_.scaledBy(sensorInfo_.analogCrop.size(), - sensorInfo_.outputSize); - scalerCrop_.translateBy(sensorInfo_.analogCrop.topLeft()); - } - request->metadata().set(controls::ScalerCrop, scalerCrop_); - state_ = State::IpaComplete; - handleState(); } @@ -1673,8 +1656,15 @@ void RPiCameraData::applyScalerCrop(const ControlList &controls) if (ispCrop != ispCrop_) { isp_[Isp::Input].dev()->setSelection(V4L2_SEL_TGT_CROP, &ispCrop); ispCrop_ = ispCrop; - /* queueFrameAction will have to update its scalerCrop_ */ - updateScalerCrop_ = true; + + /* + * Also update the ScalerCrop in the metadata with what we actually + * used. But we must first rescale that from ISP (camera mode) pixels + * back into sensor native pixels. + */ + scalerCrop_ = ispCrop_.scaledBy(sensorInfo_.analogCrop.size(), + sensorInfo_.outputSize); + scalerCrop_.translateBy(sensorInfo_.analogCrop.topLeft()); } } } @@ -1684,6 +1674,8 @@ void RPiCameraData::fillRequestMetadata(const ControlList &bufferControls, { request->metadata().set(controls::SensorTimestamp, bufferControls.get(controls::SensorTimestamp)); + + request->metadata().set(controls::ScalerCrop, scalerCrop_); } void RPiCameraData::tryRunPipeline()