From patchwork Wed Oct 19 09:01:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17631 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4E671C0DA4 for ; Wed, 19 Oct 2022 09:01:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DC4B762E4C; Wed, 19 Oct 2022 11:01:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1666170086; bh=njQ9gW5DKOXOyMQ0F7GWvUGNhge0VpeeJz1MElUPrg4=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=D4sJfp623TCg/D0eoGQHDcvHAmQ4Ftms9TfHdg4Eoe7bq9p8S+O7t6dwRc9wmkej8 5BTgOiDM4QZViCVN7Zoh40LsWL5MdlQMniNGOPu5WtNTp3TEqIgeOvZKHyZ2VpwbPU NLpWIsD6Yf31jy8UPOVUvCQzgz4P5js6ghth1qu1RvQw0RoNYFVYnBC307bfZUjcLQ niHv3AZTTbd+YLTWfwoCUp7rP78UIbROaEq79H77AOBTGeoKpmpDFfIuOLQTAMQozt fND4IaoB9c+aqkmpZotz2hxLDAW86elo+ZISduEJ9IHqBQIkuJuLzAdd545bhh/spC DF4F1bJeyClyg== Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 34E8962E46 for ; Wed, 19 Oct 2022 11:01:23 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="OctBth4T"; dkim-atps=neutral Received: by mail-wr1-x42f.google.com with SMTP id n12so27810880wrp.10 for ; Wed, 19 Oct 2022 02:01:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=W917K/nmtTKhwmLkk3bJj7stCIhhxNruF7ozW1jiSAU=; b=OctBth4TcIHemxsXPy9Nfwhu5mj2oO/9ikehYJ4WGn+B1VntgndrBEuPm07PMZ/dkn z0Assm2OE1qzRzhZG2BOIHuK1EvNehax3t2i7wIm9ZsrcEdhlOdXLF+DsvmadoG+se+7 LlzrqaXZqrWKvbroFdlLAZSh/Kpi0NMZeTgPcLg/r68v2gzR8RoSHwxzOykt/5tR5i/g fL1weqSzu1dMRXkj/L5wCdclcdGsIC6Ww8JjcwmhmFzLypdrsq2JB06ziLnZ2JljbzSX 9bPi201VTrv2Q0Kogd1oJHcIlVN/O1dPgThk0dGXWkH3i1XAjP/OStjJMxirmRnj8UCA iDaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=W917K/nmtTKhwmLkk3bJj7stCIhhxNruF7ozW1jiSAU=; b=op/m9eJUYibXR+gqr+crqI/ZFLmj0PhqWLnmjRB4YMObf0+GylUE5r9kQ7t2H//xZU JotbvTnyB074pAeS4+BDVZq/SH6Qi937ax1gaux7cFlDHhoNha9OcuNmS6h0JBmMsiDk vAtxtX+44O+AfKuk4oeX2GQBXh2FfFIMVSj2dq7J5FB1TBDhDrhuP5ZgYdTL4I+kMkxy AY6k4xm0Kb0wJLZxP6F830uH6X7pWwxrCPj620JKXYaZfDDRY3ktTeewOlaOX2UNE/V9 NjtcjxRqwWqq9rv21cNRDar61hW6wec2ievtyehqH2fAwPxsk2NG+P0M1dWpwfVo8rJY jgsQ== X-Gm-Message-State: ACrzQf36trnHUtgB66+L1FgNSceqHDF2Xw0yed9sMO5NW10e3SYHYUO0 JQuL0WAr1yMcuVqiubJtWECmaXPkcVfAPg== X-Google-Smtp-Source: AMsMyM6lviXBBuajoqIJUHRmMAhYLL7aQI/AR3GjbTNl+vNt1asBU+XNufmRvGVh0G3PtfXJb6mkJA== X-Received: by 2002:a05:6000:1051:b0:22e:5d8a:d3d5 with SMTP id c17-20020a056000105100b0022e5d8ad3d5mr4309112wrx.476.1666170082582; Wed, 19 Oct 2022 02:01:22 -0700 (PDT) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id r9-20020a05600c458900b003b47b913901sm1017595wmo.1.2022.10.19.02.01.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Oct 2022 02:01:21 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 19 Oct 2022 10:01:06 +0100 Message-Id: <20221019090107.19975-7-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221019090107.19975-1-naush@raspberrypi.com> References: <20221019090107.19975-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 6/7] pipeline: ipa: raspberrypi: Use IPA cookies X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Pass an IPA cookie from the pipeline handler to the IPA and eventually back to the pipeline handler through the setDelayedControls signal. This cookie is used to index the RPiController::Metadata object to be used for the frame. The IPA cookie is then returned from DelayedControls when the frame with the applied controls has been returned from the sensor, and eventually passed back to the IPA from the signalIspPrepare signal. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: David Plowman Reviewed-by: Kieran Bingham --- include/libcamera/ipa/raspberrypi.mojom | 4 +++- src/ipa/raspberrypi/raspberrypi.cpp | 12 +++++++----- .../pipeline/raspberrypi/raspberrypi.cpp | 19 +++++++++++++------ 3 files changed, 23 insertions(+), 12 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index 40f78d9e3b3f..bb5abd895262 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -37,6 +37,8 @@ struct ISPConfig { uint32 bayerBufferId; bool embeddedBufferPresent; libcamera.ControlList controls; + uint32 ipaCookie; + uint32 delayCookie; }; struct IPAConfig { @@ -137,5 +139,5 @@ interface IPARPiEventInterface { runIsp(uint32 bufferId); embeddedComplete(uint32 bufferId); setIspControls(libcamera.ControlList controls); - setDelayedControls(libcamera.ControlList controls); + setDelayedControls(libcamera.ControlList controls, uint32 delayCookie); }; diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 9e7792f5dfbe..aed8f68aded9 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -168,6 +168,7 @@ private: RPiController::Controller controller_; std::array rpiMetadata_; unsigned int metadataIdx_; + unsigned int lastMetadataIdx_; /* * We count frames to decide if the frame must be hidden (e.g. from @@ -324,7 +325,6 @@ void IPARPi::start(const ControlList &controls, StartConfig *startConfig) firstStart_ = false; lastRunTimestamp_ = 0; - metadataIdx_ = 0; } void IPARPi::setMode(const IPACameraSensorInfo &sensorInfo) @@ -535,6 +535,8 @@ void IPARPi::signalIspPrepare(const ISPConfig &data) * avoid running the control algos for a few frames in case * they are "unreliable". */ + lastMetadataIdx_ = metadataIdx_; + metadataIdx_ = data.ipaCookie % rpiMetadata_.size(); prepareISP(data); frameCount_++; @@ -1011,11 +1013,10 @@ void IPARPi::returnEmbeddedBuffer(unsigned int bufferId) void IPARPi::prepareISP(const ISPConfig &data) { int64_t frameTimestamp = data.controls.get(controls::SensorTimestamp).value_or(0); - RPiController::Metadata lastMetadata; RPiController::Metadata &rpiMetadata = rpiMetadata_[metadataIdx_]; Span embeddedBuffer; - lastMetadata = std::move(rpiMetadata); + rpiMetadata.clear(); fillDeviceStatus(data.controls); if (data.embeddedBufferPresent) { @@ -1048,7 +1049,8 @@ void IPARPi::prepareISP(const ISPConfig &data) * current frame, or any other bits of metadata that were added * in helper_->Prepare(). */ - rpiMetadata.merge(lastMetadata); + RPiController::Metadata &lastMetadata = rpiMetadata_[lastMetadataIdx_]; + rpiMetadata.mergeCopy(lastMetadata); processPending_ = false; return; } @@ -1147,7 +1149,7 @@ void IPARPi::processStats(unsigned int bufferId) ControlList ctrls(sensorCtrls_); applyAGC(&agcStatus, ctrls); - setDelayedControls.emit(ctrls); + setDelayedControls.emit(ctrls, metadataIdx_); } } diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 23f2460190f4..8f6c6c0ce89f 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -206,7 +206,7 @@ public: void runIsp(uint32_t bufferId); void embeddedComplete(uint32_t bufferId); void setIspControls(const ControlList &controls); - void setDelayedControls(const ControlList &controls); + void setDelayedControls(const ControlList &controls, uint32_t delayCookie); void setSensorControls(ControlList &controls); void unicamTimeout(); @@ -262,6 +262,7 @@ public: struct BayerFrame { FrameBuffer *buffer; ControlList controls; + unsigned int delayCookie; }; std::queue bayerQueue_; @@ -294,6 +295,9 @@ public: /* Have internal buffers been allocated? */ bool buffersAllocated_; + /* Frame cookie to pass to the IPA */ + unsigned int ipaCookie_; + private: void checkRequestCompleted(); void fillRequestMetadata(const ControlList &bufferControls, @@ -1064,7 +1068,8 @@ int PipelineHandlerRPi::start(Camera *camera, const ControlList *controls) * Reset the delayed controls with the gain and exposure values set by * the IPA. */ - data->delayedCtrls_->reset(); + data->ipaCookie_ = 0; + data->delayedCtrls_->reset(data->ipaCookie_); data->state_ = RPiCameraData::State::Idle; @@ -1792,9 +1797,9 @@ void RPiCameraData::setIspControls(const ControlList &controls) handleState(); } -void RPiCameraData::setDelayedControls(const ControlList &controls) +void RPiCameraData::setDelayedControls(const ControlList &controls, uint32_t delayCookie) { - if (!delayedCtrls_->push(controls)) + if (!delayedCtrls_->push(controls, delayCookie)) LOG(RPI, Error) << "V4L2 DelayedControl set failed"; handleState(); } @@ -1867,13 +1872,13 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) * Lookup the sensor controls used for this frame sequence from * DelayedControl and queue them along with the frame buffer. */ - auto [ctrl, cookie] = delayedCtrls_->get(buffer->metadata().sequence); + auto [ctrl, delayCookie] = delayedCtrls_->get(buffer->metadata().sequence); /* * Add the frame timestamp to the ControlList for the IPA to use * as it does not receive the FrameBuffer object. */ ctrl.set(controls::SensorTimestamp, buffer->metadata().timestamp); - bayerQueue_.push({ buffer, std::move(ctrl) }); + bayerQueue_.push({ buffer, std::move(ctrl), delayCookie }); } else { embeddedQueue_.push(buffer); } @@ -2168,6 +2173,8 @@ void RPiCameraData::tryRunPipeline() ipa::RPi::ISPConfig ispPrepare; ispPrepare.bayerBufferId = ipa::RPi::MaskBayerData | bayerId; ispPrepare.controls = std::move(bayerFrame.controls); + ispPrepare.ipaCookie = ipaCookie_++; + ispPrepare.delayCookie = bayerFrame.delayCookie; if (embeddedBuffer) { unsigned int embeddedId = unicam_[Unicam::Embedded].getBufferId(embeddedBuffer);