From patchwork Mon Oct 31 11:45:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17741 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 41694C3288 for ; Mon, 31 Oct 2022 11:45:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9C8A56304C; Mon, 31 Oct 2022 12:45:40 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1667216740; bh=3Jo1qVg3Sfuw1vzQ2V5aXJtvy5yWBNhAi4JDuOuniPU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=is+Eb1tkuOVY6GNp5NJbmsBCGPiEYlQphMJv2jv+cDQhMCNsIdIka5+rQOSjtKO/x mut+xA+g9K3E4QCW7Xlirp/oqU+cKZKw4/KDoOWEuqvY44fxu8DeEt9Kd+EJcCZW8b PgdpXwk99jXVITUXLZwGMpipw0zFa1hUt0tW6RtSQnu5Q5T7kFfeXPOzkLniNoJMSw ZZUNNxj2gZraNLbUP0Sq6Cg/eUkz2FRDUhUcr0TpaQQe51eWkWafluDccAEgw8PlMp HZyZSQRNQED42tMYX7ZLSvzmXIuCNw2x3aHCVoi1AcFrPSEbz0BdHVqfJvd4+FjhbK BKaKIbMLEtH3w== Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7A85863044 for ; Mon, 31 Oct 2022 12:45:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="AhDeLbUU"; dkim-atps=neutral Received: by mail-wr1-x432.google.com with SMTP id bk15so15536335wrb.13 for ; Mon, 31 Oct 2022 04:45:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5snii4aeFSeyDQEkTWsmNtv8fYZz92tBHLMau6EEI/s=; b=AhDeLbUUBigs5jAZeLKHfapt/5wozo1N4I5Ct2dW5+ieDW+vCORkC3f0pgbKX8Fskn mV9PPDuGPy1PuO3bDGzlhEiIOjjkzQv3BrXfG7SZcMaEtHevtm8807b/3yMeHeWEONKy d7eXgJZNqZyxQMkcRC4oOjExt1RGz8QsI5yHprF8Klaasdr//TPoD3BR5RLaVnm2wnSk YuMe5r4KsvSC2qjYDF9wx16piBtHTBwOmUDyvg1mmJTjSU0AaPsBgm/n0WCSTlNYAuO1 +DASMyNWOZ11LhAWIfF9tVyLLBHF2gDiPEpYWTBY/Lu1QTutoPlTM3b+SoAwXaA1sZcC UTOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5snii4aeFSeyDQEkTWsmNtv8fYZz92tBHLMau6EEI/s=; b=MKKbbFLJrMINDIv6x+OD6k08s0xpoyt5Knf3AfT/vVqY2TB+cGFo3qv7t4RH9i4zI7 +lgNx9TBaRWVtaP6d/vkXOk4565qln/vZ2yQeAksbx1o8ZRiHnXgXsxMMXuNOdljnQ3y A79VG4nY1ND5CQfvvYnnnrMntEVGYSEQMcfS/3DxGeQ+NX57KgohOJ437+eU9/kouOOZ DjOouiCIz1ECBbkg/r1nLoryyzTHAJmehwrlgNsi3KDCy4qsPP6yo40UQthrojEVsyRf ZhJTlqbAEjrPLdoUqDIpy7hsHM/x23JI+fmRrmo/MtDc53gImiug2ZaC3ipT+q5iqLV3 tdUw== X-Gm-Message-State: ACrzQf3QHEeh5UP3NTBTp4WxjJwCwiw/Vxa6kMF+owKgVOAE7tONUEIi KV8Ci+FrUgB6ybEfGftko1z8NYv4e+mGOg== X-Google-Smtp-Source: AMsMyM4oPcnPpkg7PGNAWFCC6EONDNp95A37W9JDinmGEwdNQDS2yXsN2hMOLRJlWV80kFAknjxhQQ== X-Received: by 2002:adf:c082:0:b0:236:d057:3fff with SMTP id d2-20020adfc082000000b00236d0573fffmr1979121wrf.206.1667216732784; Mon, 31 Oct 2022 04:45:32 -0700 (PDT) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id h11-20020adff18b000000b0023677e1157fsm6936354wro.56.2022.10.31.04.45.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Oct 2022 04:45:31 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Mon, 31 Oct 2022 11:45:21 +0000 Message-Id: <20221031114522.14215-7-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221031114522.14215-1-naush@raspberrypi.com> References: <20221031114522.14215-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 6/7] pipeline: ipa: raspberrypi: Use IPA cookies X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Pass an IPA cookie from the pipeline handler to the IPA and eventually back to the pipeline handler through the setDelayedControls signal. This cookie is used to index the RPiController::Metadata object to be used for the frame. The IPA cookie is then returned from DelayedControls when the frame with the applied controls has been returned from the sensor, and eventually passed back to the IPA from the signalIspPrepare signal. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: David Plowman Reviewed-by: Kieran Bingham --- include/libcamera/ipa/raspberrypi.mojom | 6 ++- src/ipa/raspberrypi/raspberrypi.cpp | 42 ++++++++++--------- .../pipeline/raspberrypi/raspberrypi.cpp | 16 ++++--- 3 files changed, 37 insertions(+), 27 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index 40f78d9e3b3f..325d2d855bc0 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -37,6 +37,8 @@ struct ISPConfig { uint32 bayerBufferId; bool embeddedBufferPresent; libcamera.ControlList controls; + uint32 ipaContext; + uint32 delayContext; }; struct IPAConfig { @@ -127,7 +129,7 @@ interface IPARPiInterface { */ unmapBuffers(array ids); - [async] signalStatReady(uint32 bufferId); + [async] signalStatReady(uint32 bufferId, uint32 ipaContext); [async] signalQueueRequest(libcamera.ControlList controls); [async] signalIspPrepare(ISPConfig data); }; @@ -137,5 +139,5 @@ interface IPARPiEventInterface { runIsp(uint32 bufferId); embeddedComplete(uint32 bufferId); setIspControls(libcamera.ControlList controls); - setDelayedControls(libcamera.ControlList controls); + setDelayedControls(libcamera.ControlList controls, uint32 delayContext); }; diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 799a4fe70000..194171a8bc96 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -126,7 +126,7 @@ public: ControlList *controls, IPAConfigResult *result) override; void mapBuffers(const std::vector &buffers) override; void unmapBuffers(const std::vector &ids) override; - void signalStatReady(const uint32_t bufferId) override; + void signalStatReady(const uint32_t bufferId, uint32_t ipaContext) override; void signalQueueRequest(const ControlList &controls) override; void signalIspPrepare(const ISPConfig &data) override; @@ -137,9 +137,9 @@ private: void queueRequest(const ControlList &controls); void returnEmbeddedBuffer(unsigned int bufferId); void prepareISP(const ISPConfig &data); - void reportMetadata(); - void fillDeviceStatus(const ControlList &sensorControls); - void processStats(unsigned int bufferId); + void reportMetadata(unsigned int ipaContext); + void fillDeviceStatus(const ControlList &sensorControls, unsigned int ipaContext); + void processStats(unsigned int bufferId, unsigned int ipaContext); void applyFrameDurations(Duration minFrameDuration, Duration maxFrameDuration); void applyAGC(const struct AgcStatus *agcStatus, ControlList &ctrls); void applyAWB(const struct AwbStatus *awbStatus, ControlList &ctrls); @@ -509,14 +509,16 @@ void IPARPi::unmapBuffers(const std::vector &ids) } } -void IPARPi::signalStatReady(uint32_t bufferId) +void IPARPi::signalStatReady(uint32_t bufferId, uint32_t ipaContext) { + unsigned int context = ipaContext % rpiMetadata_.size(); + if (++checkCount_ != frameCount_) /* assert here? */ LOG(IPARPI, Error) << "WARNING: Prepare/Process mismatch!!!"; if (processPending_ && frameCount_ > mistrustCount_) - processStats(bufferId); + processStats(bufferId, context); - reportMetadata(); + reportMetadata(context); statsMetadataComplete.emit(bufferId & MaskID, libcameraMetadata_); } @@ -540,9 +542,9 @@ void IPARPi::signalIspPrepare(const ISPConfig &data) runIsp.emit(data.bayerBufferId & MaskID); } -void IPARPi::reportMetadata() +void IPARPi::reportMetadata(unsigned int ipaContext) { - RPiController::Metadata &rpiMetadata = rpiMetadata_[0]; + RPiController::Metadata &rpiMetadata = rpiMetadata_[ipaContext]; std::unique_lock lock(rpiMetadata); /* @@ -1009,12 +1011,12 @@ void IPARPi::returnEmbeddedBuffer(unsigned int bufferId) void IPARPi::prepareISP(const ISPConfig &data) { int64_t frameTimestamp = data.controls.get(controls::SensorTimestamp).value_or(0); - RPiController::Metadata lastMetadata; - RPiController::Metadata &rpiMetadata = rpiMetadata_[0]; + unsigned int ipaContext = data.ipaContext % rpiMetadata_.size(); + RPiController::Metadata &rpiMetadata = rpiMetadata_[ipaContext]; Span embeddedBuffer; - lastMetadata = std::move(rpiMetadata); - fillDeviceStatus(data.controls); + rpiMetadata.clear(); + fillDeviceStatus(data.controls, ipaContext); if (data.embeddedBufferPresent) { /* @@ -1046,7 +1048,9 @@ void IPARPi::prepareISP(const ISPConfig &data) * current frame, or any other bits of metadata that were added * in helper_->Prepare(). */ - rpiMetadata.merge(lastMetadata); + RPiController::Metadata &lastMetadata = + rpiMetadata_[ipaContext ? ipaContext : rpiMetadata_.size()]; + rpiMetadata.mergeCopy(lastMetadata); processPending_ = false; return; } @@ -1105,7 +1109,7 @@ void IPARPi::prepareISP(const ISPConfig &data) setIspControls.emit(ctrls); } -void IPARPi::fillDeviceStatus(const ControlList &sensorControls) +void IPARPi::fillDeviceStatus(const ControlList &sensorControls, unsigned int ipaContext) { DeviceStatus deviceStatus = {}; @@ -1121,12 +1125,12 @@ void IPARPi::fillDeviceStatus(const ControlList &sensorControls) LOG(IPARPI, Debug) << "Metadata - " << deviceStatus; - rpiMetadata_[0].set("device.status", deviceStatus); + rpiMetadata_[ipaContext].set("device.status", deviceStatus); } -void IPARPi::processStats(unsigned int bufferId) +void IPARPi::processStats(unsigned int bufferId, unsigned int ipaContext) { - RPiController::Metadata &rpiMetadata = rpiMetadata_[0]; + RPiController::Metadata &rpiMetadata = rpiMetadata_[ipaContext]; auto it = buffers_.find(bufferId); if (it == buffers_.end()) { @@ -1145,7 +1149,7 @@ void IPARPi::processStats(unsigned int bufferId) ControlList ctrls(sensorCtrls_); applyAGC(&agcStatus, ctrls); - setDelayedControls.emit(ctrls); + setDelayedControls.emit(ctrls, ipaContext); } } diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 29626c0ef14e..cae1f7b9aea3 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -206,7 +206,7 @@ public: void runIsp(uint32_t bufferId); void embeddedComplete(uint32_t bufferId); void setIspControls(const ControlList &controls); - void setDelayedControls(const ControlList &controls); + void setDelayedControls(const ControlList &controls, uint32_t delayContext); void setSensorControls(ControlList &controls); void unicamTimeout(); @@ -262,6 +262,7 @@ public: struct BayerFrame { FrameBuffer *buffer; ControlList controls; + unsigned int delayContext; }; std::queue bayerQueue_; @@ -1792,9 +1793,9 @@ void RPiCameraData::setIspControls(const ControlList &controls) handleState(); } -void RPiCameraData::setDelayedControls(const ControlList &controls) +void RPiCameraData::setDelayedControls(const ControlList &controls, uint32_t delayContext) { - if (!delayedCtrls_->push(controls, 0)) + if (!delayedCtrls_->push(controls, delayContext)) LOG(RPI, Error) << "V4L2 DelayedControl set failed"; handleState(); } @@ -1867,13 +1868,13 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) * Lookup the sensor controls used for this frame sequence from * DelayedControl and queue them along with the frame buffer. */ - auto [ctrl, cookie] = delayedCtrls_->get(buffer->metadata().sequence); + auto [ctrl, delayContext] = delayedCtrls_->get(buffer->metadata().sequence); /* * Add the frame timestamp to the ControlList for the IPA to use * as it does not receive the FrameBuffer object. */ ctrl.set(controls::SensorTimestamp, buffer->metadata().timestamp); - bayerQueue_.push({ buffer, std::move(ctrl) }); + bayerQueue_.push({ buffer, std::move(ctrl), delayContext }); } else { embeddedQueue_.push(buffer); } @@ -1923,7 +1924,8 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) * application until after the IPA signals so. */ if (stream == &isp_[Isp::Stats]) { - ipa_->signalStatReady(ipa::RPi::MaskStats | static_cast(index)); + ipa_->signalStatReady(ipa::RPi::MaskStats | static_cast(index), + requestQueue_.front()->sequence()); } else { /* Any other ISP output can be handed back to the application now. */ handleStreamBuffer(buffer, stream); @@ -2168,6 +2170,8 @@ void RPiCameraData::tryRunPipeline() ipa::RPi::ISPConfig ispPrepare; ispPrepare.bayerBufferId = ipa::RPi::MaskBayerData | bayerId; ispPrepare.controls = std::move(bayerFrame.controls); + ispPrepare.ipaContext = request->sequence(); + ispPrepare.delayContext = bayerFrame.delayContext; if (embeddedBuffer) { unsigned int embeddedId = unicam_[Unicam::Embedded].getBufferId(embeddedBuffer);